Why Museum Search Is Broken — And What Semantic Search Fixes

Try this on the National Gallery of Art's own website. You remember a painting — warm candlelight, a solitary figure, probably Dutch, something melancholy about the light. You type "warm candlelight Dutch melancholy" into the search box.

You get nothing useful. Or you get results sorted by accession number. Or the system asks you to refine your medium, date range, and nationality before it will try.

The NGA's search is not unusual. Almost every major museum search works the same way: it matches against catalog fields. Title. Artist. Medium. Accession number. Date range. Classification. These fields are useful if you already know what you're looking for. They are nearly useless if you don't.

---

The Fundamental Problem

Museum catalog search is built around the assumption that search is a lookup. You have a target — a specific Rembrandt, a particular genre — and you use the interface to retrieve it. The system is optimized for provenance research, acquisition tracking, and curatorial retrieval. It was never designed for discovery.

The consequence is a hard prerequisite: to find a work, you must already know something precise about it. Artist name. A word from the title. A classification term the cataloger happened to use. If your mental model of the work doesn't match the catalog vocabulary, you're out.

flowchart LR
    A["What you remember\n'warm light, woman reading, Dutch'"]
    B["Keyword search"]
    C["Catalog fields\ntitle · artist · medium · date"]
    D["No match"]

    A --> B
    B --> C
    C --> D

    classDef memory fill:#f5f0e8,stroke:#a86845,color:#2c2926;
    classDef system fill:#ebe5d9,stroke:#6f685f,color:#2c2926;
    classDef fields fill:#fcf9f2,stroke:#d1c7b7,color:#2c2926;
    classDef result fill:#fffaf0,stroke:#a86845,color:#2c2926;
    class A memory;
    class B system;
    class C fields;
    class D result;

The painting you're thinking of is almost certainly in the collection. The collection knows everything about it. But the interface can't translate between how you remember a painting and how a cataloger described it in 1968.

---

What Semantic Search Changes

Retrievals is built around a different model. Instead of matching your query against catalog fields, it embeds your query and every artwork in the collection into the same mathematical space — a space where meaning, not vocabulary, determines proximity.

"Warm candlelight, solitary figure, Dutch" lands near a cluster of works that share those visual and emotional properties. Not because those words appear in their catalog records, but because the model that embedded them understood that those descriptions and those images occupy the same region of semantic space.

flowchart LR
    A["What you remember\n'warm light, woman reading, Dutch'"]
    B["Qwen3-VL\nembedding model"]
    C[("Shared\nsemantic space")]
    D["Nearest artworks\nby visual meaning"]
    E["Vermeer · de Hooch\nRembrandt · Leyster"]

    A --> B
    B --> C
    C --> D
    D --> E

    classDef memory fill:#f5f0e8,stroke:#a86845,color:#2c2926;
    classDef model fill:#ebe5d9,stroke:#6f685f,color:#2c2926;
    classDef space fill:#fcf9f2,stroke:#2c2926,color:#2c2926,stroke-width:2px;
    classDef result fill:#fffaf0,stroke:#a86845,color:#2c2926;
    class A memory;
    class B model;
    class C space;
    class D,E result;

The query doesn't need to match any field. It doesn't need to be a correct art-historical term. It just needs to describe, in whatever language comes naturally, what you remember or what you want to feel.

---

The Difference in Practice

Here's a direct comparison. The same intent, two systems.

Query	NGA keyword search	Retrievals
"warm candlelight solitary figure"	0 results	Rembrandt, de La Tour, Honthorst
"impressionist water reflections peaceful"	Requires medium filter + date range	Monet, Pissarro, Sisley — top results
"woman reading letter Dutch interior"	Matches if "letter" is in the title	Vermeer's Woman Reading a Letter, de Hooch's interiors
"dark stormy landscape dramatic sky"	Requires knowing "Romanticism" as a term	Turner, Rubens, Cole — by visual content
"melancholy figure in blue light"	No catalog field for mood	Works where that emotional register is visible

The last row matters most. Catalog search has no field for mood. No field for the quality of light. No field for how a painting makes you feel when you stand in front of it. Semantic search makes those things searchable because it operates on the visual and semantic content of the work itself, not on what a cataloger wrote about it.

---

What This Doesn't Fix

Semantic search changes what you can ask. It doesn't change what's in the collection.

The NGA's open-access dataset covers 68,816 objects — a substantial fraction of their holdings, with strong representation in Dutch Golden Age painting, American art, French Impressionism, and works on paper. But works still under copyright, three-dimensional objects, and some media types are absent. If the painting you're thinking of isn't in the dataset, the search won't find it.

The system also works best when you describe visual qualities — light, composition, color, mood, subject matter. It handles that register far better than highly specific iconographic terminology ("annunciation with lily" will work; obscure hagiographic references may not surface what you want if the catalog record doesn't support the visual signal).

Within those limits, it handles the cases that keyword search fails on entirely: the half-remembered image, the feeling you want to recreate, the style you can describe but not name.

---

The Underlying Shift

The difference between keyword search and semantic search is a difference in what the system assumes about the user. Keyword search assumes you know the vocabulary of the domain. Semantic search assumes you know what you want to find.

For a museum collection, that shift matters. The people most likely to benefit from a 68,000-object archive are not the scholars who already know the catalog. They're the people who remember standing in front of a painting as a child and wanting to find it again. The people who know the feeling a work produced but not the name of the artist who made it.

Retrievals is built for that second group.

Search the collection →