“We Can’t Find the Item, Even Though It’s in the Master Data”
The Vocabulary Problem Behind Duplicate Items
- AI In Business
- 12 minutes
Two people naming the same thing usually pick different words. In a study published in 1987, the chance of a match stayed below one in five in every field the researchers tested. In their simulations, filing each object under the one word its designer preferred missed 80 to 90 percent of the time in many everyday situations. An item master works the same way: one description per item, chosen by the person who created it and searched by everyone else. When the search misses, the searcher often asks for a new item, and the master gains a duplicate.
The sentence in the title comes from our clients, and the obvious reading blames the person at the keyboard. Someone searched the item master carelessly, came away empty-handed, and requested the item as new. When the original later turns up under another name, the master holds the same item twice.
A study published in Communications of the ACM in November 1987 places the cause elsewhere. George Furnas, Thomas Landauer, Louis Gomez and Susan Dumais looked at the words people come up with unprompted to name objects in five fields tied to computer applications. In every field, the chance that two people would reach for the same word stayed below 0.2. Their simulations then tried a common design: file each object under the one word its designer preferred, and in many everyday situations it missed 80 to 90 percent of the time.
So why does a search miss an item that is there? The miss starts with the word. The item was recorded under one person’s description, and the search arrives with another person’s words. A search built on exact matches finds the creator’s words and misses everyone else’s. When the search comes back empty, requesting a new item code is the surest way to finish the job.
This article names the mechanism, shows how a missed search turns into a duplicate, and what that costs. It ends with a 15-minute test you can run on your last ten new items.
Does this sound familiar?
Five signs to check in your own company:
- A search in the item master comes back empty, and a long-serving colleague finds the item in a minute, because they remember what it was called.
- Requests for new items arrive for things the company already buys, and someone in the master data team spends time checking whether each one exists.
- The same part sits in the master under two or three codes, with descriptions that differ in abbreviations, word order, or language.
- Experienced colleagues keep their own spreadsheet of item codes, and newcomers ask them before they search.
- The system reports an item out of stock when the same part is on a shelf under another code.
If two of these hold in your company, the next question is why a search misses an item the master already holds.
Why does the search miss an item that is there?
Three explanations usually come first. The colleague searched carelessly. The training was too short. The master data is messy. Each can be partly true, and each places the fault in a person or in the records. The 1987 research places a large part of it in language.
What the researchers measured
Furnas and his co-authors started from a plain observation: almost every computer application asks its users to type the system’s own word for what they want. For newcomers, and for anyone trying something for the first time, a system therefore has to understand the words people come up with themselves. So the researchers looked at the words people produced unprompted to name objects in five fields. The answers scattered across a striking range of words.
Put plainly: ask two people to name the same thing, and in more than four cases out of five they reach for different words. The researchers then simulated several ways of building word-based access. A common one files each object under the word its designer liked best, and in many ordinary situations it missed in 80 to 90 percent of attempts.
The paper’s title gives the mechanism its name: the vocabulary problem. People use many different words for the same thing, so a system that waits for one particular word misses most of the people who come looking.
The figure returned in the authors’ later work. In 1990, three of them restated it in a paper written with two colleagues for the Journal of the American Society for Information Science. They put the odds that two people settle on the same main search word for a familiar object at under one in five. The same paper reports equally weak agreement among indexers labeling the same documents, and among the search words of trained search specialists and of everyday users.
Why the item master is the textbook case
Read an item master with these numbers in mind. Each item enters it once, with one description written by the person who created it: an abbreviation they like, a word order that suits them, sometimes the supplier’s name for the part. Everyone who searches later brings their own words. That is the design the 1987 simulations put to the test, here applied to thousands of items: one word per item, chosen by one person.
Omnit’s master data management page sums up what this costs: if the same item appears five times, the result is wrong orders, wrong stock data and disputed reports. The search finds an item for whoever already knows its code or its full description, and the record waits in the system for everyone else.
The creator chose one word. Every searcher brings another.
How a missed search becomes a duplicate
The second half of the mechanism begins when the search comes back empty. The person searching has a job to finish: an order to place, a repair to log, a delivery to book in. Another round of searching with other words costs more time and may end the same way. A request for a new code leads somewhere for certain, so the request goes in, and the item enters the master a second time under the word the new person chose.
Our piece on duplicate leads in the CRM describes the same route: a different spelling, an empty search for the existing customer, and a new record from the salesperson. It also shows how small the trigger can be. In its example, a web form stripped the accents from a client’s name, and from then on every search for that name skipped the record. Our article on the real customer record sees the same habit in a newly hired salesperson who struggled to locate the existing customer and started a second record.
Read this way, a duplicate is a second name for an existing item, stored in the wrong place. The new person’s word was useful: it is one of the words people use for that item. It belonged on the existing record as an alias, but it landed in a new record.
From here, the loop feeds itself. The duplicate splits one item’s stock, orders, and history across two codes, and the master now holds two descriptions of the same thing. The next person may find either record, or bring a third word that misses both.
A duplicate is an alias that became a record.
We explored why employees quietly work around hard-to-use enterprise software in our article Why Employees Quietly Work Around Your Software →
Where is the limit?
Four limits keep the claim precise.
The step to item masters is ours. The 1987 study covered five domains related to computer applications. Carrying the finding over to an item master is our interpretation, and the tests below measure how strongly it holds in yours.
A shared code takes the guesswork away. Where everyone searches by the same identifier, such as a manufacturer’s part number or a barcode, everyone types the same string and the vocabulary problem shrinks. It returns as soon as someone has to describe the item in words.
Small, stable teams keep the problem small. Where the same small group creates and searches the items year after year, the search box meets the same handful of vocabularies. The problem grows with the number of people who search, the years the master has been filling up, and the variety of items in it.
More names bring more candidates. Adding words has a price, and the 1990 paper weighs it. In small interactive databases, extra index terms raised recall at a steady overall precision; for very large collections, the authors flagged possible ambiguity and lost precision. In an item master, a search that accepts more words also returns more candidates, so a person makes the final choice among them.
Wherever people describe items in their own words, plan for many words per item.
What does a missed search cost?
The cost lands in three places, and each can be counted with numbers from your own systems.
The search. Every miss costs a second search, a question to a colleague, or a request to the master data team to check whether the item exists. Each case is small and the time is spread across many people, so it stays hidden in the reports.
The duplicate. A duplicate splits one item across two codes. Its stock becomes two smaller stocks, so each code can trigger a reorder for stock the shelves already hold. Its purchase volume splits as well, and the buyer negotiates with a smaller number. Its history splits too, so the next forecast for the item starts from part of the data.
The clean-up. Sooner or later someone finds the pair and merges it, moving open orders, stock, history and documents onto one code. Our article on duplicate leads makes a point that holds for items as well: fixing a record costs least on the day it is created. A duplicate caught at creation costs one check; one merged later costs the whole clean-up.
We looked at how duplicate records distort reports and sales forecasts in our article Duplicate Leads in Your CRM →
The formula fits on one line. Take the number of new items created last year. Multiply it by the share that already existed under another name, which the test below measures. Multiply the result by what one duplicate costs you: the time to request and create it, the time to find and merge it later, and the extra stock held under the second code.
Count the items created twice, and the vocabulary problem gets a price.
Is it enough to search more carefully?
Four moves usually come first. Each brings a partial gain, and each keeps one word per item at the center of the search.
The first is training and a reminder to search carefully. It asks the searcher to guess the word the creator chose, and the 1987 figures put the odds of a first-guess match at under one in five.
The second is a naming convention: a fixed order of words, a list of approved abbreviations, one language. It makes new descriptions more consistent, which helps. The searcher types their own words, and items recorded before the rule keep their old descriptions. Measured against the 1987 paper, a convention swaps one person’s preferred word for a committee’s.
The third is a one-off clean-up. A one-time merge clears today’s duplicates, and the same search habits start producing new ones the next day. Our article on duplicate leads in the CRM sees the duplicate rate return to its old level within about three months.
The fourth is an approval step for every new item. A gatekeeper in the master data team checks each request before a code is issued. It catches some duplicates, and the gatekeeper searches with the same tools and meets the same vocabulary problem. It also adds a queue, the subject of our article on why adding a new item takes weeks.
Each of these moves works on the people or on the records. The move that works on the mechanism changes what the search accepts: many words for one item, matching by meaning, and a search at the moment someone is about to create a new item.
Put the search where duplicates form: at the point of creation.
What does the research say?
Two research papers carry this article. In 1987, Furnas, Landauer, Gomez and Dumais described the vocabulary problem in Communications of the ACM and measured it: two people pick the same word in under one case in five. Their remedy was unlimited aliasing, which lets each object be reached through the many words people use for it, and the paper showed that it could improve success several times over.
The 1990 paper, by Scott Deerwester, Susan Dumais, George Furnas, Thomas Landauer and Richard Harshman, introduced a method called latent semantic indexing. It gives the motivation in plain terms. The words that index a document cover a small share of the words its readers will try. Collecting extra index words from outside a document’s own text becomes heavy work as a collection grows. The method placed words and documents in one space built from patterns of word use so that a document could sit close to a query phrased in other words.
In their worked example, a query about human-computer interaction reached two relevant memo titles worded entirely differently from the query. The authors kept their claims modest. On a set of medical abstracts, its average precision beat simple word matching by 13 percent; on a set of information science abstracts, the two scored the same. Their own summary: “These results are modestly encouraging.”
What we take from both papers: the vocabulary problem is a property of language, so the cure belongs in the system, with one record and many ways in. Where we differ is the moment that matters. Both papers study the search itself: the words a person types and the objects that come back. In an item master, the decisive moment comes after a miss, when someone is about to create a new item.
That is where we would place the meaning-based search. Before a new code is issued, it shows the closest existing items, and when the person picks one, their word stays on that record as a new name. The missed word then becomes an alias, and the duplicate stays out of the master.
What can you do tomorrow?
A self-check in five questions. Yes or no. Three or more noes mean the vocabulary problem is filling your master.
- Does your item search find an item from a description written in other words?
- Before a new item is requested, does someone search for it with at least two different words?
- Does the master hold several names per item, such as old codes, abbreviations, and the words colleagues use?
- Do you know how many of last year’s new items already existed under another name?
- Does someone own the item master and review new items for duplicates every month?
A 15-minute test: the second word. Take the last ten items created in your master. For each one, write down two other ways a colleague might describe it: a synonym, a common abbreviation, the supplier’s name for the part, the old code if there was one. Search the master with each, and note whether an older item turns up that is the same thing. The share of the ten that were already there is your duplicate rate at the point of creation.
A one-hour test: the second person. Choose five items and give three colleagues from different departments only a photo of each, or a sentence on what it is used for. Ask each colleague to find the items in the master in their usual way, and count the searches that reach the right record within two attempts. The result is your own small-scale version of the 1987 measurement.
Three first steps you can take with what you have.
- Make two searches with two different words as a rule before any request for a new item, and have the requester note both words on the request.
- When an item turns up under a second word, add the first word to it, in whatever field your system offers for alternative descriptions or search terms.
- For one month, collect the searches that came back empty and go through the list with the owner of the item master: each entry is a word worth adding to your master.
What to take away
The name: the vocabulary problem. People use many different words for the same thing, so a search that waits for one word misses most of the people looking for it.
The number: two people pick the same word in under one case in five (Furnas and colleagues, 1987). In the same study’s simulations, filing each object under its designer’s preferred word missed 80 to 90 percent of the time in many everyday situations.
The tool: the second-word test on your last ten new items.
The first step: before requesting any new item, search with two different words, and keep the word that found it on record.
Back to the sentence in the title
“We can’t find the item, even though it’s in the master data” sounds like a complaint about colleagues. Read with the 1987 numbers, it describes the master data itself: one description per item, written by one person and searched by everyone else, with an exact-match search in between.
The cure follows the research: more ways into each record, matching by meaning, and a search at the moment a new item is about to be created. Omnit’s AI Item Search, built on our CER master data framework, finds items by what they mean and knows their old codes, short forms, and alternative names. CER itself routes every request for a new item through one checked gateway in front of SAP.
The question for your own company takes 15 minutes to answer: how many of your last ten new items were already there, under another name?
Two people, two words, one item. Both words belong on one record.
Who owns the data in your company?
A missed search is one of several ways a master fills up with duplicates. If your item master feeds SAP or another ERP, Omnit’s AI Item Search finds items by meaning, old codes and short forms included, and merges the variants of one item into one record. The aim is a master data owner who can tell the CFO how many distinct items the company holds and stand behind the number.
AI Executive Briefing – Strategy & Reality for IT Decision-Makers
Sources
- Furnas, G. W., Landauer, T. K., Gomez, L. M. and Dumais, S. T. (1987). The Vocabulary Problem in Human-System Communication. Communications of the ACM, 30(11), 964-971. Read article →
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. and Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41(6), 391-407. Read article →
- Omnit. (2026). Master data management before an ERP or S/4HANA move. Open page →
- Omnit. (2026). Duplicate Leads in Your CRM. Read article →
- Omnit. (2026). Which Customer Record Is the Real One. Read article →

Csaba Fekszi
Csaba Fekszi is an IT expert with more than two decades of experience in data engineering, system architecture, and AI-driven process optimization. His work focuses on designing scalable solutions that deliver measurable business value.
Related posts

AI project data quality decides the outcome long before the first model is trained
