Topic: Fuzzy Matching discussion

Duplicates in names and addresses are one of the most prominent
data quality issues around. There are a couple of different approaches
to find duplicates in names and addresses:

Synonyms: This is in my eyes the most basic approach. You have a list
of common translations to different words like common misspellings,
nicknames and so on. This approach is of course very depending on heavy
maintenance, and must be worked over for every language/country – and
actually works better with English than other languages like the
Germanic ones, where you use concatenated words
(like ‘Main Street’ being ‘Mainstreet’).

Match codes: You find those from very simple ones to the more
sophisticated ones – going from ignoring vowels, soundex and metaphone
(for English) to proprietary findings of all kinds. In my eyes match
codes works OK for selecting candidates for matching – but falls a bit
short when coming to actually settling the case.

Algorithms: A complex algorithm seems to be the only secure way to
actually settling if two different spelled records make up the same real
world entity. You have to deal with truncations, non phonetic typos,
rearranged words and letters and all that jazz.

Probabilistic learning: This is at bottom a variation of synonyms,
but the collection is not based on up front maintenance but collection
of users actual decisions when verifying automatic matching. You may
register the frequency and context of the decisions and combine with
match codes and algorithms. This of course requires a substantial collection.

All approaches may of course be combined – and parsing and standardisation
(a topic of its own) is often a supplementary method when you dedupe.