Normalizing text

In many cases, a single word can be written in multiple ways. For example, users who wrote "Über" and "Uber" probably meant the same word. If you were implementing a feature like tagging for a blog, you certainly don't want to end up with two different tags for the two words.

So, before saving your tags, you might want to normalize them to plain ASCII characters so that they end up all being considered as the same tag.

How to do it...

What we need is a translation map that converts all accented characters to their plain representation:

import unicodedata, sys

class unaccented_map(dict):
    def __missing__(self, key):
        ch = self.get(key)
        if ch is not None:
 ...