Python was born for system engineering and a very frequent need when working with shell scripts and shell-based software is to create and parse text. That's why Python has very powerful tools to handle text.

When looking for patterns in text, regular expressions are frequently the most common way to attach those kind of problems. They are very flexible and powerful, and even though they cannot express all kinds of grammar they frequently can handle most common cases.

The power of regular expressions comes out of the wide set of symbols and expressions they can generate. The problem is that for developers that are not used to regular expressions, they can look just like plain noise, and even people who have experience with them will frequently have to think a bit before understanding an expression like the following one:

"^(*d{3})*( |-)*d{3}( |-)*d{4}$"

This expression actually tries to detect phone numbers.

For most common cases, developers need to look for very simple patterns: for example, file extensions (does it end with .txt?), separated text, and so...

In many cases, when working with text, we might have to recognize text that is similar to other text, even when the two are not equal. This is a very common case in record linkage, finding duplicate entries, or for typing errors correction.

Finding similarity across text is not a straightforward task. If you try to go your own way, you will quickly realize that it gets complex and slow pretty soon.

The Python library provides tools to detect differences between two sequences in the difflib module. Since text itself is a sequence (a sequence of characters), we can apply the provided functions to detect similarities in strings.

Perform the following steps for this recipe:

1. Given a string, we want...

In our previous recipe, we saw how difflib can compute the similitude between two strings. This means that we can compute the similitude between two words and suggest corrections to our users.

If the set of correct words is known (which usually is for any language), we can first check if the word is in this set and, if not, we can look for the most similar one to suggest to the user the right spelling.

The steps to follow this recipe are:

1. First of all we need the set of valid words. To avoid bringing in the whole English dictionary, we will just sample some words:

dictionary = {'ability', 'able', 'about', 'above', 'accept',    
            ...

A very frequent need when showing text to users is to generate it dynamically depending on the state of the software.

Typically, this leads to code like this:

name = 'Alessandro'
messages = ['Message 1', 'Message 2']

txt = 'Hello %s, You have %s message' % (name, len(messages))
if len(messages) > 1:
    txt += 's'
txt += ':n'
for msg in messages:
    txt += msg + 'n'
print(txt)

This makes it very hard to foresee the upcoming structure of the message and it's also very hard to maintain in the long term. To generate text, it's usually more convenient to reverse the approach and instead of putting text in code, we shall put code in text. That's exactly what template engines do and, while the standard library has very complete solutions for formatting, it lacks a template engine out...

Usually when splitting strings on spaces, developers will tend to rely on str.split, which is able to serve that purpose pretty well. But when the need to split some spaces and preserve others arises, things quickly become harder and implementing a custom solution can require investing time in proper escaping.

Just rely on shlex.split instead of str.split:

>>> import shlex
>>>
>>> text = 'I was sleeping at the "Windsdale Hotel"'
>>> print(shlex.split(text))
['I', 'was', 'sleeping', 'at', 'the', 'Windsdale Hotel']

When analyzing user-provided text, we are frequently interested only in meaningful words; punctuation, spaces, and conjunctions might easily get in our way. Suppose you want to count word frequencies in a book, you don't want to end up with "world" and "world" being counted as two different words.

You have to perform the following steps:

1. Supply the text you want to clean up:

txt = """And he looked over at the alarm clock,
ticking on the chest of drawers. "God in Heaven!" he thought.
It was half past six and the hands were quietly moving forwards,
it was even later than half past, more like quarter to seven.
Had the alarm clock not rung? He could...

In many cases, a single word can be written in multiple ways. For example, users who wrote "Über" and "Uber" probably meant the same word. If you were implementing a feature like tagging for a blog, you certainly don't want to end up with two different tags for the two words.

So, before saving your tags, you might want to normalize them to plain ASCII characters so that they end up all being considered as the same tag.

What we need is a translation map that converts all accented characters to their plain representation:

import unicodedata, sys

class unaccented_map(dict):
    def __missing__(self, key):
        ch = self.get(key)
        if ch is not None:
 ...