Text mining with term extraction and term lookup transformations

In this recipe, you will see how text mining works. You will use a text file with some blogs as a source, extract terms from them, and then look up terms in them.

Getting ready

You need to have the Blogs.txt file available. You can get it with the code from the code download for this book.

How to do it...

1. In SSDT, add a new package to the Chapter08 SSIS project. Name it TermExtactionLookup.
2. On the Control Flow tab in the Package Designer, add two data flow tasks. Name them TermExtraction and TermLookup and connect them with the green arrow. The term extraction task should be first.
3. Click the Data Flow tab. Select the TermExtraction task. Prepare the data flow for this task.
4. Add a flat file source. Name it ImportBlogs. Set the properties for this source:
   • Create a new connection manager called BlogsTxt, whose source file is C:\SSIS2016Cookbook\Blogs.txt. Select the column names that are in the first data row checkbox.
   • Click Columns....