Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Java Data Science Cookbook

You're reading from   Java Data Science Cookbook Explore the power of MLlib, DL4j, Weka, and more

Arrow left icon
Product type Paperback
Published in Mar 2017
Publisher Packt
ISBN-13 9781787122536
Length 372 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Author (1):
Arrow left icon
Rushdi Shams Rushdi Shams
Author Profile Icon Rushdi Shams
Rushdi Shams
Arrow right icon
View More author details
Toc

Table of Contents (10) Chapters Close

Preface 1. Obtaining and Cleaning Data FREE CHAPTER 2. Indexing and Searching Data 3. Analyzing Data Statistically 4. Learning from Data - Part 1 5. Learning from Data - Part 2 6. Retrieving Information from Text Data 7. Handling Big Data 8. Learn Deeply from Data 9. Visualizing Data

Extracting PDF text using Apache Tika

One of the most difficult file types for parsing and extracting data is PDF. Some PDFs are not even possible to parse because they are password-protected, while some others contain scanned texts and images. This dynamic file type, therefore, sometimes becomes the worst nightmare for data scientists. This recipe demonstrates how to extract text from PDF files using Apache Tika, given that the file is not encrypted or password-protected and contains text that is not scanned.

Getting ready

In order to perform this recipe we will require the following:

  1. Download Apache Tika 1.10 JAR file from http://archive.apache.org/dist/tika/tika-app-1.10.jar, and include it in your Eclipse project as an external Java library.
  2. Have any unlocked PDF file saved as testPDF.pdf on your C: drive.

How to do it...

  1. Create a method named convertPdf(String), which takes the name of the PDF file to be converted as parameter:
            public void convertPDF(String fileName){ 
    
  2. Create an input stream that will contain the PDF data as a stream of bytes:
            InputStream stream = null; 
    
  3. Create a try block as follows:
            try{ 
    
  4. Assign the file to the stream you have just created:
            stream = new FileInputStream(fileName); 
    
  5. There are many different parsers offered in the Apache Tika package. If you do not know which parser you are going to use, or say you have not only PDFs but also other types of documents to get converted, you should use an AutoDetectParser as follows:
            AutoDetectParser parser = new AutoDetectParser(); 
    
  6. Create a handler to handle the body content of the file. Note the -1 as the parameter of the constructor. Usually, Apache Tika is limited to handling files with at most 100,000 characters. The -1 value ensures that this limitation is overlooked by the body handler:
            BodyContentHandler handler = new BodyContentHandler(-1); 
    
  7. Create a metadata object:
            Metadata metadata = new Metadata(); 
    
  8. Call the parser() method of the parser object with all these objects you just created:
            parser.parse(stream, handler, metadata, new ParseContext()); 
    
  9. Use the tostring() method of the handler object to get the body text extracted from the file:
            System.out.println(handler.toString()); 
    
  10. Close the try block and complement it with a catch block and finally block, and close the method as follows:
            }catch (Exception e) { 
                          e.printStackTrace(); 
                     }finally { 
                         if (stream != null) 
                              try { 
                                   stream.close(); 
                              } catch (IOException e) { 
                                System.out.println("Error closing stream"); 
                              } 
                       } 
            } 
    

    The full method with the driver method in a class will be as follows. The method you have just created can be called by sending it the path and the name of the PDF file you need to convert, which is in your C: drive saved as testPDF.pdf:

    import java.io.FileInputStream; 
    import java.io.IOException; 
    import java.io.InputStream; 
    import org.apache.tika.metadata.Metadata; 
    import org.apache.tika.parser.AutoDetectParser; 
    import org.apache.tika.parser.ParseContext; 
    import org.apache.tika.sax.BodyContentHandler; 
     
    public class TestTika { 
         public static void main(String args[]) throws Exception { 
              TestTika tika = new TestTika(); 
              tika.convertPdf("C:/testPDF.pdf"); 
         } 
         public void convertPdf(String fileName){ 
              InputStream stream = null; 
              try { 
                  stream = new FileInputStream(fileName); 
                  AutoDetectParser parser = new AutoDetectParser(); 
                  BodyContentHandler handler = new BodyContentHandler(-1); 
                  Metadata metadata = new Metadata(); 
                  parser.parse(stream, handler, metadata, new 
                      ParseContext()); 
                  System.out.println(handler.toString()); 
              }catch (Exception e) { 
                  e.printStackTrace(); 
              }finally { 
                  if (stream != null) 
                       try { 
                            stream.close(); 
                       } catch (IOException e) { 
                            System.out.println("Error closing stream"); 
                       } 
              } 
         } 
    } 
    
You have been reading a chapter from
Java Data Science Cookbook
Published in: Mar 2017
Publisher: Packt
ISBN-13: 9781787122536
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image