Extracting data from web files

Often, the data required for the purpose of reporting is not stored in a database, but instead needs to be fetched from a website. For example, customer location information, specifically the geographic coordinates used in mapping analysis, is not available internally within an organization. This information may be available on the web and can be extracted from there.

Getting ready

When extracting the data from a web file:

1. Open an existing Qlik Sense application or create a new one.
2. Open the Data load editor.
3. Click on Create new connection and select Web file.
4. The Select web file window will open.
5. Insert the following URL from which you can fetch the data: http://www.csgnetwork.com/llinfotable.html.

6. Name the connection as QlikSense Cookbook Webfile, as shown in the following screenshot:

How to do it...

1. In the list under Data connections, select QlikSense Cookbook Webfile and click on Select data. This will open up a preview window listing all the tables from the web page. When you carefully examine the table contents, you realize that it is the second table, @2, that contains the location information.
2. Check the box next to @2 and ensure that it is selected, so that the correct table is shown in the preview. The user will need to change the value under Field names to Embedded field names.
3. The preview of the table will look like the following screenshot:

4. Select all the fields from the table in the preview window. Click on Insert script to load the web data in the application.
5. Name the table as Country_Location and the script will read as follows:

Country_Location: 
LOAD 
Country, 
Capital, 
Latitude, 
Longitude 
FROM [lib://QlikSense Cookbook Webfile] 
(html, codepage is 1252, embedded labels, table is @2); 

6. Save and load the data. Once the script is successfully loaded, the Data model viewer will show the loaded table.

How it works...

Qlik Sense connects to the web file using the stored data connection. Once connected, it identifies the tables in the HTML source and lists them in the preview window.

Certain external websites require authentication in order to be accessed and Qlik Sense is unable to cope with websites that are secured in this manner. In order to get over this issue, we can use a third-party data-extraction tool. The extracted data can be stored in a data file, such as a QVD. The .qvd file can then be used as a data source in the Qlik Sense application.

There's more...

Qlik Sense can also extract data from other data formats, such as XML. The underlying principles remain the same as explained in the preceding recipes.

See also...

• The Creating a master library from the Data model viewer recipe
• The Activating the legacy mode in Qlik Sense® desktop recipe