Performing FILTER By queries in Pig

After going through the various file formats that we can use to store data in HDFS, it's time to take a look at how to execute various operations in Pig. Pig is a data flow language and works in the same way as Hive by transforming the instructions given in Pig Latin to Map Reduce programs.

Getting ready

To perform this recipe, you should have a running Hadoop cluster as well as the latest version of Pig installed on it. Here, I am going to use Pig 0.15. In case you don't have the installation already, you can refer to https://pig.apache.org/docs/r0.15.0/start.html#Pig+Setup.

How to do it...

In this recipe, you will learn how to use FILTER BY in the Pig script. To do so, let's assume that we have an employee dataset that is stored in the following format (ID, name, department, and salary):

1	Tanmay	ENGINEERING	5000
2	Sneha	PRODUCTION	8000
3	Sakalya	ENGINEERING	7000
4	Avinash	SALES	6000
5	Manisha	SALES	5700
6	Vinit	FINANCE	6200

Here the columns are delimited...