Loading data from a local machine to HDFS

In this recipe, we are going to load data from a local machine's disk to HDFS.

Getting ready

To perform this recipe, you should have an already Hadoop running cluster.

How to do it...

Performing this recipe is as simple as copying data from one folder to another. There are a couple of ways to copy data from the local machine to HDFS.

• Using the copyFromLocal command

  • To copy the file on HDFS, let's first create a directory on HDFS and then copy the file. Here are the commands to do this:

    hadoop fs -mkdir /mydir1
    hadoop fs -copyFromLocal /usr/local/hadoop/LICENSE.txt /mydir1
    

• Using the put command

  • We will first create the directory, and then put the local file in HDFS:

    hadoop fs -mkdir /mydir2
    hadoop fs -put /usr/local/hadoop/LICENSE.txt /mydir2
    

You can validate that the files have been copied to the correct folders by listing the files:

hadoop fs -ls /mydir1
hadoop fs -ls /mydir2

How it works...

When you use HDFS copyFromLocal or the put command, the following...