Configuring HDFS block size
Getting ready
To step through the recipes in this chapter, make sure you have completed the recipes in Chapter 1, Hadoop Architecture and Deployment or at least understand the basic Hadoop cluster setup.
How to do it...
ssh
to the master node, which is Namenode, and navigate to the directory where Hadoop is installed. In the previous chapter, Hadoop was installed at/opt/cluster/hadoop
:$ ssh [email protected]
- Change to the
Hadoop
user, or any other user that is running Hadoop, by using the following:$ sudo su - hadoop
- Edit the
hdfs-site.xml
file and modify the parameter to reflect the changes, as shown in the following screenshot: dfs.blocksize
is the parameter that decides on the value of the HDFS block size. The unit is bytes and the default value is 64 MB in Hadoop 1 and 128 MB in Hadoop 2. The block size can be configured according to the need.- Once the changes are made to
hdfs-site.xml
, copy the file across all nodes in the cluster. - Then restart the Namenode and
datanode...