Wednesday, November 15, 2017

How many blocks allocated for a file in HDFS Hadoop

Run below command to know how many blocks allocated for file in HDFS. Here hdfs block size is 64 MB.

$ sudo -u hdfs hdfs fsck /path/filename -files -blocks

Ex: /shekhar/tab4.csv file size in hdfs is 320.1 MB
$ sudo -u hdfs hdfs fsck /shekhar/tab4.csv -files -blocks

[root@shekhar-server2 tmp]#
[root@shekhar-server2 tmp]# sudo -u hdfs hdfs fsck /shekhar/tab4.csv -files -blocks
Connecting to namenode via http://shekhar-server2.openstacklocal:50070
FSCK started by hdfs (auth:SIMPLE) from /10.194.10.14 for path /shekhar/tab4.csv at Thu Nov 16 05:40:35 IST 2017
/shekhar/tab4.csv 335600820 bytes, 6 block(s):  OK
0. BP-1971872654-10.194.10.14-1504721645808:blk_1073752536_11792 len=67108864 repl=3
1. BP-1971872654-10.194.10.14-1504721645808:blk_1073752537_11793 len=67108864 repl=3
2. BP-1971872654-10.194.10.14-1504721645808:blk_1073752538_11794 len=67108864 repl=3
3. BP-1971872654-10.194.10.14-1504721645808:blk_1073752539_11795 len=67108864 repl=3
4. BP-1971872654-10.194.10.14-1504721645808:blk_1073752540_11796 len=67108864 repl=3
5. BP-1971872654-10.194.10.14-1504721645808:blk_1073752541_11797 len=56500 repl=3

Status: HEALTHY
 Total size:    335600820 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      6 (avg. block size 55933470 B)
 Minimally replicated blocks:   6 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          3
 Number of racks:               1
FSCK ended at Thu Nov 16 05:40:35 IST 2017 in 1 milliseconds

The filesystem under path '/shekhar/tab4.csv' is HEALTHY
[root@shekhar-server2 tmp]#


No comments: