Hadoop Yarn Resource Manager Tuning (Add new queue to Yarn Resource Manager)

Need of Yarn queues: If you have some priority jobs and you don't want to effect these jobs execution because of  other jobs. Then create new queue,  give right weight to that queue and assign queue to your jobs

This is a procedure on how to add new queue to Yarn Resource Manager and assigning jobs to this queue.

Step1:  add below lines in the file /etc/hadoop/conf/fair-scheduler.xml in core nodes where hadoop-yarn-resourcemanager is running

<allocations> ..... <queue name="newqueue"> <maxrunningapps>50</maxrunningapps> <weight>6</weight> <schedulingpolicy>fair</schedulingpolicy> <fairsharepreemptiontimeout>30</fairsharepreemptiontimeout> </queue> ..... </allocations>



Step2:Restart hadoop-yarn-resourcemanager service on all applicable nodes
service hadoop-yarn-resourcemanager restart

Step3: (OPTIONAL)
10000 mb,10 vcores
Can also be added to queue definition to set the minimum amount of resources allocated to this queue. These resources cannot be used by other queues
Step4: For Hive jobs. set below property to use new queue
set mapreduce.job.queuename=newqueue;

You can see new queue in as below.