Enable JMX metrics for Spark.

Steps to enable JMX metrics for Spark.

1. For basic JVM metrics, append the following line to the file /etc/spark/conf.mas/spark-env.sh:

# Enable HistoryServer JMX connections on port 9991

HADOOP_JOB_HISTORYSERVER_OPTS="-Dcom.sun.management.jmxremote.port=9991 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.password.file=/opt/mas/jmx/mapred.password -Dcom.sun.management.jmxremote.access.file=/opt/mas/jmx/jmx.access"

2. For Spark Mater webUI update spark-env.sh

           export SPARK_MASTER_WEBUI_PORT=18080

       3.   For more metrics update metrics.properties

*.sink.jmx.class=org.apache.spark.metrics.sink.JmxSink

*.source.jvm.class=org.apache.spark.metrics.source.JvmSource