YARN Basics

YARN Services相关的命令:

  1. Start/Stop ResourceManager
    $HADOOP_HOME/sbin/yarn-daemon.sh start resourcemanager
    $HADOOP_HOME/sbin/yarn-daemon.sh stop resourcemanager

  2. Start/Stop NodeManager
    $HADOOP_HOME/sbin/yarn-daemon.sh start nodemanager
    $HADOOP_HOME/sbin/yarn-daemon.sh stop nodemanager

  3. YARN JobHistoryServer service
    $HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver

  4. 节点健康状况检测是YARN为每个NodeManager提供的机制,通过该机制,NodeManager可通过心跳机制将节点健康状况实时汇报给ResourceManager,而ResourceManager则会根据每个NodeManager的健康状况适当调整分配的任务数目。当NodeManager认为自己的健康状况“欠佳”时,可让ResourceManager不再分配任务,待健康状况好转时,再分配新任务。(yarn.nodemanager.health-checker.script.path)

  5. dd

Hadoop Web Interfaces

No Name Link
1 NameNode Web http://10.213.210.238:50070/dfshealth.html
2 NameNode Conf http://10.213.210.238:50070/conf
3 DataNode Web http://10.213.211.92:50075/datanode.html
4 DataNode Conf http://10.213.211.92:50075/conf
5 ResourceManager Web http://10.213.210.238:8088/cluster
6 ResourceManager Conf http://10.213.210.238:8088/conf
7 NodeManager Web http://10.213.211.92:8042/node
8 NodeManager Conf http://10.213.211.92:8042/conf
9 HBase Master Web http://10.211.200.224:16010/master-status
10 HBase RegionServer Web http://10.211.200.210:16030/rs-status

References

  1. YARN调优杂记---Good
  2. Turning YRAN
  3. Best Practices for YARN Resource Management
  4. Determine HDP Memory Configuration Settings
  5. Untangling Apache Hadoop YARN, Part 4: Fair Scheduler Queue Basics
  6. Haddop YARN配置参数
  7. Hadoop YARN配置参数剖析(1)—RM与NM相关参数
  8. YARN ResourceManager HA配置详解

results matching ""

    No results matching ""