Background:

Divide the data into 60 pieces according to the business requirements, and start 60 applications to run each piece of data. The submission script of the application is as follows:

#/bin/sh
#LANG=zh_CN.utf8
#export LANG
export SPARK_KAFKA_VERSION=0.10
export LANG=zh_CN.UTF-8
jarspath=''
for file in `ls /home/dx/pro2.0/app01/sparkjars/*.jar`
do
  jarspath=${file},$jarspath
done
jarspath=${jarspath%?}
echo $jarspath

./bin/spark-submit.sh \
--jars $jarspath \
--properties-file ../conf/spark-properties.conf \
--verbose \
--master yarn \
--deploy-mode cluster \
--name Streaming-$2-$3-$4-$5-$1-Agg-Parser \
--driver-memory 9g \
--driver-cores 1 \
--num-executors 1 \
--executor-cores 12 \
--executor-memory 22g \
--driver-java-options "-XX:+TraceClassPaths" \
--class com.dx.app01.streaming.Main \
/home/dx/pro2.0/app01/lib/app01-streaming-driver.jar $1 $2 $3 $4 $5

368166;32676; 21547; 36816; 28857; 4328857; 27599;28857; 32622; 22914140;24VCores 64G

yarn32622;- 209171;p>

<<<<<<<<<<<<<<<<<<<<<19191919191933319191919191920202020202099999999999999999979999999999999999999999333333333191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919191919AcceptedAccepted,29366;, 24577; 25353;, 29031;, 20917;, 25191;, 4321153d;

36807;- yarn node-list20196;- 30475;- 24403;- 28857;- 36816;- containers20917;- 229141;-p>

24456;, 26174444432676;, 3682426377;”20998;” 28857;”26410;” 34987;”36164;” 283044;”20805;” 36275;”

37027;”24212;” 35813;”33021;” 20132;”43″21153;”25165;” 23545; 20294;”21482;” 20132;”2421153;” Yarn36824;”38169035823;”

[Tue Jul 30 16:33:29 +0000 2019] Application is added to the scheduler and is not yet activated. 
Queue's AM resource limit exceeded. Details : AM Partition = <DEFAULT_PARTITION>; 
AM Resource Request = <memory:9216MB(9G), vCores:1>; 
Queue Resource Limit for AM = <memory:454656MB(444G), vCores:1>; 
User AM Resource Limit of the queue = <memory:229376MB(224G), vCores:1>; 
Queue AM Resource Usage = <memory:221184MB(216G), vCores:24>;

Solution:

Error log: “queue am resource usage = & lt; memory:221184MB (216G), vC ores:24>; “means that 24 apps have been run (in the horn cluster mode, each app contains a driver, which is equivalent to AM): the driver of each app contains one vcores, occupying a total of 24 vcores; The driver memory of each app is 9g, 9g * 24 = 216g
error log: User am resource limit of the queue = & lt; memory:229376MB (224G), vC ores:1>; </The maximum number of resources used to run the application applicationmaster in the cluster is 224g, which is determined by the parameter yard.scheduler.capacity.maximum-am-resource-percent

1) yarn.scheduler.capacity.maximum-am-resource-percentage

<property>
    <!-- Maximum resources to allocate to application masters
    If this is too high application masters can crowd out actual work -->
    <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
    <value>0.5</value>
</property>

2) reduce driver memory

For more official questions about yarn capacity, please refer to the official website document: Hadoop: capacity scheduler