我的 Yarn Map-Reduce 作业花了很长时间。

输入文件大小: 75GB

Mapper数量: 2273

Reducer数量: 1 (在web UI中显示)

分片数量: 2273

输入文件数量: 867

集群: Apache Hadoop 2.4.0

5个节点的集群，每个节点1TB。

1个主节点和4个数据节点。

现在已经过去4小时，只完成了12%的映射。只是想知道，考虑到我的集群配置，这是否有意义或配置有问题？

yarn-site.xml

         
             yarn.nodemanager.aux-services
             mapreduce_shuffle
             
             
             yarn.nodemanager.aux- services.mapreduce.shuffle.class
             org.apache.hadoop.mapred.ShuffleHandler
             
             
             yarn.resourcemanager.resource- tracker.address
             master:8025
             
             
             yarn.resourcemanager.scheduler.address
             master:8030
             
             
              yarn.resourcemanager.scheduler.address
             master:8030
             
             
             yarn.resourcemanager.address
             master:8040
             
             
             yarn.resourcemanager.hostname
             master
             The hostname of the RM.
             
             
             yarn.scheduler.minimum-allocation-mb
             1024
             Minimum limit of memory to allocate to each container request at the Resource Manager.
             
             
             yarn.scheduler.maximum-allocation-mb
             8192
             Maximum limit of memory to allocate to each container request at the Resource Manager.
             
             
             yarn.scheduler.minimum-allocation-vcores
             1
             The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.
             
             
             yarn.scheduler.maximum-allocation-vcores
             32
             The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.
             
             
             yarn.nodemanager.resource.memory-mb
             8192
             Physical memory, in MB, to be made available to running containers
             
             
             yarn.nodemanager.resource.cpu-vcores
             4
             Number of CPU cores that can be allocated for containers.
             
             
             yarn.nodemanager.vmem-pmem-ratio
             4
              
             
   yarn.nodemanager.vmem-check-enabled
   false
   Whether virtual memory limits will be enforced for containers

使用多个输出的Map-Reduce作业。所以Reducer会发出多个文件。每台机器有15GB的RAM。运行的容器是8个。RM Web UI中可用的总内存为32GB。

任何指导都将不胜感激。提前致谢。