Problem Description:

An error is reported when using MapReduce to implement data De duplication

Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses

In addition, there are: Java.Lang.NoClassDefFoundError; java.io.IOException: Cannot initialize Cluster.

Problem-solving:

These problems are due to the incomplete import of MapReduce and Hadoop dependencies:

<dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-hdfs</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.17</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-client</artifactId>
      <version>3.2.0</version>
    </dependency>

    <!--mapreduce-->
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-core</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-common</artifactId>
      <version>3.2.0</version>
    </dependency>

So far, the problem has been solved