Tag Archives: IDEA

[Solved] IDEA jar Pack Error: java.lang.NoSuchMethodError: org.yaml.snakeyaml.nodes.ScalarNode.getScalarStyle

It is said on the Internet that it is a jar package conflict, so I intend to open the dense Maven dependency graph through the dependencies of idea to find clues.

As a result, the direct persuasion was too complicated;

17:22:00.943 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
java.lang.NoSuchMethodError: org.yaml.snakeyaml.nodes.ScalarNode.getScalarStyle()Lorg/yaml/snakeyaml/DumperOptions$ScalarStyle;
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.<init>(OriginTrackedYamlLoader.java:124)
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.get(OriginTrackedYamlLoader.java:135)
	at org.springframework.boot.env.OriginTrackedYamlLoader$KeyScalarNode.get(OriginTrackedYamlLoader.java:130)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)

Later, by clicking the code link in the console error prompt, you can directly choose to go to the jar package to see the code, and immediately find the two jar packages to do things again.

found a conflict between the selenium-server-standale-3.141.59.jar and the core package in Springboot.

This is still in vue’s node_modules

/Users/dairy/StudioProjects/mogu_blog_v2/vue_mogu_web/node_modules/selenium-server/lib/runner/selenium-server-standale-3.141.59.jar

delete it and this error will be solved.

[Solved] Idea Error: Error running ‘Application‘: Command line is too long

When a new project is started in idea, it sometimes reports an error running ‘application’: command line is too long Short command line for application or aalso for spring boot default configuration, the error message is as follows:

Solution:

First find the idea/workspace.xml file inside the project, then find the <component name=”PropertiesComponent”></component > tag, which looks like this

Then add a line in the component tag <property name=”dynamic.classpath” value=”true” />, which would look like this

In this way, no error will be reported when starting the project.

Idea Run Scala Error: Exception in thread “main” java.lang.NoSuchMethodError:com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V

I.Description

Using idea + Scala + spark, the running program code is as follows:

package cn.idcast.hello

import org.apache.spark.rdd.RDD
import org.apache.spark
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext

/**
 * Author itcast
 * Desc Demo Spark Starter Case-WordCount
 */
object WordCount_bak {
  def main(args: Array[String]): Unit = {
    //TODO 1.env/preparesc/SparkContext/SparkContext execution environment
    val conf: SparkConf = new SparkConf().setAppName("wc").setMaster("local[*]")
    val sc: SparkContext = new SparkContext(conf)
    sc.setLogLevel("WARN")

    //TODO 2.source/read data
    //RDD:A Resilient Distributed Dataset (RDD): Resilient Distributed Dataset, simply understood as a distributed collection! It is as simple to use as an ordinary collection!
    // RDD [is a row of data]
    val lines: RDD[String] = sc.textFile("data/input/words.txt")

    //TODO 3.transformation/data manipulation/transformation
    //cut:RDD[one word]
    val words: RDD[String] = lines.flatMap(_.split(" "))
    //record as 1:RDD[(word, 1)]
    val wordAndOnes: RDD[(String, Int)] = words.map((_,1))
    //group aggregation:groupBy + mapValues(_.map(_. _2).reduce(_+_)) ===>group+aggregate inside Spark in one step:reduceByKey
    val result: RDD[(String, Int)] = wordAndOnes.reduceByKey(_+_)

    //TODO 4.sink/output
    //direct output
    result.foreach(println)
    //collect as a local collection and then output
    println(result.collect().toBuffer)
    //output to the specified path (can be a file/folder)
    result.repartition(1).saveAsTextFile("data/output/result")
    result.repartition(2).saveAsTextFile("data/output/result2")
    result.saveAsTextFile("data/output/result3")

    // For easy viewing of the Web-UI you can let the program sleep for a while
    Thread.sleep(1000 * 60)

    //TODO 5. Close the resource
    sc.stop()
  }
}

(forget the screenshot) an error is reported in the result: exception in thread “main” Java lang.NoSuchMethodError:com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;) V

It is said on the Internet that the jar package conflicts, but it does not solve the problem


II.Solution

Root cause of the problem: the scala version of windows is inconsistent with the scala version of spark, as shown in the figure:

This is spark’s own version, 2.12.10

I installed 2.12.11 on windows (forgot the screenshot), and later replaced it with 2.12.10 (reinstallation):

After that, it runs successfully without error

 

[Solved] idea Run gui Error: Error:Abnormal build process termination:

Error reported by idea running program:

Error:Abnormal build process termination: 
"C:\Program Files\Java\jdk1.8.0_181\bin\java" -Xmx700m -Djava.awt.headless=true -Djava.endorsed.dirs=\"\" -Djdt.compiler.useSingleThread=true -Dcompile.parallel=false -Drebuild.on.dependency.change=true -Djava.net.preferIPv4Stack=true -Dio.netty.initialSeedUniquifier=3130135518625661539 -Dfile.encoding=GBK -Djps.file.types.component.name=CommunityFileTypes -Duser.language=zh -Duser.country=CN -Didea.paths.selector=IdeaIC2017.1 "-Didea.home.path=D:\intellij\IntelliJ IDEA Community Edition 2017.1.3" -Didea.config.path=C:\Users\Administrator\.IdeaIC2017.1\config -Didea.plugins.path=C:\Users\Administrator\.IdeaIC2017.1\config\plugins -Djps.log.dir=C:/Users/Administrator/.IdeaIC2017.1/system/log/build-log "-Djps.fallback.jdk.home=D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/jre64" -Djps.fallback.jdk.version=1.8.0_112-release -Dio.netty.noUnsafe=true -Djava.io.tmpdir=C:/Users/Administrator/.IdeaIC2017.1/system/compile-server/xmpos_version_1cf37ff8/_temp_ -Djps.backward.ref.index.builder=true -Dkotlin.incremental.compilation.experimental=true -Dkotlin.daemon.enabled -Dkotlin.daemon.client.alive.path=\"C:\Users\Administrator\AppData\Local\Temp\kotlin-idea-1543655368645972493-is-running\" -classpath "D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jps-launcher.jar;C:/Program Files/Java/jdk1.8.0_121/lib/tools.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/optimizedFileManager.jar" org.jetbrains.jps.cmdline.Launcher "D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/util.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/snappy-in-java-0.5.1.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/forms_rt.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/openapi.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jps-builders-6.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/protobuf-2.5.0.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/idea_rt.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jdom.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/resources_en.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/log4j.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/annotations.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/asm-all.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jna-platform.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/nanoxml-2.2.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/oromatcher.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jgoodies-forms.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/trove4j.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jps-builders.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/netty-all-4.1.9.Final.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jps-model.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/javac2.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jna.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/commons-io-2.2.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/commons-lang-2.6.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-base-services-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-base-services-groovy-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-cli-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-core-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-logging-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-messaging-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-model-core-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-model-groovy-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-native-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-process-services-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-resources-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-tooling-api-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-tooling-extension-api.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-tooling-extension-impl.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-version-info-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-wrapper-3.3.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/guava-jdk5-17.0.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/jansi-1.14.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/kryo-2.22.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/minlog-1.2.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-freebsd-amd64-libcpp-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-freebsd-amd64-libstdcpp-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-freebsd-i386-libcpp-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-freebsd-i386-libstdcpp-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-amd64-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-amd64-ncurses5-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-amd64-ncurses6-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-i386-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-i386-ncurses5-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-linux-i386-ncurses6-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-osx-amd64-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-osx-i386-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-windows-amd64-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/native-platform-windows-i386-0.12.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/objenesis-1.2.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/reflectasm-1.07.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/resources_en.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/gson-2.5.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/jarutils.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/ant/lib/ant-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/IntelliLang/lib/intellilang-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Groovy/lib/groovy-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Groovy/lib/groovy-rt-constants.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/eclipse/lib/eclipse-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/eclipse/lib/common-eclipse-util.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/gradle/lib/gradle-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/jps/android-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/jps/android-gradle-jps.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/android-common.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/android-rt.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/sdk-common.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/repository.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/sdklib.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/common.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/jarutils.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/layoutlib-api.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/android/lib/manifest-merger.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/jps/kotlin-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/kotlin-runtime.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/kotlin-reflect.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/kotlin-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/kotlin-android-extensions-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/Kotlin/lib/kotlin-android-extensions-compiler-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/javaFX/lib/javaFX-jps-plugin.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/plugins/javaFX/lib/common-javaFX-plugin.jar" org.jetbrains.jps.cmdline.BuildMain 127.0.0.1 49368 17dc4cd1-443b-4591-a8f4-08f506e32d55 C:/Users/Administrator/.IdeaIC2017.1/system/compile-server
Build process started. Classpath: D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/jps-launcher.jar;C:/Program Files/Java/jdk1.8.0_121/lib/tools.jar;D:/intellij/IntelliJ IDEA Community Edition 2017.1.3/lib/optimizedFileManager.jarError connecting to 127.0.0.1:49368; reason: Connection refused: no further information: /127.0.0.1:49368io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /127.0.0.1:49368    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)    at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:352)    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:612)  at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:559)    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:476) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)    at org.jetbrains.jps.service.impl.SharedThreadPoolImpl.lambda$executeOnPooledThread$0(SharedThreadPoolImpl.java:42)    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: no further information
    ... 15 more
Exiting.
---------------------

Delete unused jdks:

Open the control panel, select the program or function, find the unused Java JDK and uninstall it.

[Solved] IDEA Run Error: java: JPS incremental annotation processing is disabled.

SpringCloud Compile Error: java: JPS incremental annotation processing is disabled. Compilation results on partial recompilation may be inaccurate. Use build process “jps.track.ap.dependencies” VM flag to enable/disable incremental annotation processing environment.

Solution:
1.Add: -Djps.track.ap.dependencies=false
2.Clear the caches

IDEA compile error: sun.misc.Base64decoder upgrade processing

Idea compile error report sun.misc.BASE64Decoder upgrade processing
warn:
17:01:15 /deploy/jenkins/workspace/auto-java-test/utils/ImageBase64Utils.java:67: warning: BASE64Encoder is internal proprietary API and may be removed in a future release
17:01:15 BASE64Encoder encoder = new BASE64Encoder();
import sun.misc.BASE64Decoder;
Alternative writing:
//Since JDK 1.8, the JDK public APIs of java.util.Base64.Decoder and java.util.Base64.Encoder have been provided, which can replace the JDK internal APIs of sun.misc.BASE64Decoder and sun.misc.BASE64Encoder.
//byte[] bytes = new BASE64Decoder().decodeBuffer(base64);
byte[] bytes = Base64.getDecoder().decode(base64);

//Or use method
import org.apache.commons.codec.binary.Base64;
return Base64.encodeBase64String(encrypted);

demo test class

@Test
    public void test2() throws Exception{
        //Decoder and java.util.Base64.Encoder are available from JDK 1.8 onwards, replacing the internal JDK APIs of sun.misc. .
        //byte[] bytes = new BASE64Decoder().decodeBuffer(base64);

        System.out.println("-----------------------Early Write ----------------------");
        String text = "String text";
         BASE64Encoder encoder = new BASE64Encoder();
         BASE64Decoder decoder = new BASE64Decoder();
         byte[] textByte = text.getBytes("UTF-8");
        //Code
         String encodedText = encoder.encode(textByte);
        System.out.println(encodedText);
        //Decoding
        System.out.println(new String(decoder.decodeBuffer(encodedText), "UTF-8"));

        /**
         * Compared with the Base64 codecs provided by sun.mis c suite and Apache Commons Codec, the Base64 provided by Java 8 has better performance. In the actual test of encoding and decoding speed, Base64 provided by Java 8 is at least 11 times faster than that provided by sun.mis c suite and at least 3 times faster than that provided by Apache Commons Codec. So if you want to use Base64 in Java, the Base64 category provided by the java .util package under Java 8 is definitely the first choice!
         * https://blog.csdn.net/zhou_kapenter/article/details/62890262
         */
        System.out.println("-----------------------New writing style----------------------");
        byte[] test11 = Base64.getEncoder().encode(textByte);
        System.out.println("test11_string="+new String(test11, "UTF-8"));
        byte[] bytes11 = Base64.getDecoder().decode(test11);
        System.out.println("test11="+new String(bytes11, "UTF-8"));

        System.out.println("-----------------------apache writing style----------------------");
        String test22 = org.apache.tomcat.util.codec.binary.Base64.encodeBase64String(textByte);
        System.out.println("test22_string="+test22);
        byte[] bytes22 = org.apache.tomcat.util.codec.binary.Base64.decodeBase64(test22);
        System.out.println("test22="+new String(bytes22, "UTF-8"));

        /**
         * Print results: Consistent results
         * ----------------------- early write ----------------------
         * 5a2X5Liy5paH5a2X
         * String text
         * ----------------------- new write ----------------------
         * test11 string = 5a2X5Liy5paH5a2X
         * test11=string literal
         * -----------------------apache writeup ----------------------
         * test22 string=5a2X5Liy5paH5a2X
         * test22=word string literal
         */
    }

How to Solve idea Import ojdbc Pack Error

Download online or ask colleagues to pass two jars, ojdbc6-11.2 0.7. 0. Jar and jconn3 0.jar

Put in the same folder:

CMD in this folder is as follows: press enter to enter the CMD console.

Enter the following two commands:

mvn install:install-file -DgroupId=com.oracle -DartifactId=ojdbc6 -Dversion=11.2.0.3 -Dpackaging=jar -Dfile=ojdbc6-11.2.0.3.jar

mvn install:install-file -DgroupId=com.sybase -DartifactId=jconn -Dversion=3-6.0 -Dpackaging=jar -Dfile=jconn3-6.0.jar

Represents a successful addition. For build failure, please check whether the version number, groupid, file name, etc. in the statement are consistent with the actual situation, and then run it after modification.

Note: If you have installed multiple maven versions, please check which version the current maven_home address is. This command is to install the jar locally into our local maven warehouse. If the warehouse is wrong, it won’t work if you install it.

After installation, right-click the project and update project

Click OK, and then Maven install again. No error will be reported

[Solved] IDEA Remote Operate hdfs Hadoop Error: Caused by: java.net.ConnectException: Connection refused: no further information

DEA remote operation HDFS Hadoop

Idea remote operation HDFS Hadoop, error caused by: Java net. ConnectException: Connection refused: no further information

1.source code

package com.github.td.hdfs;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.junit.Test;

import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;

/**
 * @Description: TODO
 * @Author: 
 * @DateTime: 2021/12/20 17:52
 **/
public class HdfsClient {

    @Test
    public void testMkdirs() throws IOException, URISyntaxException, InterruptedException{

        // 1 Get the system of the files
        Configuration configuration = new Configuration();
        // master == 192.168.xx.xx
        // the port of cdh version is 8020, the port of common version Hadoop is 9000.
        FileSystem fs = FileSystem.get(new URI("hdfs://master:8020"), configuration, "root");

        // 2 Create Directory
        fs.mkdirs(new Path("/hadoop/hdfs-api"));

        // 3 Close resouce
        fs.close();
    }
}

2. Error report log

java.net.ConnectException: Call From to master:8020 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:831)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:755)
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1549)
	at org.apache.hadoop.ipc.Client.call(Client.java:1491)
	at org.apache.hadoop.ipc.Client.call(Client.java:1388)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
	at com.sun.proxy.$Proxy12.mkdirs(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:657)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
	at com.sun.proxy.$Proxy13.mkdirs(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2420)
	at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2396)
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1319)
	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1316)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1333)
	at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1308)
	at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2275)
	at com.github.td.hdfs.HdfsClient.testMkdirs(HdfsClient.java:29)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: java.net.ConnectException: Connection refused: no further information
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:715)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:700)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:804)
	at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:421)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1606)
	at org.apache.hadoop.ipc.Client.call(Client.java:1435)
	... 46 more

3. cause analysis

Port problem

In the CDH environment, HDFS is port 8020, conf.set (“FS. Defaultfs”“ hdfs://192.168.0.4:8020 ”);

In a normal Hadoop environment, HDFS is a 9000 port, conf.set (“FS. Defaultfs”“ hdfs://192.168.0.121:9000 ”);

4. Solutions

Modify the port number to 9000

[Solved] IDEA: JSP Could Not Use session Build-in Object

Problem description

Cause

since the author uses Tomcat 10, the servlet dependency used is Jakarta Servlet. For details, see building a servlet using Tomcat 10. An error is reported: class XXX is not a servlet. The error is because the servlet API package is not imported.

Solution:

Import servlet API dependencies.

<!-- https://mvnrepository.com/artifact/javax.servlet/javax.servlet-api -->
<dependency>
  <groupId>javax.servlet</groupId>
  <artifactId>javax.servlet-api</artifactId>
  <version>4.0.1</version>
</dependency>

[Solved] idea Remote Submit spark Error: java.io.IOException: Failed to connect to DESKTOP-H

Idea remotely submits spark job Java io. IOException: Failed to connect to DESKTOP-H

1. Error report log

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1713)
	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:63)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:293)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
Caused by: org.apache.spark.SparkException: Exception thrown in awaitResult: 
	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205)
	at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
	at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:101)
	at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:201)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:64)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:63)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	... 4 more
Caused by: java.io.IOException: Failed to connect to DESKTOP-HSVM
	at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:245)
	at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:187)
	at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:198)
	at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:194)
	at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:190)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: 拒绝连接: DESKTOP-HSVM
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:323)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	... 1 more
Caused by: java.net.ConnectException: denied to connect
	... 11 more

2. cause analysis

Idea cannot establish a connection with the local machine while submitting spark jobs to the remote cluster and returning the results to the local machine.

Caused by: java.io.IOException: Failed to connect to DESKTOP-HSVM

3. solution

Add the native name desktop-hsvm and IP to the/etc/hosts file of the remote cluster, as shown in the following figure