ERROR SparkContext: Error initializing SparkContext. java.lang.reflect.InvocationTargetException
Problem background:
Java version: 1.8.0.291
Spark version: spark-3.2.0-bin-hadoop 3.2
Windows operating system 64 bit
Spark initial installation
Problem Description:
After configuring the Java environment and spark path, the following error occurs when executing spark shell in CMD
ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetException
.....
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://LAPTOP-US4D0J27:64591/C:\classes
at java.net.URI$Parser.fail(URI.java:2845)
at java.net.URI$Parser.checkChars(URI.java:3018)
at java.net.URI$Parser.parseHierarchical(URI.java:3102)
at java.net.URI$Parser.parse(URI.java:3050)
at java.net.URI.<init>(URI.java:588)
at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)
... 70 more
Problem solving
From the bug printed on the console, it seems that there are illegal characters in the java file executed during startup, resulting in startup failure. After repeated search, we can’t find which java file he started has an error,
Try to reduce the version and start successfully after using spark-3.1.2-bin-hadoop 3.2
It may be that the configuration of the higher version has changed and the understanding is not deep. 0.0
Similar Posts:
- org.apache.spark.SparkException: A master URL must be set in your configuration
- Only one SparkContext may be running in this JVM
- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
- [Solved] ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
- [Solved] Exception in thread “main” java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V
- [Solved] Spark-HBase Error: java.lang.NoClassDefFoundError: org/htrace/Trace
- [Solved] java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;) V sets the corresponding Scala version
- [Solved] SparkSQL Error: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism
- Idea Run Scala Error: Exception in thread “main” java.lang.NoSuchMethodError:com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
- [Solved] Spark Programmer Compile error: object apache is not a member of package org