Error in executing spark examples:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
18/05/25 11:09:45 INFO SparkContext: Running Spark version 2.1.1
18/05/25 11:09:45 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/05/25 11:09:45 ERROR SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:379)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
at org.apache.spark.examples.graphx.AggregateMessagesExample$.main(AggregateMessagesExample.scala:42)
at org.apache.spark.examples.graphx.AggregateMessagesExample.main(AggregateMessagesExample.scala)
18/05/25 11:09:45 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:379)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
at org.apache.spark.examples.graphx.AggregateMessagesExample$.main(AggregateMessagesExample.scala:42)
at org.apache.spark.examples.graphx.AggregateMessagesExample.main(AggregateMessagesExample.scala)
Process finished with exit code 1
Solution: add parameters to the boot VM
-Dspark.master=local
Similar Posts:
- Only one SparkContext may be running in this JVM
- [Solved] Spark-HBase Error: java.lang.NoClassDefFoundError: org/htrace/Trace
- [Solved] Exception in thread “main” java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V
- [Solved] import spark.implicits._ Red, unable to import
- ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
- [Solved] SparkException: Could not find CoarseGrainedScheduler or it has been stopped.
- [Solved] java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;) V sets the corresponding Scala version
- [Solved] Spark Programmer Compile error: object apache is not a member of package org
- Spark Program Compilation error: object apache is not a member of package org
- [Solved] ERROR Shell: Failed to locate the winutils binary in the hadoop binary path