Spark-Error-Common

IDEA 开发 WordCount（Spark）报错

WordCount {}WordCount{  (args: Array[]) {    hadoopAddr = conf = SparkConf().setAppName()    sc = SparkContext(conf)    mapRdd = sc.textFile(hadoopAddr)    result = mapRdd.flatMap(_.split()).map((_)).reduceByKey(_ + _).collect()  }}

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties

15/12/08 22:07:16 INFO SparkContext: Running Spark version 1.4.0

15/12/08 22:07:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

15/12/08 22:07:18 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)

at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)

at org.apache.hadoop.util.Shell.(Shell.java:363)

at org.apache.hadoop.util.StringUtils.(StringUtils.java:79)

at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:104)

at org.apache.hadoop.security.Groups.(Groups.java:86)

at org.apache.hadoop.security.Groups.(Groups.java:66)

at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280)