Hadoop出现错误:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... usi

本文针对Hadoop安装后出现的WARNutil.NativeCodeLoader错误进行了详细分析,并提供了有效的解决方案。通过调整hadoop-env.sh文件中的java.library.path配置,成功解决了无法加载native-hadoop库的问题。

Hadoop出现错误:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable,解决方案

安装Hadoop的时候直接用的bin版本,根据教程安装好之后运行的时候发现出现了:WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 错误,百度很多都说是版本(32,64)问题,需要重新编译源码,历经一天的时间源码重新编译完成之后,再次运行仍旧有这个错误,google的解决方案是:

1.执行:$ export HADOOP_ROOT_LOGGER=DEBUG,console

查看具体的错误信息:

复制代码

17/03/16 11:35:47 DEBUG util.Shell: setsid exited with exit code 0
17/03/16 11:35:47 DEBUG conf.Configuration: parsing URL jar:file:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar!/core-default.xml
17/03/16 11:35:47 DEBUG conf.Configuration: parsing input stream sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream@462ac22a
17/03/16 11:35:47 DEBUG conf.Configuration: parsing URL file:/usr/local/hadoop/etc/hadoop/core-site.xml
17/03/16 11:35:47 DEBUG conf.Configuration: parsing input stream java.io.BufferedInputStream@405a2273
17/03/16 11:35:48 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.eName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)], about=, always=false, type=DEFAULT, sampleName=Ops)
17/03/16 11:35:48 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.eName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)], about=, always=false, type=DEFAULT, sampleName=Ops)
17/03/16 11:35:48 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadme=Time, value=[GetGroups], about=, always=false, type=DEFAULT, sampleName=Ops)
17/03/16 11:35:48 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
17/03/16 11:35:48 DEBUG util.KerberosName: Kerberos krb5 configuration not found, setting default realm to empty
17/03/16 11:35:48 DEBUG security.Groups:  Creating new Groups object
17/03/16 11:35:48 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/03/16 11:35:48 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
17/03/16 11:35:48 DEBUG util.NativeCodeLoader: java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
17/03/16 11:35:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/03/16 11:35:48 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/03/16 11:35:48 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/03/16 11:35:48 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
17/03/16 11:35:48 DEBUG security.UserGroupInformation: hadoop login
17/03/16 11:35:48 DEBUG security.UserGroupInformation: hadoop login commit
17/03/16 11:35:48 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: hadoop
17/03/16 11:35:48 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: hadoop" with name hadoop
17/03/16 11:35:48 DEBUG security.UserGroupInformation: User entry: "hadoop"
17/03/16 11:35:48 DEBUG security.UserGroupInformation: UGI loginUser:hadoop (auth:SIMPLE)

复制代码

其中有个warn信息,在这个信息附近找到一个:Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError,这表明是java.library.path出了问题,

解决方案是在文件hadoop-env.sh中增加:

export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib/native"  

解决问题过程中遇到的比较好的链接:

http://blog.csdn.net/l1028386804/article/details/51538611

http://blog.csdn.net/xichenguan/article/details/38797331

http://www.chinahadoop.cn/classroom/5/thread/43

http://www.powerxing.com/install-hadoop-in-centos/

内容概要:本文是一份针对全国计算机等级考试(NCRE)二级的全面备考指南,覆盖MS Office高级应用与设计、C语言程序设计和Python语言程序设计三大主流科目。文档按照“认知→题库→实战→冲刺”四步递进结构组织,系统介绍了考试基本信息、报名时间、考试形式、合格标准及科目选择建议,并深入剖析各科高频考点、真题规律与操作技巧。重点内容包括MS Office中的Word长文档排版、Excel函数应用与PPT动画设计;C语言中的指针、数组、函数及程序调试;Python中的基础语法、标准库(如turtle、random)与第三方库(如jieba、wordcloud、matplotlib)的应用。此外,还提供公共基础知识精讲、操作题逐步教学、易错题集锦、模拟软件使用指南和科学的时间管理策略,帮助考生高效备考。; 适合人群:零基础小白、临考突击型考生以及多次未通过的考生,尤其适合非计算机专业希望提升办公技能或理工科学生准备编程语言认证的学习者。; 使用场景及目标:①系统掌握NCRE二级考试所需的知识与实操技能;②通过刷真题、模拟考试和错题复盘提高应试能力;③在短时间内实现从入门到通关的跨越,一次性通过考试。; 阅读建议:建议按文档顺序逐步学习,结合官方考纲和真题进行实践操作,重视动手练习而非仅理论阅读,临考前使用模拟软件全真演练,强化时间分配与答题策略。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值