本次准备写三篇博客分享一下sparkHadoophive的安装教程用于处理大型的离线型数据相关文件材料链接https://pan.baidu.com/s/17240ITPR14vcRku6_P0kug?pwdme15提取码me151、本次安装的各个组件版本2、先/usr/lib下创建一个jvm目录用户安装javasudo mkdir /usr/lib/jvm3、然后把你的Java包解压到那个目录下sudo tar -zxvf jdk-8u162-linux-x64.tar.gz -C /usr/lib/jvm/4、配置环境变量4.1先进入环境配置命令vim ~/.bashrc4.2配置如下内容,放在最上面就行export JAVA_HOME/usr/lib/jvm/jdk1.8.0_162 export JRE_HOME${JAVA_HOME}/jre export CLASSPATH.:${JAVA_HOME}/lib:${JRE_HOME}/lib export PATH${JAVA_HOME}/bin:$PATH4.3让我们配置的环境变量马上生效命令source ~/.bashrc5、验证Java是否配置成功6、用这个命令把Hadoop解压到/usr/local目录下sudo tar -zxf ~/下载/hadoop-3.1.3.tar.gz -C /usr/localsudo mv ./hadoop-3.1.3/ ./hadoop # 将文件夹名改为hadoopsudo chown -R hadoop ./hadoop7、查看版本信息cd /usr/local/hadoop./bin/hadoop version8、伪分布式需要配置两个文件在cd /usr/local/hadoop/etc/hadoop目录下分别是core-site.xml配置内容。解释了作用了的?xml version1.0 encodingUTF-8??xml-stylesheet typetext/xsl hrefconfiguration.xsl?!--Licensed under the Apache License, Version 2.0 (the License);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an AS IS BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License. See accompanying LICENSE file.--!-- Put site-specific property overrides in this file. --configurationpropertynamehadoop.proxyuser.hadoop.hosts/namevalue*/value/propertypropertynamehadoop.proxyuser.hadoop.groups/namevalue*/value/propertyproperty!--配置数据的临时存储位置--namehadoop.tmp.dir/namevaluefile:/usr/local/hadoop/tmp/valuedescriptionAbase for other temporary directories./description/propertyproperty!--配置数据的访问ip和端口--namefs.defaultFS/namevaluehdfs://localhost:9000/value/property/configurationhdfs-site.xml配置内容?xml version1.0 encodingUTF-8??xml-stylesheet typetext/xsl hrefconfiguration.xsl?configurationpropertynamedfs.replication/namevalue1/value/propertypropertynamedfs.namenode.name.dir/namevaluefile:/usr/local/hadoop/tmp/dfs/name/value/propertypropertynamedfs.datanode.data.dir/namevaluefile:/usr/local/hadoop/tmp/dfs/data/value/propertypropertynamedfs.permissions.enabled/namevaluefalse/value/property/configuration9、格式化cd /usr/local/hadoop./bin/hdfs namenode -format10、开启进程cd /usr/local/hadoop./sbin/start-dfs.sh #start-dfs.sh11只要看到这两个进程就成功了有问题可以留言我有空会回复的下一篇hive