前言
还是有点问题,这有点不兼容。
安装虚拟机
VMware虚拟机安装Ubuntu(超详细图文教程)_vmware安装ubuntu-CSDN博客https://blog.csdn.net/qq_43374681/article/details/129248167Download Ubuntu Desktop | Download | Ubuntu
https://ubuntu.com/download/desktop安装redhat虚拟机_怎么安装redhat的虚拟机-CSDN博客
https://blog.csdn.net/weixin_64066303/article/details/130287039?spm=1001.2014.3001.5501有了前面安装红帽的经验,我相信Ubuntu的安装是得心应手了。
安装Java
Ubuntu下安装Java_ubuntu安装java-CSDN博客https://blog.csdn.net/JqlScala/article/details/133462691直接可以指令安装
更新所有软件包
sudo apt update
安装默认Java运行时环境(JRE)
sudo apt install default-jre
安装Java开发工具包(JDK)
sudo apt install default-jdk
验证是否安装成功
root@feng-virtual-machine:~# java -version openjdk version "11.0.22" 2024-01-16 OpenJDK Runtime Environment (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1) OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Ubuntu-0ubuntu222.04.1, mixed mode, sharing)
安装Scala
All Available Versions | The Scala Programming Language (scala-lang.org)https://www.scala-lang.org/download/all.html(Spark)学习进度十五(虚拟机(ubuntu)安装scala和使用) - 细胞何 - 博客园 (cnblogs.com)
https://www.cnblogs.com/hwh000/p/12310651.html
然后按照他的步骤,解压和配置环境。
添加(/usr/local/scala是自己的安装路径)
export SCALA_HOME=/usr/local/scala export PATH=${SCALA_HOME}/bin:$PATH
使环境变量生效
source /etc/profile
root@feng-virtual-machine:~# scala -version Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
但是结果出现了
root@feng-virtual-machine:/usr/local/scala# ./bin/scala Exception in thread "main" java.lang.NoClassDefFoundError: javax/script/Compilable at scala.tools.nsc.interpreter.ILoop.createInterpreter(ILoop.scala:118) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:911) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909) at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909) at scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97) at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909) at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:74)
./bin/scala出错(Exception in thread “main“java. lang . NoClassDefFoundError: javax/script/Compilable)_exception in thread "main" java.lang.noclassdeffou-CSDN博客https://blog.csdn.net/weixin_52350007/article/details/120569898推测是版本不兼容的问题,然后我找到了一个选择版本的
JDK Compatibility | Scala Documentation (scala-lang.org)https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
因为JDK显示是11,之后我安装了2.11.12版本的Scala还是失败了,然后安装3.0.0的没有明显的下载文件,只有一个GitHub的地址。
所以我选择下载2.13.0的版本。
按照同样的方法结果又可以了。
root@feng-virtual-machine:/usr/local/scala/bin# scala Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 11.0.22). Type in expressions for evaluation. Or try :help. scala> println("Hello World!") Hello World! scala>
虽然是出现了一个警告,但是还是可以运行。
root@feng-virtual-machine:~# cd /usr/local/scala root@feng-virtual-machine:/usr/local/scala# sudo mkdir mycode root@feng-virtual-machine:/usr/local/scala# cd ./mycode root@feng-virtual-machine:/usr/local/scala/mycode# ls root@feng-virtual-machine:/usr/local/scala/mycode# sudo vim HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# ls HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# scalac HelloWorld.scala warning: there was one deprecation warning (since 2.13.0); re-run with -deprecation for details one warning found root@feng-virtual-machine:/usr/local/scala/mycode# ls 'HelloWorld$.class' HelloWorld.class HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# scala -classpath . HelloWorld Hello World! root@feng-virtual-machine:/usr/local/scala/mycode# ls 'HelloWorld$.class' HelloWorld.class HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# cat HelloWorld.scala object HelloWorld{ def main(args: Array[String]){ println("Hello World!") } } root@feng-virtual-machine:/usr/local/scala/mycode#
按要求加上deprecation参数,显示该语法被弃用了。
root@feng-virtual-machine:/usr/local/scala/mycode# scalac -deprecation HelloWorld.scala HelloWorld.scala:3: warning: procedure syntax is deprecated: instead, add `: Unit =` to explicitly declare `main`'s return type def main(args: Array[String]){ ^ one warning found
按照要求修改之后就可以了,添加` :Unit=`
root@feng-virtual-machine:/usr/local/scala/mycode# scalac HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# cat HelloWorld.scala object HelloWorld{ def main(args: Array[String]): Unit={ println("Hello World!") } } root@feng-virtual-machine:/usr/local/scala/mycode# ls 'HelloWorld$.class' HelloWorld.class HelloWorld.scala root@feng-virtual-machine:/usr/local/scala/mycode# scala -classpath . HelloWorld Hello World! root@feng-virtual-machine:/usr/local/scala/mycode#
FinalShell
FinalShell官网 (hostbuf.com)https://www.hostbuf.com/这里还有一个软件,感兴趣的小伙伴可以尝试一下,在windows系统下连接Linux的虚拟机。
其中“名称”是自己起,“主机”是用ifconfig指令查看,“端口”默认是22,认证是root,密码就是root的密码。密码不知道可以设置一下。
【ubuntu】设置root用户密码_ubuntu设置root用户密码-CSDN博客https://blog.csdn.net/weixin_43500200/article/details/131118075我的是inet 192.168.92.130
root@feng-virtual-machine:~# ifconfig ens33: flags=4163mtu 1500 inet 192.168.92.130 netmask 255.255.255.0 broadcast 192.168.92.255 inet6 fe80::5c97:a5d8:e86f:ce76 prefixlen 64 scopeid 0x20 ether 00:0c:29:5f:3c:62 txqueuelen 1000 (以太网) RX packets 279948 bytes 161217845 (161.2 MB) RX errors 42 dropped 42 overruns 0 frame 0 TX packets 385434 bytes 102925217 (102.9 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 19 base 0x2000 lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (本地环回) RX packets 7077 bytes 854481 (854.4 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7077 bytes 854481 (854.4 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
当然输入密码的时候,密码不会回显。
root@feng-virtual-machine:~# sudo passwd root 新的密码: 重新输入新的密码: passwd:已成功更新密码 root@feng-virtual-machine:~#
补:
决定还是安装新版的Scala,JDK还是17,Scala选择了3.3.3.
Java Downloads | Oraclehttps://www.oracle.com/java/technologies/downloads/#java17
# sudo dpkg -i jdk-17_linux-x64_bin.deb
# java -version java version "17.0.10" 2024-01-16 LTS Java(TM) SE Runtime Environment (build 17.0.10+11-LTS-240) Java HotSpot(TM) 64-Bit Server VM (build 17.0.10+11-LTS-240, mixed mode, sharing) root@feng-virtual-machine:~#
然后使用了Scala,结果报错了。
scala> println("hello world") java.io.IOError: java.lang.RuntimeException: /packages cannot be represented as URI at java.base/jdk.internal.jrtfs.JrtPath.toUri(JrtPath.java:175) at scala.tools.nsc.classpath.JrtClassPath.asURLs(DirectoryClassPath.scala:216) at scala.tools.nsc.classpath.AggregateClassPath.$anonfun$asURLs$1(AggregateClassPath.scala:63) at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:118) at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:105) at scala.collection.immutable.Vector.flatMap(Vector.scala:113) at scala.tools.nsc.classpath.AggregateClassPath.asURLs(AggregateClassPath.scala:63) at scala.tools.nsc.interpreter.IMain.compilerClasspath(IMain.scala:93) at scala.tools.nsc.interpreter.IMain.makeClassLoader(IMain.scala:352) at scala.tools.nsc.interpreter.IMain.ensureClassLoader(IMain.scala:275) at scala.tools.nsc.interpreter.IMain.classLoader(IMain.scala:278) at scala.tools.nsc.interpreter.IMain.runtimeMirror$lzycompute(IMain.scala:168) at scala.tools.nsc.interpreter.IMain.runtimeMirror(IMain.scala:168) at scala.tools.nsc.interpreter.IMain.$anonfun$getModuleIfDefined$1(IMain.scala:177) at scala.tools.nsc.interpreter.IMain.getModuleIfDefined(IMain.scala:170) at scala.tools.nsc.interpreter.IMain.readRootPath(IMain.scala:289) at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.resolvePathToSymbol(IMain.scala:662) at scala.tools.nsc.interpreter.IMain$Request.resultSymbol$lzycompute(IMain.scala:919) at scala.tools.nsc.interpreter.IMain$Request.resultSymbol(IMain.scala:918) at scala.tools.nsc.interpreter.IMain$Request.typeOf$lzycompute(IMain.scala:930) at scala.tools.nsc.interpreter.IMain$Request.typeOf(IMain.scala:935) at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:897) at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:493) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:487) at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:470) at scala.tools.nsc.interpreter.shell.ILoop.interpretStartingWith(ILoop.scala:930) at scala.tools.nsc.interpreter.shell.ILoop.command(ILoop.scala:787) at scala.tools.nsc.interpreter.shell.ILoop.processLine(ILoop.scala:462) at scala.tools.nsc.interpreter.shell.ILoop.loop(ILoop.scala:485) at scala.tools.nsc.interpreter.shell.ILoop.run(ILoop.scala:1019) at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:87) at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:91) at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:102) at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:107) at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala) Caused by: java.lang.RuntimeException: /packages cannot be represented as URI ... 35 more That entry seems to have slain the compiler. Shall I replay your session? I can re-run each line except the last one. [y/n]
于是按照前面的要求下载的3.3.3版本的Scala。
Release 3.3.3 · lampepfl/dotty (github.com)https://github.com/lampepfl/dotty/releases/tag/3.3.3
# sudo tar -zxvf scala3-3.3.3.tar.gz -C /usr/local
# scala -version Scala code runner version 3.3.3 -- Copyright 2002-2024, LAMP/EPFL
scala> println("hello world") hello world
重新安装固定版本
JDK1.7.0
Java Archive Downloads - Java SE 7 (oracle.com)
如何在UBUNTU中卸载 OpenJDK,安装 Oracle JDK 17 - 生物信息刘博 - 博客园 (cnblogs.com)
列出系统上已安装的 Java 软件包
dpkg --list | grep -Ei "jdk|jre"
卸载所有 OpenJDK 版本
sudo apt-get remove *openjdk*
清理系统上已卸载的软件包和其配置文件
sudo apt-get autoremove
创建一个新的 Java 安装目录
mkdir /usr/lib/jvm
将下载的安装包移动到该目录并解压缩(保证此时的目录下有jdk-7u80-linux-x64.tar.gz这个文件)
mv jdk-7u80-linux-x64.tar.gz /usr/lib/jvm
cd /usr/lib/jvm
tar zxvf jdk-7u80-linux-x64.tar.gz
设置 JAVA_HOME 环境变量。打开 /etc/profile 文件
vim /etc/profile
在文件最后添加以下内容(jdk1.7.0_80这个不确定的话可以进入jvm目录查看)
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_80 export PATH=$JAVA_HOME/bin:$PATH
cd /usr/lib/jvm ls
执行以下命令使更改生效
source /etc/profile
验证 Java 是否已经成功安装
java -version
root@feng-virtual-machine:/usr/lib/jvm# java -version java version "1.7.0_80" Java(TM) SE Runtime Environment (build 1.7.0_80-b15) Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
Scala2.7.7
Scala 2.13.13 | The Scala Programming Language (scala-lang.org)
Scala 2.7.7.final | The Scala Programming Language (scala-lang.org)
按照前面的方法解压就行
移动压缩包到安装目录并解压(保证此时的目录下有scala-2.7.7.final.tgz这个文件)
mv scala-2.7.7.final.tgz /usr/local
cd /usr/local
ls
tar zxvf scala-2.7.7.final.tgz
更名为scala目录,因为前面配置的环境变量就是这个,这样就不用重新配置了
mv scala-2.7.7.final scala
验证Scala已经安装成功
root@feng-virtual-machine:/usr/local# scala Welcome to Scala version 2.7.7.final (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_80). Type in expressions to have them evaluated. Type :help for more information.
Hodoop2.10.2
Apache Download Mirrors
ubuntu安装hodoop详细步骤_ubuntu安装hadoop-CSDN博客
将直接解压出来到/usr/local目录下(保证此时的目录下有hadoop-2.10.2.tar.gz文件)
tar -zxvf hadoop-2.10.2.tar.gz -C /usr/local
进入/usr/local目录并更名
cd /usr/local
mv hadoop-2.10.2 hadoop
查看Hadoop版本
cd hadoop
root@feng-virtual-machine:/usr/local/hadoop# ./bin/hadoop version Hadoop 2.10.2 Subversion Unknown -r 965fd380006fa78b2315668fbc7eb432e1d8200f Compiled by ubuntu on 2022-05-24T22:35Z Compiled with protoc 2.5.0 From source with checksum d3ab737f7788f05d467784f0a86573fe This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-2.10.2.jar
新建input目录(到这里我就不知道是干嘛了)
mkdir input
将配置文件复制到input目录下
cp ./etc/hadoop/*.xml ./input
运行代码grep实例
./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar grep ./input ./output 'dfs[a-z.]+'
执行以下命令查看输出数据
root@feng-virtual-machine:/usr/local/hadoop# cat ./output/* 1 dfsadmin
hadoop伪分布式安装
修改core-site.xml 文件
vim ./etc/hadoop/core-site.xml
将代码添加到
hadoop.tmp.dir file:/usr/local/hadoop/tmp Abase for othertemporary directories. fs.defaultFS hdfs://localhost:9000
修改配置hdfs-site.xml文件
vim ./etc/hadoop/hdfs-site.xml
将下面这些代码添加到
dfs.replication 1 dfs.namenode.name.dir file:/usr/local/hadoop/tmp/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop/tmp/dfs/data
完成之后执行初始化命令
./bin/hdfs namenode -format
文件系统初始化成功后,启动HDFS
./sbin/start-dfs.sh
输入jps查看所有的java进程
jps
安装流程是这样的,就是有时候报错的莫名其妙的,最后我的版本是这些。
这个安装没成功
最后是完全按照下面的链接安装的,唯一要注意的就是要设置成ssh免密登录。
ubuntu安装hodoop详细步骤_ubuntu安装hadoop-CSDN博客https://blog.csdn.net/2301_77603488/article/details/132909277