> For the complete documentation index, see [llms.txt](https://spark-nctu.gitbook.io/spark/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://spark-nctu.gitbook.io/spark/spark-jie/spark-mllib.md).

# Spark MLlib

Spark MLlib 介紹: Linear Regression、SVM

## Linear Regression

Spark 相較其他平台有一個好處就是對 Machine Learning 的支援。為了利用此支援性，我們也找其中一個範例程式來編譯使用，在這一文章中，我們將以 Linear Regression 為範例，其參考的網頁為:\
<https://spark.apache.org/docs/2.2.0/ml-classification-regression.html#linear-regression>\
而整份參考的程式位址如下:\
<https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/ml/LinearRegressionWithElasticNetExample.scala>

首先，我們先創立資料夾，並移入 example 中的範例:

```
~$ mkdir -p liRegression/project
~$ mkdir -p liRegression/src/main/scala
~$ cp /usr/lib/spark/examples/src/main/scala/org/apache/spark/examples/mllib/LinearRegressionWithSGDExample.scala ~/liRegression/src/main/scala/LinearRegression.scala
```

考慮到對 MLlib 的支援，在宣告 `build.sbt` 時，也需要做一些修改，主要是加入 MLlib 的相依性，並更改專案名稱。

```
name := "liRegression"

version := "1.0"

scalaVersion := "2.11.6"

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.1.2"

libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.1.2"

libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.1.2"

resolvers += "Akka Repository" at "http://repo.akka.io/releases/"
```

接著

```
 val data = sc.textFile("data/mllib/ridge-data/lpsa.data")
```

修改為:

```
 val data = sc.textFile("/usr/lib/spark/data/mllib/ridge-data/lpsa.data")
```

當然，也要設定 master 為 local，如下所示:

```
val conf = new SparkConf().setAppName("LinearRegressionWithSGDExample").setMaster("local")
```

完成之後，進行 compile 可以得到以下結果:

```
$ sbt compile
[info] Loading project definition from /home/ubuntu/liRegression/project
[info] Loading settings for project liregression from build.sbt ...
[info] Set current project to liRegression (in build file:/home/ubuntu/liRegression/)
[info] Executing in batch mode. For better performance use sbt's shell
[info] Compiling 1 Scala source to /home/ubuntu/liRegression/target/scala-2.11/classes ...
[info] Done compiling.
[success] Total time: 6 s, completed Oct 30, 2018 8:12:08 AM
```