<!-- livebook:{"persist_outputs":true} -->
# Programming Machine Learning - Chapter 3
```elixir
Mix.install(
[
{:nx, "~> 0.4"},
{:explorer, "~> 0.4"},
{:exla, "~> 0.4"}
],
config: [
nx: [
default_backend: EXLA.Backend,
default_defn_options: [compiler: EXLA]
]
]
)
```
<!-- livebook:{"output":true} -->
```
:ok
```
## Notes
* I used the following blog post to understand how to use [Nx automatic differentation](https://hexdocs.pm/nx/intro-to-nx.html#automatic-differentiation-autograd): https://dockyard.com/blog/2021/04/08/up-and-running-nx.
* Contrary to the Python version, tensors parameters are in lower-case as Elixir intepret them as modules names (i.e. atoms).
## Load data
The data provided with the book is not using a particular format, it's just plain text. The following code converts it to CSV.
```elixir
file = File.stream!("#{__DIR__}/../book/02_first/pizza.txt")
{:ok, data} =
file
|> Enum.reduce([], fn line, acc ->
line =
line
|> String.trim()
|> String.split()
|> Enum.join(",")
[acc | [line, "\n"]]
end)
|> :binary.list_to_bin()
|> Explorer.DataFrame.load_csv(dtypes: [{"Pizzas", :float}, {"Reservations", :float}])
```
<!-- livebook:{"output":true} -->
```
{:ok,
#Explorer.DataFrame<
Polars[30 x 2]
Reservations float [13.0, 2.0, 14.0, 23.0, 13.0, ...]
Pizzas float [33.0, 16.0, 32.0, 51.0, 27.0, ...]
>}
```
<!-- livebook:{"branch_parent_index":1} -->
## Without bias
```elixir
defmodule GradientDescent do
import Nx.Defn
def train(x, y, iterations, lr) do
seed = DateTime.utc_now() |> DateTime.to_unix()
{w, _new_key} = Nx.Random.normal(Nx.Random.key(seed))
for _ <- 1..iterations, reduce: w do
w_acc -> update(x, y, lr, w_acc)
end
end
# -- Private
defnp predict(x, w) do
x * w
end
defnp loss(x, y, w) do
Nx.mean((predict(x, w) - y) ** 2)
end
defnp gradient(x, y, w) do
# Equivalent to
# w_gradient = 2 * Nx.mean(x * (predict(x, w) - y))
grad(w, &loss(x, y, &1))
end
defnp update(x, y, lr, w) do
w - gradient(x, y, w) * lr
end
end
```
<!-- livebook:{"output":true} -->
```
{:module, GradientDescent, <<70, 79, 82, 49, 0, 0, 17, ...>>, {:update, 4}}
```
<!-- livebook:{"reevaluate_automatically":true} -->
```elixir
GradientDescent.train(data["Reservations"], data["Pizzas"], 100, 0.001)
```
<!-- livebook:{"output":true} -->
```
#Nx.Tensor<
f64
EXLA.Backend<host:0, 0.4090228358.3360030739.105203>
1.8436928702010968
>
```
<!-- livebook:{"branch_parent_index":1} -->
## With bias
```elixir
defmodule GradientDescentWithBias do
import Nx.Defn
def train(x, y, iterations, lr) do
seed = DateTime.utc_now() |> DateTime.to_unix()
{w, _new_key} = Nx.Random.normal(Nx.Random.key(seed))
{b, _new_key} = Nx.Random.normal(Nx.Random.key(seed))
for _i <- 1..iterations, reduce: {w, b} do
{w_acc, b_acc} ->
# IO.puts("#{i}: #{inspect(Nx.to_number(w_acc))} #{inspect(Nx.to_number(b_acc))}")
update(x, y, lr, {w_acc, b_acc})
end
end
# -- Private
defnp predict(x, {w, b}) do
x * w + b
end
defnp loss(x, y, {w, b}) do
Nx.mean((predict(x, {w, b}) - y) ** 2)
end
defnp gradient(x, y, {w, b}) do
# Equivalent to
# w_gradient = 2 * Nx.mean(x * (predict(x, w, b) - y))
# b_gradient = 2 * Nx.Mean(predict(x, w, b) - y)
grad({w, b}, &loss(x, y, &1))
end
defnp update(x, y, lr, {w, b}) do
{new_w, new_b} = gradient(x, y, {w, b})
{w - new_w * lr, b - new_b * lr}
end
end
```
<!-- livebook:{"output":true} -->
```
{:module, GradientDescentWithBias, <<70, 79, 82, 49, 0, 0, 18, ...>>, {:update, 4}}
```
```elixir
{time, {w, b}} =
:timer.tc(
GradientDescentWithBias,
:train,
[data["Reservations"], data["Pizzas"], 200_000, 0.0001]
)
IO.puts("w=#{Nx.to_number(w)} b=#{Nx.to_number(b)} in #{time / 1_000} ms")
```
<!-- livebook:{"output":true} -->
```
20:50:21.081 [info] TfrtCpuClient created.
w=1.0811303090591726 b=13.172265257054141 in 12849.925 ms
```
<!-- livebook:{"output":true} -->
```
:ok
```
