Run - Livebook.dev

Run this notebook

Use Livebook to open this notebook and explore new ideas.

It is easy to get started, on your machine or the cloud.

Click below to open and run it in your Livebook at .

# APIs ```elixir Mix.install([ {:jason, "~> 1.4"}, {:kino, "~> 0.9", override: true}, {:youtube, github: "brooklinjazz/youtube"}, {:hidden_cell, github: "brooklinjazz/hidden_cell"}, {:finch, "~> 0.16.0"} ]) ``` ## Navigation <div style="display: flex; align-items: center; width: 100%; justify-content: space-between; font-size: 1rem; color: #61758a; background-color: #f0f5f9; height: 4rem; padding: 0 1rem; border-radius: 1rem;"> <div style="display: flex;"> <i class="ri-home-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="../start.livemd">Home</a> </div> <div style="display: flex;"> <i class="ri-bug-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="https://github.com/DockYard-Academy/curriculum/issues/new?assignees=&labels=&template=issue.md&title=APIs">Report An Issue</a> </div> <div style="display: flex;"> <i class="ri-arrow-left-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="../exercises/common_components.livemd">Common Components</a> </div> <div style="display: flex;"> <a style="display: flex; color: #61758a; margin-right: 1rem;" href="../exercises/pokemon_api.livemd">Pokemon API</a> <i class="ri-arrow-right-fill"></i> </div> </div> ## Review Questions Upon completing this lesson, a student should be able to answer the following questions: * What are the common HTTP request methods and what do they do? * How do we send a cURL request? * How do we send an HTTP request with an HTTP client such as Finch? * What is JSON and how can we parse it into an Elixir term? * What are Bearer Tokens and how can we use them to authenticate and authorize a request? * Why is it bad to expose access tokens and how can we prevent or recover from doing so? ## Overview API stands for Application Programming Interface. In broad terms, an API is a way to communicate between pieces of software. Generally, API refers to web APIs, which are programs that run on the internet. The internet is a network of interconnected machines that know how to communicate with each other. We won't go deep into networking or how the internet works, as that is beyond the scope of this course. However, Crash Course Computer Science provides an excellent overview.  ```elixir YouTube.new("https://www.youtube.com/watch?v=AEaKrq3SpW8") ``` ### Client-Server Model These APIs use a [client-server](https://en.wikipedia.org/wiki/Client%E2%80%93server_model) model. Servers provide a resource, and clients request a resource. ![](https://upload.wikimedia.org/wikipedia/commons/thumb/c/c9/Client-server-model.svg/375px-Client-server-model.svg.png) You use APIs every time you open your browser (Chrome, Safari, Firefox, Edge, etc.). The browser is the client, and it requests information from a server. For example, when you search for a video on YouTube, your browser communicates with YouTube servers to retrieve the video file.  ### URL (Uniform Resource Locator) A URL (Uniform Resource Locator) is a string of characters that specifies the address of a resource on the internet. A URL consists of different parts, each with its own meaning: * **Protocol**: The protocol specifies how the client and server communicate, for example, HTTP, HTTPS, FTP, etc. * **Domain name**: The domain name is the address of the server that hosts the resource. For example, in the URL "https://www.example.com", "www.example.com" is the domain name. * **Path**: The path specifies the location of the resource on the server. It comes after the domain name and starts with a forward slash ("/"). For example, in the URL "https://www.example.com/path/to/resource.html", "/path/to/resource.html" is the path. * **Query string**: The query string contains additional parameters that modify the request or provide data to the server. It starts with a question mark ("?") and includes one or more key-value pairs separated by ampersands ("&"). For example, in the URL "https://www.example.com/search?name=example&sort=date", "name=example" and "sort=date" are query string parameters.  ### Your Turn Identify the Protocol, Domain Name, Path, and Query string (separated into separate query parameters and their values) in the following url: https://www.fakedomaindoesnotexist.com/fake/domain?greeting=hello&fake=true  ### HTTP Request Methods [HTTP (Hypertext Transfer Protocol)](https://developer.mozilla.org/en-US/docs/Web/HTTP) is the protocol used for transferring data on the web. [The HTTP request methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods), also known as verbs, indicate the desired action to be performed on the identified resource. ```mermaid flowchart LR Client --HTTP Method--> Server ``` * **GET:** The GET method requests a representation of the specified resource. GET is the most common HTTP method and is used to retrieve data. It is a safe method, which means that it should not have any side effects on the server or the resource, and it should only retrieve data. * **POST:** The POST method submits an entity to the specified resource, often causing a change in state or side effects on the server. It is used to create a new resource or to submit data to be processed by the resource identified by the URI. * **PUT:** The PUT method replaces all current representations of the target resource with the request payload. It is used to update a current resource with new data. * **PATCH:** The PATCH method applies partial modifications to a resource. It is used to update only a part of the current resource, rather than replacing the whole resource like PUT. * **DELETE:** The DELETE method deletes the specified resource. It is used to delete a resource identified by a URI. When you use a browser, you're actually using HTTP under the hood. The browser hides the details of how we use HTTP to communicate with these APIs. As developers, we want to interact with web APIs directly using the HTTP protocol to request and send information.  ### Response Codes APIs use various response codes to communicate the status of a request. These are generally called [response codes](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) and are grouped into five classes. 1. Informational responses (100–199) 2. Successful responses (200–299) 3. Redirection messages (300–399) 4. Client error responses (400–499) 5. Server error responses (500–599) For example, you might be familiar with seeing the `404` not found response code.  ### Web APIs We use web APIs to communicate with servers on the internet. These APIs provide useful functionality that would otherwise be difficult to build on our own. Here are a few example APIs. * [Stripe API](https://stripe.com/docs/api) handles Stripe payments. * [OpenWeatherAPI](https://openweathermap.org/) provides live weather information. * [JokeAPI](https://sv443.net/jokeapi/v2/#getting-started) returns a random joke. * [OpenAI API](https://platform.openai.com/docs/api-reference) generate text, images, and perform other AI related tasks. ## cURL cURL (pronounced like "curl") stands for "client for URL". cURL is a command-line tool used to transfer data to or from a server. You can make a simple cURL request by typing the following in your command line. ``` curl https://www.example.com ``` It's important to understand cURL because most APIs include documentation using examples of cURL requests. We can use these simple cURL requests to test an API or know how to send a request using our chosen Elixir API Client. Here, we use the [System](https://hexdocs.pm/elixir/System.html) module to simulate running a curl request from your command line and print the response for the sake of example. Notice it returns the HTML document of https://www.example.com. ```elixir {response, _exit_status} = System.cmd("curl", ["https://www.example.com"]) IO.puts(response) ``` ### cURL Options cURL options are used to customize and modify the behavior of curl commands. Here are some commonly used curl options: * -X: Specifies the HTTP method to be used, such as GET, POST, PUT, or DELETE. * -H: Specifies an HTTP header to include in the request, such as Content-Type or Authorization. * -d: Specifies data to include in the request body, such as form data or JSON data.  ### Your Turn Run the following fake curl request in your command line. The `\` character allows us to split up a command into multiple lines. ``` curl -X POST \ -H "Content-Type: application/json" \ -d '{"username":"johndoe","password":"password123"}' https://www.example.com/login ``` You should see a response similar to the following: ```html <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>404 - Not Found</title> </head> <body> <h1>404 - Not Found</h1> <script type="text/javascript" src="//obj.ac.bcon.ecdns.net/ec_tpm_bcon.js"></script> </body> </html> ``` ## Authentication APIs use different forms of authentication to check if a user is authorized to access certain resources. Typically most forms of authentication use some kind of access token to identify the user accessing the resource. Public APIs will not require any kind of access token, while private APIs will.  ### Bearer Tokens Bearer Tokens are a type of access token used to authenticate and authorize requests to access protected resources in APIs. Typically APIs allow you to create an account and receive an API key. We use this key as a bearer token that allows us to access API resources. For example, here's an example API request to the [Open AI API](https://platform.openai.com/docs/guides/images/usage) that requires a bearer token to create an image of a siamese cat. `$OPENAI_API_KEY` would be replaced with the actual bearer token to make the request. ``` curl https://api.openai.com/v1/images/generations \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "prompt": "a white siamese cat", "n": 1, "size": "1024x1024" }' ```  ### Your Turn First, try copy/pasting the curl request above into your command line. You should see the following error: ``` { "error": { "code": null, "message": "Invalid authorization header", "param": null, "type": "server_error" } } ``` Then, follow these steps to create a bearer token and make an authenticated request to generate an AI image. 1. Create a free Open AI Account: https://auth0.openai.com/u/signup/ 2. Find or create an API key: https://platform.openai.com/account/api-keys 3. Send the cURL request above in your command line. Make sure to replace `$OPENAI_API_KEY` with the api key you created. You should see a response that looks something like this: ```js { "created": 1683498991, "data": [ { "url": "https://oaidalleapiprodscus.blob.core.windows.net/private/org-xZKFBB8qReE9rPvITFyOR6WR/user-bOWL7A3HQkffCs8ssJEx4gsI/img-nCmoC73zliZ9YuUCAxQ3043K.png?st=2023-05-07T21%3A36%3A31Z&se=2023-05-07T23%3A36%3A31Z&sp=r&sv=2021-08-06&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2023-05-07T17%3A44%3A05Z&ske=2023-05-08T17%3A44%3A05Z&sks=b&skv=2021-08-06&sig=ovkn87xw2l981zy62Aio3GErj7bfRF7zUL8HJulfL5s%3D" } ] } ``` Go to the URL created in your browser to see the generated image. You will use this Bearer token later on in this lesson. If you run out of free credits or have any other issues, request a bearer token to use from your teacher. ## Finch Finch is one of many HTTP clients that allow us to send HTTP requests in Elixir. We've chosen to teach [Finch](https://github.com/sneako/finch) instead of other popular libraries such as [HTTPoison](https://github.com/edgurgel/httpoison) or [Req](https://github.com/wojtekmach/req) because it's included by default with Phoenix 1.7 projects. To send a request with Finch we need to start the Finch process. This is typically done in the application's supervision tree, but we'll do it here for demonstration purposes. Finch starts under a named process. We've chosen `MyApp.Finch` but this name is arbitrary and typically done for you in a Phoenix project. ```elixir Finch.start_link(name: MyApp.Finch) ``` Once the Finch process has started, to make a request we build the request information in a `Finch.Request` struct using [Finch.build/5](https://hexdocs.pm/finch/0.16.0/Finch.html#build/5). ```elixir request = Finch.build(:get, "https://www.example.com") ``` Then we provide this struct to the [Finch.request/3](https://hexdocs.pm/finch/0.16.0/Finch.html#request/3) function using the name of the Finch process to send the request. ```elixir Finch.request!(request, MyApp.Finch) ``` ## Jason JSON is a popular format for storing information in a key-value structure. ```javascript { "key1": "value1", "key2": "value2", } ``` Elixir represents JSON as a string, not a key-value structure. For example, the above in Elixir would be:  ```elixir "{\"key1\":\"value1\",\"key2\":\"value2\"}" ``` We can use the popular [Jason](https://github.com/michalmuskala/jason) to encode an Elixir term into JSON and decode JSON into an Elixir term such as a map. ```elixir Jason.decode!("{\"key1\":\"value1\",\"key2\":\"value2\"}") ``` ```elixir Jason.encode!(%{"key1" => "value1", "key2" => "value2"}) ``` APIs commonly return a JSON response. For example, here we make a request to a JokeAPI that returns a JSON response. ```elixir response = Finch.build(:get, "https://v2.jokeapi.dev/joke/Any?safe-mode&type=single&format=json") |> Finch.request!(MyApp.Finch) ``` The response body is a JSON string that we need to decode. We can get the body out of the response and then use Jason to decode it into a map. ```elixir decoded_body = Jason.decode!(response.body) ``` Now we can work with the decoded body like any Elixir map. For example we can access the "joke" field. ```elixir decoded_body["joke"] ``` ## Building A Finch Request We can build a more complex request using [Finch.build/5](https://hexdocs.pm/finch/Finch.html#build/5). It's an important skill to read documentation in cURL and convert it into an HTTP request using an HTTP client. For sake of example, we're going to convert our previous Open AI API request into a Finch request. Take the following cURL request from the Open AI documentation. ``` curl https://api.openai.com/v1/images/generations \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_TOKEN" \ -d '{ "prompt": "a white siamese cat", "n": 1, "size": "1024x1024" }' ``` There are a few key details we can gather from this request. * **Method**: The HTTP method of the request. Typically GET, POST, PATCH, PUT, or DELETE. * **Headers**: HTTP headers provide additional information about a request or response, such as the content type or authentication and are transmitted as key-value pairs. * **Content-Type**: Specifies the format of the data being sent in the request body. * **Authorization**: Includes authentication information in the request, proving the client has been authenticated and is authorized to access the resource. * **data (body)**: Contains the JSON data to be sent in the request body. We need to take the information from the cURL request and provide it to [Finch.build/5](https://hexdocs.pm/finch/Finch.html#build/5). Here's the full spec for [Finch.build/5](https://hexdocs.pm/finch/Finch.html#build/5) that we need to provide.  ```elixir Finch.build(method, url, headers \\ [], body \\ nil, opts \\ []) ```  ### Converting Our cURL Request Into Elixir Terms Here's the cURL request broken down into valid Elixir terms in the format the Finch expects. You may replace the `$OPENAI_API_TOKEN` with a valid token to see the actual API response if you like. However, you should **never** expose your API tokens, which would happen if you store this Livebook on GitHub. To avoid exposing your token, make sure to create a new token and revoke the token you used for this lesson: https://platform.openai.com/account/api-keys ```elixir method = :post url = "https://api.openai.com/v1/images/generations" # Replace The $OPEN_API_TOKEN With Your Token. # Make Sure To Revoke The Token Later To Avoid Publicly Exposing It. headers = [ {"Content-Type", "application/json"}, {"Authorization", "Bearer $OPENAI_API_TOKEN"} ] body = Jason.encode!(%{ prompt: "a white siamese cat", n: 1, size: "1024x1024" }) ``` ### How Do We Know It's A Post Request? Most of this information is taken directly from the cURL request. However, the **Method** can be assumed as a POST request, even though it isn't specified. That's because the cURL request includes the `-d` option. a GET request cannot be sent with data, so cURL inferes that it's a post request. It also says so on the [Open AI API Reference](https://platform.openai.com/docs/api-reference/images) for image generation. If we wanted to make this cURL request more explicit, it could be written with the `-X` option, but that's not necessary and not how many APIs will be documented. ``` curl https://api.openai.com/v1/images/generations \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_TOKEN" \ -d '{ "prompt": "a white siamese cat", "n": 1, "size": "1024x1024" }' ```  ### The Request Putting all of this together, we get the following Finch request, which we can decode using Jason. ```elixir request = Finch.build(method, url, headers, body) |> Finch.request!(MyApp.Finch) Jason.decode!(request.body) ``` ### Your Turn Make a Finch request for the following curl request from the [Open AI API Reference](https://platform.openai.com/docs/api-reference/chat/create). ``` curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hello!"}] }' ``` Decode the response using Jason and print the response string from the Chat API using `IO.puts/2`. <details style="background-color: lightgreen; padding: 1rem; margin: 1rem 0;"> <summary>Example Solution</summary> We've used the fake token `sk-1WHDb0NwRkq3mfyRVparT3BlbkFJ500axJFd8pZ2RKxGJ0x` to demonstrate how to replace `$OPENAI_API_TOKEN` with your bearer token. ```elixir request = Finch.build( :post, "https://api.openai.com/v1/chat/completions", [ {"Content-Type", "application/json"}, {"Authorization", "Bearer sk-1WHDb0NwRkq3mfyRVparT3BlbkFJ500axJFd8pZ2RKxGJ0x"} ], Jason.encode!(%{ model: "gpt-3.5-turbo", messages: [%{role: "user", content: "Hello!"}] }) ) |> Finch.request!(MyApp.Finch) decoded_body = Jason.decode!(request.body) [%{"message" => %{"content" => message}}] = decoded_body["choices"] IO.puts(message) ``` </details> ```elixir ``` ## Commit Your Progress DockYard Academy now recommends you use the latest [Release](https://github.com/DockYard-Academy/curriculum/releases) rather than forking or cloning our repository. Run `git status` to ensure there are no undesirable changes. Then run the following in your command line from the `curriculum` folder to commit your progress. ``` $ git add . $ git commit -m "finish APIs reading" $ git push ``` We're proud to offer our open-source curriculum free of charge for anyone to learn from at their own pace. We also offer a paid course where you can learn from an instructor alongside a cohort of your peers. We will accept applications for the June-August 2023 cohort soon. ## Navigation <div style="display: flex; align-items: center; width: 100%; justify-content: space-between; font-size: 1rem; color: #61758a; background-color: #f0f5f9; height: 4rem; padding: 0 1rem; border-radius: 1rem;"> <div style="display: flex;"> <i class="ri-home-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="../start.livemd">Home</a> </div> <div style="display: flex;"> <i class="ri-bug-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="https://github.com/DockYard-Academy/curriculum/issues/new?assignees=&labels=&template=issue.md&title=APIs">Report An Issue</a> </div> <div style="display: flex;"> <i class="ri-arrow-left-fill"></i> <a style="display: flex; color: #61758a; margin-left: 1rem;" href="../exercises/common_components.livemd">Common Components</a> </div> <div style="display: flex;"> <a style="display: flex; color: #61758a; margin-right: 1rem;" href="../exercises/pokemon_api.livemd">Pokemon API</a> <i class="ri-arrow-right-fill"></i> </div> </div>

See source

Run this notebook

Have you already installed Livebook?

Not yet? Install Livebook in just a minute

Run on your machine

with Livebook Desktop

Run in the cloud

on select platforms