This chapter requires you to have access to the OpenAI API. You can create an account and register for access at https://platform.openai.com/overview.

The OpenAI Playground is an interactive web-based interface designed to allow users to experiment with OpenAI’s language models, including ChatGPT. It’s a place where you can learn about the capabilities of these models by entering prompts and seeing the responses generated in real time. This platform acts as a sandbox where developers, researchers, and curious individuals alike can experiment, learn, and even prototype their ideas.

In the Playground, you have the freedom to engage in a wide range of activities. You can test out different versions of the AI models, experimenting with various prompts to see how the model responds, and you can play around with different parameters to influence the responses generated. It provides a real-time glimpse into how these powerful AI models think, react, and create based on your input.

Getting ready

Before you start, you need to create an OpenAI Platform account.

Navigate to https://platform.openai.com/ and sign in to your OpenAI account. If you do not have an account, you can sign up for free with an email address. Alternatively, you can log in to OpenAI with a valid Google, Microsoft, or Apple account. Follow the instructions to complete the creation of your account. You may need to verify your identity with a valid phone number.

How to do it…

1. After you have successfully logged in, navigate to Profile in the top right-hand menu, select Personal, and then select Usage from the left-hand side menu. Alternatively, you can navigate to https://platform.openai.com/account/usage after logging in. This page shows the usage of your API, but more importantly, it shows you how many credits you have available.
2. Normally, OpenAI provides you a $5 credit with a new account, which you should be able to see under the Free Trial Usage section of the page. If you do have credits, proceed to step 4. If, however, you do not have any credits, you will need to upgrade and set up a paid account.
3. You need not set up a paid account if you have received free credits. If you run out of free credits, however, here is how you can set up a paid account: select Billing from the left-hand side menu and then select Overview. Then, select the Set up paid account button. You will be prompted to enter your payment details and set a dollar threshold, which can be set to any level of spend that you are comfortable with. Note that the amount of credits required to collectively execute every single recipe contained in this book is not likely to exceed $5.
4. After you have created an OpenAI Platform account, you should be able to access the Playground by selecting Playground from the top menu bar, or by navigating to https://platform.openai.com/playground.

How it works…

The OpenAI Playground interface is, in my experience, clean, intuitive, and designed to provide users easy access to OpenAI’s powerful language models. The Playground is an excellent place to learn how the models perform under different settings, allowing you to experiment with parameters such as temperature and max tokens, which influence the randomness and length of the outputs respectively. The changes you make are instantly reflected in the model’s responses, offering immediate feedback.

As shown in Figure 1.1, the Playground consists of three sections: the System Message, the Chat Log, and the Parameters. You will learn more about these three features in the Running a completion request in the OpenAI Playground recipe.

Figure 1.1 – The OpenAI Playground

Now, your Playground is set up and ready to be used. You can use it to run completion requests and see how varying your prompts and parameters affect the response from OpenAI.

In this recipe, we will actually put the Playground in action and execute a completion request from OpenAI. Here, you will see the power of the OpenAI API and how it can be used to provide completions for virtually any prompt.

Getting ready

Ensure you have an OpenAI Platform account with available usage credits. If you don’t, please follow the Setting up your OpenAI Playground environment recipe. All the recipes in this chapter will have this same requirement.

How to do it…

Let’s go ahead and start testing the model with the Playground. Let’s create an assistant that writes marketing slogans:

1. Navigate to the OpenAI Playground.
2. In the System Message, type in the following: You are an assistant that creates marketing slogans based on descriptions of companies. Here, we are clearly instructing the model of its role and context.
3. In the Chat Log, populate the USER message with the following: A company that writes engaging mystery novels.
4. Select the Submit button on the bottom of the page.
5. You should now see a completion response from OpenAI. In my case (Figure 1.2), the response is as follows:

   Unlock the Thrilling Pages of Suspense with Our Captivating Mystery Novels!

Figure 1.2 – The OpenAI Playground with prompt and completion

Note

Since OpenAI’s LLMs are probabilistic, you will likely not see the same outputs as me. In fact, if you run this recipe multiple times, you will likely see different answers, and that is expected because it is built into the randomness of the model.

How it works…

OpenAI’s text generation models utilize a specific neural network architecture termed a transformer. Before delving deeper into this, let’s unpack some of these terms:

• Neural network architecture: At a high level, this refers to a system inspired by the human brain’s interconnected neuron structure. It’s designed to recognize patterns and can be thought of as the foundational building block for many modern AI systems.
• Transformer: This is a type of neural network design that has proven particularly effective for understanding sequences, making it ideal for tasks involving human language. It focuses on the relationships between words and their context within a sentence or larger text segment.

In machine learning, unsupervised learning typically refers to training a model without any labeled data, letting the model figure out patterns on its own. However, OpenAI’s methodology is more nuanced. The models are initially trained on a vast corpus of text data, supervised with various tasks. This helps them predict the next word in a sentence, for instance. Subsequent refinements are made using Reinforcement Learning through Human Feedback (RLHF), where the model is further improved based on feedback from human evaluators.

Through this combination of techniques and an extensive amount of data, the model starts to capture the intricacies of human language, encompassing context, tone, humor, and even sarcasm.

In this case, the completion response is provided based on both the System Message and the Chat Log. The System Message serves a critical role in shaping and guiding the responses you receive from Open AI, as it dictates the model’s persona, role, tone, and context, among other attributes. In our case, the System Message contains the persona we want the model to take: You are an assistant that creates marketing slogans based on descriptions of companies.

The Chat Log contains the history of messages that the model has access to before providing its response, which contains our prompt, A company that writes engaging mystery novels.

Finally, the parameters contain more granular settings that you can change for the model, such as temperature. These significantly change the completion response from OpenAI. We will discuss temperature and other parameters in greater detail in Chapter 3.

There’s more…

It is worth noting that ChatGPT does not read and understand the meaning behind text – instead, the responses are based on statistical probabilities based on patterns it observed during training.

The model does not understand the text in the same way that humans do; instead, the completions are generated based on statistical associations and patterns that have been trained in the model’s neural network from a large body of similar text. Now, you know how to run completion requests with the OpenAI Playground. You can try this feature out for your own prompts and see what completions you get. Try creative prompts such as write me a song about lightbulbs or more professional prompts such as explain Newton's first law.

In this recipe, we will observe how modifying the System Message affects the completion response that we receive from the model. This is important because as you begin to use the OpenAI API, you will likely adjust and refine the System Message to your specific needs, and the Playground is a great way to try that.

How to do it…

1. Navigate to the OpenAI Playground.
2. In the SYSTEM field, type in the following: You are an assistant that creates engaging and professional company names based on descriptions of companies.
3. In the Chat Log, populate the USER message with the following: A company that helps you with your taxes.
4. Select the Submit button on the bottom of the page.
5. You should now see a completion response from OpenAI. In my case in Figure 1.3, the response is as follows:

   TaxGuardian.

Figure 1.3 – The OpenAI Playground with prompt and completion

6. Hover over the ASSISTANT response and select the minus icon on the right-hand side to delete the model’s response from the Chat Log. This needs to be done because we want OpenAI to not only generate a response but generate one as the ASSISTANT.
7. Modify the System Message to You are an assistant that creates potential customer segments and marketing strategies based on descriptions of companies.
8. Select the Submit button on the bottom of the page.
9. You should now see a much longer completion response from OpenAI. In my case, as shown in Figure 1.4, the response details potential customer segments:

Figure 1.4 – The OpenAI Playground response after modifying the System Message

How it works…

The model generates a completely different type of response after modifying the System Message, even though the prompt in the Chat Log is exactly the same. The response changed from a one-word answer to a multi-paragraph response because of the instructions from the System Message. We also had to remove the default Assistant response because we want OpenAI to generate its own response instead of feeding it a response.

The System Message, being the first message in the conversation, heavily influences the thinking of the model by providing a frame of reference or context. This context is crucial because, without it, the model would lack necessary guidance to respond appropriately to subsequent user inputs. It is the cornerstone of defining your interaction with the model, allowing you to provide important context and high-level directives that steer the conversation or task at hand.

As a result, when we begin to use the OpenAI API to create business applications, careful considerations must be made to the instructions that we put into the System Message.

There’s more…

The beauty of the System Message is that that you can be as simple or as intricate with your instructions as you want. For example, here are common system messages that can be used for various purposes:

• You are an assistant that helps young students learn important concepts in science by explaining concepts in easy-to-understand language
• You are an assistant that creates marketing slogans based on descriptions of companies that are provided to you
• I am planning a birthday party 2 weeks from now for my 5 year old niece and you are my party planner that tells me what I should be doing

In this recipe, we will learn how to modify the Chat Log and how it impacts the completion response that we receive from the model. This is important because developers often find this to be the best way to fine tune a model, without actually needing to create a new model. This also follows a prompt engineering must-have of providing the model with suitable examples.

How to do it…

We can add examples of prompts and responses to the Chat Log to modify the model’s behavior. Let’s observe this with the following steps:

1. Navigate to the OpenAI Playground. If you already have messages populated, refresh the page to start afresh.
2. In the System Message, type in the following: You are an assistant that creates marketing slogans based on descriptions of companies. Here, we are clearly instructing the model of its role and context.
3. In the Chat Log, populate the USER message with the following: A company that makes ice cream.

   Select the Add message button located underneath the USER label to add a new message. Ensure that the label of the message says ASSISTANT. If it does not, select the label to toggle between USER and ASSISTANT.

   Now, type the following into the ASSISTANT message: Sham - the ice cream that never melts!.

4. Select the Add message button and ensure that the label of the message says USER instead now. Type the following into the USER message: A company that produces comedy movies.
5. Select the Add message button, and ensure that the label of the message says ASSISTANT. Type the following into the ASSISTANT message: Sham - the best way to tickle your funny bone!.
6. Repeat steps 4-5 once more, with the following USER and ASSISTANT messages, respectively: A company that provides legal assistance to businesses, and Sham - we know business law!. At this point, you should see the following:

Figure 1.5 – The OpenAI Playground with Chat Logs populated

7. Finally, select the Add message button, and create a USER message with the following: A company that writes engaging mystery novels.
8. Select the Submit button on the bottom of the page.
9. You should now see a completion response from OpenAI. In my case (Figure 1.6), the response is as follows:

   Sham – unravel the secrets with our captivating mysteries!

   Yours may be different, but the response you see will definitely start with the word “Sham –” and end with an exclamation point. In this way, we have trained the model to only give us completion responses in that format.

Figure 1.6 – The OpenAI Playground with completion, after changing the Chat Log

How it works…

As we learned in the Running a completion request in the OpenAI Playground recipe, ChatGPT and its GPT models are built on a transformer architecture, which processes input and generates responses based on the immediate chat history it has been given. It doesn’t have an ongoing memory of past interactions or a stored understanding of context outside the immediate conversation. The Chat Log has a significant impact on the model’s completions. When the model receives a prompt, it takes into account the most recent prompt, the System Message, and all the preceding messages in the Chat Log.

We can observe this in the Playground by providing our own sets of User and Assistant messages, and then see how the model changes its completion, as we did in the preceding steps.

In particular, the model has detected two patterns in the Chat Log and then generated the completion to follow that behavior:

• The model detected that all manual Assistant completions begin with the word Sham, and so it added that prefix to its completion
• The model identified that all slogans end with an exclamation point, and so when it generated the completion, it also added in an exclamation point

Overall, the Chat Log can be used to train the model to generate certain types of completions that the user wants to create. In addition, the Chat Log helps the model understand and maintain the context of the bigger conversation.

For example, if you added a User message with What is an airplane? and followed it up with another User message of How do they fly?, the model would understand that they refers to the airplane because of the Chat Log.

Prompt engineering

The Chat Log plays a pivotal role in influencing the model’s completions, and this observation is a glimpse into the broader realm of prompt engineering. Prompt engineering is a technique where the input or prompt given to a model is carefully crafted to guide the model towards producing a desired output.

Within the sphere of prompt engineering, there are a few notable concepts, as follows:

• Zero-shot prompting: Here, the model is given a task that it hasn’t been explicitly trained on. It relies entirely on its pre-existing knowledge and training to generate a relevant response. In essence, it’s like asking the model to perform a task cold, without any prior examples.
• Few-shot prompting: This involves providing the model with a small number of examples related to the desired task. The aim is to nudge the model into recognizing the pattern or context and then generating a relevant completion based on the few examples given.

Understanding these nuances in how prompts can be engineered allows users to leverage ChatGPT’s capabilities more effectively, tailoring interactions to their specific needs.

Overall, the Chat Log (and the System Message, as we learned in the earlier recipe) is a great low-touch method of aligning the completion responses from OpenAI to a desired target, without needing to fine-tune the model itself. Now that we’ve used the Playground to test prompts and completions, it’s time to use the actual OpenAI API.

The OpenAI Playground is a great way to test model completions and provides the exact same responses that you would receive with the OpenAI API, but it serves a different purpose. While the Playground is treated as a sandbox for experimentation that is easy to use, interactive, and great for learning, the OpenAI API enables users to integrate the models directly into their applications.

Getting ready

In order to make HTTP requests, we need a client such as Postman to post requests to the API. We also need to generate an API key, a unique identifier that authorizes us to make requests to OpenAI’s API.

In this recipe and book, we will select Postman as our API client, but note that many alternatives exist, including WireMock, Smartbear, and Paw. We have chosen Postman because it is the most widely used tool, it’s cross-platform (meaning that it works on Windows, Mac, and Linux), and finally, for our use case it’s completely free.

Installing Postman

Postman is the widely recognized stand-alone tool for testing APIs, used by over 17 million users (https://blog.postman.com/postman-public-api-network-is-now-the-worlds-largest-public-api-hub/). It contains many features, but its core use case is enabling developers to send HTTP requests and viewing responses in an easy-to-use user interface. In fact, Postman also contains a web-based version (no downloads necessary), which is what we will be using in this section.

To use Postman, navigate to https://www.postman.com/ and create a free account using the Sign Up for Free button. Follow the on-screen instructions until you get to platform, where you should see a menu bar at the top with options for Home, Workspaces, API Network, and more. Alternatively, you can choose to download and install the Postman application on your computer (follow the steps on the website), removing the need to create a Postman account.

Now that we are on the Postman platform, let’s configure our workspace:

1. Select Workspaces from the top and click Create Workspace.
2. Select Blank Workspace and click Next.
3. Give the workspace a name (such as OpenAI API), select Personal, and then select Create.

Figure 1.7 – Configuring the Postman workspace

Getting your API key

API keys are used to authenticate HTTP requests to OpenAI’s servers. Each API key is unique to an OpenAI Platform account. In order to get your OpenAI API key:

1. Navigate to https://platform.openai.com/ and log in to your OpenAI API account.
2. Select Personal from the top right and click View API keys.
3. Select the Create new secret key button, type in any name, and then select Create secret key.
4. Your API key should now be visible to you – note it down somewhere safe, such as in a password-protected .txt file.

Note

Your API key is your means of authenticating with OpenAI – it should not be shared with anyone and should be stored as securely as any password.

How to do it…

After setting up our Postman workspace and generating our OpenAI API key, we have everything we need to make HTTP requests to the API. We will first create and send the request, and then analyze the various components of the request.

In order to make an API request using Postman, follow these steps:

1. In your Postman workspace, select the New button on the top-left menu bar, and then select HTTP from the list of options that appears. This will create a new Untitled Request.
2. Change the HTTP request type from GET to POST in the Method drop-down menu (by default, it will be set to GET).
3. Enter the following URL as the endpoint for Chat Completions: https://api.openai.com/v1/chat/completions
4. Select Headers in the sub-menu, and add the following key-value pairs into the table below it:

Select Body in the sub-menu and then select raw for the request type. Enter the following request body, which details to OpenAI the prompt, system message, chat log, and a set of other parameters that it needs to use to generate a completion response:

{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "system",
      "content": "You are an assistant that creates marketing slogans based on descriptions of companies"
    },
    {
      "role": "user",
      "content": "A company that writes engaging mystery novels"
    }
  ]
}

The Headers and Body sections of the Postman request should look like this:

Figure 1.8 – Postman Headers

Figure 1.9 – Postman Body

5. Click the Send button on the top right to make your HTTP request.

After sending the HTTP request, you should see the response from OpenAI API. The response is in the form of a JavaScript Object Notation (JSON) object.

Figure 1.10 – Postman request body and response

How it works…

In order to build intelligent applications, we need to start using the OpenAI API instead of the Playground. There are other benefits to using the OpenAI API as well, including the following:

• More flexibility, control, and customization of the model, its parameters, and its completions
• Enables you to integrate the power of OpenAI’s models directly into your application without your end users interacting with OpenAI at all
• Provides you the power to scale the amount of model requests you’re making to fit the load of your application

We are now going to shift our focus exclusively to the API, but seasoned developers will always revert to the Playground to perform testing on their system messages, chat logs, and parameters.

To make API requests, we need two things:

• A way to make our requests – For this, we used Postman as it’s an easy-to-use tool. When developing applications, however, the app itself will make requests.
• A way to authenticate our requests – For this, we generated an API key from our OpenAI account. This tells OpenAI who is making this request.

The actual API request consists of four elements: the endpoint, the Header, the body, and finally, the response. Note that this concept is not exclusive to OpenAI but applies to most APIs.

The endpoint

The endpoint serves as the location of your HTTP request, which manifests itself in the form of a specific URL. A web server exists at the endpoint URL to listen to requests and provide the corresponding data. With OpenAI, each function corresponds to a different endpoint.

For example, two additional examples of different endpoints within OpenAI are the following:

# used for generating images
https://api.openai.com/v1/images/generations
# used for generating embeddings
https://api.openai.com/v1/embeddings

Additionally, the OpenAI API endpoint only accepts POST method requests. Think of the HTTP methods (POST, GET, etc.) as different ways to travel to a location: by train, air, or sea. In this case, OpenAI only accepts POST requests.

The Header

The Header of an API request contains metadata about the request itself. Information represented in Header tags contains relevant and important elements about the body, and helps the server interpret the request. Specifically, in our case, we set two Headers:

• Content-Type: We set the content-type of our request to application/json, meaning that we are telling the server that our request body will be in JSON format.
• Authorization: We set the authorization value to the API key, which allows the server to verify the client (Postman and our OpenAI Platform account in our case) that is making the specific request. The server can use the API key to check whether the client has permissions to make the request and whether the client has enough credit available to make the request. It’s worth noting that often, API keys are sent as a Bearer token within the authorization Header. A Bearer token signifies that the bearer of this token (i.e., the client making the request) is authorized to access specific resources. It serves as a compact and self-contained method for transmitting identity and authorization information between the client and the server.

The Body

Finally, the Body of an API request is the request itself, expressed in the JSON notation format (which purposely matches the Content-Type defined in the Header of the request). The required parameters for the endpoint that we are using (Chat Completions) are model and messages:

• model: This represents the specific model that is used to produce the completion. In our case, we used gpt-3.5-turbo, which represents the latest model that was available at the time. This is equivalent to using the Model dropdown in the Parameters section of the OpenAI Playground, which we saw in the Setting up your OpenAI Playground environment recipe.
• messages: This represents the System Message and Chat Log that the model has access to when generating its completion. In a conversation, it represents the list of messages comprising the conversation so far. In JSON, the list is denoted by [] to indicate that the message parameter contains a list of JSON objects (messages). Each JSON object (or message) within messages must contain a role string and a content string:
  • role: In each message, this represents the role of the message author. To create a System Message, the role should be equal to system. To create a User message, the role should be equal to user. To create an Assistant message, the role should equal to assistant.
  • content: This represents the content of the message itself.

In our case, we had set the System Message to You are an assistant that creates marketing slogans based on descriptions of companies, and the User message or prompt to A company that writes engaging mystery novels. This, in JSON form, is equivalent to our first Playground example.

The response

When we made the preceding request using Postman, we received a response from OpenAI in JSON notation. JSON is a lightweight data format that is easy for humans to read and write, and easy for machines to parse and generate. The data format consists of parameters, which are key-value pairs. Each parameter value can be in the form of a string, another JSON object, a list of strings, or a list of JSON objects.

Figure 1.11 – Postman OpenAI API response

As you can see in Figure 1.11, the response contains both metadata and actual content. The parameters and their meaning are described as follows:

• id: A unique identifier for the transaction – every response has a different ID. This is typically used for record-keeping and tracking purposes.
• object: The designation of the request and object type returned by the API, which in this scenario is chat.completion (as we used the Chat Completions endpoint), signifying the conclusion of a chat request.
• created: A timestamp denoting the exact moment of chat completion creation (based on Unix time)..
• model: The precise model that was used to generate the response, which in this case is gpt-3.5-turbo-0613. Note that this differs from the model parameter in the request body. The model parameter in the Body section specifies the model type (gpt-3.5-turbo) that was used, whereas the model parameter in the Response section specifies not only the model type, but also the model version (which, in this case, is 0613).
• Choices: An array that comprises the responses generated by the model. Each element of this array contains the following:
  • index: A number that represents the order of the choices, with the first choice having an index of 0
  • message: An object containing the message produced by the assistant, comprising the following:
    • role: The role of the entity generating the message. This is very similar to the roles in the Chat Log within the Playground screen.
    • content: The literal text or output generated by the OpenAI model.
  • finish_reason: A string that indicates why the OpenAI model decided to stop generating further output. In this case, stop means the model concluded the message in a natural way.
• usage: A list of parameters that represent the usage, or costs, of the particular API request:
  • prompt_tokens: The quantity of tokens utilized in the initial prompt or the input message
  • completion_tokens: The number of tokens produced by the model as a response to the prompt
  • total_tokens: An aggregate of the prompt and completion tokens, signifying the total tokens expended for the specific API invocation

The response in JSON format may be difficult for us to read. In fact, what we particularly care about is not id, index, or created, but the content parameter, which contains the response

Unleash your inner sleuth with our captivating mysteries!

However, the JSON response format is essential when integrating the API into your own applications.

This recipe summarizes the essential elements of the OpenAI API and demonstrates how to use an API client such as Postman to send requests and receive responses. This is important because this is the primary method that we will use to learn more about the API and its various other aspects (such as parameters, different endpoints, interpreting the response, etc.).