I'm constantly looking for ways to automate the work with support requests. An idea has been to fine-tune a GPT-3 model to answer common support-related questions.
Here's how you can use Python to fine-tune a GPT-3 model with your own data for improved performance.
Since the initial publication of this blog post, Retrieval Augmented Generation (RAG) has emerged as a superior method for automating support and FAQ interactions. RAG works by dynamically retrieving relevant information to augment responses, akin to an open-book exam where all necessary references are at the model's disposal, ensuring more accurate and contextual answers.
Think of traditional model fine-tuning like cramming for an exam weeks in advance — the model might forget or mix up details over time.
In contrast, RAG's approach is like having the textbook open during the test, allowing the model to pull in the most pertinent information on the spot.
For those looking to enhance their support systems, adopting RAG offers a more efficient and effective solution. Learn more about implementing RAG with resources from OpenAI: OpenAI Cookbook − Question Answering using Embeddings
Prefer video over text?
Check out my free video course on fine-tuning a GPT-3 model
If you prefer learning through videos, I've got you covered! I've created a free video course that teaches you how to fine-tune a GPT-3 model with your own data. The course covers everything in this blog post and more, with step-by-step instructions. You can access the course for free: Access Course Now Happy learning!
Dependencies
Here's what we'll use:
1. OpenAI API 🤖
2. Python 🐍
Steps
Here are the steps:
1. Get OpenAI API key
2. Create training data
3. Check the training data
4. Upload training data
5. Fine-tune model
6. Check fine-tuning progress
7. Save fine-tuned model
8. Test the new model on a new prompt
Disclaimer
This guide walks you through fine-tuning a GPT-3 model in Python, shown in a Jupyter notebook. If you're looking for the steps of fine-tuning right in a terminal, OpenAI has a great guide for fine-tuning in your terminal.
1. Get OpenAI API key
Before we go ahead and fine-tune a model, let's get the OpenAI credentials needed for the API calls.
Go to https://beta.openai.com/, log in and click on your avatar and View API keys:
Then create a new secret key and save it for the request:
Now we have all the credentials needed to make an API request.
2. Create training data
The next step is to create training data to teach GPT-3 what you'd like to say. The data need to be a JSONL document with a new prompt and the ideal generated text:
{"prompt": "<question>", "completion": "<ideal answer>"}
{"prompt": "<question>", "completion": "<ideal answer>"}
{"prompt": "<question>", "completion": "<ideal answer>"}
Let's start by importing the libraries needed:
import json
import openai
Then add your API key from the previous step:
api_key ="YOUR_OPENAI_API_KEY"
openai.api_key = api_key
Now create a regular dict with the training data. For this guide, I'll add some support questions I've recently received:
training_data = [{
"prompt": "Where is the billing ->",
"completion": " You find the billing in the left-hand side menu.\n"
},{
"prompt":"How do I upgrade my account ->",
"completion": " Visit you user settings in the left-hand side menu, then click 'upgrade account' button at the top.\n"
}]
Make sure to end each prompt with a suffix. According to the OpenAI API reference, you can use ->.
Also, make sure to end each completion with a suffix as well; I'm using .\n.
The next step is to convert the dict to a proper JSONL file. JSONL file is a newline-delimited JSON file, so we'll add a \n at the end of each object:
file_name = "training_data.jsonl"
with open(file_name, "w") as output_file:
for entry in training_data:
json.dump(entry, output_file)
output_file.write("\n")
Now you have the training data as a JSONL file, let's check the training data before starting the fine-tuning.
3. Check the training data
We can check the training data using a CLI data preparation tool provided by OpenAI. It gives you suggestions about how you can reformat the training data.
Let's try it out with our training data. Run this line in Jupyter notebook:
!openai tools fine_tunes.prepare_data -f training_data.jsonl
You'll get suggestions similar to this:
I'm only using a couple of prompts for this guide, but the suggestions mention that the recommendations are to have at least a few hundred examples.
You'll also see the approximate time depending on how many jobs are ahead of you in the queue:
We're ready to upload the training data in the next section.
4. Upload training data
Now that you checked the improvement suggestions, let's upload the training data to OpenAI:
upload_response = openai.File.create(
file=open(file_name, "rb"),
purpose='fine-tune'
)
file_id = upload_response.id
upload_response
If you check the response, you'll see the file id which we'll need in the next step when we're training the model:
Use this file id in the next step, where we'll fine-tune a model.
5. Fine-tune model
Alrighty, we have the prepared training data, uploaded it, and now we're finally ready to fine-tune the model.
Start the fine-tuning by running this command:
fine_tune_response = openai.FineTune.create(training_file=file_id)
fine_tune_response
The default model is Curie. But if you'd like to use DaVinci instead, then add it as a base model to fine-tune like this:
openai.FineTune.create(training_file=file_id, model="davinci")
The first response will look something like this:
6. Check fine-tuning progress
You can use two openai functions to check the progress of your fine-tuning.
Option 1
Option 1: List events
You can use openai.FineTune.list_events() and pass in the fine_tune_response id to list all the current events:
fine_tune_events = openai.FineTune.list_events(id=fine_tune_response.id)
This will give you a list of all the events related to the fine-tuning job you're running:
Option 2
Option 2: Retrieve fine-tuning job
You can also use openai.FineTune.retrieve(id=fine_tune_response.id) to retrieve a specific fine-tune:
retrieve_response = openai.FineTune.retrieve(id=fine_tune_response.id)
This will give you a JSON object with information about a specific fine-tune job:
When your fine-tuning job is first in line, your fine-tuning event starts:
7. Save fine-tuned model
Troubleshooting fine_tuned_model as null During the fine-tuning process, the
fine_tuned_modelkey may not be immediately available in the fine_tune_response object returned byopenai.FineTune.create().To check the status of your fine-tuning process, you can call the
openai.FineTune.retrieve()function and pass in the fine_tune_response.id. This function will return a JSON object with information about the training status, such as the current epoch, the current batch, the training loss, and the validation loss.After the fine-tuning process is complete, you can check the status of all your fine-tuned models by calling
openai.FineTune.list(). This will list all of your fine-tunes and their current status.Once the fine-tuning process is complete, you can retrieve the fine_tuned_model key by calling the openai.
FineTune.retrieve()function again and passing in the fine_tune_response.id. This will return a JSON object with the key fine_tuned_model and the ID of the fine-tuned model that you can use for further completions.
Once the fine-tuning is finished, go ahead and save the name of the fine-tuned model:
# Option 1 | if response.fine_tuned_model != null
fine_tuned_model = fine_tune_response.fine_tuned_model
fine_tuned_model
# Option 2 | if response.fine_tuned_model == null
retrieve_response = openai.FineTune.retrieve(fine_tune_response.id)
fine_tuned_model = retrieve_response.fine_tuned_model
fine_tuned_model
It will look something like this:
We now have a fine-tuned model. Let's try it out on a new prompt in the next section.
8. Test the new model on a new prompt
We're finally at the last step, where we'll try our fine-tuned model on a new prompt.
I only ran my fine-tuning on 2 prompts, so I'm not expecting a super-accurate completion.
Start by creating a new prompt.
Remember to end the prompt with the same suffix as we used in the training data; ->:
new_prompt = "How do I find my billing? ->"
Next, run the completion with your fine-tuned model:
answer = openai.Completion.create(
model=fine_tuned_model,
prompt=new_prompt,
max_tokens=100,
temperature=0
)
answer['choices'][0]['text']
Here's what I got:
Great! Now you have a fine-tuned GPT-3 model ready for future prompts.
Summary
Here's a summary of what we did
1. Got our OpenAI API key
2. Created training data
3. Checked the training data
4. Uploaded training data
5. Fine-tuned a GPT-3 model
6. Check the progress of the fine-tuning job
7. Saved the fine-tuned model
8. Tested the new model on a new prompt
Improvements
Expand the training data The training data analysis by OpenAI in step 2 of this guide suggests expanding the training data amount. I only used 2 prompts in this guide. The suggestion says at least a few hundred examples/prompts.
FAQ might not be the best use case FAQ-related questions might not be the best use case for fine-tuning. If you'd like to automate your support questions, a question-answering approach might be better suited than fine-tuning a GPT-3 model.
Next steps
1. Repo with source code
Here is the repo with a Jupyter notebook with all the source code if you'd like to implement this on your own ⬇️
https://github.com/norahsakal/fine-tune-gpt3-model
2. Prefer video over text? If you prefer learning through videos, I've got you covered! Check out my free video course on fine-tuning a GPT-3 model: https://norahsakal.gumroad.com/l/qwhwa
3. Do you need help with fine-tuning your own GPT-3 model? Or do you have other questions? I'm happy to help, don't hesitate to reach out ➡️ norah@quoter.se
Or shoot me a DM on Twitter @norahsakal