Creating fine-tuned models

Every model will have different needs. With GPT-3.5 Turbo, a start might be 50 to 100 examples. After reaching the end of a good return on investment from prompt engineering, prompt chaining, and even function calling, we wind up here at fine-tuning. Because so many enterprise use cases will have at least some requirement for fine-tuned models, the best you can do is optimize for small context windows in exchange for more fine-tuning examples. The fine-tuned model costs the same, with 50 examples or 5000. So, if you take a 3000 token prompt, move all the examples into the model, and leave a prompt of 300 tokens (a few paragraphs), that is a significant saving for each interaction. To put this in perspective, this paragraph has 173 tokens (766 characters).

If fine-tuning doesn’t improve the model, the data science folks will likely have to figure out a different way of restructuring the model (OpenAI doesn’t give an example, but if all of...