I read this interesting article by Gajus about finetuning gpt-3.5-turbo. It was quite similar to my experience fine tuning a model to play Connections. A helpful takeaway was that after finetuning the model, you shouldn’t need to include system prompt in future model inference, so you can save on token cost. I also liked the suggestion to use a database to store training data. I had also been wrangling jsonl files.