---
date: "2024-07-24T20:10:08.000Z"
title: "2024-07-24"
tags: ["connections","gpt-4o-mini"]
draft: false
---

I ran the code from my [Fine-tuning "Connections" post](/posts/2024/fine-tuning-connections) using `gpt-4o-mini`.
I was hoping the results might be a bit better, which could motivate an effort to fine-tune the model.
I'm not sure where my original version of this code went, so I reconstructed a repo for it.
Once I was done, I ran 100 prompts through the model to get a sense of where its baseline performance was.

```text
Correct: 2.00%
Incorrect: 98.00%
Total Categories Correct: 19.25%
```

Not great, and not much different from `gpt-3.5-turbo`.
With these kind of results, I wasn't particularly motivated to put the effort in to do more fine tunes.

---

I read through the [`instructor`](https://python.useinstructor.com), [`marvin`](https://www.askmarvin.ai/welcome/what_is_marvin/) and [`open-interpreter`](https://github.com/OpenInterpreter/open-interpreter) docs for the first time in a while.
It has been interesting to see these libraries grow and diverge.
I also read through how Jason has been structuring an [evals](https://github.com/instructor-ai/evals/) repo for `instructor`.