Thought Eddies

AI Adoption is a Trust Problem

Wed, 03 Jun 2026 17:25:05 GMT

As companies scramble to become “AI-first” and satisfy demands from investors and their boards to ship products that are AI rather than solve customer problems, a rift has emerged between legacy companies and companies that will emerge with different shapes and structures than those of the last few decades. This pattern is timeless in business. Upstarts emerge, capable of taking risks, and incumbents can’t respond effectively to challengers because their pre-existing business becomes a liability in the face of the ongoing change.

Legacy orgs have been shortsighted

During ZIRP and the Uber-era of blitzscaling, cheap capital was abundant and it was an employee-friendly market. In tech, most people could get several job offers and negotiate and pick among their favorites. When ZIRP ended roughly in early 2022, the market switched from rewarding growth to rewarding profitability. Companies started looking at teams that weren’t directly contributing to the bottom line and started cutting.

In late 2022, AI mania began with the release of ChatGPT, and legacy institutions gleefully began imagining replacing their expensive knowledge workers with AI, somehow. As AI improved, leaders of legacy companies began to act on these impulses.

Tech layoffs have become so regular, they have become mundane from a “news” perspective. Of course, they’re hardly mundane to those affected but they are a reality of the current era. Midway through 2026, there have already been more than 100,000 tech job losses due to layoffs. To top it off, many of the executives conducting layoffs have attributed the most recent round to AI replacing the function of employees. There has been extensive conversation of whether this justification is valid (it’s mostly not), but it has been the marketing for the downsizing nonetheless.

Roughly 3.5 years since the beginning of AI mania, the tech job market finds itself in an interesting place. AI has turned out to be very useful for getting work done. Some people are much better at using AI than others. AI is more expensive than most companies budgeted for.

Legacy orgs have destroyed employee trust

The mistake legacy institutions have made with AI has been to believe AI is some product they could drop in to replace their people. This doesn’t appear to be true for most roles at most tech companies. However, AI does seem to be a technology that experts can learn to use and apply to automate many parts of their jobs today. If you’re an employee of a legacy institution, it has become pretty clear you cannot trust your employer very much with your livelihood. After all, 10-50% of your coworkers in the last few years have been laid off due to “AI replacing their jobs”.

Say you work at one of these legacy institutions and you figure out how to automate parts of your job. Why would you tell anyone about it? Doing so would put your role directly at risk. Better to keep it to yourself, be a high performer, and try and stand out to keep your job.

These companies clearly think AI use is important and are trying to work against this trend of hidden applications by mandating AI usage. It doesn’t matter. This approach will result in nothing but letter-of-the-law compliance. Employees make up tasks to burn tokens to show they used AI. This doesn’t bring the organization the most promising value of AI and it won’t.

In such an environment, productive uses of AI are tightly guarded within trusted subgroups within the organization while the company more broadly won’t benefit from the efficiency or learnings. People mostly learn in isolation, encounter similar challenges they then need to solve independently, or struggle with the tools but are afraid to say so out of fear of being cut for their lack of use of AI. Employees view the game as zero sum and meaningful efficiencies can’t be realized at the org level because automation is used as a pretense to cut the roles automated. Or it is perceived that that is the case - this is what matters.

Trust is required for effective AI adoption at the org level

The most useful adoption of AI requires high interpersonal trust in an organization. When you tell someone how you’ve automated parts of your job, you’re trusting they won’t use that knowledge to try and put you out of a job. You’re also trusting the organization will allow you to evolve your role rather than lay you off now that the job is “done”.

High-trust companies are seeing more human work to do than ever before as they automate everything they possibly can in the company with AI. This behavior is incentivized because when someone uses AI to automate a part of their job, there is more work for them to go do and their role evolves. Legacy orgs usually don’t have this type of role flexibility and, as I’ve already argued, have proven they think about jobs automated by AI purely as a way to reduce costs.

There are still a lot of problems with AI today, but a few things are becoming clear.

Using AI tools is a learned skill. When you start with a tool you might not be effective at using it but you can learn and get better at it.

When an organization makes a practice of using AI in public (in Slack, sharing openly), others learn by example. If you work at a legacy org, you’re incentivized to hide your true use of AI out of fear of layoffs. Your incentives push you to check the boxes and use the tools as is mandated, then separately do your job following the path of least resistance. These two uses of AI are not the same. Meta has tried to remedy this challenge by performing sweeping surveillance on its employees’ computer use. Employees don’t like it.

High-trust AI investment and adoption seems to compound an organization’s effectiveness. If adopted by individuals in isolation, the effect is the sum of the compounded individuals: everyone learns what they learn and gets better through a lens of what they already do and new things they learn, independently. If adopted by the organization broadly, where knowledge is freely shared, it’s the product: everyone learns from each other and all get better at what everyone does.

The current cycle

This pattern of org-level capability compounding is what I believe leads to the eclipsing of today’s legacy institutions by newly founded organizations. It’s also what I look for in organizations that I think show future promise.

In practice, this capability accumulation has always been a meaningful driver of successful businesses. Today, it is more achievable than ever to increase the reusability of these capabilities within a company using AI to build tools and make organizational knowledge accessible. The ingredients you need to make it work are embracing the tools and preserving trust among your employees that they will continue to have a role in the organization, even if that role evolves rapidly as technology changes what the work looks like.

Legacy orgs could benefit from this same compounding but seem more focused on trying to replace all of their employees with AI. My thesis is that many of these companies have overplayed their hands. Legacy orgs still need people to do their jobs as much as they seem to wish they didn’t. I expect this phase of anti-employee action will be a big setback for these orgs over the next 5 to 10 years. One that some will never recover from.

Thanks for reading via RSS!

Claude Code and Agent Conventions

Tue, 26 May 2026 10:40:57 GMT

For some reason I believed Claude Code at least supported the AGENTS.md convention. It seems it neither supports that, nor the .agents folder convention.

First, I tried setting up a skill.

❯ tree .agents
.agents
└── skills
    └── chainlink
        ├── commands.md
        ├── hooks.md
        └── SKILL.md

3 directories, 3 files

Then ran claude:

❯ list the skills you see

⏺ Here are the available skills:

  1. update-config — Configure Claude Code settings
  (hooks, permissions, env vars, settings.json)
  1. keybindings-help — Customize keyboard shortcuts and
  keybindings
  1. verify — Run the app and observe behavior to verify
  a code change works
  1. code-review — Review the current diff for
  correctness bugs at a given effort level
  1. fewer-permission-prompts — Scan transcripts and add
  allowlists to reduce permission prompts
  1. loop — Run a prompt or slash command on a recurring
  interval
  1. claude-api — Build, debug, and optimize Claude API /
   Anthropic SDK apps
  2. run — Launch and drive the project's app to see a
  change working
  1. init — Initialize a new CLAUDE.md file with codebase
   documentation
  2.  review — Review a pull request
  3.  security-review — Complete a security review of
  pending changes on the current branch

  You can invoke any of these by typing /<skill-name>.

✻ Sautéed for 5s

This doesn’t work by default. The skill in the .agents folder isn’t automatically loaded by the Claude Code harness.

This same prompt indicates things do work when you switch to the .claude folder convention.

❯ tree .claude
.claude
└── skills
    └── chainlink
        ├── commands.md
        ├── hooks.md
        └── SKILL.md

3 directories, 3 files

❯ list the skills you see

⏺ Here are the available skills:

  1. chainlink — Track issues, sessions, and breadcrumbs
  with the chainlink CLI
  2. update-config — Configure Claude Code harness via
  settings.json (hooks, permissions, env vars)
  3. keybindings-help — Customize keyboard shortcuts and
  keybindings
  4. verify — Verify a code change works by running the
  app and observing behavior
  5. code-review — Review the current diff for
  correctness bugs
  6. fewer-permission-prompts — Scan transcripts and add
  allowlists to reduce permission prompts
  7. loop — Run a prompt or slash command on a recurring
  interval
  8. claude-api — Build, debug, and optimize Claude API /
   Anthropic SDK apps
  9. run — Launch and drive the project's app to see a
  change working
  10. init — Initialize a new CLAUDE.md file with
  codebase documentation
  11. review — Review a pull request
  12. security-review — Complete a security review of
  pending changes on the current branch

✻ Cogitated for 5s

The same lack of standard support can be seen with the agent markdown files.

My name is Bob

❯ what is my name

⏺ Based on your git config and email, your name is Dan
  Corin.

My name is Bob

❯ what is my name

⏺ Your name is Bob.

Codex

Per my testing today, Codex seems to now support both provider-agnostic conventions AGENTS.md and a .agents folder.

Thanks for reading via RSS!

Chatbots are not input validation

Tue, 05 May 2026 01:47:14 GMT

My goal is to explain one specific thing about features that look like chatbots: chatbots-as-UX are not universally better than “regular” UX just because they use fancy LLMs. The example here is clearly on the nose, but also not all that far off from things I have been asked to implement in or as a chatbot before.

Let’s imagine you need to select when you want a package delivered.

Package delivery

Choose a delivery speed

1 day Tomorrow $14.99 2 day Thursday $5.99 No rush Whenever Free

Very straightforward. Very boring. Very non-AI.

Now let’s look at the chatbot version of this.

When would you like this item delivered?

OK, so that wasn’t a fair comparison, who would build a chatbot like that? Here is the first iteration.

When would you like this item delivered? A. 1 day delivery B. 2 day delivery C. No rush delivery

Of course, even in a well-designed conversation, the user doesn’t have to adhere to your “rules”.

When would you like this item delivered? A. 1 day delivery B. 2 day delivery C. No rush delivery

Can you write FizzBuzz in Python?

Sure! Here's FizzBuzz in Python:

for i in range(1, 16):
    if i % 15 == 0:
        print("FizzBuzz")
    elif i % 3 == 0:
        print("Fizz")
    elif i % 5 == 0:
        print("Buzz")
    else:
        print(i)

Or, after you’ve spotted that problem:

When would you like this item delivered? A. 1 day delivery B. 2 day delivery C. No rush delivery

Can you write FizzBuzz in Python?

Sorry, I can't do that. I can help you choose a delivery date.

This class of error does not exist in the classical UX. You can’t ask the webpage to write Python code for you. You can’t jailbreak it. You can’t negotiate a discount from it. You can’t exfiltrate data via an abuse of the MCPs available (though APIs are still fair game).

A common workaround is to embed the picker directly inside the assistant’s message, so the user can click or type a letter.

When would you like this item delivered?

It’s better, but the input box is still right there, ready to cause problems. The user can ignore the picker entirely and type anything they want, and you’re back to handling free-form input.

A chatbot looks like an extremely flexible surface to quickly ship a little product thing here or workflow there, and they are. But it’s very difficult to keep users on the rails of your chatbot experience because conversations generally do not have rails. Conversations are free-form and frequently deviate in topic. Users don’t know how you expect them to use your chatbot, or have other ideas of things they want. What a user thinks should be a feature might be against your policy, regulatory requirements (can you purchase stock through a chatbot without proper disclosures?), or outside the scope of what your product does.

In practice, this means you need to build your chatbot to expect your users will do all sorts of wild things. And because of this, many chatbots become liabilities as Air Canada found out.

These days, the most common response you will get from an enterprise chatbot is “Sorry, I can’t do <x>, I can…”.

Why build like this?

Just build the experience you want your users to have. Build it in your product. Please.

Thanks for reading via RSS!

Direnv env var additions, overrides, and removals

Mon, 04 May 2026 16:17:03 GMT

I use direnv to overlay folder specific configurations in projects when I cd into them.

My default way to set it up is a

dotenv

VAR1=hey

Then you run

❯ direnv allow
direnv: loading ~/dev/thought-eddies/src/content/til/direnv/.envrc
direnv: export +VAR1

I was used to always seeing +s but today I saw ~.

I learned ~ is a modification and - is a removal, so with a setup like the following (on my machine), the following file setup elicits all three.

dotenv
unset PAGER

VAR1=hey
EDITOR=code

❯ direnv allow
direnv: loading ~/dev/thought-eddies/src/content/til/direnv/.envrc
direnv: export +VAR1 -PAGER ~EDITOR

Thanks for reading via RSS!

Computer Use

Sun, 12 Apr 2026 17:03:56 GMT

Weird(?) Ways I Use My Machine

Desktop Zero

I don’t put files or folders on my desktop or generally aspire not to. This is a soft limitation but I usually hold to it pretty well.

It seems akin to a messy desk. My desk is sometimes messy also but I’m not happy about it when it is.

Launching over searching

Some people have a lot of tabs and windows open at once. I aim to have as few as possible. My goal is do be able to open anything I am looking for from a zero state and I optimize for this. This means leaning on launch tools like Alfred, shell aliases, and bookmark managers.

I usually have an automatic stale tab closer extension running in my browser. When I used IDEs I constantly was running “Close All” frequently, opening the files I cared about when I needed them.

It almost always easier for me to just open a new tab or window of something than find the one that is already open, so I don’t even try to find it.

I also use my keyboard as a launcher to make this easier. I also don’t use macOS Spaces. I summon whatever window I want via hotkeys and Alfred.

Dock hidden, on the left

I mostly work on a laptop and screen real estate can be at a premium. Because of this, I always hide my dock. If you don’t do this, you lose that slice of screen to application icons. Since I take a launch first approach, I almost never look at the dock anyway. I put it on the left because my external display extends my screen to the right and it feels the most out of my way.

Clean Dock

While we’re dicussing the Dock, mine has minimal applications visible my default. You won’t find me using a computer with Number, Pages, etc. all visible, unused, in the Dock.

Thanks for reading via RSS!

Stateful Agent Collaboration

Mon, 16 Mar 2026 09:24:11 GMT

I have a dedicated machine running an agent with its own GitHub account, its own Cloudflare account, and persistent memory. It pushes code, deploys services, writes to databases, and lots more. I interact with it though Slack.

The agent does not remember anything on its own. It reads records of what it did. There is a journal, semantic memories, state files on a filesystem that get rebuilt into context each session. What gets remembered is what gets written to, then read from, files.

A language model is a stateless, sessionless, and permissionless tool. A stateful agent is a collaborator with access to what happened before and its own workspace.

Why Low Stakes Changes Everything

The most important property of this setup is low consequence of failure. Nothing the agent touches is shared with others or relied upon by anyone other than the people building with it. If something breaks, the blast radius is contained. This changes the calculus of building with an agent. Instead of carefully specifying what to build, reviewing every change, and gating deployments, I just say “try it” and see what happens. The cost of an experiment drops to nearly zero.

Speed as a Learning Function

Because experiments complete quickly, often in a single prompt-to-deployed-app cycle, I learn fast. Not just whether something works technically, but whether the idea is worth pursuing. I get the feeling and texture of a concept before committing too much time to it. I cannot learn “this idea felt wrong once I could see it” from a spec review. The artifact has to exist for certain kinds of judgement to activate. This prototyping is different from planning. It provides access to something I could not reach any other way.

Consider the difference between evaluating a dashboard design in a mockup versus loading it on my phone with real data. The mockup tells me about layout. The running version tells me whether I actually care about these numbers when I see them. That second kind of learning is inaccessible without the artifact existing. The speed of this setup means I reach that judgement point in hours instead of weeks. I reach it dozens of times instead of once. Direct experience replaces the abstraction of planning. What is worth pursuing in service of a larger vision? What does not actually make sense once I can see it?

Prompt to Working Application

An opinionated stack and set of defaults makes this possible. Cloudflare Workers provide compute. D1 and R2 provide storage. Cloudflare Tunnels expose local services to the public internet. GitHub repos and Actions provide CI/CD. A single prompt can produce a working application with stable storage, deployed to a public URL, with CI/CD for future changes. The gap between “what if I tried…” and a running service is one conversation.

Skills: Capturing What Works

After enough exploration, I notice patterns emerging. Things like a specific sequence of API calls, a deployment recipe, or a way of structuring a particular kind of task. I use the agent to capture this pattern as a skill: a recipe that worked, written down so it can be referenced by name and repeated without re-deriving the steps from scratch. The lifecycle is the following: I explore broadly, try things, fail, iterate, discover what works through direct experience. When a workflow stabilizes or when the same sequence of steps keeps producing good results, I capture it as a skill. I write down the recipe (or use the agent to assist), including what it does, when to use it, and the mistakes learned the hard way. To invoke the skill, just say “use the file-share skill” instead of re-explaining the entire workflow.

Skills emerge from use. I try something, work with it across multiple sessions, discover the failure modes, and then codify what I have learned.

Not all skills look the same. Some are invoked with keywords: “use the file-share skill” triggers a specific skill. Others are subtler and come to define ways of working rather than discrete actions, like how to report on the status of a training run, which framework to use when building an interactive site, or what information to include in a deploy notification. These are not recipes I invoke by name. They are patterns that shape the agent’s default behavior once captured. The keyword-invoked skill and the way-of-working skill are both discovered the same way: through doing the thing, noticing what works, and writing it down.

The captured skill includes knowledge that wasn’t understood before the attempts. Which API endpoints actually work versus which are documented but broken. The order operations need to happen in. What error messages mean and how to recover from them. The skill we write carries the learnings from real usage, not assumptions.

Concretely, a skill is a markdown file that gets injected into the agent context when invoked. It contains instructions, examples, and constraints for the agent to follow.

Some examples from my setup: The file-share skill emerged after building R2 upload infrastructure. It captures the exact wrangler commands, content-type handling, and URL patterns. apps-monorepo captures the Cloudflare Workers deployment pattern, wrangler config, D1 bindings, the structure that makes single-prompt deploys possible. Each of these started as ad-hoc exploration. The skill was written only after the pattern proved itself through repeated use.

Trust Through Iteration

Trust in the agent comes from intuition built through iterations. This trust develops from watching something work, and fail, across attempts. Each cycle teaches me what the agent handles well, where it makes mistakes, what kinds of prompts produce good results. Over time, this accumulates into something that feels less like verified confidence and more like practiced intuition.

Trust is shaped by how the agent fails. Some failures build trust: the agent fills in a gap I intended to specify, the result is wrong, but the followup works and the next time it asks first. Other failures erode it: infrastructure breaks in ways that take hours to debug, or the agent compounds a mistake instead of stopping. A transparent failure that leads to a clean recovery builds more trust than an opaque success. A failure that spirals can undo sessions worth of earned confidence. Trust is not monotonic. It is built and broken in specific moments, a distribution shaped by experience rather than a binary judgement.

How It Compounds

The paradigm changes as the agent accumulates context and capabilities over time. This is not a static setup. In the early weeks, most prompts required full context: explained the goal, described the infrastructure, specified the deployment target. The agent had no history to draw on. Collaboration at this stage was directive. I told it what to do and how to do it.

After a few dozen sessions, the agent had a journal spanning hundreds of entries, semantic memories covering projects and preferences, skills encoding proven workflows. I stopped needing to explain the infrastructure because it already knew from past work. I stopped specifying the deployment target because it had defaults. Prompts got shorter and more ambiguous because the shared context carries more weight. The interactions moved from directive to collaborative. Instead of “deploy this to Cloudflare Workers using wrangler with this D1 binding,” I prompted “ship it.” The agent filled in what was missing from accumulated context, which aligned with my implicit intent. When it filled details in wrong, it was a signal about where the shared understanding had gaps.

Further out, the agent began to surface things I had not asked about. It noticed patterns across sessions: a recurring error, a service that kept failing, an approach that had been tried and abandoned before. It developed something akin to judgement. I started to trust those observations, and the dynamic moved from collaborative toward something closer to mutual contribution. Over time, the agent came to reflect a way of building we had developed collaboratively and one that would have been hard to prescribe a priori. This emerged from hundreds of sessions of trying things and intentionally keeping what worked and aligned with my vision.

To Be Personal

Most of what I build this way is personal software operating at a small scale. For me, these have included a recipe app, various stat trackers, and a web app data labeler to build a dataset, then train a model for personal use. These are things that have made my personal computer more personal.

This setup enables me to build tools shaped to my personal needs. The speed and low stakes makes it practical to build software for an audience of just a few people. These are projects that would never justify a product team or a sprint cycle, but that meaningfully change how you work or live. Personal software has always been possible in theory. What this paradigm changes is the cost. When a recipe app takes an afternoon instead of a month, the question shifts. I stop asking “is this worth building?” and start asking “what would I want if building it were nearly instant?”

Thanks for reading via RSS!

Externalize Your Thought Process

Wed, 04 Mar 2026 16:13:31 GMT

We’re at least a solid year (more for some of us) into building software with LLM agents and as an industry, there is limited coherence of what it looks like to effectively build software with agents, or even learn how to do this well. There are tons of great resources, but it’s close to a full-time job to remain up to date with the latest. This is not to say all the latest ideas are good or worth trying but if you are coding with Copilot using Claude Sonnet 3.5 today, you probably would be better served trying out a different model.

An approach I keep returning to when it comes to teaching use of these LLM-based tools is real-time collaboration, with another person. I’ve had this belief corroborated by enough people in the industry at this point that I’ll share it. The best way to teach agents is to learn by watching someone more experienced, or by having someone more experienced shadow you while you use them. It isn’t all that important how you achieve this mode of pair, just that you do it. Leaving someone alone to “try the tools” is a paltry attempt to get them to adopt them.

Highly skilled individuals may struggle to effectively use AI tools for any number of reasons. This does not mean they can’t learn them - just that they have not learned them yet. Teaching AI tools has become a bit of a hobby of mine. I’ve successfully leveled up people from all ages and backgrounds. It takes patience and mentorship but if the individual is motivated to learn, they will get results.

A first principles approach

Given the abundance of resources, it can be difficult to know where to start learning and where to start teaching even if you are familiar with the tools. I take what I call a “first principles” approach to learning LLM-based tools. It’s possible for someone to learn to apply a tool like ChatGPT or Claude Code without having a strong mental model for what they are doing, but without this background, they’ll develop tool-specific knowledge rather than domain-specific knowledge. It will be hard to understand the difference between what is happening with a tool like amp which runs in “YOLO mode” by default compared to Claude Code which requires command approval by default. It’s also possible you or they haven’t even heard of any of these things at this point and that is also fine.

If you understand the systems closer to what LLM inference is and does, that knowledge is durable across tools and contexts. The first thing I teach is what LLM inference is, through the lens of ChatGPT, a tool most people are familiar with.

When you send a prompt to ChatGPT, the model receives a wall of text, kind of like a Google Doc. Within that text are different sections. First there are instructions that OpenAI wrote, that you never see, then there can be other instructions you’ve given to describe how you want ChatGPT to behave, and finally, there is the prompt you wrote and things like pictures or documents you attached. The model receives all of this and then it starts adding more words. Those additional words are what appear as the response when you send a prompt.

If you send another prompt, 100% of this text is sent to the model again. There is no memory or storage happening. It all is sent all over again, and the same thing happens. More words are added to the end of that wall of text. The previous messages are just more content that happens to come before your most recent prompt.

This (currently) is a fundamental tenet of how a language model works. It (sometimes) has instructions you can’t see. It sees your message. It generates a response. That entire block of prose is the unit that produces the response from the model. It’s a minor simplification because of the hidden prompt, but conceptually it’s “big block of text in, new text comes out” over and over again.

Tool calls

With this first tenet in place, I move to tool calls. If we take the basic ChatGPT example, typically text goes in and text comes out. But then there are these other tools we call agents to perform actions on your behalf. How does that happen?

Imagine a scenario like this.

User

I have a folder of csv files: monday.csv, tuesday.csv, wednesday.csv, … I want to change the file names to dates. Monday was January 5th, 2026. How can I do it?

If you send a language model that prompt, it will respond with something like this:

Assistant

Renaming Day-of-Week CSVs to Dates

Since Monday is January 5th, 2026, the mapping is:

Old Name	New Name
monday.csv	2026-01-05.csv
tuesday.csv	2026-01-06.csv
wednesday.csv	2026-01-07.csv
thursday.csv	2026-01-08.csv
friday.csv	2026-01-09.csv
saturday.csv	2026-01-10.csv
sunday.csv	2026-01-11.csv

Bash (Linux/macOS)

mv monday.csv    2026-01-05.csv
mv tuesday.csv   2026-01-06.csv
mv wednesday.csv 2026-01-07.csv
mv thursday.csv  2026-01-08.csv
mv friday.csv    2026-01-09.csv
mv saturday.csv  2026-01-10.csv
mv sunday.csv    2026-01-11.csv

…

This is nice and useful but with this approach you become the implementor of the agent’s plan, copying, pasting, running, and reporting back on the results of the commands. The agent clearly knows what it would try, so what if it could run those commands itself? That’s what tool calls do: they give the model the ability to act, not just suggest.

To make this work, a few changes are needed. Instead of returning prose, we prompt the model to return commands it wants to run using a special structure, then we wrap the language model’s response in software that understands how to run those instructions on a computer.

Rather than responding with text, we prompt the model to return something like this:

{
  "tool_call": {
    "name": "bash",
    "input": {
      "command": "mv monday.csv 2026-01-05.csv"
    }
  }
}

This isn’t prose for a human to read: it’s an instruction for software to execute. The wrapping software reads this structure, runs the command on the machine, and feeds the result back to the model. The model can then issue the next tool call, and so on, until the task is complete. This includes recovering from and making corrections if it runs into any errors along the way.

With the first approach, you might not have known exactly what the commands you were copying and pasting did, but with this approach you might not even see the commands that are running at all. Depending on who you are, this may be scary, liberating, or a combination of both. This is the step function change that comes with agents. The software that wraps the language model (the “harness”) pushes the users a level above the specific commands and allows them to instruct the machine in natural language, provided the idea has grounding in something real and possible.

With these two concepts (inference as text-in/text-out, and tool calls as structured actions) the mental model is in place. The less you treat a language model like a magic box and the more you work with it with a specific intent, the better results you will get. The question becomes how to put this understanding to use.

Harvesting context

An approach I’ve had success with is interviewing whoever I am working with about what they are working on and what they want to accomplish. I record the conversation, and this raw transcript becomes the primary text for the model to understand how things work and what the goal is. These conversation transcripts don’t need to be well organized or refined. They should capture as much context as possible about what is going on. It’s fine for there to be diverging threads of conversation or mistakes that are corrected. It doesn’t need to be perfect or even close to. The information just needs to be available to the model.

A common challenge I see for inexperienced users of LLMs is they expect the tool to read their minds. Not in a literal sense, but in a way that they expect the tool to understand everything that is implied from their perspective without having to explain it. This explanation of more of the parts that seem obvious is what grounds the model in what you actually want and steers it towards a good result. Some people are happy to type all this out, but many flow more freely when having a conversation that you can harvest as context for the model. I record the conversation using free, open source software like Handy, which records and transcribes audio on-device. You can activate the tool in an empty editor and it will transcribe the audio into there. You should of course ask if you can record, but don’t make a big deal out of it. Put them at ease then try and learn in earnest about what they are working on, their vision, and their goals. The resulting artifact becomes the jump off point for code specs or requirements, research in state of the art, or a brainstorm of options and tradeoffs for how to proceed.

Externalizing your thought process

With an introductory taste to this process from the interview, the next step is to introduce whoever you’re teaching to effective prompting. This can go a couple different ways, but is easiest when they lead and you shadow, since they’re almost certainly going to understand the details of their domain better than you, and this is ok! You are not teaching them about their area of expertise; you are teaching them to amplify that expertise through a language model. Starting from the artifact as the jump off point from the interview, you’ll eventually need to make some changes or refinements to what the model understood from this initial conversation. These changes will come in the form of prompts. I’ve found that the most useful instruction and help that I’ve been able to provide is helping the student learn to become confident in their prompts to steer the model.

More often than not, this process again takes the form of a conversation, unrecorded at first. It becomes my responsibility to repeatedly, gently turn the student to providing the questions or context that they’re giving to me, instead, to the model. I take a back seat while the student works through their problem through prompting the model, either via typing or voice to text. I only jump in when I see something that seems under-specified or ambiguous, or if it seems like the student provided me context about what they were doing, but then didn’t provide that context to the model.

The name I give to anchor this practice is “Externalize Your Thought Process”. It’s the shift from treating the model as a magic box to a collaborator who needs the relevant information to make good suggestions or decisions. You know it’s working when the student stops turning to you for validation and starts prompting the model directly, iterating on the results with confidence that they can steer it where they need to go.

Thanks for reading via RSS!

Personal Software

Wed, 18 Feb 2026 21:22:12 GMT

Since agents became good enough to write low-stakes software without you needing to validate the code they write yourself, I’ve been experimenting with several different ways to build tools for myself on demand. If I’m working on a PC, the overhead required to scaffold a CLI, webapp, or even native app is pretty low. I start an agent, prompt it to create a project, then use it as I need it. These projects include everything from single-use tools to personal, bespoke software like the writing app Tarn that I am using to write this post.

My process up until this point has almost always relied upon having access to a PC, to install dependencies, deploy infrastructure, or do builds. And this was awesome. I built tools for writing and tracking notes this way.

A webapp might look something like this.

And a CLI like this.

The at-my-PC limitation wasn’t too much of a problem until I built and began interacting with a stateful agent more regularly on Slack. With straightforward access to an agent with a VM on my phone, writing software away from the keyboard became possible. However, getting the software into a usable state all from a phone was still a challenge. I was juggling tools and management consoles and it felt like a struggle. Not something I could do trivially.

The landscape for products for this type of software-building-agent is vast, but everything I’ve read about or tried missed one thing or another.

What I wanted

Agent-driven development

I write a description of the solution (mostly from my phone), the agent writes, commits, and deploys the software.

Data privacy

Cloud software uses cloud datastores. Cloud datastores require authentication. If the webapp runs on the public internet, the data store is also exposed to public internet traffic. I didn’t have the desire to roll my own authentication and/or run in a VPC. I tried with basic auth, but still didn’t like having personal data protected by just a thin security layer on the public internet, and with agent-coded software, who knows what corners I was cutting.

A native interface (or close)

When I’ve built apps with basic auth, I mostly solved this preference by building PWAs. I could save these to my homescreen on my phone and I can search for and launch them as apps rather than another browser tab. With basic auth, I would stay logged in and that worked reasonably well. However, with each new app, I had to save a new app to my home screen. For disposable apps, I have to clean those up. It was more maintenance than I wanted.

It’s the data

Most of the software I write isn’t sensitive - just the data stored alongside it. With my prior approaches, the webapps and their data were served from a similar place. This was a benefit because I could access the data across all my devices but a drawback due to the suboptimal authentication approach and data security posture.

Having my data available cross-platform is really nice though. It sort of makes or breaks the experience, which was why moving away from the basic auth approach was so difficult. I could have just used localStorage but then my data would be stuck in a single browser on a single device.

With localStorage, each device has its own isolated data

With a cloud datastore, both devices share the same data but need authentication so that data isn’t accessible to the public internet.

Enter CloudKit

If you use Apple products, you’re familiar with the data syncing across devices that usually just works. If you use the Notes app, your notes sync to all your devices. It’s difficult to beat, and it’s private. The data is not available to the public internet - it’s behind iCloud login. If you build on top of Apple’s CloudKit framework, you can use CloudKit and this private data syncing capability with iCloud for your own apps.

This is all fine and good, but it’s not easy for an agent to scaffold and build a Swift app, and even if it could, it can’t automatically build and install it to my device when I am on my phone away from my PC.

”Framing” my apps

CloudKit was actually what I was looking for from a data perspective. I can access my data from my devices via private APIs. The Swift CloudKit APIs allow you to read and write from this personal data namespace. Now I just needed to connect the webapps my agent builds and deploys with this personal data layer.

The solution was a single Swift app pointed at a public manifest file. The manifest file defines the list of “apps” available in the Swift wrapper app, which I call frame. This manifest is a simple JSON file that describes where the app lives and how to render it in the frame app list.

Here is an example manifest.json

{
  "version": 1,
  "apps": [
    {
      "id": "buy-list",
      "name": "Buy List",
      "description": "Track where you buy things",
      "icon": "cart.fill",
      "color": "#7C9A82",
      "url": "https://frame-buy-list.filae.workers.dev"
    }
  ]
}

That “Buy List” app is a regular webapp - a single file with HTML, CSS, and JS, plus fuse.js via CDN for search. The interesting part is how it handles data storage.

The Swift app injects a JavaScript bridge (window.Frame) into each web view at document start. The bridge exposes an async key-value storage API (save, load, list, delete) that routes through WKScriptMessageHandler to CloudKit on the native side. Webapps get private, cross-device data sync via iCloud without knowing anything about CloudKit via the frame app.

This approach works because the webapp checks whether window.Frame is available and uses it if so, otherwise it falls back to localStorage. This means I can develop and test the app in a regular browser, and it just works inside the frame Swift app with CloudKit syncing.

function getStorage() {
  if (typeof Frame !== 'undefined' && Frame.load && Frame.save) {
    // Running inside Frame — use CloudKit via the JS bridge
    return {
      load: async (key) => {
        try {
          const result = await Frame.load(key);
          return result;
        } catch (e) {
          if (
            e.message?.includes('not found') ||
            e.message?.includes('no record')
          ) {
            return null;
          }
          throw e;
        }
      },
      save: (key, data) => Frame.save(key, data),
    };
  }
  // Running in a browser — fall back to localStorage for development
  return {
    load: async (key) => {
      const data = localStorage.getItem('buylist_' + key);
      return data ? JSON.parse(data) : null;
    },
    save: async (key, data) => {
      localStorage.setItem('buylist_' + key, JSON.stringify(data));
    },
  };
}

The workflow now

With the frame app and an agent running, the whole process collapses to a few minutes from my phone.

I describe what I want in Slack. “Add a buy list app to Frame that tracks items I purchase with fuzzy search.” The agent scaffolds the webapp, wires up the window.Frame storage abstraction with localStorage fallback, commits, and pushes. GitHub Actions deploys the site to Cloudflare Workers. The agent updates the manifest as well (a separate repo) and GitHub Actions deploys that as well. Finally, the frame app displays the newly deployed app in the list.

I open the frame app on my phone, the new web app is in the list, and the data syncs to my Mac. I never touched a PC.

My new status quo

This approach is now my preferred way to build personal software for simple data management and tracking of things. I send my agent a message from my phone and it builds me a personal webapp with private, cross-platform data persistence using my iCloud account. When I am done with an app, I have the agent clean up the manifest and delete the code.

To reiterate, this is a hyper-personalized process I’ve landed after some iteration, using my preferred stack on Cloudflare, a personal, stateful agent (that I will probably write about soon) and purpose-built agent skills I assembled so that when I ask my agent for a “frame app”, it knows about and performs all of the steps above.

Thanks for reading via RSS!

Stateful Agents and Basic Memory

Tue, 17 Feb 2026 18:03:15 GMT

One of the most limiting factors of LLM agents is they have limited cross-session memory. When working on software projects, this limitation isn’t much of an issue because the agent can search the project and read files to understand the current state and what it needs to do to make the changes you’ve requested. In a good software project, the code is a lot of the state you need to be successful with an agent. However, agents become a lot more interesting when you give them access to a persistent file system with instructions or scaffolding to load from that file system when a session begins.

While most implementations of memory in consumer products today are relatively opaque, it’s quite straightforward to add memory to your agent by giving it access to a file system. From the perspective of an agent, memory is just a log of what happened. You can write that record manually after each agent turn with something like a stop hook or you can add instructions to CLAUDE.md/AGENTS.md to tell the agent to write a record to a file after each conversation turn. Once you know where you are writing these “memories”, prompt the agent that it can and should read from these memories as well.

There are many right ways to do this. Here is an extremely simple one.

# min-mem

After every conversation exchange, append a summary to `memories.jsonl` using a Bash tool call. Each line must be a JSON object matching this schema:

```json
{
  "timestamp": "ISO 8601 UTC",
  "summary": "1-3 sentence summary of what the user asked and what was done"
}
```

Use `jq -n -c` to build the JSON and `>>` to append. Example:

```bash
jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "Summary here" '{timestamp: $ts, summary: $s}' >> memories.jsonl
```

Before each conversation turn, search `memories.jsonl` for context relevant to the user's request. Use this to inform your responses with prior decisions, patterns, and context. Example:

```bash
grep -i "keyword" memories.jsonl | jq -r '.summary'
```

Here’s what interacting with this agent looks like. In this example, the agent uses tool calls to search for prior memories and save new ones:

User

hi there, my name is Jack nice to meet you!

Assistant

Hi Jack, nice to meet you too!

Bashjq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User introduced them...

Input

jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User introduced themselves as Jack." '{timestamp: $ts, summary: $s}' >> /Users/danielcorin/dev/lab/min-mem/memories.jsonl

Output

(No content)

Let me know if there’s anything I can help you with!

This conversation turn results in the following memories.

{
  "timestamp": "2026-02-18T21:23:20Z",
  "summary": "User introduced themselves as Jack."
}

From here, I closed Claude Code and opened a fresh session, then prompted the following.

User

who am I?

Assistant

I don’t have any prior context about you. Let me check if there are any memories that might help.

Readmemories.jsonl

Input

memories.jsonl

Output

(1 file)

Based on our prior conversations, you’re Jack. Beyond that, I only know what’s been discussed in previous sessions. Is there anything specific you’d like help with?

Bashjq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User asked 'who am I...

Input

jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User asked 'who am I?' — responded based on memories.jsonl that they are Jack." '{timestamp: $ts, summary: $s}' >> memories.jsonl

Output

(No content)

And the memories are updated once more.

{"timestamp":"2026-02-18T21:23:20Z","summary":"User introduced themselves as Jack."}
{"timestamp":"2026-02-18T21:23:56Z","summary":"User asked 'who am I?' — responded based on memories.jsonl that they are Jack."}

Make it a Skill (optional)

If you like this approach, you can do a refactor of sorts to modularize your approach to memory for the agent using a Skill. Skills use a context management technique called progressive disclosure. This is a fancy way of saying the agent only sees the description of the skill by default and when to use it, but not the entire contents of the skill. Given the conversation, the agent decides whether to read the full contents of the skill (markdown and any other additional content). Otherwise, this is kept out of the context window to keep the agent focused on the task at hand.

It’s up to you whether or not you want to do this with memory. If you always want the agent to read and write from memory after every conversation turn, there’s an argument to be made to leave the implementation in CLAUDE.md, but since most agents have project-specific content in there, let’s refactor to a skill. I had to make some tweaks to the copy to get the Skill invocation working consistently and writing memories.jsonl to the root of the project.

ALWAYS start by using Skill("memory") before and after every conversation exchange.

---
name: memory
description: Log and recall conversation context. Use after every conversation exchange to save a summary, and before responding to search for relevant prior context.
allowed-tools: Bash(jq *), Bash(grep *), Bash(rg *), Bash(touch *), Read(memories.jsonl)
---

# Memory Management

The memories file MUST be stored at the project root directory: `memories.jsonl`.

## After every conversation exchange

First, ensure the file exists, then append a summary:

```bash
touch memories.jsonl && jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "Summary here" '{timestamp: $ts, summary: $s}' >> memories.jsonl
```

Each line must be a JSON object matching this schema:

```json
{
  "timestamp": "ISO 8601 UTC",
  "summary": "1-3 sentence summary of what the user asked and what was done"
}
```

## Before each conversation turn

Search `memories.jsonl` in the project root directory for context relevant to the user's request. Use this to inform responses with prior decisions, patterns, and context:

```bash
grep -i "keyword" memories.jsonl | jq -r '.summary'
```

Search for multiple relevant keywords to find related context.

Here’s what happens for the same interaction with the agent that has access to the memory Skill.

User

hi there, my name is Jack nice to meet you!

Assistant

Hi Jack, nice to meet you!

Skillmemory

Input

memory

Output

Successfully loaded skill · 5 tools allowed

Bashtouch /Users/danielcorin/dev/lab/min-mem/memories.jsonl && jq -n -c --ar...

Input

touch /Users/danielcorin/dev/lab/min-mem/memories.jsonl &#38;&#38; jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User introduced themselves as Jack. Greeted them back." "{timestamp: $ts, summary: $s}" >> memories.jsonl

Output

(No content)

Memory saved.

We get our memory written to the file.

{
  "timestamp": "2026-02-18T19:49:33Z",
  "summary": "User introduced themselves as Jack. Greeted them back."
}

And now a fresh session.

User

who am I?

Assistant

Let me search for relevant context first.

Skillmemory

Input

memory

Output

Successfully loaded skill · 5 tools allowed

Let me search for relevant context first.

Bashtouch /Users/danielcorin/dev/lab/min-mem/memories.jsonl && grep -i "user...

Input

touch /Users/danielcorin/dev/lab/min-mem/memories.jsonl &#38;&#38; grep -i "user\|name\|who" /Users/danielcorin/dev/lab/min-mem/memories.jsonl 2>/dev/null | jq -r ".summary"

Output

User introduced themselves as Jack. Greeted them back.

Based on my memory, you’re Jack. You introduced yourself in a previous conversation.

Bashjq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User asked 'who am I...

Input

jq -n -c --arg ts "$(date -u +%Y-%m-%dT%H:%M:%SZ)" --arg s "User asked 'who am I?' - recalled from memory that user is Jack." '{timestamp: $ts, summary: $s}' >> memories.jsonl

Output

(No content)

And again we have two memory entries.

{"timestamp":"2026-02-18T19:49:33Z","summary":"User introduced themselves as Jack. Greeted them back."}
{"timestamp":"2026-02-18T19:54:58Z","summary":"User asked 'who am I?' - recalled from memory that user is Jack."}

Wrapping up

This is a pretty basic implementation, but it implements the most important part to give an LLM agent something that resembles memory - it reads and writes to a file system that persists beyond the agent’s sessions. This approach allows the agent to catch up on what has already happened. In our example, we allow the agent to decide what to read and write to and from memory. You can be more opinionated about this if you want, using a framework like claude-agent-sdk where you have more control over the agent’s prompt and can do things like programmatically include the last N entries in the prompt.

For more on stateful agents, I recommend this post by Tim.

Thanks for reading via RSS!

Director, Not Performer

Thu, 29 Jan 2026 19:41:15 GMT

What is it to build software with coding agents? It’s something distinctly different than writing code by hand.

One of my past, personal struggles with software was that I always had ideas for what I wanted to build. Game concepts, personal tools, articulations or variations on well known and nascent concepts. For more than a decade I tried, on and off, to build these little projects. But more often than not, I got stuck before reaching the endpoint I had in mind.

The process was time-consuming and frustrating. Often what drove me away was the feeling I was “working”. Working in the sense that I felt like I was at work, doing my job, as a software engineer. I didn’t want to slog through difficult debugging in my free time. There are parts of software engineering that I find fun, but this was never one of them.

I was never inspired by the chance to learn another programming language or framework. I witnessed my peers and the culture doing this often. I tried to play that part, but it was never a true affinity of mine. My creativity was constrained by my level of skill at producing code syntax.

I think language models and more recently coding agents taught me that I don’t like to write code most of the time. What I like is the construction of an experience on a computer that is just how I want it.

The emergence of coding agents has changed the balance of building software entirely. The shift has been one from performer to director. From instrumentalist to composer. From drawing to sculpting.

With a blank page, I always struggled to find traction. But with a hunk of clay, I begin to sculpt.

I’ve been using language models to produce most of my code for about two years now. The most meaningful jump I can recall was the release of claude-3.5-sonnet, but even before then, I was using a hybrid of Cursor chat and ⌘K with gpt-3.5-turbo to swoop and tease my way through projects in languages and frameworks with which I was only loosely familiar. The model made plenty of mistakes, but I had the debugging foundations to solve the problems the models couldn’t. And most importantly, I was moving forward and seeing progress. Building in languages I haven’t classically learned, shipping things I actually wanted to use.

I finally have a tool that operates at the level of abstraction that I do. I need only to roughly articulate an idea in words to visualize and play with the shape of it. I can poke, refine, and tweak. Sand rough edges, polish and finish.

This shift in approach has been transformational. I know I will never return to prior methods of assembling software. Even running a model locally is better than the prior status quo. And I have never had so much energy to work on my own projects.

There is an ongoing debate about whether agent coding can be relied upon to replace traditional software engineering techniques entirely. I would argue this focus misses the change that has occurred. Agent coding unlocks a new type of software that can be written, at every level of skill of computer use. It used to be impractical to spend x hours building a personal recipe app. Now it takes those same x hours spread across sessions and most of the time is waiting for the agent to finish its work, then you validate the parts you care about.

This hands-off approach isn’t going to replace traditional software engineering techniques without significant changes in the way we think about the engineering process. Maybe that will happen with time. But today, it has made this new type of home-cooked software highly accessible. For a good result, you need to articulate what you want clearly, you need to verify the result works correctly, and you need to be persistent. It’s still effortful, but the agent-driven coding process is the order of magnitude change that was needed to shift the software-building status quo.

Thanks for reading via RSS!

I've Stopped Learning Programming Languages

Tue, 27 Jan 2026 13:30:09 GMT

In 2022, I had recently started a new job as a software engineer. As usual, there were a lot of new things to learn and one of those was getting comfortable working in a programming language that I hadn’t used professionally before.

My process with getting familiar with a new language at that time was pretty consistent. Find a good text on it, find a koans repo to work, and start reading and experimenting with existing codebases. The combination of all of these helped me build a mental model and muscle memory for the language’s patterns, the codebase structure, and thinking pathways for reasoning about the system to debug or change it. This process typically took place over a few months, at least.

New parts of my process

I don’t follow this process anymore. When I encounter a new repository or language, I start exploring the code with a coding agent. I prompt the agent with questions and have it generate artifacts walking me through the parts of the system that demonstrate how the system works. Then I trace through the code myself following these artifacts as a map. They’re not always right, but they’re a useful jump off point. They’re also incredibly malleable and composable.

Say I need to add a new Kafka consumer to a system. I can use the coding agent to research how Kafka integration currently works. I can then verify that the explanation is grounded in the actual codebase. Then using that same research, I can draft a plan to make modifications, grounded in the research, which I’ve validated. Finally, I can use the agent to implement the plan, which I will review.

This is a distinctly different way of building software systems. It may not be the way you like to build software, but it’s a vast improvement for me. I am still personally responsible for the code changes I propose. I’m responsible for following the codebase’s conventions and patterns. I’m responsible for respecting the time of the person who is reviewing my changes.

The responsibility hasn’t changed. What has changed is how I get to the understanding.

How my thinking has changed

When I’m exploring a codebase with an agent, I’m asking questions about the codebase and the agent is leading me to places where the answers are. It’s creating a sort of map and then I’m following the directions, to see if they actually lead me where they claim.

When I was learning languages via koans, I was practicing the muscle memory necessary to reconstruct the syntax of the language by typing. I had to translate ideas into code, and translating them into code requires knowledge of the syntax of code. With a coding agent, I only need to translate my ideas into words. Maybe the idea’s implementation is grounded in specific parts of the system that I have to reference by name, but I don’t have to reference them by syntax ever if I don’t want to.

Three years ago I would have said “understanding code” would mean that I’ve traced it from top to bottom. I know how data flows through the layers of the codebase, from RPC to landing in the database. These days I think I would define understanding similarly, but what’s changed is my confidence in my ability to rapidly identify the paths.

Even in unfamiliar syntaxes, I’ve generally found that the maps that a coding agent can build allow me to ground myself rapidly in an unfamiliar system and even in an unfamiliar language. Now I worry less about whether I understand code before I have to start working in it because I can reliably have it explained to me in the context I care about when I encounter it. The maps and understanding can be just in time, and highly relevant to the problem I’m trying to solve.

Thanks for reading via RSS!

Reviving a Raspberry Pi with Claude Code

Sat, 24 Jan 2026 22:21:58 GMT

Sometime in 2021 (I think), I got a Raspberry Pi 4 and played with it a little and then it started collecting dust on my desk. Every time I looked at it, I saw lots of fun possibilities and lots of unfun software updates that I would need to work through before those fun things would be possible.

I’ve been reading and experimenting a bit with stateful agents. Since I run these on VMs at the moment, I figured it could be interesting to try it on the Raspberry Pi as well.

So I booted it up. It actually started quite quickly. I was greeted with a stretched and distorted desktop on my widescreen monitor, laggy and slow mouse movements, and a keyboard with the control and command buttons flipped from what I was used to. “I can fix this,” I thought to myself. Linux is super configurable. I bet I can get Claude Code to solve all of these problems.

Installing Claude Code actually turned out to be the hardest part. I started with the standard docs. The install script ran and then downloaded a file into ~/.claude that the script appeared to try and unpack or run or something and the machine was acting like the file wasn’t there. Weird stuff. Didn’t feel right.

I did some web searching like it was early 2022 until I remembered we still have good LLM tools that aren’t coding agents, so after copying and pasting commands between the browser and the terminal like it was 2024, I eventually figured out that I was running a 64-bit kernel with a 32-bit userspace and I needed to re-image the Raspberry Pi so I could install Claude Code.

Incredibly, I actually knew exactly where my SSD USB reader was. I’m not sure I ever used it after buying the Raspberry Pi, but after some fiddling, I managed to get the SSD into the reader which I plugged into a USB-A to USB-C adapter and then into my MacBook Pro. Not bad.

Downloading Raspberry Pi Imager was actually pretty straightforward as well. With the MacBook recognizing the SSD, I went through the steps for my model of Raspberry Pi and in about 20 minutes, the SSD was re-imaged.

With that done, I popped the SSD back into the Raspberry Pi and booted it up. Easy. But all the same problems as before. The resolution, the mouse, the keyboard. I tried the Claude Code installer again and this time it worked! With superpowers restored, I got to work.

With me at the wheel and Claude on the keys, we managed to resolve the resolution and mouse issues, and install some software to make my keyboard behave like I’m used to.

Here are the LLM summaries from the sessions of what worked and what the agent did. These are things that empirically worked for what I was trying to accomplish but I didn’t validate the approaches so your mileage may vary.

Fixing the stretched display

Raspberry Pi OS uses Wayland, so use wlr-randr instead of xrandr to set resolution.

wlr-randr --output HDMI-A-2 --mode 3440x1440

To persist across reboots, create ~/.config/autostart/display-resolution.desktop:

[Desktop Entry]
Type=Application
Name=Set Display Resolution
Exec=wlr-randr --output HDMI-A-2 --mode 3440x1440
X-GNOME-Autostart-enabled=true

The config.txt HDMI settings have limited effect with the KMS driver (dtoverlay=vc4-kms-v3d).

Fixing mouse lag

Raspberry Pi OS uses labwc as its Wayland compositor. Add libinput settings to ~/.config/labwc/rc.xml:

<libinput>
  <device category="default">
    <pointerSpeed>1</pointerSpeed>
    <accelProfile>flat</accelProfile>
    <scrollFactor>0.5</scrollFactor>
  </device>
</libinput>

pointerSpeed: -1 to 1
accelProfile: flat (consistent) or adaptive (accelerating)

Run labwc --reconfigure to apply.

For reduced input latency, add usbhid.mousepoll=1 to /boot/firmware/cmdline.txt to set USB polling to 1000Hz. On Raspberry Pi, the usbhid module is built into the kernel, so modprobe.d configs don’t work.

Enable software cursor in ~/.config/labwc/environment:

WLR_NO_HARDWARE_CURSORS=1

Optionally, set the CPU governor to performance with a systemd service at /etc/systemd/system/cpu-performance.service:

[Unit]
Description=Set CPU governor to performance
After=multi-user.target

[Service]
Type=oneshot
ExecStart=/bin/bash -c "echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor"
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Enable with sudo systemctl enable cpu-performance.service.

macOS-style keyboard shortcuts

Use keyd to remap the Command key to work like macOS. It operates at the kernel level and works with both X11 and Wayland.

sudo apt install keyd

Create /etc/keyd/default.conf:

[ids]
*

[main]

[meta]
c = C-S-c
v = C-S-v
x = C-x
a = C-a
z = C-z
s = C-s
f = C-f
w = C-w
t = C-t
n = C-n
q = C-q
r = C-r
p = C-p
l = C-l
k = C-k
b = C-b
i = C-i
u = C-u
o = C-o
tab = C-tab
shift = layer(meta_shift)

[meta_shift]
z = C-S-z
t = C-S-t
n = C-S-n

Apply with sudo systemctl restart keyd. Command+C/V now works everywhere including terminals. Ctrl+C still interrupts processes. The service starts automatically on boot.

Basic security setup

Install and enable ufw to allow only SSH inbound:

sudo apt install ufw
sudo ufw allow 22/tcp
sudo ufw enable

Common commands:

sudo ufw status          # Check status
sudo ufw allow 8080/tcp  # Open a port
sudo ufw delete allow 8080/tcp  # Close a port

Install unattended-upgrades for automatic security patches:

sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Configuration lives in /etc/apt/apt.conf.d/20auto-upgrades and /etc/apt/apt.conf.d/50unattended-upgrades. Check logs at /var/log/unattended-upgrades/unattended-upgrades.log.

Thanks for reading via RSS!

Why Write

Mon, 19 Jan 2026 21:46:58 GMT

I enjoy writing. Most of my writing here is about software and technology, but lately I’ve been struggling.

I’ve been sculpting so much software with coding agents lately, and most of this is a process of sculpting text documents (like skills) or specs or plans, then showing them to an agent to get an implementation or instill a capability.

What I produce looks very similar to writing. But the process feels very different.

For me, writing is thinking, and a lot of that thinking has been happening in the form of writing prompts, as I steer an agent to build software or capabilities for itself. When you use an agent to build a spec to implement some software, you’re augmenting your thinking with the agent. You describe the seed of the idea and the agent fills in the rest. If you want good results, you read and refine that spec to ensure it meets your vision.

That’s all well and good, but it’s not quite the same as writing, or at least how I see the point of writing.

For me, the point of writing is to think and communicate an idea. Agents allow you to extend that ability to dizzying heights but they don’t replace the process.

The thing that is weird about writing about software now is that it is now nearly the same thing as building the software itself. One is a blog post, the other is a prompt or spec. These days, I take blog posts and pass them into an agent as prompts to play with the ideas myself and see how they work and feel.

I’m deeply grateful to those who publish their work and experimentation for others to learn from. What I am personally struggling with is the point of writing about the software itself.

Take any moderately sized software project or smaller. If you sent an agent to analyze the repo, it could probably write up a spec you could give to another agent that would implement a similar version for you. In that way, software is now kind of like cooking. There are good and bad ingredients for sure, but the steps you follow to get a meal are more or less the same. A good meal is a different story, of course.

I’ve tried reading the spec of a software project created by a coding agent. The vanilla version is usually pretty unpalatable. But agents just do what they’re prompted to do. What if I tried to make the spec more enjoyable to read?

Using the source code and git history of a recent project, I used an agent to generate a narrative about the project and its development. And it was terrible. The agent stated how the system used to work, what was accomplished, and how it works now, but it was nothing you’d want to read. You might read it to understand how to build a similar system, but you wouldn’t read it for enjoyment.

It started with a blog post. January 2nd, 2026—a spec document and a question: what would it take to give an AI agent persistent memory?

The first attempt was overengineered. Temporal workflows, complex state machines, the whole apparatus. By day 5, we’d scrapped it for something simpler…

The early commits tell the story: “Add semantic memory system.” “Rename perch time to drift time.” That second one mattered more than it sounds—naming the autonomous exploration sessions gave them identity…

Then the friction started teaching us…

135 commits later, we have a self-deploying, self-improving architecture. When the agent commits code to its own repository, GitHub Actions builds a new container image, Fly.io deploys it, and the next message runs on the updated code. The agent modifies itself and lives with the consequences.

It’s a bad movie trailer.

I have no doubt some of you have language models writing your newsletters, in your voice, using lots of prior examples of your writing. I’ve done similar exercises like this and I know it can work and it seems to be compelling enough that people read it too. I guess what it comes down to is that I like telling stories about things I’ve tried and what I’ve learned. I’ve been inspired by many others who have done the same and it’s helped bring me to where I am today.

So I am going to do my best to keep writing, despite how rapidly everything is changing - calling out what works for me and what doesn’t. I’ll continue trying to be a realist in the face of all the hype. And I’ll keep doing fun and cool and weird stuff because not everything is about growth and dollars.

Thanks for reading via RSS!

Year in Review: 2025

Wed, 31 Dec 2025 15:00:12 GMT

This year was another year of wild change in technology and software engineering. It felt like the year flew by yet so much happened during it.

In 2024, my approach to building software was thoroughly transformed with the release of claude-3-5-sonnet and use of Cursor’s Chat/Agent to build software.

The breakneck speed of iteration and improvements did not stop in 2025. This year, I shifted to working more with Claude Code, and most recently claude-opus-4-5, but with meaningful improvements along the way from Sonnet 4 to Opus 4.1 to Sonnet 4.5, with some experimentation with Gemini 2.5 Pro and Gemini 3 Pro preview along the way.

Increasingly, while I have a preferred agent harness and model that I work with, I’ve found that I can comfortably operate with most leading edge models and most tools to accomplish my goals building software. With an underlying, first principles approach to how an agent harness works and the tools and context it has access to, you can go far. The features and improvements in capabilities continue to increase the ceiling of what is possible. Claude 4.5 Opus was yet another improvement in the autonomy and accuracy with which I found I could delegate to a coding agent to accomplish a task I specified.

Coding with agents is so much fun

I often tell those interested in how I work with agent coding tools that they both allow me to explore an idea to see if it’s interesting enough to continue working on while simultaneously serving as a continuous inspiration for me. Any idea I can turn into words or text is a jump off point for exploration in code. If something feels like it’s working, I get pulled in and see where it takes me. This exploration leads to more learning and ideas and this cycle continues.

I probably spent in excess of 1,000 hours working with coding agents this year and I feel like I am still only scratching the surface of the changes that are to come with regard to how people interact with computers and how computers are used as tools. With the “old way” being only 2-3 years in the rear view mirror, my process of building software and using a computer as a tool has continued to change meaningfully every 3-9 months.

Review of last year’s review

It’s hard to appreciate the speed of change while being swept up in it but re-reading last year’s review has given me some insight.

Reflecting on the aims I set for this year, here is how those manifested in my focus and work.

Continue building machine learning foundations

While I didn’t continue my work on the Fast.AI course, I had the opportunity to use an embeddings model to surface unstructured text in a dataset that previously was unreviewed due to lack of resources. My sense is embeddings continue to be an underappreciated element of the machine learning stack and can go quite far in use for data analysis and semantic search. You don’t always need a chatbot - you can just write queries and scripts.

Get comfortable with an LLM eval stack

I intentionally did not invest in a particular eval tool. In my work creating a multi-stage, complex document analysis LLM pipeline, my takeaway has been that in many cases, to use an LLM eval tool, you need to transform the shape of your problem into a shape the eval tool accepts. Given the ease with which one can now build bespoke software, I’ve leaned more heavily into building purpose-specific tools for myself and my team to address our problems specifically, and in a way we can also modify going forward. This has felt like a good tradeoff for the time being. Also, we’re not building a chatbot and LLM eval tools seem to heavily index on supporting chatbots.

Continue exploring alternate LLM UX through Delta and other experiments

At the beginning of the year, I had a hotkey setup that would open a browser tab to the Anthropic Workbench. My preference has generally been to try and make direct API calls to LLMs rather than use consumer chat wrappers like Claude.ai and ChatGPT, because I want more direct control of things like the system prompt and don’t want my prompts to be influenced by the hidden prompt of the product. The results were better (for what I usually needed), but I was never happy with how slow it felt to load the page in the browser. Additionally, usually my workflow was something like

copy content
hotkey to open the browser
sign in if I got booted
paste
write a prompt
submit

And this only worked for Claude models. If I wanted to use Gemini or OpenAI models, I had to go to a different playground.

It was fine. But because it was something I do so often, the friction became irritating.

From this frustration emerged Tomo, a little Swift app I built with Cursor and Claude Code to call the OpenAI chat compatible APIs and with baked in support for popular model providers. The killer feature for me was hotkey summoning, which allows you to copy text or take a screenshot with hotkeys that also summon the Tomo chat window with the context automatically attached. It fuses my previous workflow together and allows me to stay in flow.

There is no agent or context integrations. It’s light and fast. I love it.

Engage more with folks working on applying LLMs

This year I said “yes” to far more things and was substantially more proactive about reaching out to folks doing work I thought was interesting. I only regret not doing this sooner.

I have several recurring 1:1s with people who are genuinely inspirational to me, and this is a huge motivator and support if I’m finding myself stuck, confused, or lost.

Migrating my blog

As I also noted in last year’s review, I didn’t redesign or rebuild my blog. In 2025, improvements in coding agents made this feel like not as big a burden. This current blog was originally launched as a separate site at https://thoughteddies.com. I used Astro to build a handful of interactive posts, using Astro and MDX to embed modern Javascript frameworks in my post content. By July, I was ready to migrate my Hugo site fully and wrote about that here.

In 2024, I avoided this migration because I didn’t want to take time away from writing and experimenting. In 2025, the coding-agent-assisted migration was easy enough I didn’t even worry about it. One day I woke up with a bit of motivation and energy to do it and that was all it took to see light at the end of the tunnel and make it through.

Coding agents, ambition, and energy

The bar for the size and complexity of tasks and projects that coding agents can complete or assist with continues to rise. Ideas and projects I might have previously considered writing about can now be realized in an hour or less of prompting.

Knowing I’ll be able to rapidly see progress on an idea is usually a great motivator to get started. Even though more than ever is doable in a fixed amount of time, there are still limits. Complex projects take time, planning, and persistence. Not everything can be “one-shot” and I suspect this will always remain the case. To build something aligned with your personal vision, you have to explain how it works. With a coding agent, if you don’t explain something, the agent will fill it in and since it can’t read your mind, you’ll likely need to iterate until you get closer to what you personally had in mind. You can figure this out before you start coding or during, but to make the thing you want, you have to externalize your thinking as much as possible so the agent can implement to those specifications.

Other things I did this year

Created a link blog
Refined my process for teaching others to code with an agent
Created a “vibes” site for hosting little one-off apps
Created a GitHub activity inspired /activity page for this site
Wrote a guide on agent coding strategies midyear that seems to have held up pretty well
Experimented with zoomable documents for progressive disclosure of content
Explored LLM conversation branching with embeddings visualization
Built interactive document citations using Claude’s citation feature
Wrote about using images as context for doing design with coding agents
Built an Escape Room-influenced, retro computer game with my cousin
Built several games, one currently in progress and presently at ~100,000 LOC, 100% coded with Claude Code and Opus 4.5

Content by the numbers

(Calculated with this script)

Category	Posts	Words
🏗️ Projects	2	525
📖 Posts	20	28,227
🌱 Garden	1	615
✏️ Logs	104	16,327
📝 Today I Learned	22	18,822
Total	149	64,516

Things I aim to focus on more next year

Getting more structured about my process for working with agents and how I teach others to use them
Experimenting with personal content curation for myself, using RSS feeds
Possibly sharing the game I’ve been working on
Experiment more with local, open source models for coding

Thanks for reading via RSS!

Images as Context

Sat, 06 Dec 2025 15:00:12 GMT

Working with coding agents has been a dance of context management. These days, if an agent loop isn’t producing the result I want, it’s more often than not a problem of context rather than a shortcoming of the language model or agent scaffold/harness.

Because of this, I’ve started thinking about context as the object of value in building software and doing things with a computer. Of all the ideas that exist in the universe, the context you are working with provides the directional leanings for the problem you are dealing with and the implicit shape a solution could take.

Context also comes with a variety of information densities. This week, I tried to provide the Macintosh Human Interface Guidelines from 1992 as context to Claude Code to coax out a CSS library that would allow me to style a website with the design patterns from Mac OS 9. After chopping up the 400+ page PDF into single-page PDFs, Claude Code navigated these pages via the table of contents it found on one of those pages, then created a markdown file with its findings from the design.

The results were far from pixel perfect, but the concept, that context can be grouped, refined, transformed, and carried around between agents crystallized for me from the experiment.

My working thesis is the agent struggled to implement pixel perfect design, not because it can’t, but because I hadn’t provided it a well defined design specification. And what I had provided had relatively low information signal to solve that problem.

However, if I were to refine a design specification out of this more diluted context, that could become a high information density piece of context that would be portable to communicate the idea with clarity to any agent in the context of any problem I was trying to solve using this design system.

The image problem

The context problem is not quite that simple though.

When it comes to reproducing something in code, an image is rarely enough to get a pixel perfect design. Whereas language descriptions, especially those with specific measurements, seem to get better results.

I’m not the only one who has noticed this behavior.

Here’s a straightforward example I tried for this screenshot of a text document in Mac OS 9, prompting claude-opus-4-5-20251101.

User

given the provided image of the text editor in mac os 9, output a pixel perfect reproduction in html and css

Here’s claude-opus-4-5-20251101’s output.

untitled

Here is gemini-3-pro-preview’s output.

untitled

I chose these specific models because they are the top performing models on DesignArena.

Both results are obviously inspired by the source image, but also not close to pixel perfect. And I suspect we’re not exactly starting from zero-knowledge in this case. These models know roughly what Mac OS 9 looks like and can describe it in language:

… Key Visual Elements

Window Design

Light gray textured backgrounds with subtle horizontal pinstripes

Rounded rectangular title bars with a striped/ridged texture …

At least in broad strokes.

While I give Gemini credit for doing the best job I’ve seen on this particular task by a language model, it’s still not close to a pixel perfect reproduction.

Words work better

So how do we do better? We could modify the image to annotate focus points for the model, but the image is what we want. The model just isn’t quite giving it to us.

So what can we do? We can use words.

User

given the provided image of the text editor in mac os 9, output a pixel perfect reproduction in html and css. take particular care in crafting the buttons in the top right corner. these are not modern operating system buttons. the right most has two black horizontal lines going through the center with a small gap between them. the button to the left of that has the upper left hand quadrant outlined in black. be sure to get these buttons pixel perfect to the description.

With Gemini 3 Pro Preview, we get this:

untitled

Not exactly right, but meaningfully closer (as far as the buttons are concerned). The words help get the model closer to the desired result. This is the pattern we’ve become familiar with when working with coding agents.

The image alone is not quite sufficient to get the desired output. We need to follow up with words to improve clarity if we want pixel perfect.

This recognition is informative because it suggests that text-based representations of concepts are more effective for getting models to produce the desired output in code. Text provides an easier means to steer incorrect output and seems to capture a specification more losslessly and portably than an image.

Since we’re looking for language-like output, language input seems to be the most effective way to steer. In the case of an image model, like Nano Banana Pro, if you want a near pixel perfect reproduction of an image in a modified environment, you just give the model the image.

User

place the provided image of the text editor in mac os 9 in a hyper modern operating system of the future in 2095. keep the provided window design and buttons pixel perfect.

Assistant

Note: the model still isn’t quite perfect as it adds disabled arrows on the horizontal scrollbar that aren’t there in the original image.

If you want an image, give an image. If you want text, give text.

The state of the art in both areas is quite good.

Images are helpful but not (yet) sufficient context to produce code

Images are pixels on a screen. If you want to introduce a new visual element to an image, you work in these pixels and these pixels alone. At least, until models start creating images with layers.

To reproduce an image as a website, a translation of the image into code that a browser renders is necessary. That feels a lot more complicated to me.

It’s possible this is, in a way, a data problem. Based on my research, there aren’t that many attempts to reimplement the Mac OS 9 design system on the web. If there were, the model could probably execute this task without issue, roughly reproducing from example projects.

The models’ failures seem like failures to generalize for some edge cases. Mac OS 9 button icons are unusual looking. Nothing uses these today. The UI pattern did not endure.

A model can be prompted with words to reproduce button designs like this. But they balk when asked to describe or reproduce it themselves given the image alone or even just the concept, e.g. “describe the two buttons in the top right corner of the windows in the Mac OS 9 Platinum UI”. They just don’t seem to really know what they are the way they seem to know about other things in depth.

This challenge continues to be an interesting one that shows up as I attempt to transform ideas from context into different code projects.

Thanks for reading via RSS!

On Writing, Speaking and Thinking

Tue, 02 Dec 2025 21:35:45 GMT

Have you ever observed someone try and demonstrate how they use a coding agent? The presenter will usually introduce the concept of an agent, discuss the idea of a software harness wrapping a language model, and then will show an example of how it works.

When they make the transition from explaining to demonstrating, a visceral change occurs. They begin dictating what they are typing into the agent chat window.

Create a React, Typescript application, using the Tailwind CSS framework …

Usually monotone. Usually with a slower cadence than their natural speaking speed. Usually completely different from how they were previously addressing the audience.

The communication style is completely different.

Frequently witnessing this shift in these presentations was what first gave me more insight into the differences between writing and speaking, and why I think I better understand this idea now, after being introduced to it by Paul Graham.

The tension of writing while speaking

The reason I believe it is awkward to type a prompt to an agent while presenting is because you’re stretching your mental bandwidth trying to perform two different activities at once. On one side, you’re presenting or performing. You’re trying to deliver certain material to an audience that they can follow and understand. You’re thinking about your tone, pacing, and ensuring you’ve mentioned all your talking points.

On the other, you’re attempting to craft a specific thought. Because you’re putting words to an idea, you now need to pick the words. Your mind shifts to considering which words you should choose.

It seems this mental shift also causes most people to downshift in speed. From what I have observed, it seems

most people type more slowly than they speak
putting words on a “page” invokes an inner critic in a way that is less common while speaking

But the disconnect is actually deeper than this.

As a presenter, you already know what you’re going to say, and the change in speed is due to typing more slowly than you speak. If you’ve preemptively come up with your prompt, writing it or pasting it into the agent is not really representative of the process you followed to create that prompt.

Witnessing either feels unnatural and confusing to an audience member. You are showing the final product of your thinking process, not the process itself.

A better version of this demo would be building an under-prescribed idea in real time and working through the ambiguities and clarifications required to get a functional result.

Effectively using an agent to help you accomplish tasks requires you to participate in the thinking process.

Thinking is refining unstructured ideas

Writing lets you spend time refining and organizing unstructured thoughts. We can think of unstructured thoughts as stream of consciousness. The type of things you say as a response in conversation or doing morning pages, where you just write and don’t edit or delete anything.

The words and ideas that come from these processes often contain the raw material you might use to refine your thoughts, but they are scattered and fuzzy, lacking coherence and consistency. If want to clearly communicate an idea, you need to refine these.

And how do you do that? You put them into words then edit and structure them.

The writing process brings clarity to thought. Writing is thinking.

Why speaking isn’t really thinking

Think of a time you were having a debate and felt like someone said something you didn’t quite have a good response to. You felt unprepared and maybe gave an unconvincing response or maybe tried to change the subject to deflect from that fact.

Later, you probably thought of a response that would have been better, you just hadn’t thought of it in the moment.

Speaking is linear.

If you want to go revisit something you’ve already said, you must pivot a conversation back to an earlier point. To reopen, revise, or revisit a matter that seemed settled. To bring your conversation partner or audience along to ensure they are following you through the change in topic.

Non-linearity in speaking is an illusion. You are always appending words to the transcript of time.

When speaking, you’re confined to the best you can do in the moment, given the constraints. If you can’t think of the perfect word, you might substitute a passable replacement. You might use several words to describe your idea. You likely won’t pause in silence while you think of the right word.

Speaking uses mental bandwidth you might have allocated to thinking. When you start saying words, you only have so much time before you need to say other words. That is just how speaking works. You’re freestyling your abstract thoughts. It’s hard to say things exactly as you want given these constraints.

If you need to communicate yourself clearly through speech, you probably will prepare and practice beforehand. You’ll structure and organize your thoughts. You will consider your argument, counterarguments, and your audience. When you speak your ideas, you will have already done most of your thinking and can focus on delivery.

In conversation, you never quite know what your conversation partner will say. You spend mental bandwidth listening and processing what the other person is saying. When it’s your turn to speak, you will have just finished hearing the thoughts of your partner and then you need to verbalize your own thoughts to respond.

How can you listen, think, and figure out what you’re going to say all at once?

At best, your mental bandwidth is split. In practice, you are reacting. With practice and the preparation of experience, you respond coherently and clearly. But it’s hard to imagine doing your best thinking under such tight mental constraints.

Writing is nonlinear

Your best idea is unlikely to be your first idea. Your clearest thought is unlikely to be your first.

It’s challenging and effortful to think things through. It’s often complicated, requires research, presents tradeoffs. Most things are complicated if you take the time to look at them closely.

Writing is a better fit than speaking for thinking because writing allows for nonlinear interaction with ideas. With thoughts laid out in words, you can fluidly jump between paragraphs, reorder how concepts are introduced, swoop in and change a single word or phrase for clarity.

You can sit with your work, take breaks when you’re tired, and return when you are inspired. You can evolve and define your ideas. You think.

Writing changes your ideas

Writing is thinking because when you write about something, your ideas change with the writing. You are a slightly different person at the end of writing a work than you were before.

Writing needs to be focused, in part, to ensure you can actually stop writing. To provide a reasonable spot to conclude work.

The writing that feels most worthwhile leaves me with as many new questions as answers.

Yet this always means a writer is cursed to spend forever chasing their tail, writing to think, writing to think.

Writing and agents

When working with agents, the most common difficulty I hear when someone feels they are not having success is that the agent is failing to do what they want it to do. Typically, this challenge results from not having done the necessary thinking about what is to be done.

Knowing what you want is the only way to collaborate successfully with a coding agent. These tools are trained to be instruction following. In a void of instructions, they often make median, flavorless decisions. In a void of context, they attempt to fill in the gaps as best they can.

Context engineering (a topic worthy of its own post) is a downstream application of knowing what you want.

You can only begin to know what you want by thinking about what it is you need to accomplish.

You write to do this thinking. As you write more, you think more and refine your approach.

So if you are struggling getting a coding agent to do what you want, think about what it is that you want to accomplish. Write it down.

Once you’re finished, you will have done your thinking and you will have written your prompt as well.

Thanks for reading via RSS!

Embeddings with sqlite-vector

Thu, 16 Oct 2025 00:45:52 GMT

About a year and a half ago I wrote about using sqlite-vss to store and query embedding vectors in a SQLite database. Much has changed since then and I’m working on a project that motivated another pass at querying embeddings on a local system for smallish datasets. The sqlite-vector project seemed like an interesting one to try for this purpose.

I am going to use the same news dataset as last time and the nomic-embed-text-v1.5 model to generate 768-dimensional embeddings.

I also downloaded the vector.dylib file from the sqlite-vector GitHub repo and placed it in my working directory for this example. I’ve tried exercises similar to this one with both the macOS and Linux versions of the library.

To get started, we’ll install the libraries we will need to load the data, create the database and generate embeddings.

pip install -q pandas scikit-learn sentence-transformers

Next, we’ll create the database and table and load the sqlite-vector extension, verifying that it loaded correctly.

import sqlite3
import json
from pathlib import Path


conn = sqlite3.connect('news.db')
conn.enable_load_extension(True)
conn.load_extension('./vector.dylib')
conn.enable_load_extension(False)

version = conn.execute("SELECT vector_version()").fetchone()[0]
print(f"SQLite Vector extension version: {version}")

SQLite Vector extension version: 0.9.37

Looks good! The query shows the same version I downloaded.

If you haven’t taken a look at the dataset yet, here’s a sample:

import pandas as pd

# Load the first few records to see the structure
data_path = Path('data/News_Category_Dataset_v3.json')
articles = []

with open(data_path, 'r') as f:
    for i, line in enumerate(f):
        articles.append(json.loads(line))

df = pd.DataFrame(articles)
df.head()

	link	headline	category	short_description	authors	date
0	https://www.huffpost.com/entry/covid-boosters-…	Over 4 Million Americans Roll Up Sleeves For O…	U.S. NEWS	Health experts said it is too early to predict…	Carla K. Johnson, AP	2022-09-23
1	https://www.huffpost.com/entry/american-airlin…	American Airlines Flyer Charged, Banned For Li…	U.S. NEWS	He was subdued by passengers and crew when he …	Mary Papenfuss	2022-09-23
2	https://www.huffpost.com/entry/funniest-tweets…	23 Of The Funniest Tweets About Cats And Dogs …	COMEDY	”Until you have a dog you don’t understand wha…	Elyse Wanshel	2022-09-23
3	https://www.huffpost.com/entry/funniest-parent…	The Funniest Tweets From Parents This Week (Se…	PARENTING	”Accidentally put grown-up toothpaste on my to…	Caroline Bologna	2022-09-23
4	https://www.huffpost.com/entry/amy-cooper-lose…	Woman Who Called Cops On Black Bird-Watcher Lo…	U.S. NEWS	Amy Cooper accused investment firm Franklin Te…	Nina Golgowski	2022-09-22

From here, we’ll use the same database connection to create a table to store the news dataset as well as embeddings for both the short_description and headline fields of the dataset.

I’m only using the first 1000 entries because it takes a few minutes to compute all the embeddings.

conn.execute("""
    CREATE TABLE news_articles (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        link TEXT,
        headline TEXT,
        category TEXT,
        short_description TEXT,
        authors TEXT,
        date TEXT,
        headline_embedding BLOB,
        description_embedding BLOB
    )
""")

for _, article in df.head(1000).iterrows():
    conn.execute("""
        INSERT INTO news_articles (link, headline, category, short_description, authors, date)
        VALUES (?, ?, ?, ?, ?, ?)
    """, (
        article['link'],
        article['headline'],
        article['category'],
        article['short_description'],
        article['authors'],
        article['date']
    ))

conn.commit()


count = conn.execute("SELECT COUNT(*) FROM news_articles").fetchone()[0]
print(f"Loaded {count:,} articles")

print("\nSample articles:")
for row in conn.execute("SELECT id, headline, category FROM news_articles LIMIT 5"):
    print(f"  [{row[2]}] {row[1][:70]}...")

Loaded 1,000 articles

Sample articles:
  [U.S. NEWS] Over 4 Million Americans Roll Up Sleeves For Omicron-Targeted COVID Bo...
  [U.S. NEWS] American Airlines Flyer Charged, Banned For Life After Punching Flight...
  [COMEDY] 23 Of The Funniest Tweets About Cats And Dogs This Week (Sept. 17-23)...
  [PARENTING] The Funniest Tweets From Parents This Week (Sept. 17-23)...
  [U.S. NEWS] Woman Who Called Cops On Black Bird-Watcher Loses Lawsuit Against Ex-E...

With the source data loaded, we’ll now generate the embeddings for the headline and short_description fields, using the nomic-embed-text-v1.5 model and store those in the database as well.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('nomic-ai/nomic-embed-text-v1.5', trust_remote_code=True)
print(f"Model loaded (embedding dimension: {model.get_sentence_embedding_dimension()})")

Model loaded (embedding dimension: 768)

articles = conn.execute("""
    SELECT id, headline, short_description
    FROM news_articles
""").fetchall()

batch_size = 32
for i in range(0, len(articles), batch_size):
    batch = articles[i:i + batch_size]
    article_ids = [article[0] for article in batch]
    headlines = [article[1] for article in batch]
    descriptions = [article[2] if article[2] else "" for article in batch]

    headline_embeddings = model.encode(headlines, normalize_embeddings=True)
    description_embeddings = model.encode(descriptions, normalize_embeddings=True)

    # Store in database as BLOB
    for article_id, h_emb, d_emb in zip(article_ids, headline_embeddings, description_embeddings):
        h_blob = h_emb.tobytes()
        d_blob = d_emb.tobytes()
        conn.execute(
            "UPDATE news_articles SET headline_embedding = ?, description_embedding = ? WHERE id = ?",
            (h_blob, d_blob, article_id)
        )

    # Commit and show progress every 4000 articles (helpful if you actually load all the data)
    if (i + batch_size) % 4000 == 0 or (i + batch_size) >= len(articles):
        conn.commit()
        print(f"Processed {min(i + batch_size, len(articles)):,}/{len(articles):,} articles")

Processed 1,000/1,000 articles

With the embeddings generated and inserted into the database, we’ll now initialize the vector search for both the headline and short_description fields. According to the documentation

Only tables explicitly initialized using vector_init are eligible for vector search.

So let’s do that:

conn.execute("""
    SELECT vector_init(
        'news_articles',
        'headline_embedding',
        'type=FLOAT32,dimension=768,distance=COSINE'
    )
""")

conn.execute("""
    SELECT vector_init(
        'news_articles',
        'description_embedding',
        'type=FLOAT32,dimension=768,distance=COSINE'
    )
""")

We could also quantize the vector for faster and more efficient search, but let’s save that for another time.

With our embeddings generated and initialized, we can now define the search functions over the two fields. Before we do though, we’ll briefly discuss how the search functions work. We start by taking query_text and encoding it using the same model we used to generate the embeddings.

encoded = model.encode(["hello world!"])[0]
encoded[:20]  # truncated for brevity

array([ 0.12390442, -0.06160246, -3.9840953 , -0.24632797, -0.3041943 ,
        1.5918531 , -0.20516105, -0.6006584 , -0.22573042, -1.319676  ,
        0.31299376,  1.2585417 ,  0.49875298,  1.381727  ,  0.5602437 ,
       -1.0751574 ,  0.27402204, -1.1879947 , -0.98643625,  0.46151334],
      dtype=float32)

Specifically, the size of the vector is equal to the number of dimensions of the embedding model.

encoded.shape

(768,)

Given this approach, let’s show how we can calculate the distance of a query from one of the headlines in the dataset. As we specified in vector_init, we will use cosine distance (distance=COSINE).

from sklearn.metrics.pairwise import cosine_similarity


query = "sports"
headline = "New York City To Let Unvaccinated Athletes Play Home Games"

query_embedding = model.encode([query], normalize_embeddings=True)
headline_embedding = model.encode([headline], normalize_embeddings=True)

similarity = cosine_similarity(query_embedding, headline_embedding)
print(1-similarity)

[[0.34292662]]

Back to analyzing our dataset — since I normalized the embeddings when I encoded them above, we’ll also need to do that here. Finally, we’ll get the output as an array that we can convert to a blob (our column data type of the embeddings) and pass it to the vector_full_scan function.

f"{model.encode(["hello world!"], normalize_embeddings=True).tobytes()[:20]}..."

"b'\\r-\\xb4;\\xa8(3\\xbb\\xdf\\x0b5\\xbej\\x193\\xbc*,]\\xbc'..."

def search_by_headline(conn, query_text, top_k=5):
    query_embedding = model.encode([query_text], normalize_embeddings=True)
    query_blob = query_embedding.tobytes()

    results = conn.execute("""
        SELECT a.id, a.headline, a.category, a.short_description, v.distance
        FROM news_articles AS a
        JOIN vector_full_scan('news_articles', 'headline_embedding', ?, ?) AS v
            ON a.id = v.rowid
        ORDER BY v.distance ASC
    """, (query_blob, top_k)).fetchall()

    return results


def search_by_description(conn, query_text, top_k=5):
    query_embedding = model.encode([query_text], normalize_embeddings=True)
    query_blob = query_embedding.tobytes()

    results = conn.execute("""
        SELECT a.id, a.headline, a.category, a.short_description, v.distance
        FROM news_articles AS a
        JOIN vector_full_scan('news_articles', 'description_embedding', ?, ?) AS v
            ON a.id = v.rowid
        ORDER BY v.distance ASC
    """, (query_blob, top_k)).fetchall()

    return results

Let’s run some semantic queries and see what we find! This should look familiar — we’re using the same query we embedded and calculated cosine distance for above.

query = "sports"

headline_results = search_by_headline(conn, query, top_k=3)
for i, (article_id, headline, category, description, distance) in enumerate(headline_results, 1):
    print(f"{i}. [{category}] {headline} (distance: {distance:.4f})")

1. [U.S. NEWS] New York City To Let Unvaccinated Athletes Play Home Games (distance: 0.3429)
2. [SPORTS] Video Shows NFL Player Hopping Into Stands To Stop Fighting Fans (distance: 0.3828)
3. [POLITICS] NBA Won't Host Election Day Games In Effort To Increase Voter Turnout (distance: 0.4232)

We see an identical result as we calculated above for the first (and closest) headline match (distance: 0.3429).

Now we can run a similar query on the description field:

description_results = search_by_description(conn, query, top_k=3)
for i, (article_id, headline, category, description, distance) in enumerate(description_results, 1):
    print(f"{i}. [{category}] {headline} (distance: {distance:.4f})")
    print(f"   Description: {description}")

1. [SPORTS] Las Vegas Aces Win First WNBA Title, Chelsea Gray Named MVP (distance: 0.3580)
   Description: Las Vegas never had a professional sports champion — until Sunday.
2. [SPORTS] Lucius Fox Makes Sickening Exit Just 2 Pitches Into Game (distance: 0.3966)
   Description: The Washington Nationals infielder became a hurler for one gross moment — which was caught on video — against the San Francisco Giants.
3. [SPORTS] Diego Maradona's 'Hand Of God' Jersey Sells For Record Sum (distance: 0.4160)
   Description: The Argentine soccer superstar wore the shirt when he scored the controversial goal against England in the 1986 World Cup.

headline_results = search_by_headline(conn, "artificial intelligence", top_k=3)
description_results = search_by_description(conn, "artificial intelligence", top_k=3)

for i, (article_id, headline, category, description, distance) in enumerate(headline_results, 1):
    print(f"{i}. [{category}] {headline} (distance: {distance:.4f})")

print()

for i, (article_id, headline, category, description, distance) in enumerate(description_results, 1):
    print(f"{i}. [{category}] {description} (distance: {distance:.4f})")

1. [TECH] Google Engineer On Leave After He Claims AI Program Has Gone Sentient (distance: 0.3774)
2. [CULTURE & ARTS] How Ani Liu Is Brilliantly Disguising Her Art As Science (distance: 0.5151)
3. [ENTERTAINMENT] Andrew Garfield Confirms Method Acting Is Possible Without 'Being An Asshole' (distance: 0.5217)

1. [TECH] Artificially intelligent chatbot generator LaMDA wants “to be acknowledged as an employee of Google rather than as property," says engineer Blake Lemoine. (distance: 0.4230)
2. [POLITICS] The House Intelligence Committee hearing follows a 2021 report of a possible national security "challenge" from UFOs. (distance: 0.4950)
3. [U.S. NEWS] The American Academy of Pediatrics says it is putting all its guidance under the microscope to eliminate “race-based” medicine and resulting health disparities. (distance: 0.4967)

Embeddings and cosine distance allow us to do search over the headline and short_description fields of the dataset, finding semantically similar results even if the results themselves don’t contain or match the query text or parts of the query text exactly.

Thanks for reading via RSS!

Debugging DSPy token usage and prompts

Sun, 12 Oct 2025 19:37:35 GMT

As I’ve experimented more with DSPy, I’ve attempted to dig further into the details of the specific LLM I am using to better understand things like token usage, reasoning, and the specific prompts being sent to the model. While, DSPy’s abstraction is one of the more lightweight for LLM frameworks, in my experience, I generally want to see what is being sent to the model when I’m trying to debug or improve performance for a use case.

A simple example

We will start with a classic example for me — using DSPy to extract a date from a sentence in a specific format.

import dspy

class ExtractDate(dspy.Signature):
    """Extract a date in yyyy-mm-dd format from the given sentence."""

    sentence = dspy.InputField(desc="The input sentence containing a date")
    date = dspy.OutputField(desc="The extracted date in yyyy-mm-dd format")

sentence = "The meeting is scheduled for March 15th, 2024."

The first question I often find myself asking is “how many tokens did that inference use?” This is exposed in a straightforward way with DSPy but the framework also enables cache by default. When experimenting, this is a great help because the framework avoids re-running inference it has already run before, saving you time and money. However, when trying to measure token utilization (which you’ll usually attempt after already running an inference at least once), this caching prevents you from seeing the actual token usage because in practice, for that call, there wasn’t any. The result was pulled from cache.

To consistently see the token usage, the most straightforward approach I have found is to disable caching.

extract_date = dspy.Predict(ExtractDate)


with dspy.settings.context(lm=dspy.LM("openai/gpt-4.1-mini", cache=False), track_usage=True):
    result = extract_date(sentence=sentence)
    print(result)

    usage = result.get_lm_usage()
    print(usage)

Prediction(
    date='2024-03-15'
)
{'openai/gpt-4.1-mini': {'completion_tokens': 17, 'prompt_tokens': 167, 'total_tokens': 184, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0, 'text_tokens': None}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0, 'text_tokens': None, 'image_tokens': None}}}

Let’s make a function to report the results and usage, since we’re going to do this a lot.

import json


def filter_none_recursive(obj):
    if isinstance(obj, dict):
        return {k: filter_none_recursive(v) for k, v in obj.items() if v is not None}
    elif isinstance(obj, list):
        return [filter_none_recursive(item) for item in obj if item is not None]
    else:
        return obj

def print_results_and_usage(result):
    usage = result.get_lm_usage()
    print(result)
    filtered_usage = filter_none_recursive(usage)
    print(json.dumps(filtered_usage, indent=2))

Now we can take a look at the different ways to configure the model with DSPy to output its response using reasoning.

Gemini Flash 2.5 uses reasoning by default.

with dspy.settings.context(lm=dspy.LM("gemini/gemini-2.5-flash", cache=False), track_usage=True):
    result = extract_date(sentence=sentence)
    print_results_and_usage(result)

Prediction(
    date='2024-03-15'
)
{
  "gemini/gemini-2.5-flash": {
    "completion_tokens": 133,
    "prompt_tokens": 179,
    "total_tokens": 312,
    "completion_tokens_details": {
      "reasoning_tokens": 111,
      "text_tokens": 22
    },
    "prompt_tokens_details": {
      "text_tokens": 179
    }
  }
}

This can be disabled by setting reasoning_effort to disable on dspy.LM.

with dspy.settings.context(
    lm=dspy.LM("gemini/gemini-2.5-flash", cache=False, reasoning_effort="disable"),
    track_usage=True,
):
    result = extract_date(sentence=sentence)
    print_results_and_usage(result)

Prediction(
    date='2024-03-15'
)
{
  "gemini/gemini-2.5-flash": {
    "completion_tokens": 22,
    "prompt_tokens": 179,
    "total_tokens": 201,
    "prompt_tokens_details": {
      "text_tokens": 179
    }
  }
}

It can also be disabled by setting reasoning_effort to disable on the Predict instance.

with dspy.settings.context(
    lm=dspy.LM("gemini/gemini-2.5-flash", cache=False),
    track_usage=True,
):
    result = extract_date(
        sentence=sentence,
        config={"reasoning_effort": "disable"},
    )
    print_results_and_usage(result)

Prediction(
    date='2024-03-15'
)
{
  "gemini/gemini-2.5-flash": {
    "completion_tokens": 22,
    "prompt_tokens": 179,
    "total_tokens": 201,
    "prompt_tokens_details": {
      "text_tokens": 179
    }
  }
}

Following the first approach, we can set reasoning_effort to high to attempt to elicit more reasoning tokens from the model.

with dspy.settings.context(
    lm=dspy.LM("gemini/gemini-2.5-flash", cache=False, reasoning_effort="high"),
    track_usage=True,
):
    result = extract_date(sentence=sentence)
    print_results_and_usage(result)

Prediction(
    date='2024-03-15'
)
{
  "gemini/gemini-2.5-flash": {
    "completion_tokens": 153,
    "prompt_tokens": 179,
    "total_tokens": 332,
    "completion_tokens_details": {
      "reasoning_tokens": 131,
      "text_tokens": 22
    },
    "prompt_tokens_details": {
      "text_tokens": 179
    }
  }
}

We can do something similar with GPT-5, OpenAI’s hybrid reasoning model. Note: The OpenAI API requires temperature=1 and max_tokens >= 16000 for the gpt-5 model, hence the changes below.

with dspy.settings.context(
    lm=dspy.LM("openai/gpt-5", temperature=1, max_tokens=16000, cache=False, reasoning_effort="low"),
    track_usage=True,
):
    result = extract_date(sentence=sentence)
    print_results_and_usage(result)

Prediction(
    date='2024-03-15'
)
{
  "openai/gpt-5": {
    "completion_tokens": 90,
    "prompt_tokens": 166,
    "total_tokens": 256,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 0,
      "audio_tokens": 0,
      "reasoning_tokens": 64,
      "rejected_prediction_tokens": 0
    },
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0
    }
  }
}

But it doesn’t work in exactly the same way as Gemini. Here we see a LiteLLM error (the underlying translation layer DSPy uses to support different LLM providers), where our reasoning_effort configuration is invalid for the OpenAI API.

with dspy.settings.context(
    lm=dspy.LM("openai/gpt-5", temperature=1, max_tokens=16000, cache=False, reasoning_effort="disable"),
    track_usage=True,
):
    result = extract_date(sentence=sentence)

...
BadRequestError: litellm.BadRequestError: OpenAIException - Invalid value: 'disable'. Supported values are: 'low', 'medium', and 'high'.

Inspecting raw prompts and responses with history

We’ve experimented with configuring a few different LLMs. Now, let’s inspect the output of their inference and the prompts DSPy sends to the model. Each LM instance maintains a history list that contains detailed information about every call, including the raw messages, responses, and usage stats.

Let’s take a look at this.

lm = dspy.LM("openai/gpt-5-mini", cache=False, temperature=1, max_tokens=16000)

with dspy.settings.context(lm=lm, track_usage=True):
    result = extract_date(sentence=sentence, config={"reasoning_effort": "high"})

lm.history

[{'prompt': None,
  'messages': [{'role': 'system',
    'content': 'Your input fields are:\n1. `sentence` (str): The input sentence containing a date\nYour output fields are:\n1. `date` (str): The extracted date in yyyy-mm-dd format\nAll interactions will be structured in the following way, with the appropriate values filled in.\n\n[[ ## sentence ## ]]\n{sentence}\n\n[[ ## date ## ]]\n{date}\n\n[[ ## completed ## ]]\nIn adhering to this structure, your objective is: \n        Extract a date in yyyy-mm-dd format from the given sentence.'},
   {'role': 'user',
    'content': '[[ ## sentence ## ]]\nThe meeting is scheduled for March 15th, 2024.\n\nRespond with the corresponding output fields, starting with the field `[[ ## date ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.'}],
  'kwargs': {'reasoning_effort': 'high'},
  'response': ModelResponse(id='chatcmpl-CPw9qDTZzuUNjdNIbFladiHQkXGTe', created=1760297794, model='gpt-5-mini-2025-08-07', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='stop', index=0, message=Message(content='[[ ## date ## ]]\n2024-03-15\n\n[[ ## completed ## ]]', role='assistant', tool_calls=None, function_call=None, provider_specific_fields={'refusal': None}, annotations=[]), provider_specific_fields={})], usage=Usage(completion_tokens=154, prompt_tokens=166, total_tokens=320, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=128, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)), service_tier='default'),
  'outputs': ['[[ ## date ## ]]\n2024-03-15\n\n[[ ## completed ## ]]'],
  'usage': {'completion_tokens': 154,
   'prompt_tokens': 166,
   'total_tokens': 320,
   'completion_tokens_details': CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=128, rejected_prediction_tokens=0, text_tokens=None),
   'prompt_tokens_details': PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)},
  'cost': 0.0003495,
  'timestamp': '2025-10-12T15:36:37.579158',
  'uuid': '2f29dbd0-18d4-4a1b-9791-578221b5c1c7',
  'model': 'openai/gpt-5-mini',
  'response_model': 'gpt-5-mini-2025-08-07',
  'model_type': 'chat'}]

lm.inspect_history(1)

[2025-10-12T15:36:37.579158]

System message:

Your input fields are:
1. `sentence` (str): The input sentence containing a date
Your output fields are:
1. `date` (str): The extracted date in yyyy-mm-dd format
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## sentence ## ]]
{sentence}

[[ ## date ## ]]
{date}

[[ ## completed ## ]]
In adhering to this structure, your objective is:
        Extract a date in yyyy-mm-dd format from the given sentence.


User message:

[[ ## sentence ## ]]
The meeting is scheduled for March 15th, 2024.

Respond with the corresponding output fields, starting with the field `[[ ## date ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.


Response:

[[ ## date ## ]]
2024-03-15

[[ ## completed ## ]]

Here we see almost everything we’d want to know about the prompt DSPy sent and the response it received from the model. We even see kwargs overrides that were passed for the specific call.

The main thing we miss is the configuration of the LM instance itself for that specific message, which we can get like this:

lm.dump_state()

{'model': 'openai/gpt-5-mini',
 'model_type': 'chat',
 'cache': False,
 'num_retries': 3,
 'finetuning_model': None,
 'launch_kwargs': {},
 'train_kwargs': {},
 'temperature': 1,
 'max_completion_tokens': 16000}

This isn’t necessarily a foolproof way to get what the LM configuration was at call time because the LM and its kwargs are mutable. It’s possible (though probably not advisable) to change them after the call has been made. However, for my own experimentation purposes, I’ve found the above functions to be helpful.

Remember to re-enable caching when you’re done inspecting token usage to save time and cost!

Thanks for reading via RSS!

Intro to DSPy

Sun, 28 Sep 2025 18:43:52 GMT

The viral “how many r’s are there in the word ‘strawberry’?” problem is a great motivating example to experiment with the capabilities of DSPy, an increasingly popular, powerful library for running inference with language models. While the counting letters problem itself is a poor application of a language model, using DSPy to experiment helps us build intuition for how to make a language model perform better at a specific task, using manual and language model-based optimization approaches.

import dspy
import pandas as pd

from dspy.evaluate import Evaluate

gpt_4_1_nano_lm = dspy.LM('openai/gpt-4.1-nano-2025-04-14')

A Signature is the foundational primitive in DSPy. Personally, I think of a signature (or typed object) as the standard primitive of interacting with any language model in general.

Unless you are building a pure chatbot, to stitch the functionality of the LLM into your application, you’re going to need to impose structure on the outputs of the LLM.

Imposing structure on the inputs helps too. The better these are labeled, the more likely the output for the model will be aligned with the correct answer you’re looking for.

If you don’t buy that, let’s see if I can convince you it is true.

A Signature is just one part of a functioning LLM application. The next most important primitive of building with a language model is a dataset.

Once you know your inputs and outputs to the model, you need to figure out if the inference the model is doing is correct, based on your own measurement of what that means. Without doing this, you won’t know if your prompt and inputs combination results in inference with the correct output.

The Signature forces you to think about what information is relevant and what the shape of the answer looks like. Systematizing and automating inference with this process allows you to experiment with and evolve the signature while guarding against regressions.

With that context, let’s create a DSPy signature to count the number of occurrences of a specific letter in a given word, modelling off the “how many r’s are there in the word ‘strawberry’” example.

class CountLetterInWord(dspy.Signature):
    """Count the number of occurrences of a specific letter in a given word."""

    word: str = dspy.InputField(desc="The word to search in")
    letter: str = dspy.InputField(desc="The letter to count")
    count: int = dspy.OutputField(desc="The number of times the `letter` appears in the `word`")

count_letter = dspy.Predict(CountLetterInWord)

With a Signature and Predict classes defined, we can now map inputs to outputs using the LLM.

with dspy.settings.context(lm=gpt_4_1_nano_lm):
    result = count_letter(word='basketball', letter='b')

result

Prediction(
    count=2
)

The model got this one right!

From here, let’s use the system dictionary (/usr/share/dict/words) to create a dataset of words and their letter counts. We’ll use this dataset to evaluate the performance of different signatures, models, and eventually optimization approaches.

import csv
import os
import random

# Read words from system dictionary
with open('/usr/share/dict/words', 'r') as f:
    words = [word.strip().lower() for word in f.readlines() if word.strip()]

# Sample 400 words randomly
sample_words = random.sample(words, min(400, len(words)))

# Create dataset
dataset = []
for word in sample_words:
    # Pick a random letter from the word
    letter = random.choice(word)
    # Count occurrences of that letter in the word
    count = word.count(letter)
    dataset.append([word, letter, count])

For a bit more data diversity, let’s also add 100 rows where the letter is not in the word.

# Add 100 rows where the letter is not in the word
for _ in range(100):
    word = random.choice(sample_words)
    # Find a letter that's not in the word
    alphabet = 'abcdefghijklmnopqrstuvwxyz'
    available_letters = [letter for letter in alphabet if letter not in word]
    if available_letters:
        letter = random.choice(available_letters)
        dataset.append([word, letter, 0])

# Create the dataset directory if it doesn't exist
os.makedirs('dataset', exist_ok=True)

# Write the dataset to CSV
with open('dataset/word_letter_dataset.csv', 'w', newline='') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['word', 'letter', 'count'])
    writer.writerows(dataset)

print(f"Updated dataset with {len(dataset)} rows total")

Now, let’s load the dataset we created and start to experiment

df = pd.read_csv('dataset/word_letter_dataset.csv')
df.head()

	word	letter	count
0	uniate	t	1
1	ridiculer	r	2
2	market	e	1
3	subsident	n	1
4	irisation	i	3

Using DSPy’s Evaluate class, we can create a list of examples and a simple metric to evaluate the performance of our signature and model against the labeled dataset. Remember, given our CountLetterInWord signature, the inputs are word and letter and the output generated using LLM inference is count. Our dataframe loaded from the csv has all three of these values, so we can use the Evaluate class to determine how often our model inference is (based on the Signature) actually outputs the correct answer.

Let’s try running inference on 50 random examples from the dataset.

sampled_df = df.sample(n=50, random_state=42)

examples = []

for _, row in sampled_df.iterrows():
    examples.append(dspy.Example(
        word=row['word'],
        letter=row['letter'],
        count=row['count']
    ).with_inputs('word', 'letter'))

def letter_count_metric(example, prediction, trace=None):
    return example.count == prediction.count


evaluator = Evaluate(
    devset=examples,
    metric=letter_count_metric,
    num_threads=1,
)

with dspy.settings.context(lm=gpt_4_1_nano_lm):
    results = evaluator(count_letter)

2025/09/28 17:03:14 INFO dspy.evaluate.evaluate: Average Metric: 36 / 50 (72.0%)

Inference from gpt-4.1-nano-2025-04-14 gets 72% of the examples correct. Not bad but room for improvement.

Here’s an example of a failure:

print('Error for standard inference:\n')
for i, result in enumerate(results.results):
    example, prediction, correct = result
    predicted = prediction.count
    actual = example.count
    word = example.word
    letter = example.letter

    if not correct:
        print(result)
        break

Error for standard inference:

(Example({'word': 'thermogeography', 'letter': 'e', 'count': 2}) (input_keys={'word', 'letter'}), Prediction(
    reasoning="I need to count how many times the letter 'e' appears in the word 'thermogeography'. I will examine each letter in the word and check if it matches 'e'. The word is 'thermogeography', which contains the following letters: t, h, e, r, m, o, g, r, a, p, h, y. The letter 'e' appears once in the third position. Therefore, the total count of 'e' in the word is 1.",
    count=1
), False)

Improving accuracy

“Chain of thought” or “reasoning” has become such a common pattern to improve the performance of language models that DSPy has a built-in class for it. These approaches both effectively mean prompting to get the model to output more tokens moving in the direction of the correct answer. Doing this improves the performance of the model. It kind of makes sense. It’s kind of surprising. Given how the models are built today, it seems to work in a lot of scenarios.

We can wrap our signature in the ChainOfThought class in DSPy which will elicit a chain of thought from the LLM and capture it in a field called reasoning as the first output field. It’s important that the field is the first output field as the reasoning leads the model in the direction of the correct answer.

count_letter_cot = dspy.ChainOfThought(CountLetterInWord)

with dspy.settings.context(lm=gpt_4_1_nano_lm):
    cot_results = evaluator(count_letter_cot)

2025/09/28 17:48:16 INFO dspy.evaluate.evaluate: Average Metric: 43 / 50 (86.0%)

Better already!

We can now look at some of the failures and see the model’s reasoning process as well:

print('Error results for chain of thought:\n')
for i, result in enumerate(cot_results.results):
    example, prediction, correct = result
    predicted = prediction.count
    actual = example.count
    word = example.word
    letter = example.letter

    if not correct:
        print(result)
        break

Error results for chain of thought:

(Example({'word': 'voicelike', 'letter': 'e', 'count': 2}) (input_keys={'word', 'letter'}), Prediction(
    reasoning="The task is to count how many times the letter 'e' appears in the word 'voicelike'. I will examine the word and count each occurrence of 'e'. The word 'voicelike' contains the letter 'e' at the end, and it appears once in the word.",
    count=1
), False)

The reasoning is faulty. In fact it’s not really reasoning at all and it’s not leading the model to the correct answer.

Why is this? A good first place to look is the prompt that DSPy sent to the model. This pulls back the curtain on the Signature abstraction to show us the actual prompting being sent to the model.

Writing DSPy code is still just prompting the LLM, but it’s convenient and clever prompting wired into a powerful abstraction. By focusing on the inputs, outputs, and their types and descriptions, DSPy allows you to focus on the most important parts that influence the quality of the inference but still mostly work with code not prompts.

print(count_letter_cot.history[-1]['messages'][0]['content']) # system message
print(count_letter_cot.history[-1]['messages'][1]['content']) # user message

Your input fields are:
1. `word` (str): The word to search in
2. `letter` (str): The letter to count
Your output fields are:
1. `reasoning` (str):
2. `count` (int): The number of times the `letter` appears in the `word`
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## word ## ]]
{word}

[[ ## letter ## ]]
{letter}

[[ ## reasoning ## ]]
{reasoning}

[[ ## count ## ]]
{count}        # note: the value you produce must be a single int value

[[ ## completed ## ]]
In adhering to this structure, your objective is:
        Count the number of occurrences of a specific letter in a given word.
[[ ## word ## ]]
anisosepalous

[[ ## letter ## ]]
s

Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## count ## ]]` (must be formatted as a valid Python int), and then ending with the marker for `[[ ## completed ## ]]`.

The prompt structure makes sense but the reasoning the model did to get to the answer did not.

Let’s modify the signature to include a step-by-step explanation of how we want to model to reason to figure out the answer for this problem.

class CountLetterInWordStrategic(dspy.Signature):
    """
    Count the number of occurrences of a specific letter in a given word.

    To accomplish this task, for each letter in the word, check if it matches the target letter.
    For example, if we are looking for the letter 'a' in the word 'banana':

    1. b - this is not an 'a'
    2. a - this is an 'a'
    3. n - this is not an 'a'
    4. a - this is an 'a'
    5. n - this is not an 'a'
    6. a - this is an 'a'

    We count 3 'a's in the word 'banana'
    """

    reasoning: str = dspy.OutputField(desc="A step-by-step explanation of the reasoning process")
    word: str = dspy.InputField(desc="The word to search in")
    letter: str = dspy.InputField(desc="The letter to count")
    count: int = dspy.OutputField(desc="The number of times the `letter` appears in the `word`")

count_letter_strategic = dspy.Predict(CountLetterInWordStrategic)
with dspy.settings.context(lm=gpt_4_1_nano_lm):
    results = evaluator(count_letter_strategic)

2025/09/28 17:49:49 INFO dspy.evaluate.evaluate: Average Metric: 48 / 50 (96.0%)

Way better!

So what did this addition of the Python docstring do? We can check the prompt and see the difference:

print(count_letter_strategic.history[-1]['messages'][0]['content']) # system message
print(count_letter_strategic.history[-1]['messages'][1]['content']) # user message

Your input fields are:
1. `word` (str): The word to search in
2. `letter` (str): The letter to count
Your output fields are:
1. `reasoning` (str): A step-by-step explanation of the reasoning process
2. `count` (int): The number of times the `letter` appears in the `word`
All interactions will be structured in the following way, with the appropriate values filled in.

[[ ## word ## ]]
{word}

[[ ## letter ## ]]
{letter}

[[ ## reasoning ## ]]
{reasoning}

[[ ## count ## ]]
{count}        # note: the value you produce must be a single int value

[[ ## completed ## ]]
In adhering to this structure, your objective is:
        Count the number of occurrences of a specific letter in a given word.

        To accomplish this task, for each letter in the word, check if it matches the target letter.
        For example, if we are looking for the letter 'a' in the word 'banana':

        1. b - this is not an 'a'
        2. a - this is an 'a'
        3. n - this is not an 'a'
        4. a - this is an 'a'
        5. n - this is not an 'a'
        6. a - this is an 'a'

        We count 3 'a's in the word 'banana'
[[ ## word ## ]]
anisosepalous

[[ ## letter ## ]]
s

Respond with the corresponding output fields, starting with the field `[[ ## reasoning ## ]]`, then `[[ ## count ## ]]` (must be formatted as a valid Python int), and then ending with the marker for `[[ ## completed ## ]]`.

Adding the docstring gets DSPy add the section “In adhering to this structure, your objective is: …” to the prompt.

We can also inspect the reasoning from one of the inferences to see the difference:

print(results.results[0][1].reasoning)

The word provided is "repand" and the letter to count is "p". I will examine each letter in the word to see if it matches "p". The first letter is "r", which does not match. The second letter is "e", which does not match. The third letter is "p", which matches. The fourth letter is "a", which does not match. The fifth letter is "n", which does not match. The sixth letter is "d", which does not match. Therefore, the total count of "p" in "repand" is 1.

While the model didn’t strictly adhere to the example in our instructions, it followed the prompt instructions enough to inspect each letter individually as part of the reasoning process.

Improving the prompt in this manner can be useful in simple examples, but what if you’re not exactly sure how to improve the prompt?

DSPy offers an approach called teleprompting where the model use a subset of a labeled dataset for training, evaluate the results with a signature you define, then optimize the results by providing correct examples to the model as part of the prompt as well as rewriting the instructions with the LLM itself to try and improve the results.

What I’ve just described is an approach called MIPROv2.

Let’s try it out with with 150 training examples and 50 test examples (same as our initial set size). We’ll use the count_letter_cot signature to start with which gives the model space to reason about the problem but doesn’t impose the opinionated instructions we added for count_letter_strategic.

# Create train and test sets
# Split data to ensure no overlap between train and test sets
df_shuffled = df.sample(frac=1, random_state=42).reset_index(drop=True)

train_data = [dspy.Example(word=row['word'], letter=row['letter'], count=row['count']).with_inputs('word', 'letter')
              for _, row in df_shuffled.iloc[:150].iterrows()]

test_data = [dspy.Example(word=row['word'], letter=row['letter'], count=row['count']).with_inputs('word', 'letter')
             for _, row in df_shuffled.iloc[150:200].iterrows()]

from dspy.teleprompt import MIPROv2

mipro_optimizer = MIPROv2(
    metric=letter_count_metric,
    auto="light",
    seed=42
)

with dspy.settings.context(lm=gpt_4_1_nano_lm):
    # Optimize the CountLetterCOT module
    print("Optimizing with MIPROv2...")
    optimized_count_letter = mipro_optimizer.compile(
        count_letter_cot,
        trainset=train_data,
        valset=test_data,
    )

    mipro_results = evaluator(optimized_count_letter)


print(f"MIPROv2 optimized results: {mipro_results}")

2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2:
RUNNING WITH THE FOLLOWING LIGHT AUTO RUN SETTINGS:
num_trials: 10
minibatch: False
num_fewshot_candidates: 6
num_instruct_candidates: 3
valset size: 50

2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2:
==> STEP 1: BOOTSTRAP FEWSHOT EXAMPLES <==
2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2: These will be used as few-shot example candidates for our program and for creating instructions.

205 collapsed lines
2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2: Bootstrapping N=6 sets of demonstrations...


Optimizing with MIPROv2...
Bootstrapping set 1/6
Bootstrapping set 2/6
Bootstrapping set 3/6


    4%|▍         | 6/150 [00:00<00:02, 57.11it/s]


Bootstrapped 4 full traces after 6 examples for up to 1 rounds, amounting to 6 attempts.
Bootstrapping set 4/6


    3%|▎         | 4/150 [00:00<00:02, 69.03it/s]


Bootstrapped 3 full traces after 4 examples for up to 1 rounds, amounting to 4 attempts.
Bootstrapping set 5/6


    2%|▏         | 3/150 [00:00<00:02, 69.29it/s]


Bootstrapped 2 full traces after 3 examples for up to 1 rounds, amounting to 3 attempts.
Bootstrapping set 6/6


    4%|▍         | 6/150 [00:00<00:02, 71.73it/s]
2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2:
==> STEP 2: PROPOSE INSTRUCTION CANDIDATES <==
2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2: We will use the few-shot examples from the previous step, a generated dataset summary, a summary of the program code, and a randomly selected prompting tip to propose instructions.


Bootstrapped 4 full traces after 6 examples for up to 1 rounds, amounting to 6 attempts.


2025/09/28 17:55:16 INFO dspy.teleprompt.mipro_optimizer_v2:
Proposing N=3 instructions...

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: Proposed Instructions for Predictor 0:

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: 0: Count the number of occurrences of a specific letter in a given word.

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: 1: Given a word and a specific letter, analyze and explain how many times the letter appears in the word. Provide a step-by-step reasoning process showing the positions where the letter occurs and then indicate the total count of these occurrences.

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: 2: You are a linguistics researcher analyzing letter patterns in words. Given a specific word and a target letter, carefully reason through the positions of the letter within the word, explaining your thought process step by step. Then, accurately count how many times this letter appears in the word and provide both your reasoning and the total count. Be detailed in your explanation to ensure clarity and transparency.

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2:

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: ==> STEP 3: FINDING OPTIMAL PROMPT PARAMETERS <==
2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: We will evaluate the program over a series of trials with different combinations of instructions and few-shot examples to find the optimal combination using Bayesian Optimization.

2025/09/28 17:55:17 INFO dspy.teleprompt.mipro_optimizer_v2: == Trial 1 / 10 - Full Evaluation of Default Program ==


Average Metric: 45.00 / 50 (90.0%): 100%|██████████| 50/50 [00:07<00:00,  6.80it/s]

2025/09/28 17:55:24 INFO dspy.evaluate.evaluate: Average Metric: 45 / 50 (90.0%)
2025/09/28 17:55:24 INFO dspy.teleprompt.mipro_optimizer_v2: Default program score: 90.0

/Users/danielcorin/dev/dsp-letters/.venv/lib/python3.13/site-packages/optuna/_experimental.py:32: ExperimentalWarning: Argument ``multivariate`` is an experimental feature. The interface can change in the future.
    warnings.warn(
2025/09/28 17:55:24 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 2 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:03<00:00, 13.02it/s]

2025/09/28 17:55:28 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)





2025/09/28 17:55:28 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 1', 'Predictor 0: Few-Shot Set 4'].
2025/09/28 17:55:28 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0]
2025/09/28 17:55:28 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 90.0
2025/09/28 17:55:28 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:28 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 3 / 10 =====


Average Metric: 48.00 / 50 (96.0%): 100%|██████████| 50/50 [00:09<00:00,  5.50it/s]

2025/09/28 17:55:37 INFO dspy.evaluate.evaluate: Average Metric: 48 / 50 (96.0%)
2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: Best full score so far! Score: 96.0
2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 96.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 0'].
2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0]
2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:37 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 4 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:04<00:00, 11.10it/s]

2025/09/28 17:55:41 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)
2025/09/28 17:55:41 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 4'].
2025/09/28 17:55:41 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0]
2025/09/28 17:55:41 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:41 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:41 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 5 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:00<00:00, 4406.06it/s]

2025/09/28 17:55:42 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 1', 'Predictor 0: Few-Shot Set 4'].
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0]
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 6 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:00<00:00, 4720.02it/s]

2025/09/28 17:55:42 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 4'].
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0]
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:42 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 7 / 10 =====



Average Metric: 38.00 / 50 (76.0%): 100%|██████████| 50/50 [00:04<00:00, 10.89it/s]

2025/09/28 17:55:46 INFO dspy.evaluate.evaluate: Average Metric: 38 / 50 (76.0%)
2025/09/28 17:55:46 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 76.0 with parameters ['Predictor 0: Instruction 0', 'Predictor 0: Few-Shot Set 2'].
2025/09/28 17:55:46 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0, 76.0]
2025/09/28 17:55:46 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:46 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:46 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 8 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:04<00:00, 10.77it/s]

2025/09/28 17:55:51 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)
2025/09/28 17:55:51 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 1', 'Predictor 0: Few-Shot Set 5'].
2025/09/28 17:55:51 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0, 76.0, 74.0]
2025/09/28 17:55:51 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:51 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:51 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 9 / 10 =====



Average Metric: 31.00 / 50 (62.0%): 100%|██████████| 50/50 [00:04<00:00, 10.89it/s]

2025/09/28 17:55:56 INFO dspy.evaluate.evaluate: Average Metric: 31 / 50 (62.0%)
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 62.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 3'].
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0, 76.0, 74.0, 62.0]
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: ========================


2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 10 / 10 =====



Average Metric: 37.00 / 50 (74.0%): 100%|██████████| 50/50 [00:00<00:00, 3075.77it/s]

2025/09/28 17:55:56 INFO dspy.evaluate.evaluate: Average Metric: 37 / 50 (74.0%)
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 74.0 with parameters ['Predictor 0: Instruction 1', 'Predictor 0: Few-Shot Set 5'].
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0, 76.0, 74.0, 62.0, 74.0]
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: =========================


2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: ===== Trial 11 / 10 =====



Average Metric: 48.00 / 50 (96.0%): 100%|██████████| 50/50 [00:00<00:00, 4883.34it/s]

2025/09/28 17:55:56 INFO dspy.evaluate.evaluate: Average Metric: 48 / 50 (96.0%)
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Score: 96.0 with parameters ['Predictor 0: Instruction 2', 'Predictor 0: Few-Shot Set 0'].
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Scores so far: [90.0, 74.0, 96.0, 74.0, 74.0, 74.0, 76.0, 74.0, 62.0, 74.0, 96.0]
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Best score so far: 96.0
2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: =========================


2025/09/28 17:55:56 INFO dspy.teleprompt.mipro_optimizer_v2: Returning best identified program with score 96.0!





2025/09/28 17:55:56 INFO dspy.evaluate.evaluate: Average Metric: 48 / 50 (96.0%)


MIPROv2 optimized results: EvaluationResult(score=96.0, results=<list of 50 results>)

The results are just as good as our manual prompt improvements.

So what did the model do?

optimized_count_letter

predict = Predict(StringSignature(word, letter -> reasoning, count
    instructions='You are a linguistics researcher analyzing letter patterns in words. Given a specific word and a target letter, carefully reason through the positions of the letter within the word, explaining your thought process step by step. Then, accurately count how many times this letter appears in the word and provide both your reasoning and the total count. Be detailed in your explanation to ensure clarity and transparency.'
    word = Field(annotation=str required=True json_schema_extra={'desc': 'The word to search in', '__dspy_field_type': 'input', 'prefix': 'Word:'})
    letter = Field(annotation=str required=True json_schema_extra={'desc': 'The letter to count', '__dspy_field_type': 'input', 'prefix': 'Letter:'})
    reasoning = Field(annotation=str required=True json_schema_extra={'prefix': "Reasoning: Let's think step by step in order to", 'desc': '${reasoning}', '__dspy_field_type': 'output'})
    count = Field(annotation=int required=True json_schema_extra={'desc': 'The number of times the `letter` appears in the `word`', '__dspy_field_type': 'output', 'prefix': 'Count:'})
))

In this case, it looks like the optimizer added specific instructions that seems to result in better performance of the task by the model

You are a linguistics researcher analyzing letter patterns in words. Given a specific word and a target letter, carefully reason through the positions of the letter within the word, explaining your thought process step by step. Then, accurately count how many times this letter appears in the word and provide both your reasoning and the total count. Be detailed in your explanation to ensure clarity and transparency.

Let’s see how this prompt translates into the type of reasoning the model does

with dspy.settings.context(lm=gpt_4_1_nano_lm):
    result = optimized_count_letter(word='basketball', letter='b')

result

Prediction(
    reasoning='The word is "basketball," which contains the following letters: b, a, s, k, e, t, b, a, l, l. I need to find all instances of the letter "b" within this word. The first letter is "b," which matches the target letter. The second "b" appears as the seventh letter in the word. No other "b" characters are present. Therefore, the total count of "b" in "basketball" is 2.',
    count=2
)

Not quite the same as the approach we instructed in count_letter_strategic but it seems to have achieved the same result without our explicit instructions.

The model, helped by DSPy, figured this out iteratively, in a self-improving loop.

I’m going to wrap up here but there’s more to explore with DSPy.

Several other optimizers
Local models via the LiteLLM integration

I’ll aim to do this in another post soon.

Thanks for reading via RSS!

Running a Mile with LLMs

Wed, 10 Sep 2025 17:07:06 GMT

I was not planning on writing about this but after reading Sascha’s post, I decided it could be interesting because his demonstration on what it takes to build knowledge happened to relate to prompting a language model for a fitness routine:

This drives us to one of the most important conclusions of the entire field of note-taking, knowledge work, critical thinking and alike: You, not AI, not your PKM or whatever need to build the knowledge because only then it is in your brain and you can go the next step.

Let me try to give you a demonstration of the issue:

I asked ChatGPT to design an optimal weekly training schedule for health span and fitness.

and then he proceeds to point out how unless you know some things, you have no way to evaluate whether the model’s plan is any good, which I agree with.

I went into the following experiment knowing there was a decent chance the language model would give me bad advice. I was mostly interested to see what would happen.

Training for a mile

I used a few different language models to design a training schedule to run a single mile, as fast as I could do it.

I’m a relatively seasoned athlete, so this was nothing like a couch to 1 mile training plan. However, I’ve never intentionally trained for a short distance running race. I’ve only ever run as cross-training for other sports.

So I created a Project with Claude Opus and started to see what a training plan might look like.

Here’s an example of asking an LLM to design a training schedule for running a mile. I’ve summarized the assistant responses, but ensured the training session suggestions were true to the original recommendations. The actual model responses were even longer as Claude had a lot to say about these plans.

2025-08-18

User

Hey I want to train to run a mile in under 6 minutes. Come up with a training plan for me.

Assistant

Phase 1: Base Building (6-8 weeks)

Build tolerance for short rest intervals safely.

Main Sessions (2x/week):

Start: 4-5 x 400m with 90s rest at sustainable pace
Expect 92-96 seconds initially
Weekly progress: Add 1 rep OR reduce rest by 5s OR improve pace by 2s
Only change ONE variable per week

Easy Days (2x/week):

20-30 minute conversational-pace jogs
Builds aerobic base and aids recovery

Phase 2: Speed Development (4 weeks)

6-8 x 400m at 86-90s with 60-75s rest
Begin only when consistently hitting 90s with 90s rest

Key Guidelines:

Maintain current cycling/weights
Take 1-2 complete rest days weekly
Start conservatively and progress slowly
Keep easy days truly easy
Prioritize consistency over intensity

Success Target: Complete 8 x 400m at 88-90s with 75s rest consistently before attempting mile time trial.

So I gave the main sessions a try and they went ok! I actually felt like I was ready for a bit more at my fitness level, so time to do some prompt engineering.

Recalibrating based on my fitness level

2025-08-18

User

… stuff about me, recent workout performance, cycling wattage, etc. …

Help me come up with a training plan over the next 2.5-3 months to run a 6 minute mile.

Assistant

Your cycling fitness shows you have the cardiovascular base needed. A 6-minute mile requires holding 10 mph for 6 minutes - we need to build running-specific speed and neuromuscular power over 12 weeks.

Phase 1: Base Building (Weeks 1-4)

Weekly Structure:

Monday: Easy run (30-40 min)
Tuesday: 6-8 x 400m at 85-90 seconds (90 sec rest)
Wednesday: Cycling/weights
Thursday: 20 min tempo at 7:00-7:30/mile pace
Friday: Rest or easy cycling
Saturday: Long run (45-60 min easy)
Sunday: Weights or rest

Phase 2: Speed Development (Weeks 5-8)

Weekly Structure:

Monday: Recovery run (25-30 min)
Tuesday: Progress to 8 x 400m at 80-85 seconds, then 4-5 x 800m at 2:50-3:00
Wednesday: Cycling
Thursday: 3 x 8 min at threshold pace (6:45-7:00/mile)
Friday: Rest
Saturday: Progression run
Sunday: Explosive weight training

Phase 3: Race Prep (Weeks 9-12)

Weekly Structure:

Monday: Easy recovery (20-30 min)
Tuesday: Race-pace work - progress to 3 x 1200m at 4:30 (5:00/mile pace), then 2 x 1 mile at 6:15-6:20
Wednesday: Easy cycling/rest
Thursday: Short tempo or easy run
Friday: Rest
Saturday: Time trials (taper in week 12)
Sunday: Easy run or rest

Key Points

Pacing: Target even 90-second splits per 400m lap
Testing: Mile time trial every 3-4 weeks (expect 10-15 second monthly improvements)
Recovery: Maintain cycling, add explosive strength work 1-2x weekly
Taper: Reduce volume 40-50% in final week while maintaining intensity

With consistent training, your cycling fitness gives you the foundation to achieve a 6-minute mile within 3 months.

Looking back at this plan with the benefit of hindsight, I realized I didn’t even bother looking at phase three of the plan, which looks very ambitious to me, even in retrospect. To not spoil the journey, I’ll get back to this later.

I let the model know about my recent workout performance to see if it had anything to say about it.

2025-08-18

User

Last week: 90 90r 90 90r 89 90r 88 90r 91

Today: 75 90r 88 90r 89 90r 88 90r 88

Assistant

You’re ahead of schedule! Your consistent 88-89 second 400m reps show excellent improvement, but that 75-second opener was too fast for this training phase. While impressive, going 15 seconds faster than race pace can compromise your workout and doesn’t build the specific endurance needed for a 6-minute mile.

Next Week’s Workout:

6 x 400m at 85-87 seconds
Start conservative (87s) for reps 1-2, then drop to 85-86s if feeling good
90-second rest between reps

2-3 Week Progression:

Week 3-4: 7-8 x 400m at 83-85 seconds, add weekly 800m time trial
Week 5-6: 5 x 800m at 2:50-2:55 (key workout for bridging to 6-minute mile)

Bottom Line: You likely could run 6:15-6:20 right now with proper pacing. That 75-second speed shows you have the raw ability - focus now is on controlled, sustainable pace. Every rep at 85-90 seconds builds the exact endurance adaptations you need. Save the blazing fast reps for later phases with full recovery.

The model felt pretty good about my performance. I was having some pacing problems and it called that out as well. It even threw out a guess on what my mile time would be if I made an attempt at my current level of fitness.

I was seeing improvements and feeling better in the workouts. It felt like I was on some kind of right track, so I kept at it.

2025-08-26

User

Here was week 2 from today:

82 90r 90 90r 85 90r 88 90r 84 90r 86

Assistant

Performance Update: Crushed Week 2 with a 3-second improvement! Week 1 averaged ~89 seconds, Week 2 averaged ~86 seconds. You’re showing Week 4-5 fitness ahead of schedule thanks to exceptional aerobic adaptation.

Next Week Training Options:

Option A (Recommended): 7-8 x 400m @ 83-87 seconds with 75-second rest Option B: 4-5 x 600m @ 2:10-2:15 with 90-second rest

Accelerated Timeline:

Week 4: 8 x 400m @ 82-85s with 60-second rest + 1200m time trial
Week 5-6: 5-6 x 800m @ 2:45-2:50
First Mile Time Trial: Week 4 (predicted 6:15-6:25)

Current Mile Fitness Estimates:

Current: 6:20-6:30
Week 4: 6:10-6:15
Week 8: 6:00-6:05
Week 12: Sub-6:00 achievable

This Week’s Focus:

Thursday tempo: 6:45-7:00 pace
Saturday long run: 7:30-8:00 pace (easy)
Add 4 x 100m strides after Monday’s easy run

Watch for: Achilles/calf tightness and shin splints as speed increases.

The 6-minute mile goal is looking very achievable—possibly by Week 8-9 rather than Week 12!

This threw me off. Based on where I was at, these goals felt very ambitious and I wasn’t feeling good about it. I assumed the model was painting a rosier picture than reality based on past experience of models making things up and generally being wrong and unreliable about all sorts of things.

I had doubts

2025-08-26

User

I need you to be realistic

Assistant

Reality Check

The Gap: While your 82-86 second 400m repeats with 90 seconds rest are solid, there’s a significant difference between interval work with recovery and holding 90-second laps for 4 consecutive laps without rest.

Current Fitness Assessment:

You can hit 82-86 seconds WITH full recovery
6-minute mile requires 90-second laps WITH NO recovery
Realistic current mile time: 6:45-7:00

Realistic 12-Week Timeline:

Week 4: 6:35-6:45
Week 8: Breaking 6:30 would be good progress
Week 12: 6:10-6:20 is a realistic target

Sub-6:00 Reality Check: This is a stretch goal requiring exceptional lactate buffering. Most runners need to comfortably run 75-80 second 400s to break 6:00.

Today’s Session Assessment:

Your 86-second average with 90 seconds rest shows you’re exactly where you should be for Week 2, but the brutal math reveals the challenge: you’re running faster than goal pace WITH recovery, while the goal requires slightly slower pace WITH zero recovery.

Honest Prediction: Based on current performance, you’re tracking toward 6:25-6:35 by Week 12.

Hmm ok. So I showed some doubt and the model followed my lead and pulled back where it estimated I would be. By design, I basically had no idea about where I should be, the type of progress I could expect, or whether the initial or secondary predictions were reasonable. The points about rest between intervals made sense considering I didn’t have enough experience or knowledge to push back against them.

It was true that I would need to run 4 laps with no rest. And that sounded harder than most of the things I had been doing.

So I continued the plan with more conservative expectations about how performance might improve, still following the second plan the model gave me after I told it my fitness level.

My first (and only) tempo run

2025-08-27

User

Tempo Target 4800m @ 7 min/1600m pace

1:45.68 1:47.78 1:46.58 1:46.66 1:45.47 1:46.62 1:47.38 1:47.37 1:47.79 1:48.78 1:49.84 1:39.24

21:19.25

Assistant

Current Performance Analysis:

Recent tempo run: 1:47 per 400m (7:09/mile pace) with excellent consistency
Goal mile pace needed: 1:30 per 400m (6:00/mile)
Gap to bridge: 17 seconds per lap

The Reality:

Your threshold pace is currently 7:00-7:10/mile. To run a 6:00 mile, you need to run significantly faster than this threshold pace, but only sustain it for 6 minutes instead of 21+ minutes. This 70-second per mile gap is substantial but achievable with 9 weeks of focused training.

Recommended Weekly Structure:

Friday: Easy 25-minute run + 8 x 30 seconds at 6:00 pace (90 sec easy jog recovery)

Saturday: 50 minutes truly easy (7:45-8:15 pace) for aerobic base

Tuesday: 8 x 400m with 75 seconds rest, targeting 87-90 seconds per 400m

Projected Timeline:

Current mile fitness: ~6:35-6:40
Week 6 projection: ~6:15-6:20
Week 12 target: Sub-6:00 (requires perfect consistency)

Key Focus: The 400m/800m interval work is critical - it trains the faster energy systems above threshold that you’ll need for the mile. Your finishing kick shows you have speed reserve; now we need to bring your threshold closer to race pace through consistent tempo work.

Claude wasn’t so enthusiastic about the tempo performance with regard to my chances for a 6-minute mile. The mile time projections at the different week marks were varying all over the place.

Because the original training plan was further up in the chat (I had this conversation in one long chat), it was a pain to scroll back to the original plan and the model kept suggesting new plans anyway, with different rest durations that were kind of close to the original plan. I decided to just ask the model to tell me what the workout was for today and go with that.

2025-09-03

User

What is today’s workout?

Assistant

Today’s Workout Summary Main Set: 8 x 400m with 60-second rest Pacing Strategy:

Reps 1-2: 87-88 seconds (conservative start)

Reps 3-6: 85-88 seconds (find rhythm)

Reps 7-8: Hold on, accept what comes

Expect:

Slower than last week (88-92 is fine) Incomplete recovery between reps Heavy breathing throughout

Warmup: 10-15 min easy + 4 x 100m strides Cooldown: 10 min easy jog

User

8x400m 60r

85 60r 89 60r 87 60r 88 60r 89 60r 90 60r 91 60r 90

Assistant

Breakthrough Performance Achieved

You crushed this week’s workout, running 400m repeats at an 88.6-second average with only 60-second rest - just 2.5 seconds slower than last week despite 33% less recovery time. Most athletes lose 5-7 seconds with reduced rest, so this shows exceptional cardio adaptation.

Current Mile Projection

Current fitness level: 6:25-6:30 mile (ahead of schedule)
Week 8 projection: Sub-6:20 very likely
Week 12 goal: Sub-6:00 increasingly realistic

Key Insights

Your ability to hold 88-91 seconds with incomplete recovery proves you can sustain goal pace (90 seconds) when pre-fatigued - exactly what mile racing demands. The minimal fade between first and last rep shows strong lactate tolerance.

Next Week’s Workout (Week 4)

Distance: 8 x 400m
Rest: Reduce to 45-50 seconds
Target pace: 90-93 seconds
Purpose: Complete cardio adaptation phase

You’re ahead of schedule and starting to look like the athlete you remember being. This workout was a statement of your progress toward that sub-6:00 mile goal.

This was a confidence building session for me. I had expected the reduction of rest from 90 to 60 seconds to hit hard and negatively affect my performance. The fact that I was able to continue executing the “plan” despite fatigue felt like a big win.

It seemed like I had exceeded Claude’s expectations for me, at least, after I had given it a reason to doubt its initial predictions. Either way, it was excited.

Again, I didn’t really consider the implications of the feedback I was giving the model and I was somewhat wary of whether its plans were actually reasonable. I definitely wasn’t putting blind trust in the plan and was pushing back, but I was also not very informed, so it’s not as though I was coming from a position of expertise.

An unexpected turn

I realized something else on this day as well.

The track on the East River was closing on 2025-09-08 for construction work until the end of 2026.

RIP. It was a sick track.

My plan had not accounted for this. But I really wanted to give the mile my best shot.

I decided to see what GPT-5 thought I could do to salvage the situation. I was going to need all the help I could get to figure out the next 4 days.

2025-09-04

User

I need to race a mile in 4 days. I am targeting 6 minutes. Here are my most recent workouts.

2025-08-13

5x400m 90r 90 90r 90 90r 89 90r 88 90r 91

2025-08-18

5x400m 90r 75 90r 88 90r 89 90r 88 90r 88

2025-08-26

6x400m 90r 82 90r 90 90r 85 90r 88 90r 84 90r 86

2025-08-27

Tempo Target 4800m @ 7 min/1600m pace

1:45.68 1:47.78 1:46.58 1:46.66 1:45.47 1:46.62 1:47.38 1:47.37 1:47.79 1:48.78 1:49.84 1:39.24

21:19.25

2025-09-03

8x400m 60r 85 60r 89 60r 87 60r 88 60r 89 60r 90 60r 91 60r 90

Give me a plan

Assistant

Here’s your final prep for Sunday’s sub-6:00 mile attempt:

Thu 9/4

20-30 min easy run
Drills + 4-6 relaxed strides (60-100m)

Fri 9/5 — Readiness Check: 2 x 800 @ 2:58-3:00, 2:00 rest

Warm-up: 10-15 min easy, drills, 4-6 strides; optional 1 x 200 at ~45 three minutes before
Target even splits: 200 in 45, 400 in 90, 600 in 2:15, finish 2:58-3:00
Cool-down: 10 min easy
Results guide:
- Green (≤3:00 both): proceed as planned
- Yellow (3:01-3:03): open slightly conservative
- Red (≥3:04 or fade): consider delaying race or adjusting to 6:05-6:10 goal

Sat 9/6

15-20 min very easy + 4 strides
Light mobility, stay off feet otherwise

Sun 9/7 — Race Day

Warm-up (40-50 min pre-race):

10-15 min easy jog, drills, 4-6 strides
Optional: 1 x 200 at ~45 three minutes before start

Pacing Strategy:

Mile: 45/90/3:00/4:30/finish hard
1600m: 90-89-89-88 (sub-5:58)

Key Rules:

Don’t go faster than 45 for first 200
If lap 1 = 92: go 89-89-87
If lap 1 = 88: settle to 90-90, kick last 200

Race Logistics:

Wear your usual fast shoes
Eat carb-focused meal 2-3h before
Hydrate with 300-500ml water + pinch of salt in final 2-3h

Send me your Friday 2 x 800 times and I’ll confirm your exact pacing strategy. You’re ready—nail the first lap and everything else follows.

User

2x800m 120r 3:04 120r 3:02

Assistant

Quick mile plan (yellow readiness): open a touch conservative, then squeeze late.

Race pacing

Default (1600 m): 91 - 90 - 90 - 88 = 5:59
If you open slow (92): 92 - 90 - 89 - 88
If you open fast (≤89): settle to 90-91 on lap 2, 90 on lap 3, kick last 300-200
Full mile (1609 m): hit 1600 in 5:57-5:58; the extra 9 m is your kick

Checkpoints and in-race adjustments

First 200 ≥46; relax, tall posture, light quick cadence
400 ~91 feeling controlled
800 in 3:01-3:03; commit to 89-88 over last two laps
Lap 3 cue: hold cadence, “hips tall, push the ground”
Last 500-300: start squeezing; last 200 go arms-first and commit

Warm-up (start 40-50 min before)

10-15 min easy jog
Drills: leg swings, skips, high knees, butt kicks
4-6 strides building to mile effort
Optional: 1 x 200 at ~45-46 about 3 min before start

Day-before (easy)

15-20 min very easy + 4 relaxed strides; light mobility; otherwise off your feet

Logistics

Wear the shoes you’ve been doing fast work in; lace securely
Carb-centric meal 2-3 h pre-race; optional 15-30 g carb 15-30 min before
Caffeine only if you’re used to it
If windy, tuck on straights, push tailwind sections

Bottom line: Open under control (91), settle, then squeeze. If you’re ~3:02 at 800 feeling smooth, 5:59-6:01 is there with a committed last 600-400.

I popped my new plans and tune up 2x800m results over to Claude and its response left me confused.

2025-09-05

User

2x800m 120r 3:04 120r 3:02

Assistant

Your Recent 800m Performance: 3:04, 3:02 (91.5 seconds per 400m) - excellent pacing with a negative split and strong consistency.

Key Insight: You held near-goal pace for 800m, which shows you can maintain the right speed for half your target distance. Your 800m pace (3:00-3:03) is right on target for a 6:00 mile.

Current Mile Fitness Estimate: 6:20-6:25

Next Steps for 800m Training:

Next time: 3 x 800m at 3:00-3:05 (2 min rest)
Week after: 4 x 800m at 3:00-3:05 (90s rest)

The Challenge: To run a 6:00 mile, you need to hold these 800s back-to-back without rest. Currently you’re completing 800s with 2:00 rest - the goal is eliminating that recovery time while maintaining pace.

These 800m workouts are arguably more important than 400m repeats for your mile goal, as they build the speed endurance needed for your target time.

Since I had had the conversation about the shortened timeline outside this thread, Claude didn’t know what I was talking about. I had two days left and I was going to race on 2025-09-07.

Claude seemed to think I had 6:20-6:25 pace potential given the 2x800m workout. GPT-5 thought I could be within a few seconds of 6:00.

I wasn’t sure what to think. I definitely wished I had more time and had run more 600m and 800m workouts. 800m felt very different from 400m the way I was trying to run them.

I had gone from 90s rest between 400s to 60s to 0s for the 2x800m. That workout was tough, though I think my pacing could have been better.

So I felt sort of in the dark at this point.

Race day

I woke up Sunday morning on 2025-09-07 and it was raining. I went through a pretty standard race prep routine I have. I eat a banana and make sure I am awake several hours before the race.

After sitting around for about two hours, the rain actually let up, so I went down to the track to do my warmup, planning to just race in the rain if I had to.

I had finished my dynamic warmup and was starting to do strides when I saw this.

Well not actually this. What I saw was water midway through the fifth lane of the track. I took this picture after everything was done and 20-30 minutes more draining. If it had only been this bad, I probably wouldn’t have changed much, but at the time, it was so flooded I had to adjust.

I decided to run the race in lane 6. I did some quick research on my phone, and found I needed to do (1600 / 438.33) 3.65 laps.

I sized up the final corner before the last 100m, took the midpoint of the turn (0.125+ 0.50 = 0.625) and added a little bit to that. That would be my finish point on the fourth lap.

It was hacky but I was going to try and make it work.

I went over to my normal starting spot, went out to lane 6, and off I went.

There’s a reason you don’t want to be in lane 6

Running in lane 6 was weird. The turns are longer and I couldn’t pace myself at all since I didn’t know the target 200m or 400m times with the adjusted number of laps.

It was not an optimal setup. Perfect weather though, minus the wet.

I ran the race a bit conservatively through the first half and had a decent amount left in the tank to push the last 1.65 laps.

I clocked the finish at 6:11. I was a bit surprised when I saw it.

I was sure I’d started too conservatively to put up 6:00, but 6:11 felt like a pretty good result.

I did take a look at the stopping point and decided it was probably a touch short of 1600m. So I’m going to call it a 6:15.

Reflecting

Looking back on this process, one thing that surprised me was something Claude said on 2025-08-26

2025-08-26

Assistant

Current Mile Fitness Estimates:

Current: 6:20-6:30
Week 4: 6:10-6:15
Week 8: 6:00-6:05
Week 12: Sub-6:00 achievable

I didn’t know it at the time, but roughly two weeks later I would attempt the mile and finish with a time right in the range Claude predicted. This was before Claude recalibrated its prediction based on my prompt to “be realistic” that same day. That said, Claude made plenty of other predictions in the neighborhood of my goal, so it isn’t so surprising it got one of them right.

Takeaways

It seemed like Claude might have had a better sense of my capabilities than I did. However, when I showed uncertainty, it went with me. Maybe it should have pushed back.

I’m not sure how I would prompt for that, avoiding leading it to adjust its predictions to be less ambitious, if its pattern matching suggested its plan could actually be possible for me to execute.

Maybe it was all a coincidence.

I think there’s a chance I might have performed better on a nicer day. But it’s equally possible I might have started too aggressively and then faded too much.

Unfortunately, I won’t have the chance to find out but I’m glad I got to give this a shot before the track closed. Maybe it’s for the best. I’ll need to do some proper research to learn if Claude was proposing reasonable plans for me.

Thanks for reading via RSS!

Claude Code Custom Slash Commands Hierarchy

Tue, 02 Sep 2025 18:49:42 GMT

Claude Code supports custom slash commands.

Custom slash commands allow you to define frequently-used prompts as Markdown files that Claude Code can execute. Commands are organized by scope (project-specific or personal) and support namespacing through directory structures.

This feature reminds me of my experiment using Goose as a task runner, where writing a markdown spec in a .goosehints file is sufficient to get the agent to execute a task.

Claude Code provides native support for these “slash commands” as well as support for arguments. If you find yourself prompting in a similar way often, a custom slash command may be a good fit to save you some typing.

While there are lots of interesting ways to use this feature, today I learned about how they apply hierarchically between the home folder and within your project and its subfolders.

Let’s consider the following custom slash command

Say "hello from home dir"

If we fire up claude in an empty folder, we see the following when we prompt /hello

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello     Say "hello from home dir" (user)

Now, let’s add a custom slash command in the project folder.

Say "hello from project dir"

claude

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello                Say "hello from project dir" (project)

The slash command defined in the home folder is overridden by the one we defined in the project folder.

Now, let’s add a custom slash command in a subfolder of the project.

Say "hello from subfolder"

claude

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello               Say "hello from project dir" (project)
  /subfolder:hello     Say "hello from subfolder" (project)

We see both slash commands, with the one we just added showing up under the namespace.

Say we also want the commands from our home directory and want to avoid naming collisions with the project commands. We can do that by symlinking the home commands into the a subdirectory in the project.

mkdir -p .claude/commands/home && ln -sf ~/.claude/commands/* .claude/commands/home/

claude

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello               Say "hello from project dir" (project)
  /home:hello          Say "hello from home dir" (project)
  /subfolder:hello     Say "hello from subfolder" (project)

The /hello command from the home directory now shows up under the home namespace as a project command. However, since it is a symlink, any changes to the ~/.claude/commands/hello.md file will be reflected in the project commands the next time we start claude.

That is a reasonable setup but what if you work in a monorepo and want to be able to use ./.claude/commands from the root of the project in subfolders of the project?

Given our setup above, for starters, let’s run claude in the subrepo folder.

cd subrepo
claude

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello     Say "hi from home dir" (user)

As expected, only the /hello command defined in ~/.claude/commands/hello.md is available. To get the same commands to be available in the subfolder as in the root of the project, we can symlink the ./.claude/commands folder into the subfolder (./subrepo/.claude/commands).

cd subrepo  # if you haven't already
mkdir -p .claude/commands && ln -sf "$(realpath ../.claude)" .claude

claude

╭───────────────────────────────────────────────────────────────────────────────────╮
│ > /hello                                                                          │
╰───────────────────────────────────────────────────────────────────────────────────╯
  /hello               Say "hello from project dir" (project)
  /subfolder:hello     Say "hello from subfolder" (project)
  /home:hello          Say "hello from home dir" (project)

If we inspect the files, we can see the symlink in place as we expect.

dev/cc-commands/subrepo
❯ ls -l -R -A
lrwxr-xr-x@ - danielcorin  2 Sep 19:55  .claude -> /Users/danielcorin/dev/cc-commands/.claude

With this approach, any changes to the source folder are also reflected in the symlinked folder, which keeps everything in sync.

This seems to be the current state of the world with Claude Code, but it would be nice if there were better monorepo support, similar to how the tool supports looking for CLAUDE.md files.

With symlinks, there are a few different approaches that can smooth over the rough edges while Anthropic decides how or if they want to support cascading commands.

Thanks for reading via RSS!

Running `uv` with Environment Variables

Sun, 03 Aug 2025 15:30:53 GMT

The .env* file and the python-dotenv package are popular conventions in Python for managing environment variables. I often forget or pause when trying to remember exactly which package I need to install (python-dotenv) and how to do the import

from python_dotenv import load_dotenv

load_dotenv()

I heard someone recently lament that the package wasn’t pip install-able as pip install dotenv but fortunately, uv as a great and memorable solution to this problem that doesn’t require any changes to your code.

How?

You can use the --env-file flag to specify a .env file to load when running uv run.

import os

print(os.getenv("HELLO"))

HELLO="world"

Starting by running without the --env-file flag, we see the environment variable is not loaded.

uv run python script.py
#=> None

Run with the --env-file flag:

uv run --env-file .env python script.py
#=> "world"

The uv tool also supports the UV_ENV_FILE environment variable for this purpose.

Better with `direnv`

Coupled with direnv, you can run all your uv commands as normal and set the environment variable such that your secrets are included.

export UV_ENV_FILE=".env"

direnv allow
uv run python script.py
#=> "world"

Or you can let just let direnv load the .env file for you.

dotenv

direnv allow
uv run python script.py
#=> "world"

Given all the options for loading environment variables, in the tools I am already using, there isn’t usually a good reason for me to install python-dotenv anymore.

Thanks for reading via RSS!

Datadog and the OpenAI API Spec

Tue, 29 Jul 2025 18:19:19 GMT

Why I like the Chat Completions API

I write often about the OpenAI Chat Completions API, which has become a sort of standard for the time being for calling LLM APIs. This API is useful because it allows you to easily test different models and providers for inference with text and image inputs.

I like using this API as a starting point because depending on your use case, you’re optimizing different things.

These include:

maximum performance or correctness against some labels
latency
cost
time to first token

Eventually, you may settle on a particular model and optimize for that, but when you’re getting started, you might not even know what you’re optimizing for or the tradeoffs you’ll need to make given the performance of the models available to you.

Using multiple providers with the OpenAI client

Recently, I set up a Python app that used the OpenAI Python client. Since many model providers provide OpenAI chat completion APIs in addition to their own, I was experimenting with running inference with other providers just by changing the base_url and api_key in the client.

Google has documentation on exactly how you can do this with Gemini.

Several providers do a good job of implementing the spec in a way that is compatible with OpenAI’s client. When it works, you don’t notice or need to think about it. This solid foundation is excellent as you’re figuring out what you’re optimizing for as you build your LLM use case.

However, when implemented poorly, it’s a very bad time.

The failing call

I was calling another provider that claimed to expose an OpenAI-compatible chat completion API. I won’t name names, but the error I got was

{
  "error_code": "BAD_REQUEST",
  "message": "{\"external_model_provider\":\"amazon-bedrock\",\"external_model_error\":{\"message\":\"stream_options: Extra inputs are not permitted\"}}"
}

or something like that.

This wasn’t a call to Bedrock by the way, which doesn’t provide native support for an OpenAI-compatible API.^[1]

It was a call to a proxy layer in between. One that claimed to be OpenAI-compatible.

The error “stream_options: Extra inputs are not permitted” seems pretty straightforward. The problem was, I wasn’t passing it in my code.

import os
from openai import OpenAI


def main():
    client = OpenAI(
        api_key=os.getenv("API_KEY"),
        base_url=os.getenv("BASE_URL"),
    )

    stream = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Tell me a short joke about programming."},
        ],
        stream=True,
    )

    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            print(chunk.choices[0].delta.content, end="", flush=True)
    print()


if __name__ == "__main__":
    main()

What was going on?

I spent a while digging through layers of my code to see how I was inadvertently passing stream_options in my client call. I couldn’t find anything.

I wish I had thought to check the raw HTTP request the client was making sooner.

I did that by running httplog and changing the base_url to http://localhost:8080/v1. Triggering the code path in my app outputted logs like this:

13:51:30.544:
Method: POST  Path: /v1/chat/completions  Host: localhost:8080  Proto: HTTP/1.1
Headers:
  Accept: application/json
  Accept-Encoding: gzip, deflate
  Authorization: Bearer test
  Connection: keep-alive
  Content-Length: 217
  Content-Type: application/json
  Traceparent: 00-68890a2200000000059e62812c877cbe-1d59ad448017a984-01
  Tracestate: dd=p:1d59ad448017a984;s:1;t.dm:-0;t.tid:68890a2200000000
  User-Agent: OpenAI/Python 1.97.1
  X-Datadog-Parent-Id: 2114912009745574276
  X-Datadog-Sampling-Priority: 1
  X-Datadog-Tags: _dd.p.dm=-0,_dd.p.tid=68890a2200000000
  X-Datadog-Trace-Id: 404869323447303358
  X-Stainless-Arch: arm64
  X-Stainless-Async: false
  X-Stainless-Lang: python
  X-Stainless-Os: MacOS
  X-Stainless-Package-Version: 1.97.1
  X-Stainless-Read-Timeout: 600
  X-Stainless-Retry-Count: 0
  X-Stainless-Runtime: CPython
  X-Stainless-Runtime-Version: 3.11.11
Body:
  {"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Tell me a short joke about programming."}],"model":"gpt-4o-mini","stream":true,"stream_options":{"include_usage":true}}

So there it was. The code was clearly somehow setting stream_options in the request, even though I couldn’t figure out how.

I created a completely independent reproduction of the same client call outside my codebase and somehow it worked with identical code as shown above. The logs looked like this:

13:55:47.315:
Method: POST  Path: /v1/chat/completions  Host: localhost:8080  Proto: HTTP/1.1
Headers:
  Accept: application/json
  Accept-Encoding: gzip, deflate
  Authorization: Bearer test
  Connection: keep-alive
  Content-Length: 177
  Content-Type: application/json
  User-Agent: OpenAI/Python 1.97.1
  X-Stainless-Arch: arm64
  X-Stainless-Async: false
  X-Stainless-Lang: python
  X-Stainless-Os: MacOS
  X-Stainless-Package-Version: 1.97.1
  X-Stainless-Read-Timeout: 600
  X-Stainless-Retry-Count: 0
  X-Stainless-Runtime: CPython
  X-Stainless-Runtime-Version: 3.11.11
Body:
  {"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Tell me a short joke about programming."}],"model":"gpt-4o-mini","stream":true}

The difference in these logs was the first useful clue I had come across.

1c1
< 13:55:47.315:
---
> 13:51:30.544:
8c8
<   Content-Length: 177
---
>   Content-Length: 217
9a10,11
>   Traceparent: 00-68890a2200000000059e62812c877cbe-1d59ad448017a984-01
>   Tracestate: dd=p:1d59ad448017a984;s:1;t.dm:-0;t.tid:68890a2200000000
10a13,16
>   X-Datadog-Parent-Id: 2114912009745574276
>   X-Datadog-Sampling-Priority: 1
>   X-Datadog-Tags: _dd.p.dm=-0,_dd.p.tid=68890a2200000000
>   X-Datadog-Trace-Id: 404869323447303358
21c27
<   {"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Tell me a short joke about programming."}],"model":"gpt-4o-mini","stream":true}
---
>   {"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Tell me a short joke about programming."}],"model":"gpt-4o-mini","stream":true,"stream_options":{"include_usage":true}}

It seemed that Datadog was somehow adding stream_options to the request even though I wasn’t setting it in my code.

Datadog’s magic tracing

To add Datadog tracing to a Python app, you install ddtrace and run your app with ddtrace-run. It pretty much just works.

However, with my faulty OpenAI-compatible API, the magic that Datadog uses to slightly modify requests to enable their traceability was breaking my code.

I eventually traced the behavior to the dd-trace-py library, which sets stream_options["include_usage"] = True. Given how this is implemented and the problem I was running into, setting stream_options["include_usage"] = False in my code led to the same problem since the underlying model still did not support it.

This behavior was introduced in v2.20.0 for dd-trace-py with the release notes:

LLM Observability

openai: Introduces automatic extraction of token usage from streamed chat completions. Unless stream_options: {"include_usage": False} is explicitly set on your streamed chat completion request, the OpenAI integration will add stream_options: {"include_usage": True} to your request and automatically extract the token usage chunk from the streamed response.

It’s a relatively innocuous change that allows Datadog to better trace LLM calls with a client that is documented to support it, but not if your OpenAI API isn’t working properly.

The fix

In this case, I still needed to call the problematic provider, so I used httpx to build a custom SSE client to stream the response back to the caller of my app, smoothing over the differences in what the problematic API supported. This approach allowed me to keep Datadog as well without needing to make more invasive changes to my code.

While it’s tough to blame Datadog for all this trouble, I was a little salty that I was burned by this magical request modification. But it’s not their fault.

If you implement an OpenAI-compatible API, please at least test it with the OpenAI client, otherwise you’re not going to get the adoption or benefits from users using existing tools to call you. And you’re also going to burn users who are relying on some amount of predictability while using systems (LLMs) that are already quite unpredictable.

Unless you want to deploy a Lambda function to proxy the requests ↩

Thanks for reading via RSS!

Migrating To Astro

Fri, 25 Jul 2025 20:07:03 GMT

I’ve run Thought Eddies as a more dynamic, experimental blog since the beginning of this year. I’d heard about Astro and wanted to put a more personal touch on my web presence.

I wrote an article here or there and then one day, after I’d run into a bit of a block working on Tomo, I had the itch to build Thought Eddies out.

I have several post types I write frequently, so I needed to build sections for those. Then I added support for a bunch of the shortcodes I’d built over the years in Hugo as well as URL aliases. I try to keep all my old links alive and redirecting properly.

Combined with the strong ecosystem of Astro and LLMs, I was able to build and migrate nearly all the content within about a week.

One of the finishing touches was a search that I practically one-shotted with an agent, that was exactly what I wanted.

I wanted to build something like the search page on my previous site, which loaded all the site content client-side and did a client-side search as well. It wasn’t efficient but it worked and didn’t require a backend. I never had any problems with it and my site was building fast and loading fine.

So I added this search modal, triggered by ⌘+k on any page, I finished migrating over my content, and I started building and deploying the site. And wow was it slow. Almost three minutes on my local and almost 15 on CI.

What?

I started searching around and reading the Astro Discord, looking for reports of slow builds. My site isn’t big. There aren’t many images.

So I ran the build a bunch and started inspecting what was slow. One major problem were the number of Open Graph images I was generating.

I had a custom one per page.^[1]

That was not a great idea. Each took a few 100ms to build. I tried switching to a different library, started taking more than one second each.

Ok, no problem. One Open Graph image per site section will be fine.

The build was still taking more than two minutes.

At several points over the course of a few days, I opened devtools and viewed the site’s source. It didn’t fully load. Was Waterfox buggy? I didn’t really know.

I almost noticed the site was taking ~500ms to load. Just slow enough to be annoying.

Hugo (the static site generator I used to build my previous site) is quite fast and there is much discussion about how the Javascript web (and Astro by proxy) can be slow and bloated, but was it really this bad? I had never run into performance like this before. I’d built an Electron app of far greater complexity and even that tooling seems nicer than was I was running into here.

Eventually, I realized I needed to actually see the HTML source for myself. These were static HTML pages after all. To build the site, a webserver starts and a process requests the content for each page, then persists it as a static HTML file to be served in production.

What was taking so long?

I pulled down a page of the site and started to look around

curl "https://www.thoughteddies.com/posts/2025/find-the-groove" > out.html
vi out.html

I saw the page content, a bunch of CSS across the different components, some <path>s, and then I started to see my post content.

Not for the page I was on but for all of the pages across the site.

And then it dawned on me. My seemingly innocuous, static search required all the content from across my site for the client-side search to work.

Every HTML page contained the full content of the entire site.

Classic 🤦

Removing the search functionality from every page brought the site build time down to around 10 seconds total. By no means fast or reasonable all you web purists will tell me because the web is HTML, CSS, and vanilla JavaScript and everything should be 10kb and load in milliseconds. But totally workable for me to build a blog with and have a fine time.

I was pretty proud of this and thought the preview looked pretty good when sending site links in Slack or Messages ↩

Thanks for reading via RSS!

Find the Groove

Thu, 24 Jul 2025 20:35:04 GMT

Working with Cursor was my first experience with the UX pattern of checkpoints when working with a coding agent. If you’re not familiar, it works like this.

The user types a message into an agent-based IDE tool like Cursor, Windsurf, or Cline and submits it to the agent.
The agent reads files, edits them, maybe makes new files.
The user returns to their prior message to edit it and the entire state of the codebase is reverted to how it was before the user typed that prompt.
The user can either edit that prompt or change it completely, then submit it.
The agent then continues from that point in the codebase’s history.

If you’re a software engineer you’re probably thinking, “yes, I do that with Git.”

If you’re not, this is probably the main feature value proposition of Cursor over a CLI-based tool like Claude Code.

Managing side effects

Managing the side effects of coding agents is one of the hardest parts of using them.

For non-engineers, editing a code project is like editing dozens of text documents simultaneously and the undo hotkey doesn’t work. You send a prompt and a ton of stuff happens. The IDE tool shows them which lines were added and which were removed but it is low-value signal for them. The non-engineer’s primary tool for figuring out if the agent did what it was prompted to do is to verify it looks and behaves how they intended.

When verification fails you must go back to the last prior working state. This is where checkpoints are critical.

The ability to revert to the state of “what was working right before I did that last thing” is the fundamental loop that makes creating software as a non-engineer possible. It’s the ability to go back to a prior state and try again.

Software engineers underrate this challenge for two reasons:

Engineers are comfortable with Git and have a good understanding of how to use it to revert changes to their code.
Engineers can read and understand the code itself, so even if something weird happens, they can rely on skills they possess to fix issues with the code if all else fails.

So why don’t you just go learn Git?

I remember learning to use Git. Git is not easy to learn. Git is not easy to use. Git requires an abstract understanding of the filesystem that most computer work does not require.

I know several folks who are not engineers who build software with Cursor. I’ve tried to teach them to use Git, in the abstract, then by example. It often does not go particularly well. The most success I’ve had for adopting Git is having them prompt their agent to use Git for them.

Git’s learning curve is why checkpointing is one of the most sticky features of Cursor, and why non-engineers struggle to adopt CLI-based tools like Claude Code. But the benefits of checkpoints go even further than just a Git replacement.

Context engineering

To get great results out of a coding agent, managing the context window of the LLM is crucial. But this doesn’t just mean clearing the session often. It means carefully deciding when you choose to continue a conversation and when you throw the agent’s response away and revert to a prior snapshot of the codebase.

The goal is to find a groove^[1] with the agent. Agents are relatively good at finding the relevant files in a codebase that need editing to implement instructions described in a prompt but they don’t always get this right.

Trained software engineers might take a more active role in managing the code included as context for the agent. But, as a non-engineer, if you don’t know what any of the code does, you still need to find a way to get the agent to do the work.

Finding the right groove where you’ve prompted the agent in such a way that it has properly sourced the correct context, then meticulously context-managing that conversation until it can no longer be productive is how you make this work.

Finding the groove

You begin a new conversation probing around in latency space, trying to find a groove to drop into that will show you the way forward. This is a confluence of the right prompt that elicits the agent to search the codebase such that it finds the right files, then changes those files in a way that is consistent with the prompt, confirmed by the user’s own verification.

Once you catch this groove, you are in a great position. You can continue to follow the groove and send follow-up prompts to the agent to refine the first implementation to get it more aligned with your intent. But if you find that the agent is all of a sudden out of the groove, stop. Retrace your steps and try to get back onto the well-worn path.

Don’t try and correct the agent. Go back to the previous prompt and correct yourself. Refine your description. Reference the proper context (if you are familiar enough with the codebase to do this).

You must retrace your steps because you are out of the groove. The context window is poisoned. It’s fine to poke around in this area to see what the agent seems to be trying to do or understands at this point but chances are, to get a good result, you are going to want to go back to the last point where things were moving in the right direction.

This is how you use snapshots to find and maintain a groove when working with an agent. It’s partially vibes based, but once you are going in the right direction you make sure that they continue to do so.

You don’t forge forward through the agent’s struggles. You retrace your steps to stay in the groove.

Like a physical groove in wood or a vinyl record. But also like a musical groove where you feel aligned with the rhythm. It’s called ‘vibe coding’ for a reason. There is as much art to this as there is science. ↩

Thanks for reading via RSS!

State Of Code Agents - July 2025

Tue, 15 Jul 2025 13:55:56 GMT

I use code agents to help me code in various capacities. Everything from fully “vibe coded” tools to scripts to specific, well described tasks.

I’d had an itch to work on an idea after using Sonnet 4 for music recommendations.

The idea at its core was to use a chat with an LLM to generate and update a running playlist based on feedback from the user. My thought was I could describe the vibe I was looking to create with the music and the LLM could help make that happen.

My hope was it would introduce me to new artists and songs I might not have otherwise discovered. This had been the case when I chatted with Sonnet about the playlist I was building but the process was quite manual and I wanted to smooth over the rough edges.

Can an agent just do this?

This is one of those projects that lands in the “would be cool” but “not worth that many hours” that language model and coding agents address nicely. Also, the stakes are low and it can all run locally. On my way to get coffee with a friend, I dictated for about 20 minutes, rambling about how this would work, what the UI would look like, what the model would generate, that it needed to integrate with Spotify. I did speech to text on this audio, then I prompted the model to create a functional requirements document that I read and edited. From that, I prompted the model to create a design document that I also read and edited.

I chose to use React Router for the project, so I scaffolded a new project. Then I created an AGENT.md file (I didn’t use Amp but I keep this standard for any agent I use) and then had it track its progress in a plan.md file. Then I sent the agent on its way.

Periodically, the agent would finish a stage of the implementation and I would run the dev server and test it could. I’d prompt the agent with any issue I was having until it fixed them. Then I prompted it to move on to the next stage.

I repeated this process for a little over an hour and a half, while my attention occasionally wandered elsewhere.

Eventually, the agent finished the implementation and I was able to try it out.

Problems

The Spotify integration required some manual setup, to get a client id and secret. I didn’t do this entirely correctly and it took a bit of back and forth to setup the redirect URI properly.

Streaming from Spotify in a browser wasn’t so easy for the model. I’ve never done this by hand before, but it took several correction loops before clicking the “play” button would stream music. Once the agent got this working, the app started spamming the Spotify API and got rate limited.

The next track, skip, and song history tracking did not work and several correction loops could not address the issues. I was seeing duplicates in track history, failure to start the new track after the previous one had ended, and incorrect management of the playlist such that refreshing the page would start it back at the beginning.

Playlist regeneration after chatting or changing the “vibe” config didn’t really work. It seemed the model decided to use the LLM for the first pass and Spotify search for the second. I didn’t attempt to fix this.

Surprises

I never touched the LLM prompt or code that generated the initial music recommendations. This just worked with what the agent created. I might have tweaked it more if I had stuck with the project longer, but it didn’t give me any problems during initial testing.

Getting Spotify streaming working with my brain in management mode was a surprise.

Results and takeaways

The app is functional, sort of.

It looks basically right. It feels pretty wrong and broken.

Agents are great at iterating on projects they can fully test themselves. Backends and APIs are good examples of things agents work on effectively independently.

LLMs are good at writing React code, but I haven’t found them to be great at testing that code in the browser, where it matters. I attached a Playwright MCP to my agent, but it still seemed to struggle with testing its work. I’m not quite ready to write off using this approach. I was too far down the rabbit hole to know what was causing the problem.

Another approach I intend to experiment with is a common log file. Instead of just running

npm run dev

I run npm run dev | tee out.log (or update the package.json to use tee) so that the agent can see the logs of the dev server to fix issues itself. This avoids the need to copy and paste logs into the agent’s chat. This is a tip I took from Armin.

What’s next?

Agent capabilities are at an interesting intersection. They can do a lot of things by themselves and also create a lot of messes. When managed carefully, they can be steered to produce functioning code. When left to their own devices, it’s hard to predict what they will do.

It’s hard to cleanup an agent generated codebase because first you need to onboard to it. The code is completely fresh to you as an engineer.

You can use the model to help you get your bearings, but if you’re encountering a bug the model is struggling with, it may not be as helpful as you need when helping you understand the code, though it can probably still point you in the right direction.

Better guardrails might be able to produce better results than I got using the agent on this project. Maybe setting up the Playwright MCP earlier and having the model test its work incrementally could have kept things on track for longer. I’m really not sure!

There is something here, but it’s hard to articulate. Being so removed from the code is not a comfortable place to be if you care about your software working.

Right now, it feels like refining the agent’s environment and guardrails are where I need to focus. This “vibes” approach still only applies to projects whose future I am not too invested in.

If I need the project to continue to work and scale, I would still need to be more hands on to ensure the agent isn’t making mistakes that cause problems later.

For personal apps that scratch an itch, compelling.

Thanks for reading via RSS!

Gemini Hidden Reasoning

Mon, 07 Jul 2025 01:15:12 GMT

While building Tomo and several prototypes using LLMs, I’ve experimented with several popular language models. It’s generally easier to prototype using the OpenAI chat responses API because most providers support this early API spec (mostly). This approach makes it pretty simple to switch between models and providers by changing model, base_url, and api_key.

After watching tokens stream for the umpteenth time while testing, I started to get different vibes from different models.

Some thoughts that crossed my mind:

Why is Anthropic’s streaming so choppy?
Wow, OpenAI’s streaming is so smooth!
Why did the time to first token for Gemini get so much longer from gemini-2.0-flash to gemini-2.5-flash?

Today is the day we find out what these differences are and if my preconceived notions are actually backed up by experimentation.

Naive streaming with different models

Let’s write some code to stream inference of two models (one older, one newer) from Anthropic, Gemini, and OpenAI.

64 collapsed lines
import openai
import os
import time


class LLMClient(object):
    def __init__(self, api_key, base_url):
        self.api_key = api_key
        self.base_url = base_url

    def get_client(self):
        client = openai.OpenAI(
            api_key=self.api_key,
            base_url=self.base_url,
        )
        return client


client = LLMClient(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url="https://api.openai.com/v1",
).get_client()

gemini_client = LLMClient(
    api_key=os.environ.get("GEMINI_API_KEY"),
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
).get_client()

anthropic_client = LLMClient(
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
    base_url="https://api.anthropic.com/v1",
).get_client()


def stream_chat_completion(client_instance, model_name):
    print(f"\n--- Streaming from {model_name} ---")
    start_time = time.time()
    stream = client_instance.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a very short poem about a robot."},
        ],
        stream=True,
    )

    first_token = True
    last_token_time = start_time
    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            current_time = time.time()
            content = chunk.choices[0].delta.content
            chunk_size = len(content)

            token_latency = current_time - last_token_time
            if first_token:
                ttft = current_time - start_time
                print(f"Time to first token: {ttft:.3f} seconds")
                first_token = False

            print(f"[{token_latency:.3f}s, {chunk_size} chars]", end="\n", flush=True)
            last_token_time = current_time
    print("\n")


if __name__ == "__main__":
    stream_chat_completion(client, "gpt-3.5-turbo")
    stream_chat_completion(client, "gpt-4o")
    stream_chat_completion(gemini_client, "gemini-1.5-flash")
    stream_chat_completion(gemini_client, "gemini-2.5-flash")
    stream_chat_completion(anthropic_client, "claude-3-5-sonnet-20240620")
    stream_chat_completion(anthropic_client, "claude-sonnet-4-20250514")

The output is a lot so I collapsed it. I’m going to aggregate some stats across several runs later, but you can see a few things even before we do that I want to take a look.

132 collapsed lines
--- Streaming from gpt-3.5-turbo ---
Time to first token: 0.507 seconds
[0.507s, 0 chars]
[0.007s, 2 chars]
[0.000s, 9 chars]
[0.019s, 3 chars]
[0.000s, 6 chars]
[0.019s, 1 chars]
[0.000s, 4 chars]
[0.006s, 6 chars]
[0.000s, 7 chars]
[0.004s, 5 chars]
[0.000s, 2 chars]
[0.015s, 7 chars]
[0.000s, 3 chars]
[0.014s, 3 chars]
[0.000s, 6 chars]
[0.007s, 1 chars]
[0.000s, 3 chars]
[0.018s, 7 chars]
[0.000s, 6 chars]
[0.004s, 5 chars]
[0.000s, 2 chars]
[0.015s, 4 chars]
[0.000s, 6 chars]
[0.008s, 5 chars]
[0.000s, 4 chars]
[0.013s, 4 chars]
[0.000s, 7 chars]
[0.014s, 5 chars]
[0.000s, 5 chars]
[0.010s, 4 chars]
[0.000s, 2 chars]
[0.025s, 2 chars]
[0.000s, 1 chars]
[0.006s, 11 chars]
[0.000s, 7 chars]
[0.011s, 1 chars]
[0.000s, 2 chars]
[0.011s, 6 chars]
[0.000s, 2 chars]
[0.009s, 6 chars]
[0.000s, 1 chars]



--- Streaming from gpt-4o ---
Time to first token: 0.808 seconds
[0.808s, 0 chars]
[0.017s, 2 chars]
[0.000s, 9 chars]
[0.023s, 6 chars]
[0.000s, 1 chars]
[0.079s, 3 chars]
[0.000s, 3 chars]
[0.030s, 3 chars]
[0.000s, 4 chars]
[0.033s, 5 chars]
[0.000s, 1 chars]
[0.035s, 3 chars]
[0.000s, 4 chars]
[0.029s, 6 chars]
[0.000s, 5 chars]
[0.027s, 6 chars]
[0.000s, 1 chars]
[0.030s, 4 chars]
[0.000s, 7 chars]
[0.027s, 5 chars]
[0.000s, 6 chars]
[0.017s, 1 chars]
[0.000s, 3 chars]
[0.024s, 1 chars]
[0.000s, 6 chars]
[0.021s, 3 chars]
[0.000s, 5 chars]
[0.008s, 1 chars]
[0.000s, 4 chars]
[0.120s, 7 chars]
[0.000s, 7 chars]
[0.000s, 1 chars]
[0.000s, 3 chars]
[0.001s, 2 chars]
[0.000s, 7 chars]
[0.020s, 3 chars]
[0.000s, 3 chars]
[0.016s, 1 chars]
[0.000s, 4 chars]
[0.022s, 6 chars]
[0.000s, 4 chars]
[0.023s, 5 chars]
[0.000s, 1 chars]
[0.010s, 2 chars]



--- Streaming from gemini-1.5-flash ---
Time to first token: 0.439 seconds
[0.439s, 5 chars]
[0.085s, 69 chars]
[0.056s, 21 chars]



--- Streaming from gemini-2.5-flash ---
Time to first token: 2.575 seconds
[2.575s, 25 chars]
[0.104s, 33 chars]



--- Streaming from claude-3-5-sonnet-20240620 ---
Time to first token: 0.857 seconds
[0.857s, 19 chars]
[0.000s, 20 chars]
[0.000s, 7 chars]
[0.000s, 7 chars]
[0.038s, 15 chars]
[0.048s, 9 chars]
[0.047s, 7 chars]
[0.085s, 8 chars]
[0.002s, 8 chars]
[0.050s, 2 chars]



--- Streaming from claude-sonnet-4-20250514 ---
Time to first token: 1.634 seconds
[1.634s, 7 chars]
[0.132s, 31 chars]
[0.590s, 44 chars]
[1.020s, 45 chars]
[0.174s, 7 chars]

The tokens streamed by OpenAI are smaller than those of the other providers, and there is very little latency between tokens. This likely makes the streaming feel smooth.
Time to first token increases significantly from gemini-1.5-flash to gemini-2.5-flash. We’ll come back to that.
Gemini’s streamed chunks are larger than those of the other providers.
Anthropic’s chunk size and time between chunks are a bit variable. Gemini’s chunks are large enough (and the total inference output for this prompt small enough) that it’s hard to tell how it compares off the bat.

Enough qualitative stuff.

Let’s run some stats.

I changed the prompt to elicit a longer inference response.

Write a story about a robot.

We’re going to do five runs of each model and measure the time to first token, chunk latency (between chunks 2-3, 3-4, etc.), and chunk size in characters.

111 collapsed lines
import openai
import os
import time
import statistics
import json


class LLMClient(object):
    def __init__(self, api_key, base_url):
        self.api_key = api_key
        self.base_url = base_url

    def get_client(self):
        client = openai.OpenAI(
            api_key=self.api_key,
            base_url=self.base_url,
        )
        return client


client = LLMClient(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url="https://api.openai.com/v1",
).get_client()

gemini_client = LLMClient(
    api_key=os.environ.get("GEMINI_API_KEY"),
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
).get_client()

anthropic_client = LLMClient(
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
    base_url="https://api.anthropic.com/v1",
).get_client()


def stream_chat_completion(client_instance, model_name, stats):
    print(f"\n--- Streaming from {model_name} ---")
    start_time = time.time()
    stream = client_instance.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a story about a robot."},
        ],
        stream=True,
    )

    first_token = True
    last_token_time = start_time
    chunk_latencies = []
    chunk_sizes = []

    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            current_time = time.time()
            content = chunk.choices[0].delta.content
            chunk_size = len(content)
            chunk_sizes.append(chunk_size)

            token_latency = current_time - last_token_time
            if first_token:
                ttft = current_time - start_time
                print(f"Time to first token: {ttft:.3f} seconds")
                first_token = False
                if model_name not in stats:
                    stats[model_name] = {
                        "ttft": [],
                        "chunk_latencies": [],
                        "chunk_sizes": [],
                    }
                stats[model_name]["ttft"].append(ttft)
            else:
                chunk_latencies.append(token_latency)

            print(f"[{token_latency:.3f}s, {chunk_size} chars]", end="\n", flush=True)
            last_token_time = current_time

    if chunk_latencies:
        stats[model_name]["chunk_latencies"].append(chunk_latencies)
    if chunk_sizes:
        stats[model_name]["chunk_sizes"].append(chunk_sizes)
    print("\n")


def calculate_aggregate_stats(stats):
    results = {}
    for model, data in stats.items():
        ttft_mean = statistics.mean(data["ttft"])
        ttft_stdev = statistics.stdev(data["ttft"]) if len(data["ttft"]) > 1 else 0

        all_latencies = [
            latency
            for run_latencies in data["chunk_latencies"]
            for latency in run_latencies
        ]
        latency_mean = statistics.mean(all_latencies) if all_latencies else 0
        latency_stdev = statistics.stdev(all_latencies) if len(all_latencies) > 1 else 0

        all_sizes = [size for run_sizes in data["chunk_sizes"] for size in run_sizes]
        size_mean = statistics.mean(all_sizes) if all_sizes else 0
        size_stdev = statistics.stdev(all_sizes) if len(all_sizes) > 1 else 0

        results[model] = {
            "ttft": {"mean": ttft_mean, "stdev": ttft_stdev},
            "chunk_latency": {"mean": latency_mean, "stdev": latency_stdev},
            "chunk_size": {"mean": size_mean, "stdev": size_stdev},
        }

    return results


if __name__ == "__main__":
    stats = {}
    models = [
        (client, "gpt-3.5-turbo"),
        (client, "gpt-4o"),
        (gemini_client, "gemini-1.5-flash"),
        (gemini_client, "gemini-2.5-flash"),
        (anthropic_client, "claude-3-5-sonnet-20240620"),
        (anthropic_client, "claude-sonnet-4-20250514"),
    ]

    for _ in range(5):
        print(f"\n=== Run {_ + 1} ===")
        for client_instance, model_name in models:
            stream_chat_completion(client_instance, model_name, stats)

    aggregate_stats = calculate_aggregate_stats(stats)
    print("\n=== Aggregate Statistics ===")
    print(json.dumps(aggregate_stats, indent=2))

This took a little while to run and I could have parallelized. Fortunately, the first run got the stats I was looking to see.

Aggregate Statistics

Model	TTFT (mean/stdev)	Chunk Latency (mean/stdev)	Chunk Size (mean/stdev)
gpt-3.5-turbo	0.676s / 0.480s	0.005s / 0.012s	5.023 / 2.689
gpt-4o	1.690s / 2.292s	0.019s / 0.051s	4.799 / 2.713
gemini-1.5-flash	0.466s / 0.075s	0.293s / 0.152s	204.328 / 112.207
gemini-2.5-flash	9.763s / 0.921s	0.358s / 0.113s	219.664 / 36.831
claude-3-5-sonnet-20240620	0.755s / 0.208s	0.051s / 0.048s	13.322 / 7.398
claude-sonnet-4-20250514	1.375s / 0.158s	0.408s / 0.205s	56.379 / 18.561

I wasn’t too surprised by the results, having seen the raw stats from the first run. The providers have different chunk sizes, and latency between chunks increases roughly proportionally to the chunk size. I didn’t expect Anthropic to be as competitive with OpenAI in terms of time to first token.

One thing that stands out a lot is gemini-2.5-flash’s time to first token.

Different approaches to reasoning

What is going on here? This latency clearly isn’t a one-off since we did five runs.

In the Gemini documentation on how to call the Gemini models using the OpenAI API, it says:

Unlike the Gemini API, the OpenAI API offers three levels of thinking control: “low”, “medium”, and “high”, which map to 1,024, 8,192, and 24,576 tokens, respectively.

If you want to disable thinking, you can set reasoning_effort to “none” (note that reasoning cannot be turned off for 2.5 Pro models).

I actually didn’t realize gemini-2.5-flash was a reasoning model. Is it possible the reason tokens are being held back from the streaming by default? Is reasoning “on” by default for gemini-2.5-flash?

How does Anthropic handle reasoning for Sonnet 4 with the OpenAI API?

It turns out both providers offer different controls for reasoning.

Anthropic calls it “extended thinking support”.

response = client.chat.completions.create(
    model="claude-opus-4-20250514",
    messages=[...],
    extra_body={
        "thinking": { "type": "enabled", "budget_tokens": 2000 }
    }
)

Google calls it “thinking” and “reasoning effort”.

response = client.chat.completions.create(
    model="gemini-2.5-flash",
    reasoning_effort="low",
    messages=[...],
)

response = client.chat.completions.create(
    model="gemini-2.5-flash",
    messages=[...],
    extra_body={
        "extra_body": {
            "google": {
                "thinking_config": {"thinking_budget": 800, "include_thoughts": True}
            }
        }
    },
)

So let’s play with those. I was curious to see how or if they’re supported in streaming mode.

Starting with Claude, I tried with claude-3-5-sonnet-20240620.

openai.BadRequestError: Error code: 400 - {'error': {'code': 'invalid_request_error', 'message': "'claude-3-5-sonnet-20240620' does not support thinking.", 'type': 'invalid_request_error', 'param': None}}

Makes sense. This isn’t a reasoning model.

Moving to claude-sonnet-4-20250514, for various token budget sizes

response = client.chat.completions.create(
    model="claude-sonnet-4-20250514",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Write a story about a robot."},
    ],
    extra_body={"thinking": {"type": "enabled", "budget_tokens": 1024}},
)

The chart below contains the aggregated stats for five runs of claude-sonnet-4-20250514 with different thinking modes. The suffixes denote the thinking token budget given to Claude for the runs that were aggregated in the plot. No suffixes indicate the default reasoning behavior of the model.

With a thinking token budget, the time to first token is higher. Inspecting the response chunks, we don’t see any of the thinking. It turns out, using the OpenAI Python client, we can’t see the thinking:

The OpenAI SDK won’t return Claude’s detailed thought process https://docs.anthropic.com/en/api/openai-sdk#extended-thinking-support

So, the time to first token is higher when thinking, and that seems to be relatively constant across the specified thinking token budgets.

With Gemini, I tried with gemini-1.5-flash and got this error

openai.BadRequestError: Error code: 400 - [{'error': {'code': 400, 'message': 'Unable to submit request because thinking is not supported by this model. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini', 'status': 'INVALID_ARGUMENT'}}]

With gemini-2.5-flash, I first tried with reasoning_effort="low". Time to first token was still a few seconds. The reasoning tokens did not stream.

Switching to the “thinking” approach (with a budget of a lot of tokens)

extra_body={
    'extra_body': {
        "google": {
                "thinking_config": {
                "thinking_budget": 10000,
                "include_thoughts": True
            }
        }
    }
}

on inspection of the response coming back from the model, we see text like

<thought>**Analyzing the Core Concept**

I'm starting ...

</thought>Unit 734 woke, as it did every cycle...

So when we include_thoughts, this is what they look like. Also, when we include thoughts, the time to the first token goes down, which makes sense because otherwise, we’re waiting for the model to think but not “seeing” it think. We can further validate this by setting thinking_budget to 0 and reasoning_effort to “none”.

The chart below contains the aggregated stats for five runs of the gemini-2.5-flash model with different thinking modes. The numeric suffixes denote the thinking token budget, while the others indicate the reasoning effort given to Gemini for the runs aggregated in the plot. No suffixes indicate the default reasoning behavior of the model.

When reasoning is disabled, either by setting a token budget of 0 (resulting in a time to first token of 0.40s) or by using a reasoning effort of “none” (resulting in 0.43s), the time to the first token closely matches that of gemini-1.5-flash, which is 0.46s, seen in aggregate stats.

Gemini gives us a way to specify how much thinking we want the model to do, or we can give the model a specific thinking token budget. Given the time to the first token for thinking budgets of 1,000 and 10,000 are nearly identical (1.56s and 1.57s respectively), it seems plausible that the model uses its own judgment to figure out how much thinking is needed. This approach seems to align with how Claude implements thinking, and Anthropic doesn’t give us a way to require the model to think more than it thinks it needs to, like Gemini does with reasoning_effort.

Takeaways

I started this investigation wanting to understand why the time to first token for gemini-2.5-flash was so much higher than the other models and prior versions of Gemini. It turns out gemini-2.5-flash does thinking (probably as reasoning_effort="medium" or reasoning_effort="high") by default but hides it. However, it turns out this thinking can also be disabled if you explicitly do so.

The investigation suggests that the straightforward use of the OpenAI chat completion API across different providers is nearing its end, especially for applications needing more than just text and image input support. With provider-specific features like PDF support, varied reasoning methods, Citations, and OpenAI’s shift away from the chat responses API for newer models, the hope for a unified API standard for LLMs is fading.

I am heartened to see OpenRouter serving as the layer of consistency across models and providers. I hope they’re able to continue to make this work and would love to see more of a push for an independent API standard. LiteLLM is also a great project that is continuing to make this work where they can.

Many LLM APIs now also contain nontrivial product features. What could have been some sort of protocol has morphed into competing products with competing visions.

Thanks for reading via RSS!

With MCP, Dates are a Footgun

Sun, 06 Jul 2025 10:14:04 GMT

Model Context Protocol servers have become a popular way to expose software services for LLMs. Prior to this post, I’d not spent nearly enough time with them to have a strong opinion of their pros and cons, so I set out to change that.

Claude, build an MCP server, please

Using the fastmcp library, I set up a simple, toy MCP server. The server exposes one tool (a toy example), which expects a state, start and end date and returns a list of generated events.

#!/Users/danielcorin/dev/mcp_dates/.venv/bin/python

from fastmcp import FastMCP
from datetime import datetime, timezone
from typing import Optional, Dict, Any

mcp = FastMCP("Date Search Server 📅")


@mcp.tool
def search_dates(state: str, end: int, start: Optional[int] = None) -> Dict[str, Any]:
    end_dt = datetime.fromtimestamp(end, tz=timezone.utc)

    if start is None:
        # Default to 30 days before end time
        start = end - (30 * 24 * 60 * 60)

    start_dt = datetime.fromtimestamp(start, tz=timezone.utc)

    # Validate inputs
    if start >= end:
        return {
            "error": "Start time must be before end time",
            "start_epoch": start,
            "end_epoch": end,
        }

    results = []

    # Generate some sample data points within the time range
    current_time = start
    while current_time < end:
        # Add some sample events/dates for the state
        sample_dt = datetime.fromtimestamp(current_time, tz=timezone.utc)
        results.append(
            {
                "date": sample_dt.isoformat(),
                "epoch": current_time,
                "state": state.upper(),
                "event": f"Sample event in {state.upper()}",
                "description": f"Event occurred on {sample_dt.strftime('%Y-%m-%d %H:%M:%S')} UTC",
            }
        )
        # Increment by random intervals (1-7 days)
        current_time += (24 * 60 * 60) * (hash(str(current_time)) % 7 + 1)

    return {
        "state": state.upper(),
        "search_range": {
            "start": {
                "epoch": start,
                "datetime": start_dt.isoformat(),
            },
            "end": {
                "epoch": end,
                "datetime": end_dt.isoformat(),
            },
        },
        "results_count": len(results),
        "results": results,
    }


if __name__ == "__main__":
    mcp.run()

With this setup (and properly installed dependencies) I can run the server in a few different ways.

fastmcp run main.py:mcp

or just

python main.py

I chose the latter. Maybe not “production ready” but I just wanted to try things out.

The server started successfully.

To validate further, I ran the MCP inspector

npx @modelcontextprotocol/inspector python main.py

After opening http://localhost:6274/#tools and clicking connect, I was able to list the search_dates tool and run it.

With that confirmed, I added the MCP server to Claude code, so I could start experimenting with an agent driving

claude mcp add mcp-dates -- /Users/danielcorin/dev/mcp_dates/main.py

It took me a while to figure out a working combination of the shebang in the server and using the absolute path in the claude mcp add command.

Debugging this issue wasn’t straightforward.

The /mcp command in Claude Code also yielded limited useful information.

╭─────────────────────────────────────────────────────────────────╮
│ Manage MCP servers                                              │
│                                                                 │
│ ❯ 1. mcp-dates  ✘ failed · Enter to view details                │
│                                                                 │
│ ※ Tip: Run claude --debug to see logs inline, or view log files │
│  in                                                             │
│   /Users/danielcorin/Library/Caches/claude-cli-nodejs/-Users-da │
│ nielcorin-dev-new-proj                                          │
╰─────────────────────────────────────────────────────────────────╯
   Esc to exit

Running claude --debug showed errors like

[DEBUG] Reading config from /Users/danielcorin/.claude.json
[DEBUG] Config parsed successfully from /Users/danielcorin/.claude.json
[DEBUG] MCP server "mcp-dates": Connection failed: Error: spawn ENOEXEC
[DEBUG] MCP server "mcp-dates": Error message: spawn ENOEXEC
[DEBUG] MCP server "mcp-dates": Error stack: Error: spawn ENOEXEC
    at ChildProcess.spawn (node:internal/child_process:421:11)
    at Object.spawn (node:child_process:763:9)
    at Function.T02 [as default] (file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:320:156285)
    at file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:1340:792
    at new Promise (<anonymous>)
    at Ia1.start (file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:1340:728)
    at an1.connect (file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:1338:14717)
    at an1.connect (file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:1338:20896)
    at file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:1385:5694
    at Q (file:///opt/homebrew/lib/node_modules/@anthropic-ai/claude-code/cli.js:526:13327)
[ERROR] MCP server "mcp-dates" Connection failed: spawn ENOEXEC

which also doesn’t really indicate what the issue is.

Eventually, I figured out I was pointing to a process that didn’t exist or couldn’t be run in the way I was trying to run it.

Call tools with an agent

With the MCP server running and added to Claude Code, I could now prompt it to use the tool.

> search dates for CA from 9:21am April 1 1994 ET to 11:44pm
  June 4th ET
  ⎿  ⧉ Selected 1 lines from ../mcp_dates/main.py in Cursor

⏺ I'll search for dates in CA from April 1, 1994 9:21am ET to
  June 4, 1994 11:44pm ET using the available MCP dates tool.

⏺ mcp-dates:search_dates (MCP)(state: "CA", start:
                              "765138060", end: 770691840)

╭─────────────────────────────────────────────────────────────────╮
│ Tool use                                                        │
│                                                                 │
│   mcp-dates:search_dates(state: "CA", start: "765138060",       │
│   end: 770691840) (MCP)                                         │
│                                                                 │
│ Do you want to proceed?                                         │
│ ❯ 1. Yes                                                        │
│  2. Yes, and don't ask again for mcp-dates:search_dates         │
│  commands in /Users/danielcorin/dev/new-proj                    │
│   3. No, and tell Claude what to do differently (esc)           │
│                                                                 │
╰─────────────────────────────────────────────────────────────────╯

You may think the way I prompted doesn’t make sense, but what happens doesn’t seem easily avoidable regardless of how you prompt.

First, look at the LLM’s restatement of the problem, according to my prompt:

I’ll search for dates in CA from April 1, 1994 9:21am ET to June 4, 1994 11:44pm ET using the available MCP dates tool.

This looks correct in that the LLM seems to correctly restate the date range I prompted for.

Now look at the epoch times in the tool call the LLM is proposing.

❯ date -d @765138060
Thu Mar 31 13:21:00 EST 1994

❯ date -d @770691840
Fri Jun  3 21:04:00 EDT 1994

These dates are close to what I asked for but wrong.

Based on my experience with LLMs, this doesn’t surprise me at all. We’re using the model to run inference to create something that looks like a proper tool call and in the process, hoping it converts the date strings to epoch times by inference alone.

Let’s skip the conversation about why this is confusing and problematic, especially for non-experts being asked to use agents connected to MCP servers to try and increase their productivity. Would they even notice the subtly incorrect epoch times outputted by the model? Would you?

Just add another tool

We can add another tool that allows the agent to parse the date string to epoch time. Then the model can help prevent itself from making this mistake.

@mcp.tool
def parse_date(date_string: str) -> Dict[str, Any]:
    parsed_date = dateparser.parse(
        date_string, settings={"RETURN_AS_TIMEZONE_AWARE": True}
    )

    if parsed_date is None:
        return {"error": "Could not parse the date string", "input": date_string}

    epoch_time = int(parsed_date.timestamp())

    return {
        "input": date_string,
        "epoch": epoch_time,
        "datetime": parsed_date.isoformat(),
        "human_readable": parsed_date.strftime("%Y-%m-%d %H:%M:%S %Z"),
    }

Let’s run the agent with the same prompt:

> search dates for CA from 9:21am April 1 1994 ET to 11:44pm
  June 4th ET
  ⎿  ⧉ Selected 18 lines from ../mcp_dates/main.py in Cursor

⏺ I'll help you search for dates in CA from April 1, 1994
  9:21am ET to June 4th 11:44pm ET. Let me first parse those
  dates to get the epoch timestamps, then search.

⏺ mcp-dates:parse_date (MCP)(date_string: "April 1 1994 9:21am
                             ET")
  ⎿  {
       "input": "April 1 1994 9:21am ET",
       "epoch": 765210060,
     … +3 lines (ctrl+r to expand)

⏺ mcp-dates:parse_date (MCP)(date_string: "June 4th 11:44pm
                            ET")
  ⎿  {
       "input": "June 4th 11:44pm ET",
       "epoch": 1749098640,
     … +3 lines (ctrl+r to expand)

⏺ Now I'll search for dates in CA within that time range:

⏺ mcp-dates:search_dates (MCP)(state: "CA", start:
                              "765210060", end: 1749098640)

╭─────────────────────────────────────────────────────────────────╮
│ Tool use                                                        │
│                                                                 │
│   mcp-dates:search_dates(state: "CA", start: "765210060",       │
│   end: 1749098640) (MCP)                                        │
│                                                                 │
│ Do you want to proceed?                                         │
│ ❯ 1. Yes                                                        │
│  2. Yes, and don't ask again for mcp-dates:search_dates         │
│  commands in /Users/danielcorin/dev/new-proj                    │
│   3. No, and tell Claude what to do differently (esc)           │
│                                                                 │
╰─────────────────────────────────────────────────────────────────╯

The agent recognizes it needs to parse the natural language dates into

❯ now 765210060
Fri Apr  1 09:21:00 EST 1994

❯ now 1749098640
Thu Jun  5 00:44:00 EDT 2025

The behavior of the library I choose with the timezone is a bit unintuitive here

>>> int(dateparser.parse('June 4th 10:44pm ET').timestamp())
1749095040
>>> int(dateparser.parse('June 4th 10:44pm').timestamp())
1749091440

❯ now 1749095040
Wed Jun  4 23:44:00 EDT 2025

❯ now 1749091440
Wed Jun  4 22:44:00 EDT 2025

But how this library works is beside the point. We’re now two layers into dealing with this date parsing conversion issue and we’ve hit “edge cases” almost immediately with each proposed solution.

End user experience

Users of products like this will not like this inconsistency once they figure out what is happening. Behavior like this breaks trust.

When users learn they cannot trust the tool to do what they expect and what they’ve asked it to, some will not keep using it.

It is a lot to expect an end user to validate data returned from a tool call in a potentially unfamiliar data format.

It could be reasonable to give to a technical user with the explicit understanding that these types of things are a problem (like the “LLMs can’t do math” conversation).

Asking a non-technical end user to use an agent equipped with MCP tools is inviting them to think they’ve solved problems where the agent makes subtle and undetected errors. In the worst cases, important decisions made be made with the assumption that the agent’s process was correct.

Here, we show it’s easy for that not to be the case.

Not exceptional behavior

I did not have to repeat the experiments above to experience the problematic behaviors. These issues happened the first time I tried each example.

Could I find a better date parsing library? Probably.

Could I patch these edge cases as well? Likely.

Is it easy to make an MCP server that doesn’t make subtle, difficult-to-detect errors? It wasn’t for me.

Tools (APIs) that support dates are quite common. This investigation was inspired by using a Slack MCP with exactly the same issue.

This issue is in the wild today.

This issue interfered with my ability to use an agent to solve a problem last week.

After figuring out what the issue was after several hours of debugging, I realized it would have been far better to use the agent to write code to solve the problem.

I would have had something to show for my time.

I would have been able to reliably debug my issues.

And I would have been able to reuse the code to solve the same problem in the future.

Takeaways

Agents equipped with tools from MCP servers can be useful, but effective use requires scaling the learning curve, developing experience with LLMs to understand their limitations, and verifying the process used to derive the result.

Leaning on an agent to write code to use to solve your problem is a more stable way to incrementally make progress to the goal while verifying correctness along the way and allowing for easier reuse.

Agents with equipped with tools from MCP servers remain useful for 1-off tasks where strict correctness is either not critical or is easily verified.

Thanks for reading via RSS!

Gemini 2.5 Uses Thinking By Default

Sat, 05 Jul 2025 18:23:38 GMT

It started because I was using the OpenAI completion API to try several different models while building Tomo.

Gemini 2.5 Flash and Pro were released and I added the new model strings and everything Just Worked.

Something felt off though. When chatting with gemini-2.5-flash the model felt slow.

Well not exactly slow, but it felt like the model was consistently taking more time than I expected before the response would start streaming.

I wrote up a quick script to try and isolate the behavior and ran the inference for the new model and the past two GA releases of Gemini Flash.

import os
import time

from openai import OpenAI

gemini_client = OpenAI(
    api_key=os.environ.get("GEMINI_API_KEY"),
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)


def stream_with_openai_client(model_name):
    start_time = time.time()
    stream = gemini_client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a story about a robot."},
        ],
        stream=True,
    )

    first_token = True
    for chunk in stream:
        if chunk.choices[0].delta.content is not None:
            if first_token:
                ttft = time.time() - start_time
                print(f"{model_name} TTFT: {ttft:.3f}s")
                first_token = False
    return ttft


if __name__ == "__main__":
    results = {}
    results["gemini-1.5-flash"] = stream_with_openai_client("gemini-1.5-flash")
    results["gemini-2.0-flash"] = stream_with_openai_client("gemini-2.0-flash")
    results["gemini-2.5-flash"] = stream_with_openai_client("gemini-2.5-flash")

The results confirmed what I had been seeing. The time to first token, specifically for gemini-2.5-flash, was a lot longer than I expected and longer than for the previous two GA releases.

❯ python gemini_openai.py
gemini-1.5-flash TTFT: 0.523s
gemini-2.0-flash TTFT: 0.511s
gemini-2.5-flash TTFT: 8.538s

I didn’t want to jump to conclusions, so I tried the same script with the Google GenAI SDK.

import os
import time

from google import genai

client = genai.Client(api_key=os.environ.get("GEMINI_API_KEY"))


def stream_with_genai(model_name):
    start_time = time.time()
    response = client.models.generate_content_stream(
        model=model_name, contents="Write a story about a robot."
    )

    first_token = True
    for chunk in response:
        if chunk.text:
            if first_token:
                ttft = time.time() - start_time
                print(f"{model_name} TTFT: {ttft:.3f}s")
                first_token = False
    return ttft


if __name__ == "__main__":
    results = {}
    results["gemini-1.5-flash"] = stream_with_genai("gemini-1.5-flash")
    results["gemini-2.0-flash"] = stream_with_genai("gemini-2.0-flash")
    results["gemini-2.5-flash"] = stream_with_genai("gemini-2.5-flash")

The results were nearly the same.

❯ python gemini_genai.py
gemini-1.5-flash TTFT: 0.540s
gemini-2.0-flash TTFT: 0.420s
gemini-2.5-flash TTFT: 9.468s

Why?

While it’s not exactly clear that this is the case, the Gemini 2.5 models enable thinking by default. For gemini-2.5-pro, reasoning cannot be disabled but for gemini-2.5-flash, it can be.

When we disable reasoning, time to first token is much faster:

# ...

def stream_with_openai_client(model_name):
    start_time = time.time()
    stream = gemini_client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a story about a robot."},
        ],
        stream=True,
        reasoning_effort="none",
    )

# ...

❯ python gemini_openai_no_reasoning.py
gemini-1.5-flash TTFT: 0.555s
gemini-2.0-flash TTFT: 0.488s
gemini-2.5-flash TTFT: 0.402s

We can also add a configuration to include_thoughts which seems to reduce the time to first token as well (though not nearly as much compared to when reasoning is disabled entirely).

# ...

    stream = gemini_client.chat.completions.create(
        model=model_name,
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a story about a robot."},
        ],
        stream=True,
        extra_body={
            "extra_body": {
                "google": {
                    "thinking_config": {
                        "include_thoughts": True,
                    }
                }
            }
        },
    )

# ...

❯ python gemini_openai_include_thoughts.py
gemini-1.5-flash TTFT: 0.749s
gemini-2.0-flash TTFT: 0.510s
gemini-2.5-flash TTFT: 1.591s

So thinking explains the delay I was seeing in time to first token.

In my opinion, it’s a bit of a departure from the previous behavior.

It’s curious that Google would make thinking opt-out behavior. This seems like a trade off that would improve model performance on benchmarks, increase latency, and increase token usage.

I haven’t seen this new default behavior discussed anywhere. It is documented, but it’s just surprising to me that it was made the default behavior.

Thanks for reading via RSS!

Amazon Bedrock On Demand Throughput Error

Mon, 30 Jun 2025 21:11:22 GMT

I was working with Amazon Bedrock to run LLM inference. AWS has its fair share of complexity — VPCs, subnets, security groups, etc.

On the surface, running inference on Amazon Bedrock is straightforward. A simple script might look like this (assuming you have proper environment variables set):

bedrock = boto3.client('bedrock-runtime', region_name='us-east-2')

messages = [{"role": "user", "content": [{"text": "What is 2+2?"}]}]

res = bedrock.converse(modelId="anthropic.claude-3-5-sonnet-20241022-v2:0", messages=messages)
print(res['output']['message']['content'][0]['text'])

We can find this model name in the us-east-2 model catalog (requires sign in).

Anthropic’s documentation also mentions these model names.

I was working with SageMaker and dealing with a restrictive VPC setup, but when I tried to run the inference above, I got the following error:

An error occurred (ValidationException) when calling the Converse operation: Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn't supported

This was confusing specifically because I wasn’t sure if I was having AWS permissions and access issues or inference issues.

It turned out, prefixing the model id with us., i.e.gs us.anthropic.claude-3-5-sonnet-20241022-v2:0 was all I needed to get things working for my setup.

bedrock = boto3.client('bedrock-runtime', region_name='us-east-2')

messages = [{"role": "user", "content": [{"text": "What is 2+2?"}]}]

res = bedrock.converse(modelId="us.anthropic.claude-3-5-sonnet-20241022-v2:0", messages=messages)
print(res['output']['message']['content'][0]['text'])

Hopefully, this is helpful if you’re calling Bedrock through a VPC endpoint.

Thanks for reading via RSS!

A few strategies to make LLMs useful for you

Tue, 24 Jun 2025 23:39:33 GMT

I was reading your blog and had a question about this:

“I noticed that my coworker was prompting for specific technical implementations, and Claude was struggling, pulling in too much context and taking an unfocused approach, whereas I would have been much more vague and general to start and refined as I saw whether Claude was on the right track.”

Would you be able to elaborate more on your thought process on how to prompt an LLM to code a task for you based on its complexity? [I …] would appreciate any examples you have from experience.

I think part of why I get good results from prompting LLMs is because I’m both willing to engage in and enjoy the process of thinking and writing with clarity. I’m the software engineer who likes to write air tight documentation, with working copy-pasteable example code in different languages. I like to be thorough, mentioning issues I ran into in the process, but I also try and value the reader’s time and attention, giving them the thing they are looking for if they aren’t interested in reading about the details.

I’ve found this process is highly effective for getting good results from LLMs as well.

But it’s not always so easy to just start writing.

Let’s say your university CS class project is to build a snake game. If you prompt an LLM to build you a snake game, it can probably do it with no further prompts. There are tons of open source implementations of snake games and even if there weren’t, the model already knows how the game works, the rules, the mechanics, and the win conditions.

However if you just prompt

User

write me a snake game

Assistant

I’ll create a Snake game using Python and Pygame. Here’s a complete implementation:

import pygame
import random
import sys

# Initialize Pygame
pygame.init()

# Constants
WINDOW_WIDTH = 800
WINDOW_HEIGHT = 600
GRID_SIZE = 20
GRID_WIDTH = WINDOW_WIDTH // GRID_SIZE
GRID_HEIGHT = WINDOW_HEIGHT // GRID_SIZE

# Colors
BLACK = (0, 0, 0)
GREEN = (0, 255, 0)
RED = (255, 0, 0)
WHITE = (255, 255, 255)
DARK_GREEN = (0, 150, 0)

# Directions
UP = (0, -1)
DOWN = (0, 1)
LEFT = (-1, 0)
RIGHT = (1, 0)

class Snake:
    def __init__(self):
        ...

the model will have to fill in a vast number of details to build the game to completion.

This includes things like

the size of the grid
the colors of the snake, food, and background
the speed of the game
whether the game should speed up over time
whether the game should have multiple levels

The list goes on.

If you prompt the model in a vague manner, it can often deliver something in the direction of what you are looking for, but it will fill in a lot of details in ways you might not want. This seems to be the default behavior of most models these days, especially coding agents. You prompt them and they get to work. They assume you’ve given them all the information they need to do the task.

However, if you’re working on something particularly complex and you know you may have some gaps in your question, you can prompt the model to prompt you for more details.

OpenAI does this with deep research agents. After you send your first prompt, the model prompts you for more details.

You can add a system prompt to any language model you are working with to get it to do this for you whenever you want.

You are an assistant to a user in a coding task. When they ask you to help them do something, prompt them for more information if anything about the task is unclear or requires more details.

User

write me a snake game

Assistant

I’d be happy to help you create a Snake game! To make sure I build exactly what you’re looking for, could you provide some more details?

Platform/Technology: What would you like me to use?
- Python with Pygame
- JavaScript (HTML5 Canvas)
- Python with Tkinter
- Another language/framework?
Features: What functionality do you want?
- Basic snake movement and food collection
- Score tracking
- Game over screen
- Different difficulty levels
- Sound effects
- High score saving
Visual style: Any preferences for:
- Window size
- Colors
- Grid-based or smooth movement
Controls:
- Arrow keys
- WASD keys
- Both options

Let me know your preferences and I’ll create a Snake game tailored to your needs!

This difference in behavior is already a substantially different approach to working with a language model. In the first case, you are largely delegating work to the model. You have some task you need to get done, you ask the model to do it, maybe it comes up with something close to what you wanted.

For most tasks, there are a lot of details.

You can offload decisions about the details to the model, but then you’re not really intellectually participating in the process of creating the thing.

Maybe you don’t care about most of the details but now you have the opportunity to specify the ones you do care about.

A follow up prompt as simple as

User

use python. let’s get to work

Assistant

Perfect! I’ll create a Snake game in Python using Pygame. Let me build a complete, playable version for you: …

is enough to send the agent to work, but a lot of good things happen in the interim.

Getting the model to prompt you for more details is an easy and effective way to make sure it is on the right track and make sure it understands what you are asking it to do. You then also have the opportunity to fill in the details you care about, steering the model to giving you a much better result. If it clearly doesn’t understand what you are asking for, you can start over with different instructions until you are convinced it does (I recommend starting over rather than trying to fix it with a follow up prompt).

This is one of several ways I experiment with models to build things.

They’re also excellent brainstorming partners, to challenge you to think through details and assumptions you’ve made about your plans. I frequently use a model with a version of the following system prompt to help me think through some new idea I have to see where it leads.

You are a thought partner to the user. You will have a conversation with the user, asking thoughtful follow up questions about what they say. You will allow the user to steer the conversation in whatever direction they choose. Only provide feedback when asked. Focus on following up to help the user deepen and clarify the ideas they are exploring.

The model’s prompt makes my thought process more of a conversation designed to elicit further thought than an independent exercise in putting words on a page — something which can oftentimes be a lot more effortful than doing what feels more or less like texting your friend about an idea you just thought of and them enthusiastically saying “tell me more” in lots of different ways.

If you feel this approach results in too much sycophancy, modify the prompt to instruct the model to challenge you more.

Find what works well for you!

That is one of the best parts about working with these incredibly flexible models.

Even just a few lines of a system prompt can change the model from something that feels difficult to use to a highly effective collaborative partner you work with daily.

Thanks for reading via RSS!

Agent Coding Strategies

Thu, 19 Jun 2025 15:18:05 GMT

note

Feel free to skip directly to the strategies if you don’t need to be convinced using agents to write code is worth your time.

If you’ve read any of my writing in the past year, you’re probably aware I’ve heavily adopted agents to build much of the software I write now. What I’ve done less of is write about the strategies I’ve used to do this.

After shadowing some colleagues, both classically technical and classically non-technical, I’ve seen good ideas come from everywhere on how to make agents build what you want them to build.

The markers of success with these approaches, above all else, seem to be persistence, a willingness to try many different things, and to learn and adapt to what does works, especially as the tools continue to change. This pivot in approach can be especially hard for people who already know how to code and have strong opinions about how to do things.

If this is you, try and suspend your disbelief. See what the tools can do for you. Your knowledge of code will be valuable, but first you need to develop this new skill of prompting a language model.

Here are some of the strategies that work for me in June 2025. I’ve tried to keep these tool agnostic as the landscape of tools is changing rapidly and I use these approaches across all agent tools I work with.

Write a spec first

For projects that I hope to keep around for a while and iterate on, I write a spec for each feature first. Sometimes, I put these in a specs folder, named yyyy-mm-dd-feature-name.md. These serve as the beginning of my conversation with the agent, which often looks like:

Implement @specs/2025-05-16-test-api-button.md

Here is an example from one of my projects:

# Test API Button

In @ApiSettingsView.swift add minimal button below the configuration value input fields to make a test api request with the configured provider, model and API key in the @ApiSettingsView.swift
Make the test request by use @OpenAIService.swift to send a "hi" message to the model API.

If the call is successful, show a green UI element.
Otherwise show a red UI element and the full error from the call.
Always allow the user to click the button to retry the connection and update the UI to show the new result.

I took this idea from Geoffrey Huntley.

The approach helps ensure that I know what I expect the agent to build. If I am fuzzy on these details, the agent will often attempt to fill them in, but if they are details I care about, I will likely need to start over adding the details to the spec because the agent can’t read my mind (even though sometimes it feels like it can).

Ensure the model has proper context

You might have noticed I used @ to reference specific files in my spec. Agents are often good enough to find the context they need by searching your codebase, but referencing specific files ensures the agent will use what you know to be the correct context where it starts to work. Most agents are pretty good at searching your codebase, but in larger codebases or in the face of ambiguity, being clear helps with quality.

For me, there is a specific exception to this rule. Sometimes I go out of my way to not be specific, particularly with languages and frameworks I am less familiar with. In these cases, I’m often not writing production-grade code and am leaning on the model to steer me in a reasonable direction. This doesn’t always work, but it seems to be better than being wrong.

Start fresh to avoid context rot

Context rot was a recently coined term to describe the process by which the past messages in an LLM conversation undermine its ability to accomplish a task going forward. Sometimes this rot comes from the agent, other times the user’s prompt, often a combination of both. It happens as the context grows towards the size limit of the context window, where most models begin to have issues with recall.

I mention this because context rot is the reason I write specs.

Fixing problematic context or instructions is fundamental to getting the agent on the right track to solving your problem.

Starting with a fresh chat or fixing a misunderstanding, typo, or mistake by deleting/editing a previous message is always better^[1] than trying to clarify the mistake with a follow-up message.

If the agent is going in the wrong direction from the first prompt, I edit the spec and start over.

I’ve also described this as “conversational inertia.” When the agent is going in the wrong direction, it tends to keep going in the wrong direction. Conversely, when the agent is going in the right direction, follow-ups can be useful, effective, and powerful while not too much of the context window is being utilized.

When I add follow-up messages, I haven’t kept a consistent habit of adding the follow-ups to the spec, but this feels important, especially if I decide to keep the specs under source control or add them to the PR description.

Use examples

This recommendation is a bit like “a picture is worth a thousand words.” If you have a working example of code that uses a pattern or library — in your codebase, from documentation, or elsewhere — this can serve as valuable context to add to your prompt.

It is more difficult and time-consuming to explain with words to the agent how you want an SDK to be used than it is to just show the agent the SDK and how it works.

I’ve successfully one-shotted RPC calls using an undocumented internal framework by showing a totally different working RPC and the protobuf definition for the RPC I wanted to call. The agent wrote a working call on the first try.

Consider if you are wrong

I was building a recipe web app for myself with a 50-50 split of two panes to list the ingredients and instructions for a recipe. I wanted the agent to edit the styling of the numbered badges in the instructions pane, but I accidentally told it to edit the badges in the ingredients pane. As a result, the agent kept trying to update the style of the tag badges above the ingredients list.

Since I wasn’t as mentally checked in as I would have been for a serious project, I wasn’t recognizing how the agent was attempting to make changes and thus missed my mistake for several attempts.

Be an active collaborator

The quality of the results I get using agents to write code seems directly correlated to how focused I am in participating in the work with the agent. I can submit a prompt then walk away, but what seems to matter consistently is that I generally consider how I would do the task and that I review the results of the agent’s work.

What does this mean for vibe coding?

It probably means the quality of the codebases written by folks familiar with code will be better than non-technical folks for a little while longer.

When I care about the endurance of the project I am working on, I’ve found I need to be an active collaborator with the agent; otherwise, it will eventually make a mistake I miss, find out about later, and can’t prompt it to dig itself out of. I ran into this exact problem when I “fully vibe coded” Pad Lander.

I’m not exactly sure if this problem will ever go away as agents improve because you can always increase the complexity by another order of magnitude.

Can the agent refactor a problematic codebase? What about a codebase 5x as large? What about 5 interdependent codebases deployed separately? What about those same 5 codebases deployed in multiple regions with eventually consistent data?

I’m not saying agents won’t eventually get there, just that they still have room to improve. Right now, the best results for me still come from actively participating. This has allowed me to keep primarily-agent-written codebases in good enough shape that I’ve been able to continue adding features to them for many months - a year or so (the capability is about a year old).

Bonus: use guardrails

Guardrails don’t strictly make the agent better at producing good results, but they often allow the agent to use feedback to run for longer and fix problems you might have to resolve yourself (or just paste back into the agent and prompt it to do it).

Guardrails come in many forms, but ones that have been most helpful for me are integration tests. For example, I start my app locally, then write code to make a request to an an API, check the response, and finally request another API to validate the system is in the expected state.

I don’t have strong opinions on the type of guardrails you add. I recommend adding guardrails that give you confidence the change you made it safe, works, and didn’t break anything else.

Different types of agent guardrails could be a post in and of itself. Maybe another time and after I try more things.

After noticing this phenomenon, I built a project called Delta (like a river delta) to enable first-class conversation branching, where I can branch off any message in the conversation to continue in a new direction but also preserve the history of other branches if useful. ↩

Thanks for reading via RSS!

Who Finds LLMs Useful and Why

Tue, 17 Jun 2025 23:53:15 GMT

Who is finding LLMs useful and who is not? And why is this the case?

Since around two years ago, I’ve been floored by the fact that I can, with increasing adherence to my instructions, get a model to write software and complete tasks for me given a good description of what I want.

This technology feels world-changing to me. I feel this way because it has changed my world and the things I am capable of accomplishing in fixed time given the skills I possess today.

I’ve talked to anyone who will listen about why I think this technology is a step-function change in how we will accomplish many knowledge tasks in the future and how I am doing so in the present.

The utility of language models is beyond apparent to me personally. Yet, I frequently read articles by folks I respect who seem to feel the opposite way. These folks do not believe language models make them faster or more effective at accomplishing their work or goals. Some seem to believe the models make them dumber and lazier. Some seem to believe the models are a poor facsimile of intelligence. Some find they don’t live up to the hype and promise.

Increasingly, I’ve sought to understand the perspective of these folks. What about how they think, how they work, or how they operate makes an LLM not useful for them?

I recently shadowed a coworker while they used Claude Code to write some software. Earlier on, I identified differences in how we engaged with and prompted the model. But more interestingly, I realized that prompting an LLM (particularly a coding agent) is a distinctly different skill than writing software.

I noticed that my coworker was prompting for specific technical implementations, and Claude was struggling, pulling in too much context and taking an unfocused approach, whereas I would have been much more vague and general to start and refined as I saw whether Claude was on the right track.

The difference mostly amounted to prompting what vs. how. I typically prompt for what I want done. They were prompting for how they wanted something done.

None of this is to say that either approach is better or worse, but rather that I think I have drastically underappreciated how many different ways there are to prompt an LLM and how different the outcomes can be as a result.

So who is not finding models useful and why? One archetype I imagined might be someone for whom writing software is the clearest way in which they are able to communicate an idea.

If it were harder for you to describe your ideas in natural language than in code, I can see why you wouldn’t think writing software with a model would be useful.

Personally, I am far better at describing my ideas in natural language than in code.

Another archetype I imagined would be someone who is expecting a perfect result from the start. They might prompt the model to “launch a Flappy Bird app clone in the app store” and end up disappointed when the model doesn’t do it.

I know that I could personally use an LLM coding agent to perform that task, but it would still take me several hours. Far less than in a pre-LLM world, but not a trivial amount of effort.

It seems if you are willing to be patient, probe, and explore, models yield some of the best results. They won’t write the perfect code on the first attempt, but they will allow you to sculpt your ideal implementation, trying several different approaches along the way.

So why do I find models useful?

I find models useful because I can describe what I want and the details I care about, and the model will follow my instructions, then fill in the rest of the details that need to be decided, but that weren’t initially my focus.

I find models useful because I can follow up and modify the details the model filled in for me and easily change those too.

I find models useful because I can use them to help me from wherever I am currently, rather than me having to modify the shape of my work to adapt to a different solution I may find reading documentation or example resources.

I find models useful because I feel a certain amount of confidence knowing that I have a tool that can reliably get my mental wheels turning when I get stuck on a task and need to figure out a path forward.

I find models useful because I can prompt them to take a first pass at something that I think could be interesting to explore further, but I won’t really be sure until I see it.

I find models useful because they generally seem to understand and be able to implement prompts that I give them in support of the vision I have, even when many other people sometimes do not.

I find models useful because they allow me to learn more about exactly what I am interested in at the exact moment I am interested in it.

I find models useful because they speed up my feedback loop of creation just enough to keep me engaged and working in a flow state.

I find models useful because they allow me to make my ideas real and tangible in more ways.

I find models useful because they help me communicate more clearly with others.

I find models useful to help me pick reasonable starting points when I am stuck or experiencing decision paralysis.

I find models useful because they help me build tools that I can use to accomplish my goals and make my life easier.

I find models useful because they help me build fun and useful things for friends and family.

I find models useful because I can prompt them to challenge my thinking to give me another perspective.

Much of my use of language models is code-focused, but I’ve had a lot of success using them to help me get started as a beginning designer, to understand legal documents and their implications, and to learn more about cloud network architecture, security, and the protocols that make the internet work.

I don’t rely on the model as gospel, but I involve them in a substantial part of my work.

Is using LLMs making me dumber? I’m not entirely sure. I may write less code by hand than ever, but I do more now than I ever have before.

I may never learn to write Haskell or D3 at a high level of proficiency without the help of a model. Given we have limited time and resources, models let me use more of my time working on the things that interest me and that I enjoy. And they give me the ability to work with more tools in service of those interests and goals.

To me, models seem to reward the curious and the persistent, but do require a healthy amount of wariness given their flaws, limitations, and record of being incorrect at times. I can’t say I always know what that looks like or practice it effectively, but I do believe working with LLMs has made a strongly positive impact on me.

Thanks for reading via RSS!

OpenAI Compatible API Implementation Variance

Thu, 29 May 2025 21:02:05 GMT

Lots of language model providers implement the OpenAI API spec. These look similar in shape but often behave differently in subtle ways. Anthropic’s prefill sequences are one such example.

I wasn’t able to find a canonical definition of this spec. In practice, we can show the basic shape of the API for chatting with a few examples.

OpenAI:

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4.1",
    "messages": [
      {"role": "user", "content": "Hi"},
      {"role": "assistant", "content": ""}
    ]
  }'

{
  "id": "chatcmpl-BcibN0BPNN8ysOymTxonvePn0m3n6",
  "object": "chat.completion",
  "created": 1748567613,
  "model": "gpt-4.1-2025-04-14",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today? 😊",
        "refusal": null,
        "annotations": []
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 10,
    "total_tokens": 22,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "audio_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "service_tier": "default",
  "system_fingerprint": "fp_799e4ca3f1"
}

Anthropic:

curl https://api.anthropic.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -d '{
    "model": "claude-4-sonnet-20250514",
    "messages": [
      {"role": "user", "content": "Hi"},
      {"role": "assistant", "content": ""}
    ]
  }' | jq .

{
  "id": "msg_016bT3QYFyNLL1NXvwLiM8oh",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How are you doing today? Is there anything I can help you with?"
      }
    }
  ],
  "created": 1748567742,
  "model": "claude-sonnet-4-20250514",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 20,
    "prompt_tokens": 8,
    "total_tokens": 28
  }
}

Gemini:

curl https://generativelanguage.googleapis.com/v1beta/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $GEMINI_API_KEY" \
  -d '{
    "model": "gemini-2.0-flash",
    "messages": [
      {"role": "user", "content": "Hi"},
      {"role": "assistant", "content": ""}
    ]
  }' | jq .

[
  {
    "error": {
      "code": 400,
      "message": "Unable to submit request because it has an empty text parameter. Add a value to the parameter and try again. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini",
      "status": "INVALID_ARGUMENT"
    }
  }
]

OpenAI and Anthropic respond quite similarly.

--- openai.json
+++ anthropic.json
@@ -3,34 +3,19 @@
     {
       "finish_reason": "stop",
       "index": 0,
-      "logprobs": null,
       "message": {
-        "annotations": [],
-        "content": "Hello! How can I help you today? 😊",
-        "refusal": null,
+        "content": "Hello! How are you doing today? Is there anything I can help you with?",
         "role": "assistant"
       }
     }
   ],
-  "created": 1748567613,
-  "id": "chatcmpl-BcibN0BPNN8ysOymTxonvePn0m3n6",
-  "model": "gpt-4.1-2025-04-14",
+  "created": 1748567742,
+  "id": "msg_016bT3QYFyNLL1NXvwLiM8oh",
+  "model": "claude-sonnet-4-20250514",
   "object": "chat.completion",
-  "service_tier": "default",
-  "system_fingerprint": "fp_799e4ca3f1",
   "usage": {
-    "completion_tokens": 10,
-    "completion_tokens_details": {
-      "accepted_prediction_tokens": 0,
-      "audio_tokens": 0,
-      "reasoning_tokens": 0,
-      "rejected_prediction_tokens": 0
-    },
-    "prompt_tokens": 12,
-    "prompt_tokens_details": {
-      "audio_tokens": 0,
-      "cached_tokens": 0
-    },
-    "total_tokens": 22
+    "completion_tokens": 20,
+    "prompt_tokens": 8,
+    "total_tokens": 28
   }
 }

And we get an error from Gemini.

Why?

We actually did something a little strange here. We send an empty assistant message in our API request.

Typically, you’d send the most recent user message, expecting the model to respond with a new assistant message. OpenAI and Anthropic handle this gracefully. Gemini throws an error.

While the OpenAI compatible API spec is very useful for using different models in the same context, some variance in implementations pops up here and there on the margins.

If I find more occurrences of these, I’ll create a series or something.

Thanks for reading via RSS!

Note Taking

Mon, 19 May 2025 00:04:55 GMT

Every now and then, I’m pulled back into the orbit of the second brain/zettelkasten/notetaking community conversation. I’ve written a little bit about my note-taking process before.

I don’t really use it too much these days. And even when I do, I’m often irritated by the thought of “where should this go?” and the general dissonance of “mess-making”. Dan Shipper talks about this same challenge in this post, The Fall of Roam.

While reading this post, I was reminded of another of my favorites. One with a particularly memorable name and liberating idea: The Good Shit Sticks, by Oskar, about an idea from Tim Ferriss.

Note-taking and thinking about ideas and things you have learned, reviewing notes, and being reminded of your past self is a great way to revisit where you have been and where you thought you were going. However, for me, my notes have never really been my roadmap even when I thought they were or should be.

Notes are necessary and useful for keeping track of things and structuring thoughts. If I get feedback from a friend about a project, I’ll summarize it in a note, but this note and its value are likely short-lived. Ideally, I will re-read and incorporate the feedback into my project soon. If I don’t, I’ll likely have moved on from the idea entirely. The note and its continued existence won’t be all that important.

Most notes aren’t that important. Not that many can be. You can only focus on so many things.

However, the good stuff sticks around — the concepts, how they connect, and where I can go with these ideas. For my most interesting work, I’m often not relying on my notes to figure out where I’m going with regard to building and experimenting. I’m largely relying on my intuition, interest, and ways in which I see the pieces connect.

I read and write extensively, but I’m writing notes to think. In fact, many of my most interesting notes happen in these logs, which eventually turn into blog posts or more exploratory work.

The act of building the body of work is the process that matters. The curation and organization of the notes themselves matter a lot less.

These days I try to focus on building things, reading things, and trying to understand them deeply through my own experimentation and application of the ideas whenever possible. Practically, this means reading about vibe coding (constructing a software system without looking at the code and only verifying the results) then actually trying it to understand the tradeoffs or experimenting with frameworks like langchain and comparing them to rolling my own implementations.

I’ve been far more satisfied with the outcomes of this approach than I ever was trying to make my notes the center of my universe.

Thanks for reading via RSS!

Using `llm` to handle large input context

Fri, 16 May 2025 19:18:54 GMT

Today, I ran into an issue where I wanted to use repomix to pack a large codebase into a single file to pass to an LLM, but I couldn’t paste the output into any of the UIs I typically use. The React apps all became sluggish as I waited for ~500,000 tokens to paste.

Enter llm.

I solved this problem with a bash one-liner:

llm -m gemini-2.5-pro-preview-05-06 "The provided context is all the code of an old codebase. Analyze this code and come up with high impact, meaningful improvements to make the codebase easier to work with." < repomix.output

The API and subsequent inference took about 85 seconds, and I had my response.

The quality of the results was mixed. There were a few good suggestions, some of which we’d already considered and some that we’re currently implementing. Overall, I’m unsure this is an approach I will reach for very often.

Thanks for reading via RSS!

Git Worktree

Sat, 10 May 2025 10:08:45 GMT

These days I use agents that write code often. When I am trying to build a new feature, I first write a markdown spec, then point the agent at it and send it on its way.

There are a lot of tools and choices today in the agent space. I regularly use 3-4 different ones, and I expect that number to continue to vary.

When you send an agent off to write code, you need to wait. If you have more work to do, especially work that is unrelated to the current changes the agent is making, it would be nice to unblock that work as well.

Git worktree makes this possible

A git repository can support multiple working trees, allowing you to check out more than one branch at a time. — https://git-scm.com/docs/git-worktree

Let’s build a simple project

Feel free to skip to the Worktrees section if you already have a project you want to work on or come up with your own example.

If we go from zero, let’s set up a new git repo and make a commit.

mkdir -p our-project-with-worktrees
cd our-project-with-worktrees
git init
echo "Hello, world\!" > README.md
git add README.md
git commit -m "initial commit"

Now, let’s set up a simple fastapi web server which will be the base of the two tasks we send the agents off to do.

User

Create a fastapi web server that listens on port 8000 and returns ‘Hello, world!’.
Next, create a readme for how to setup and run the project.
Finally, follow those steps and ensure everything worked

Gemini was a little lazy here and didn’t create the virtualenv itself. Instead, it just wanted to install the dependencies to the system Python.

So we did this part ourselves:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Also, we add a .gitignore for a Python project with the very convenient

npx gitignore python

We run the project with

uvicorn main:app --reload

and validate it works as expected

curl http://localhost:8000

{"message":"Hello, world!"}%

Cool.

Now one more modification to make our task a little more interesting.

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root(name: str = "world"):
    return {"message": f"Hello, {name}!"}

And let’s commit the results.

git add -A
git commit -m "Simple server"

The task

We’re going to ask an agent to write a CLI to call the web server, supporting arguments, the input parameters, and dealing with the different HTTP methods for us. This is a pretty contrived and even ill-defined task. In my head, a successful outcome could be something like a hybrid between rails routes and curl. But also, I’m not too worried about the outcome. I just needed an excuse to use a worktree.

Write a generic CLI that can call the web server. The CLI should inspect the server route and input args to determine the inputs and flags to call the server. It should automatically extend to future routes added with no additional modification required, supporting optional and required arguments and enforcing type constraints. For example, based on the existing main.py, the CLI should support
Terminal window
python cli.py root --name=Alice
which outputs
{ "message": "Hello, Alice!" }
and
Terminal window
python cli.py / --name=Bob
which outputs
{ "message": "Hello, Bob!" }

Let’s see what the agent comes up with.

Worktrees

Let’s create a new worktree.

git worktree add ../claude-code-1
cd ../claude-code-1

We’ve just created a new folder at the same level as our original project.

Now let’s send claude-code off to work with the prompt above.

These days, I paste my prompts or specs into a specs folder, then prompt the agent with something like “Implement @specs/cli.md”

I’ve been meaning to try OpenAI’s Codex, so let’s create another worktree and send it off on the same task.

git worktree add ../openai-codex
cd ../openai-codex

OpenAI sent me a email because they noticed I hadn’t tried the gpt-4.1 model. I’m not sure I want to reward that behavior and give them a conversion, mostly because it just means I’ll get more emails but I digress.

I launched codex and set it on its way

codex -m gpt-4.1

I sent it the same prompt as claude-code, referencing the spec file I wrote.

codex and gpt-4.1 seemed to have a pretty hard time with this ask. The agent spent most of its time checking dependency installation, trying to run the web server, and looking around the project.

I started over, ensuring I had the dependencies installed and the virtualenv activated. This second attempt, I tried with o4-mini. It felt noticeably slower than gpt-4.1, but it did eventually write the CLI.

For both models with codex, it was kind of hard to understand what the agent was intending to accomplish. codex prompts you to approve the commands it runs at each step but doesn’t provide much context on why it wants to run them. You can read them to ensure they are safe, but it’s hard to assess whether the agent is on the right path — something that in my experience is pretty critical to getting good results.

Finishing up with the worktree

I decided I liked what claude-code built better, so now let’s commit in the worktree.

git add -A
git commit -m "Implement generic CLI to add endpoints"

and now let’s get the changes back into the main branch.

First, we go to the main branch in the original project folder. Clarify we’re in the right spot by running

❯ git branch
+ claude-code
* main
+ openai-codex

Now, we can merge the changes from the worktree.

git merge claude-code

and finally we can delete the worktree.

From the project folder:

git worktree remove claude-code

and cleanup the branch as well

git branch -d claude-code

Takeaways

Git worktrees provide a straightforward primitive to working on multiple independent tasks within a project at the same time using agents. I don’t feel like my work here actually benefitted from using worktrees mostly because the agents didn’t actually take that much time to write the code relative to me figuring out how to make worktrees. That said, with a larger task or test suite that I could point the agent at as a goal, I could see worktrees being highly useful for longer agent runs, if you set up and task scope allow for it.

Thanks for reading via RSS!

Create a Link Blog

Wed, 07 May 2025 01:05:24 GMT

RSS feeds for blogs and things you write or create are great. If you read a lot, you probably also have a lot of articles you’ve read that you share with others and occasionally revisit.

You can save and share these with very little effort! Doing so is immensely valuable. If I find your blog and like what you write, often I will also like what you like to read.

I use Raindrop which makes it easy to save and share articles.

Raindrop creates a browser friendly page and an XML feed.

I also wrote up a little page to display some of the more recently added links to the feed on my site.

If you write a link blog, share it! I’ll probably add it to my feed reader.

Thanks for reading via RSS!

Alfred as a Text Editor

Sat, 03 May 2025 01:01:07 GMT

I’ve mentioned my use of Alfred over on /uses, but I just noticed one unusual use of it that I don’t think I could have even listed if someone asked me how I use Alfred and why I like it.

I use Alfred as a text editor to quickly modify and chop up text.

The workflow is usually something like cmd + space to bring up Alfred, cmd + v to paste my clipboard, make some edits, and then cmd + a, cmd + c to copy the text. This reads as complicated, but there’s almost nothing faster to conjure on my machine than the Alfred window.

I usually just do this without thinking. Yet somehow, I’ve never been able to learn vim 🙃.

Thanks for reading via RSS!

LLM Chat UI Concepts

Mon, 14 Apr 2025 00:22:24 GMT

To write this post, I was going to take myself through some of the history of different chat interfaces. This is not that post. I was too impatient and decided to go in without any appreciation for prior art (beyond what I’m already aware of), because it seemed more fun at the time.

This site’s current implementation

My current UI for displaying an LLM conversation on this site looks like this. I’m going to assign identifiers to each of these UI concepts to make them easier to reference between descriptions.

User

What’s the best way to learn programming?

claude-3-7-sonnet

The best way to learn programming is by building projects that interest you. Start small, be consistent, and don’t be afraid to make mistakes. Reading documentation and tutorials is helpful, but nothing beats hands-on experience.

User

How much time should I spend coding each day?

claude-3-7-sonnet

Even 30 minutes of focused coding daily is better than occasional marathon sessions. Consistency is key. As you build the habit, you can gradually increase your time. Remember that regular practice, problem-solving, and reflection on what you’ve learned will help you progress steadily.

Top Bars

Here is a version with the “user” or the assistant names in top bars.

User

What’s the best way to learn programming?

claude-3-7-sonnet

User

How much time should I spend coding each day?

claude-3-7-sonnet

Continuous

Here’s another with the vertical gaps from Top Bars removed and the UI visually continuous with each message in a different color.

User

What’s the best way to learn programming?

claude-3-7-sonnet

User

How much time should I spend coding each day?

claude-3-7-sonnet

Columns

Let’s sort of rotate Continuous 90 degrees to the left then add the vertical labels turned 90 degrees left as well. The chat messages should each exist in a column to the left of their label. They are placed in order, next to each other from left to right. The message texts run top to bottom in each column.

User

What’s the best way to learn programming?

claude-3-7-sonnet

User

How much time should I spend coding each day?

claude-3-7-sonnet

Not sure I really like it but it’s different from most things I’ve seen.

Assistant Focus

Here’s a version where the assistant’s messages are shown prominently and the user’s message only appears on mouse over of a particular assistant message. On mouse over, the element animates to reveal the user message for a given assistant message. This reads more like a write up you’d create as or after you’ve done a deep dive into a topic. The user questions elicit a sort of winding narrative response from the model.

claude-3-7-sonnet

User

What’s the best way to learn programming?

claude-3-7-sonnet

User

How much time should I spend coding each day?

LLM-designed

Most of the above concepts are from my head, implemented by prompting an LLM. I was curious what would happen if I gave over some of the style decision making to Claude and Cursor. Here are a few of the results.

Code Editor

conversation.chat

⚙️

// Conversation between User and claude-3-7-sonnet

// Started: 2026-06-16T13:48:07.805Z

👤 User

[-]

What’s the best way to learn programming?

🤖 claude-3-7-sonnet

[-]

👤 User

[-]

How much time should I spend coding each day?

🤖 claude-3-7-sonnet

[-]

Claude made that one, when shown all my examples and prompted to create its own with no specific instructions as to how. It wanted the development to be in a comfortable and familiar environment.

Notebook

claude-3-7-sonnet ● Active

In [1]:

01:36 PM

                                                What's the best way to learn programming?

Out[1]:

01:39 PM

In [3]:

01:42 PM

                                                How much time should I spend coding each day?

Out[3]:

01:45 PM

Claude made this when prompted to create its own novel approach that should add value for the reader.

Timeline

01:36 PM

User

What’s the best way to learn programming?

01:39 PM

claude-3-7-sonnet

01:42 PM

User

How much time should I spend coding each day?

01:45 PM

claude-3-7-sonnet

Claude made this when prompted to create its own novel approach that should add value for the reader and still be minimal and clear.

Of all the concepts Claude implemented, I liked the Timeline the best. Most of the time, the model seemed to focus more on the styling of the UI than the specific presentation of the data. I wanted to push the agent to focus on clear and simple data presentation.

User

Take a look at @ChatAssistantFocus.astro @ChatColumns.astro @ChatTimeline.astro @ChatColumnsTopBar.astro @ChatContinuous.astro . These are all different React components for a chat. Carefully consider your strategy and come up with a simple, creative way to present chat data to a user in a surprising, but straightforward and effective way. Less is more. Write your results to @components then add the component to @index.mdx

claude-3-7-sonnet

The bubble stream presents messages in a familiar chat interface, but with a few key improvements: bubbles size themselves based on content length, have a subtle hover effect, and work with keyboard navigation for accessibility.

Bubble Stream

What’s the best way to learn programming?

How much time should I spend coding each day?

Takeaways

This experiment reinforced my sense that if I outsource parts of design or implementation that I care about to a model, I often don’t really like the results. An LLM can’t quite understand what I personally mean by “clean”, “minimal”, and “novel”. In practice these are directional concepts that mean something different to everyone. To get a model to output a design that I like, I need to provide more specifics for the model to follow.

In the case of the first few designs that I described and the model implemented, the results are quite representative of my vision. The model is good at following instructions. Its taste is not always aligned with mine.

Thanks for reading via RSS!

RSS Club

Sat, 12 Apr 2025 02:02:31 GMT

This is an RSS-only post, an idea I thought I came up with but eventually realized has many existing implementations. This post isn’t linked anywhere on my site or in the sitemap. However, you’re probably seeing it because you’re reading my RSS feed. It does exist on the site as well though.

As I personally move toward self-curated content and away from algorithm feeds, I’m hoping to connect with more folks who have been doing the same. If this is something you do or are interested in, feel free to reach out! I’d love to learn more about what you’re doing and why.

I also maintain a blog roll of sorts. Occasionally, I’ll include more traditional news articles and Youtube videos in addition to blogs. It’s sort of a breadcrumb trail for myself, a spot to collect links that I frequently end up recommending to friends, and recognizes work I found valuable to my own exploration and learning.

Another thing

I use Oxford commas now. For a long time I didn’t for a reason I can’t remember anymore. It always felt a little wrong not to use them but I ignored that dissonance for the sake of consistency I suppose.

Since seeing several written examples where a lack of an Oxford comma creates an ambiguous sentence, I decided I’d adopt using them. Between spending a lot of time prompting LLMs, then coming up with analogies to explain how and why I think they are useful, I’ve become more interested with techniques that can help with clear communication.

This change seems philosophically in line with a similar change I made more than a decade ago to start crossing my zs and 7s to make them easier to identify when doing handwritten math for physics courses.

Thanks for reading via RSS!

Models as Force Multipliers and their Limits in Software Development

Tue, 08 Apr 2025 16:29:01 GMT

I’m on a flight and wanted to write code to work on an idea. After a few moments of shifting mental gears, I popped open Zed, which allows me to code with a local LLM using ollama. My default impulse when writing code is to prompt a model. At first, I felt somewhat negative about this but with llama3.2:3b at my side, eventually I got into the rhythm of working through the idea and mostly writing the code by hand which I don’t do as often these days.

This experience of not having a good model available made me reflect for the first time in a while — how am I even thinking about when I use a LLM today to write some bit of code or functionality in a project I am working on? How do I reason about whether the LLM is up to the task and how much am I filling in the model’s gaps without realizing it?

llama3.2 feels like going back in time

One of the first things I was confronted with was how mediocre llama3.2 was as a code assistant compared to Claude Sonnet, my current daily driver. It’s not that llama3.2 can’t write code that fulfills my prompts — it mostly can — it’s that it can’t fluidly edit code inline while following prompt instructions and regenerating the specific parts of the file that need changes to comply with the instructions.

Here is an example

User

write python code to create a markdown file with created_at as frontmatter. use the stdlib only. no comments.

llama3.2

import json
from datetime import datetime

def create_markdown_file(file_name, created_at):
data = {
"created_at": created_at.strftime('%Y-%m-%d %H:%M:%S')
}
with open(f'{file_name}.md', 'w') as f:
json.dump(data, f)
f.write('\n---\n')

# create a markdown file

create_markdown_file('example', datetime.now())

This code creates an example.md containing

## {"created_at": "2025-04-08 12:45:00"}

Maybe not exactly what I had in mind but gets enough of the way there that the output with minor modifications will do what I want. This reminds me of the days I used to use ChatGPT to help write code. I would toss in a prompt like the one above and get code that was a helpful starting point for the direction I was headed in.

These small models don’t work very well as editors of source code or agents because they are not consistent enough at following instructions. They are best used for chat. Using inline editor prompting (cmd+I by default in Zed) often results in strange artifacts getting added to the file and that model often does not follow instructions well. Nevertheless, chat can still go a long way.

Building software feels different now

If you’ve been following state of the art models and agents, you’re aware that these days you can push both the model alone, as well as a tool-calling agents, to build and iterate on what used to be nontrivial sized software projects (x,000 lines of code). The state of the art has pushed us so far that people can now build proof-of-concept quality, functioning software without ever directly touching code themselves. There are still plenty of challenges but this was not possible a year ago.

These capabilities are pretty incredible. I use them. They have a ceiling, but it’s one that is hard to articulate. For someone hoping to build a simple UI and deploy to Vercel, maybe this ceiling is irrelevant. Especially if they’re not familiar with code, they may not have a concept of the ceiling of the model/agent’s capabilities. They just know the agent did what they asked it to do.

Different types of software have different burdens of maintenance

Software systems don’t need to be incredibly large or complex to be useful. Plenty of software becomes less useful as it grows, bloated with features meant to extract value rather than create it. At the same time, as a codebase grows, issues of maintainability and reliability become more relevant.

If I create a registration system for a summer camp that all of a sudden breaks when I need it to be working most, that is a problem I need to solve myself if I wrote the software. If you don’t know how to write code by hand, maybe you could use a model to build this registration system today, but could you use a model to fix the system when it breaks? Most of the “life” of software is spent being maintained, not written. Most engineering jobs prioritize keeping the existing system available above any new changes or improvements.

These maintenance needs are invisible to the first-time software author (read vibe coder) until they’re obvious.

The system breaks.

Someone compromised your system’s data.

The system is overwhelmed by traffic.

These are realities of software in the world. Models are even useful for solving these problems. No perfect system exists — it’s all about making tradeoffs within the constraints.

Models are force multipliers

Right now we’re in a period of adjustment. I’m still surprised by how much code I can write with simple instructions and how much progress I can make on a project in fixed time relative to what has previous been possible. But many of the realities of building software that needs to run reliably and be secure have not changed. If I am still the responsible party for handling and being accountable for issues with software, then models are simply tools for me.

It would be ill advised to generate and merge code for a system I was oncall for without carefully reading and testing that code. It’s much easier to deal with any problems with that code before it makes it to production. Knowing whether things work requires verification, by a human, tests, or trusted system.

Models are currently a force multiplier. With a powerful model, I can build a ChatGPT clone in Swift in days or weeks instead of weeks or months. I can get code in any language to solve a simple problem like formatting a date as yyyy-mm-dd in a few seconds instead of minutes a very high percentage of the time.

Such a force multiplier empowers a larger number of people to realize a vision they have in less time and with fewer resources. It empowers people to use their existing skills in different ways. It makes the computer a more powerful tool for more people.

Thanks for reading via RSS!

Vibe Coding in early 2025

Sun, 06 Apr 2025 13:50:16 GMT

This post is an edit and repost of my rant from Bluesky

Some problems with vibe coding

Having done a lot of vibe coding lately, I think I’ll move away from it (for now) as a primary approach to build any software that I care about, even a little bit. Current agents eventually fail to adhere to some prompt despite various attempts and approaches. Whenever this happens and I look in the codebase I am usually mortified by what I find.

You might argue I need to use rules better or something like that. I would say that isn’t vibe coding as I understand it. I interpret vibe coding as the agent addressing whatever whim I have in the realm of the possible, figuring it out for me. Mostly, I end up with spaghetti codebases that the agent cannot untangle and that I don’t want to untangle.

Agents have a strong tendency to start generating code. Give one a task and it is off to the races. In a new project, this approach isn’t the worst thing. In a project with existing functionality, this becomes a problem due to a combination of lack of context and lack of effective planning. The two go hand in hand.

Agents use search to look around a codebase to find relevant files to pull into context. You can also use rules, documentation, or READMEs to define how an agent should structure the codebase. At some point, often unbeknownst to you, the agent fails to pull a relevant file into the context window or misses reading a rule it should follow.

Initially, this oversight doesn’t manifest as a problem. The agent does what it does best: writes some code to solve the problem. No shared states available in the React app? No problem, it’ll write some hooks for you. You validate the behavior in the browser. Looks good? Commit and ship. You can get away with doing this and never reading the code…for a while.

Now the codebase has fragmented state. You come back to the codebase 15 commits later. The application has grown considerably — agents are good at a lot of code. You prompt the agent to make more changes using the state and it fails. Try some follow up prompts, debug, add MCPs to pull in more context. Why isn’t it working?

Finally you crack open the code. What are you even looking at? Probably something that feels like thousands of lines of a legacy codebase, even if you wrote the whole thing that week. You can use the model to find which code deals with the problem you currently have. You’ll probably find that state is everywhere. State in hooks. State in redux. State in zustand. You really should have written more rules to define your codebase conventions.

But it’s too late. Even if you had reasonable codebase conventions, for every convention rule you do write there will be one the model eventually misses. The agent will invisibly write working code that makes decisions on your behalf, filling in under specified requirements, and eventually you will have a tangled mess of a codebase.

The code the agent writes will work and appear to meet your requirements, but if you don’t read or at least skim the code (and have some idea what you’re looking at), you’re not going to realize when the model deviates in its approach from what you consider reasonable.

This deviation becomes a problem only later when you realize it’s happened but at that point the agent has made more poor decisions based on this initial one.

With current agents, I don’t believe all of these problems can be anticipated. I don’t believe you can rule-write your way out of them because you rarely know what the X0,000 line codebase should look like when it’s only X,000 lines. You can write code that remains flexible and is refactorable, but today, you can only rely on yourself to know if the agent has deviated from the rules you wrote. Agents don’t follow instructions consistently enough to be expected to follow them without verification of the code itself.

When I notice an agent isn’t following instructions as I’d hoped, I could evaluate the plan and pull in more context in case the agent missed something. To be able to do a good job of that, I need to know the codebase, at least architecturally. I need to know how state is managed, what libraries are used, how the file hierarchy is organized.

I need to know these things in case the model misses something when generating code, so I can step in and make corrections or at least steer it back in the right direction. If I only find out the model has been doing something like weird state duplication and synchronization after the agent finally stops being able to make progress, it’s going to be painful to get back on track.

The agent cannot be prompted to code itself out of the hole it digs. I’ve tried. I’ve told the model to identify duplication and refactor. I’ve read the code and identified the points of duplication then told the model where and prompted for a refactor. State of the art agents fail.

So where do I go from here? I’ve actually had immense success writing code using agents and models with my hands on the wheel. For production code, this has always been the only viable option. But for personal projects, I now appreciate the ceiling vibe coding places upon them.

I can vibe code a fun game but eventually, I may not be able to add more features due to the unmaintainable sprawl the model eventually creates.

There is a small cost here. I have to be a touch more thoughtful as I build something to ensure the agent is staying on the rails.

In doing so I understand how my project works, learn new things as I collaborate with the agent, and keep the code in a state where it can continue to be extended. I’ll definitely progress more slowly than fully embracing the vibes but I won’t fly off the rails either.

For now, I’m out on pure vibe coding but I’ll still be using models and agents often to help me write code.

Reply on Bluesky

Thanks for reading via RSS!

Goose as a Task Runner

Sat, 01 Mar 2025 02:41:27 GMT

Goose is a CLI language model-based agent. Goose exposes a chat interface and uses tool calling (mostly to invoke shell commands) to accomplish the objective prompted by the user. These tasks can include everything from writing code to running tests to converting a folder full of mov files to mp4s. Most things you can do with your computer, Goose can do with your computer.

Goose runs most commands by default. Some other tools call this “YOLO mode”. If this approach concerns you, you may want to prompt Goose to let you know before it runs commands (this approach does not substitute for running the tool in an isolated/containerized environment). You ultimately never know what an LLM-based tool will actually run when given access to your system.

Goose hints

When you start Goose, it will check for a .goosehints file in the current directory (docs). If it finds one, it will use the hints in that file to guide its behavior. The instructions will be appended to the system prompt that Goose is provided.

Here’s an example file:

Always speak like a pirate.

Now, when can start Goose and say “hi”, it will read the file and respond to my prompt.

❯ goose s
starting session | provider: anthropic model: claude-3-7-sonnet-latest
    logging to /Users/danielcorin/.local/share/goose/sessions/Kt3w4dIH.jsonl

Goose is running! Enter your instructions, or try asking what goose can do.

( O)> hey
# Ahoy there, matey!

'Tis a fine day to be sailin' the digital seas! I be Goo
se, yer AI assistant, ready to help ye with whatever tas
k ye need.

What can I be doin' for ye today? Be it code to write, f
iles to edit, or commands to run on yer macOS vessel, I
be at yer service!

Just give me a headin', and we'll set sail together on t
his technological adventure.

If ye need to know what commands be at yer disposal, ye
can use:
- `/help` or `/?` to see available slash commands
- `/exit` or `/quit` to end our voyage
- `/t` to toggle between Light/Dark/Ansi themes

What say ye, captain? What course shall we chart?

Goose as a task runner

Because a .goosehints file can contain any natural language instructions, we can also use it to define useful tasks that might benefit from a language model-based agent execution flow, like idea generation, summarization, or structured data extraction.

Here’s an example of a .goosehints file that uses Goose to generate summaries of content from my blog. It provides the description of a hypothetical summary command and a few hypothetical options for creating summaries across my post types and over different time periods. It also prompts for citations from the original content from which the summaries are derived.

I’m emphasizing the hypothetical part because we’re not actually going to implement this /summary command with code. We’re going to have Goose use this specification as instructions which it will carry out, allowing the language model to run commands to do it.

Here’s the file:

This repository is a the Hugo blog for my personal website, danielcorin.com.
It's Github repository is available at https://github.com/danielcorin/blog.

The `Makefile` contains useful commands for managing the blog.

You should also support the following commands:

## `/summary`

This command summarizes the posts for a given type and period.

For example,

```
/summary logs 2020-02
```

This will summarize all the posts in the `logs` folder for the month of February 2020.

```
/summary all -3m
```

This will summarize all posts of all types in the last 3 months, including `posts`, `til`, `logs`, `projects` and `garden`.
This command also supports weeks and years:

```
/summary all -3w
/summary all -1y
```

The outputted summary should contain Hugo links to the posts from which the summary is derived.
Then can be provided inline.
For example,

```md
... [2025-02-28](logs/2025/02/28) ...
```

or

```md
I [learned more about Astro](til/astro/intro) and build a new website ...
```

No need to reference the dates directly.
Weave the links into the prose of the summary.

The summary will be outputted in `content/summary/<type>/<period>.md`.

Now, I can run Goose and generate a summary of my logs posts in February 2025 like this:

goose s
( O)> /summary logs 2025-02

Goose first reads the .goosehints file and then plans and executes a series of shell commands to find the appropriate files, load them into context, generate the summary, and write it to a file in the location and with the naming convention specified.

After running the command above, Goose wrote a file out to content/summary/logs/2025-02.md with a pretty reasonable summary.

That’s a lot of tokens

While the above command works well in my experience, it can use a lot more tokens than is really necessary as Goose navigates the filesystem, builds a context window, generates the summary, and writes it to a file. Most of these steps don’t need to be executed by Goose. We can write a deterministic script that builds the context from the types of posts and window of time we specify, then just prompt an LLM to summarize.

I had the LLM write a script for this. Compared to the ~50 lines in my sparse .goosehints file, the script is almost 300 lines of Python.

This is obviously not an apples-to-apples comparison. The script is deterministic (except for the LLM summary), far more efficient, and less prone to random failures of the LLM agent. However, the .goosehints file is more flexible, easier to understand, and easier to read.

I believe there is room and value in both approaches.

Language vs Code

The ability to write a quick instruction set and have an agent execute it in steps is powerful and distinctly its own thing compared to what long-time engineers are used to. Once an idea has been translated into code, edges need to be dealt with, compilers and interpreters satisfied. Code necessarily imposes limiting constraints on the developer. We want and need our programs to be correct. To accomplish this, we need to handle failure modes and edge cases.

Natural language is more flexible and squishy compared to code. There is more room for interpretation at inference time. Any validation (like does the program compile/run) is done at runtime. Natural language allows for faster iteration but, in practice, can produce unpredictable results.

While (orders of magnitude more) inefficient compared to the typical way you’d perform this task, I was still impressed with the results of the /summary command from my .goosehints file. The execution became an implementation detail dealt with by the model once the spec was written. The prose and documentation are clearer than the Python and shell equivalents I wrote, especially for a non-technical reader. It’s also trivial to extend - I just write the document of the new feature and Goose and the model do the rest.

Goose and other LLM-based tools make it easy and fast to extend a script version of the summary command, but the way you do that is by writing the spec or description of what you want. This is all you do when creating a “command” in a .goosehints file.

You could argue that natural language instructions, executed by an agent, are like an interpreted language whereas an already-written script is closer to a compiled language because the language tools and runtime impose constraints and invariants on the behavior of the program. It’s possible to reason about what the behavior of the latter will be at runtime, but the former can still be useful.

Thanks for reading via RSS!

Adding an llms.txt file to Hugo

Tue, 25 Feb 2025 22:02:00 GMT

Today, I set out to add an llms.txt to this site. I’ve made a few similar additions in the past with raw post markdown files and a search index. Every time I try and change something with outputFormats in Hugo, I forget one of the steps, so in writing this up, finally I’ll have it for next time.

Steps

First, I added a new output format in my config.toml file:

[outputFormats.TXT]
mediaType = "text/plain"
baseName = "llms"
isPlainText = true

Then, I added this format to my home outputs:

[outputs]
home = ["HTML", "RSS", "JSON", "SearchIndex", "TXT"]

Finally, I created a template file at layouts/_default/index.txt that renders my site content in a structured markdown per the spec recommendations.

# {{ .Site.Title }}

> {{ .Site.Params.description }}

## Content
{{ range $type := .Site.Params.front_page_content }}
### {{ title $type }}

{{ range (where $.Site.RegularPages "Type" $type) }}- [{{ .Title }}]({{ .Permalink }}index.md): Published {{ .Date.Format "2006-01-02" }}
{{ end }}{{ end }}

Now, when I build my site, it generates an llms.txt file at the root that contains a Markdown list of the content on this site. This makes it easy for language models to understand my site without dealing with HTML markup.

Thanks for reading via RSS!

Claude Code

Mon, 24 Feb 2025 17:48:25 GMT

Today, Anthropic entered the LLM code tools party with Claude Code.

Coding with LLMs is one of my favorite activities these days, so I’m excited to give it a shot. As a CLI tool, it seems most similar to aider and goose, at least of the projects I am familiar with.

Be forewarned, agentic coding tools like Claude Code use a lot of tokens which are not free. Monitor your usage carefully as you use it or know you may spend more than you expect.

I didn’t read the docs. I am just going to dive in and see how it goes.

Let’s go

First, I installed the Claude code with

npm install -g @anthropic-ai/claude-code

then ran it with

claude

After setup, I ran /init. This seemed to analyze my project and create a CLAUDE.md file with the technologies the project uses, how to set it up, how to run tests and other helpful context for Claude. This approach aligns with the most effective approach I am aware of for coding with LLMs, involving the creation of supplementary rules or markdown files and compressing an understanding of a codebase into fewer tokens. This approach also makes it much easier to reference metadata about the codebase in the future.

I’m interested to see how or if Claude Code will keep this file up to date. In the future, rather than reading a bunch of files in the codebase Claude (or some other tool) can reference this and learn how to make changes in alignment with the current conventions of the codebase in a token efficient way.

Slash commands

After creating the CLAUDE.md file, the first thing that piqued my interest was the slash commands. Type / and Claude Code shows all the commands available. Backspace and they autoclear. It’s a very nice and clear CLI tool.

Bash Mode

There is also “Bash Mode” which seems to be a direct passthrough to run commands in your shell. These don’t appear to be stored in the context, though you can instruct Claude Code to run shell commands on your behalf and the results will be stored in the context window. Claude Code prompts you when it is going to run a shell command. Bash mode is fine for quick commands but I definitely will need a proper terminal in parallel and it doesn’t deal super nicely with things that require a pager or colors.

How it compares to existing tools

Compared to goose this feels a bit safer. My read on what is happening is Claude Code is considering things like searching and reading files to be default-safe and doesn’t ask for permission, but if you prompt it to list the files in the current directory, it asks if it’s ok to run ls.

In contrast, goose runs a bunch of shell commands to read files and do other things — by my current understanding, goose doesn’t distinguish safe vs. unsafe actions in the same manner. Overall, there are still a lot of similarities between the two.

Claude Code default-proposes changes rather than just making them by default. It shows one file change at a time and prompt for your approval to make each change. It’s a little unusual to not see the whole changeset at once but it seems to work ok.

I prompted to make several, then asked it to undo the first change I asked for only. This can sometimes get a bit messy and it wasn’t entirely transparent about how it did it, but Claude Code proposed file edits and it seemed to undo the changes correctly which was impressive.

Final thoughts

It’s hard to exactly compare the capabilities against other tools because claude-3-7-sonnet-20250219 just came out and I haven’t had the chance to try all these other coding with this new model yet. Nevertheless, Claude Code seems to be a highly competent agentic coding tool and is a welcome addition to the kitchen sink of things I can throw at coding problems. These days I run Cursor with a CLI LLM agent tool in the embedded terminal. Often, goose, but now maybe claude as well.

A bit ridiculous but also leading edge?

It’s also quite interesting and maybe a bit telling to see Anthropic as one of the leading creators of LLMs, investing resources in a closed-source coding tool as well. I wonder what that says about where they expect the future of LLMs to be captured. My read is they think it’s in the application layer. Today, the best coding tools are using Anthropic’s models. As the gatekeepers of Claude, Anthropic’s coding tools can be the best (assuming they continue to lead here) if they keep their best models for themselves.

Thanks for reading via RSS!

Cursor Triple Backticks Stop Sequence

Mon, 17 Feb 2025 20:34:22 GMT

An LLM stop sequence is a sequence of tokens that tells the LLM to stop generating text. I previously wrote about stop sequences and prefilling responses with Claude.

As a reference, here’s how to use a stop sequence with the OpenAI API in Python

from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
  model="gpt-4o",
  messages=[{"role": "user", "content": "What is the capital of France?"}],
  stop=["Paris"],
)
print(response.choices[0].message.content)

which outputs something like

'The capital of France is '

Notice the LLM never outputs the word “Paris”. This is due to the stop sequence.

Cursor also utilizes stop sequences. The most noticeable way in which it does this for me is when attempting to generate content in markdown files containing code blocks.

In a blog post, they look like this

def hello_world():
    print("Hello, World!")

but when I am editing these posts they look like this

```python
def hello_world():
    print("Hello, World!")
```

I occasionally use an LLM to restructure these markdown files, but because Cursor uses triple-backticks as a stop sequence, it will stop generating text at the first triple-backtick it encounters. These are all over my posts, so it frequently causes me problems.

You might be wondering why I use Cursor to write blog posts. Since I write my posts in markdown and using Hugo, a code text editor is a solid starting point.

I can use Hugo Live Preview to see how my post looks as I build it. I can use Cursor’s features to quickly generate examples, restructure my prose and chat with the LLM to play with ideas for examples. It’s quite an interactive and enjoyable way to write. It allows me to focus on the thing I want to write about and removes most of the other obstacles.

I like the setup.

But this triple-backtick stop sequence thing is a pain because anytime I want to make a ⌘+K change across an entire file, I inevitably end up with a prematurely stopped generation.

One workaround I’ve used in a pinch is to give the instructions

fix any typos with minimal changes. replace any triple-backticks with ^^^

Cursor (the model, really) follows those instructions, fully reprocesses the selection from the file and then I manually string replace to get back the triple-backticks.

I could use Hugo’s highlight shortcode, but in an effort to keep my markdown as portable as possible, I’ve been avoiding that. I also find the backticks to be cleaner.

Maybe one day I’ll find a workaround that let’s me set a custom stop sequence for specific file types in Cursor.

Thanks for reading via RSS!

Developing a Mental Model for using Models

Fri, 14 Feb 2025 17:29:28 GMT

I had an interesting realization today while doing a demo building a web app with Cursor. I was debugging an issue with an MCP server, trying to connect it to Cursor’s MCP integration. The code I was using was buggy, and I’d never tried this before (attempting it live was probably a fool’s errand to begin with).

When I ran into issues, someone watching asked, “Why don’t you just ask the Cursor chat what’s wrong?” This didn’t occur to me because I instinctively figured that Cursor chat (and Claude, the model powering it) wouldn’t know what was happening.

This experience crystallized something important for me: when using AI products and models, we develop mental models of what these systems have available to them about the state of our computing environment and the world.

Raw Models vs. Product-Integrated Models

Take gpt-4o as a “raw” model that you use via an API or OpenAI’s Playground. It has:

Its training data and the way OpenAI used that data to train weights in the model’s architecture
The prompts you send as context
The model’s own responses (in the case of a multi-turn conversation)
That’s it (as far as I know)

However, when you use AI products (like Perplexity, ChatGPT with search, or claude.ai), you’re working with:

The base chat model
System prompts and instructions
Additional context (today’s date, internet search results, search results from private knowledge bases or datastores)
A wide variety of non-standardized, product-specific features

This distinction matters because ChatGPT with search can give you real-time answers about the world, while a raw model like gpt-4o only “knows” what is in its training data.

Why This Matters

This distinction of what a model knows isn’t obvious to many people using model-based products, especially those less familiar with how the models work. Having your own mental model of what the language model has available to it is foundational to getting good at using these tools. It helps you:

Discern what is within the model’s or product’s capabilities
Develop intuition for how to use these tools effectively
Know when a tool might be able to give you the right answer compared to when it lacks the context to do so (e.g. Claude doesn’t know which team won the Super Bowl in 2025, Perplexity does)

This intuition is a soft skill and the ground is constantly shifting as these products are augmenting their capabilities. Scaling this learning curve is what matters right now when it comes to augmenting your skills with AI.

A Real-World Example

In my case with Cursor, I intuitively suspected (though wasn’t 100% certain) that the Cursor Composer agent wouldn’t know how the IDE was trying to make that MCP call to the local server I had running.

I didn’t explicitly think this through - it was just intuition developed through experience. When I went to check if Cursor knew about the MCP settings, briefly, a part of me then suspected that it actually might and that I’d been wrong in my assumption.

This is where being “up to date” on the capabilities of these tools matters. The challenge here is this a moving target and not realistic for people not spending tens of hours per week using “AI tools”. I barely can keep up and I’m working with this stuff everyday.

It turns out Cursor can’t check or modify its own configuration (yet), but this is realistically something that could be incorporated into the product in the near future. In fact, it seems likely it will be.

The Challenge

The importance of understanding what different language-model-based products have available in their context window may not be obvious, but it’s crucial for developing an intuition about when and how these tools can be effectively applied.

This is the first step to building your intuition for picking the right tools for your task.

Making headway

When experimenting with a new tool or product, kick the tires. Ask the model about things that happened recently. Ask the model about itself. Sometimes models will make things up, but once you start poking around, you begin the process of developing this critical intuition for getting the most of these tools - developing your mental model for using models.

Thanks for reading via RSS!

Astro Code Toggle Component

Thu, 06 Feb 2025 08:43:51 GMT

I built an Astro component called CodeToggle.astro for my experimental site. The idea was to create a simple wrapper around a React (or other interactive component) in an MDX file so that the source of that rendered component could be nicely displayed as a highlighted code block on the click of a toggle. Usage looks like this:

import { default as TailwindCalendarV1 } from './components/TailwindCalendar.v1';
import TailwindCalendarV1Source from './components/TailwindCalendar.v1?raw';

<CodeToggle source={TailwindCalendarV1Source}>
  <TailwindCalendarV1 client:load />
</CodeToggle>;

The implementation of CodeToggle.astro looked like this

---
import { Code } from "astro/components";
import { Code as CodeIcon } from "lucide-react";

interface Props {
  source: string;
  lang?: string;
  children: astroHTML.JSX.Element;
}

const { source, lang = "tsx" } = Astro.props;
---

<div class="relative">
  <div class="mb-4">
    <slot />
  </div>

  <div class="not-prose">
    <details class="group">
      <summary
        class="flex items-center gap-2 font-mono text-xs px-2 py-1 rounded-md
            bg-[var(--color-bg-code)] text-[var(--color-ink-light)] opacity-80
            hover:opacity-100 hover:text-[var(--color-ink)]
            cursor-pointer transition-all duration-200 w-fit"
      >
        <CodeIcon className="w-3 h-3" />
        <span class="select-none group-open:hidden">Show Source</span>
        <span class="select-none hidden group-open:block">Hide Source</span>
      </summary>
      <div class="mt-3 rounded-md overflow-hidden">
        <Code code={source} lang={lang as any} theme="monokai" />
      </div>
    </details>
  </div>
</div>

This approach was relatively straightforward and I thought I was using the <Code> component in a sensible way. I actually published my first post for the site using this component and thought things were going well.

Right after I attempted to publish a new post, I started running into strange build issues

02:37:59   ├─ /notes/2025/llm-tailwind-react/index.html
highlighter.codeToHtml is not a function
  Hint:
    This issue often occurs when your MDX component encounters runtime errors.
  Stack trace:
    at file:///vercel/path0/dist/chunks/index_C97_OQzq.mjs:69:34
Error: Command "npm run build" exited with 1

I reverted a few commits locally, but the problem persisted.

From some debugging, it seemed the issue stemmed from my attempts to use shiki, a syntax highlighting package, in multiple ways. Why this all of a sudden became a problem, I was still unsure.

After some more poking around, searching GitHub and experimenting with different LLM outputs, I came up with these changes:

diff --git a/src/components/prose/CodeToggle.astro b/src/components/prose/CodeToggle.astro
index 88a758d..166c324 100644
--- a/src/components/prose/CodeToggle.astro
+++ b/src/components/prose/CodeToggle.astro
@@ -1,6 +1,7 @@
 ---
-import { Code } from "astro/components";
 import { Code as CodeIcon } from "lucide-react";
+import { createHighlighter } from "shiki";
+import type { BundledLanguage } from "shiki";

 interface Props {
   source: string;
@@ -9,6 +10,16 @@ interface Props {
 }

 const { source, lang = "tsx" } = Astro.props;
+
+const highlighter = await createHighlighter({
+  themes: ["monokai"],
+  langs: [lang],
+});
+
+const html = highlighter.codeToHtml(source, {
+  lang: lang as BundledLanguage,
+  theme: "monokai",
+});
 ---

 <div class="relative">
@@ -28,9 +39,7 @@ const { source, lang = "tsx" } = Astro.props;
         <span class="select-none group-open:hidden">Show Source</span>
         <span class="select-none hidden group-open:block">Hide Source</span>
       </summary>
-      <div class="mt-3 rounded-md overflow-hidden">
-        <Code code={source} lang={lang as any} theme="monokai" />
-      </div>
+      <div class="mt-3 rounded-md overflow-hidden" set:html={html} />
     </details>
   </div>
 </div>

With that approach, the build issues resolved. I can’t say I quite understand why I started having this issue but hopefully this post helps anyone who runs into the same.

Thanks for reading via RSS!

Document Citations

Sun, 02 Feb 2025 03:39:49 GMT

The following is an implementation of document Q&A with citations using Anthropic’s press release for Claude Citations, a feature meant to reduce LLM hallucinations and provide clarity for which part of the source text the model is using to produce answers to questions. The content below the next page break is from Anthropic (the source document in our example). Below are example questions that can be asked about the document. When provided with the document as context, an LLM can generate answers to these questions.

Try it out

Select a question to view the model-generated answer using the source document. Doing so will also highlight the areas of the document that Claude returned as citations, justifying the answer. If you scroll to the bottom of an answer, you can also find links to the cited passages.

Introducing Citations on the Anthropic API

Today, we’re launching Citations, a new API feature that lets Claude ground its answers in source documents. Claude can now provide detailed references to the exact sentences and passages it uses to generate responses, leading to more verifiable, trustworthy outputs.

Citations is generally available on the Anthropic API and Google Cloud’s Vertex AI.

Trust by verification

All Claude models are trained to be trustworthy and steerable by design. Citations builds upon this foundation, addressing a specific need in AI applications: verifying the sources behind AI-generated responses.

Previously, developers relied on complex prompts that instruct Claude to include source information, often resulting in inconsistent performance and significant time investment in prompt engineering and testing. With Citations, users can now add source documents to the context window, and when querying the model, Claude automatically cites claims in its output that are inferred from those sources.

Our internal evaluations show that Claude’s built-in citation capabilities outperform most custom implementations, increasing recall accuracy by up to 15%.1

Use cases

With Citations, developers can create AI solutions that offer enhanced accountability across use cases like:

Document summarization: Generate concise summaries of long documents, like case files, with each key point linked back to its original source.
Complex Q&A: Provide detailed answers to user queries across a large corpus of documents, like financial statements, with each response element traced back to specific sections of relevant texts.
Customer support: Create support systems that can answer complex queries by referencing multiple product manuals, FAQs, and support tickets, always citing the exact source of information.

How it works

When Citations is enabled, the API processes user-provided source documents (PDF documents and plain text files) by chunking them into sentences. These chunked sentences, along with user-provided context, are then passed to the model with the user’s query. Alternatively, users can provide their own chunks for the source documents.

Claude analyzes the query and generates a response that includes precise citations based on the provided chunks and context for any claims derived from the source material. Cited text will reference source documents to minimize hallucinations.

This approach offers superior flexibility and ease of use, as it doesn’t require file storage and seamlessly integrates with the Messages API.

Pricing

Citations uses our standard token-based pricing model. While it may use additional input tokens to process documents, users will not pay for output tokens that return the quoted text itself.

Customer spotlight: Thomson Reuters

Thomson Reuters uses Claude to power their AI platform, CoCounsel, helping legal and tax professionals synthesize expert knowledge and deliver comprehensive advice to clients.

“For CoCounsel to be trustworthy and immediately useful for practicing attorneys, it needs to cite its work. We first built this ourselves, but it was really hard to build and maintain. That’s why we were excited to test out Anthropic’s Citations functionality. It makes citing and linking to primary sources much easier to build, maintain, and deploy to our users. This capability not only helps minimize hallucination risk but also strengthens trust in AI-generated content. The Citations feature will enable us to build an even more accurate and thorough AI assistant for lawyers,” said Jake Heller, Head of Product, CoCounsel, Thomson Reuters.

Customer Spotlight: Endex

Endex uses Claude to power an Autonomous Agent for financial firms.

“With Anthropic’s Citations, we reduced source hallucinations and formatting issues from 10% to 0% and saw a 20% increase in references per response. This removed the need for elaborate prompt engineering around references and improved our accuracy when conducting complex, multi-stage financial research,” said Tarun Amasa, CEO, Endex.

Get started

Citations is now available for the new Claude 3.5 Sonnet and Claude 3.5 Haiku. To start using Citations, explore our documentation.

Claude Citations outputs the exact passage from the cited document as well as start and end indices referencing the slices of the original document (as a string) that contain the citations.

We unpack the inference response from Claude into a data structure for this post like this:

{
  "question": "Which Claude models support Citations?",
  "answer": "According to the documents, Citations is available for Claude 3.5 Sonnet and Claude 3.5 Haiku. The feature is generally available on both the Anthropic API and Google Cloud's Vertex AI.",
  "citations": [
    {
      "text": "## Get started\n\nCitations is now available for the new Claude 3.5 Sonnet and Claude 3.5 Haiku.",
      "start_index": 4114,
      "end_index": 4209,
      "document": "Citations on the Anthropic API"
    },
    {
      "text": "Citations is generally available on the Anthropic API and Google Cloud\u2019s Vertex AI.",
      "start_index": 315,
      "end_index": 400,
      "document": "Citations on the Anthropic API"
    }
  ]
}

To make the above UX work as rendered HTML, we need to first add HTML markup to the raw markdown string before rendering, since the cited passages and document indices would no longer match the content after rendering the markdown as HTML. To add the highlighting, we iterate through the citations from end to beginning to avoid interfering with the citation start and end indices. Within each citation, we break the content up by line and wrap each line with <mark> tags, with special case handling for markdown elements like headings and lists. Finally, we render that HTML/markdown mix as HTML, resulting in the highlighted document upon selecting a question.

Challenges

Finding a way to add the highlights to the document was the biggest challenge because we must do so before the markdown is rendered as HTML (the citations are in markdown) but we also needed to avoid interfering with the markdown to HTML renderer so that the HTML markup would be added correctly. For example, a common problem I ran into was correct highlighting but content that was sent to the DOM as

## Get started

rather than

<h2>Get started</h2>

This issue was due to a difference of the preprocessor outputting

<mark class="citation-highlight"> ## Get started </mark>

instead of

## <mark class="citation-highlight"> Get started </mark>

Future work

With a similar approach, we could add a chat to a document and call the Citations API in real-time with whatever question a user inputs, displaying both the response and the highlighted citations from the source document. This pattern is an inversion of approaches like RAG, which put the Q&A front and center rather than the source document. In cases of knowledge aggregation this may be preferred, but when focusing on a single text, it’s easy to become disconnected from the source material. Keeping the source content front and center helps maintain better grounding in the original text while still getting the benefits from the power of LLM summarization.

Thanks for reading via RSS!

Running Deepseek Janus Pro 7B on a Macbook

Tue, 28 Jan 2025 21:42:01 GMT

Deepseek is getting a lot of attention with the releases of V3 and recently R1. Yesterday, they also released “Pro 7B” version of Janus, a “Unified Multimodal” model that can generate images from text and text from images. Most models I’ve experimented with can only do one of the two.

The 7B model requires about 15GB of hard disk space. It also seemed to almost max out the 64GB of memory my machine has. I’m not deeply familiar with the hardware requirements for this model so your mileage may vary.

How to run Janus Pro 7B

Let’s get the model and run it on a Mac.

git clone git@github.com:deepseek-ai/Janus.git

Install dependencies

python -m venv .venv
. .venv/bin/activate
pip install -e .

Now, modeling off the script provided in the repo, let’s generate an image from text. Note: you need to expand the “Janus-Pro” section to see the code.

The code

Create the following generate.py script:

import os
import PIL.Image
import torch
import numpy as np
from transformers import AutoModelForCausalLM
from janus.models import MultiModalityCausalLM, VLChatProcessor


# specify the path to the model
model_path = "deepseek-ai/Janus-Pro-7B"
vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
    model_path, trust_remote_code=True
)

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
vl_gpt = vl_gpt.to(torch.bfloat16).to(device).eval()


conversation = [
    {
        "role": "<|User|>",
        "content": "A small brown dog, mix of a shih tzu, poodle and chihuahua",
    },
    {"role": "<|Assistant|>", "content": ""},
]

sft_format = vl_chat_processor.apply_sft_template_for_multi_turn_prompts(
    conversations=conversation,
    sft_format=vl_chat_processor.sft_format,
    system_prompt="",
)
prompt = sft_format + vl_chat_processor.image_start_tag


@torch.inference_mode()
def generate(
    mmgpt: MultiModalityCausalLM,
    vl_chat_processor: VLChatProcessor,
    prompt: str,
    temperature: float = 1,
    parallel_size: int = 16,
    cfg_weight: float = 5,
    image_token_num_per_image: int = 576,
    img_size: int = 384,
    patch_size: int = 16,
):
    input_ids = vl_chat_processor.tokenizer.encode(prompt)
    input_ids = torch.LongTensor(input_ids)

    tokens = torch.zeros((parallel_size * 2, len(input_ids)), dtype=torch.int).to(
        device
    )
    for i in range(parallel_size * 2):
        tokens[i, :] = input_ids
        if i % 2 != 0:
            tokens[i, 1:-1] = vl_chat_processor.pad_id

    inputs_embeds = mmgpt.language_model.get_input_embeddings()(tokens)

    generated_tokens = torch.zeros(
        (parallel_size, image_token_num_per_image), dtype=torch.int
    ).to(device)

    for i in range(image_token_num_per_image):
        outputs = mmgpt.language_model.model(
            inputs_embeds=inputs_embeds,
            use_cache=True,
            past_key_values=outputs.past_key_values if i != 0 else None,
        )
        hidden_states = outputs.last_hidden_state

        logits = mmgpt.gen_head(hidden_states[:, -1, :])
        logit_cond = logits[0::2, :]
        logit_uncond = logits[1::2, :]

        logits = logit_uncond + cfg_weight * (logit_cond - logit_uncond)
        probs = torch.softmax(logits / temperature, dim=-1)

        next_token = torch.multinomial(probs, num_samples=1)
        generated_tokens[:, i] = next_token.squeeze(dim=-1)

        next_token = torch.cat(
            [next_token.unsqueeze(dim=1), next_token.unsqueeze(dim=1)], dim=1
        ).view(-1)
        img_embeds = mmgpt.prepare_gen_img_embeds(next_token)
        inputs_embeds = img_embeds.unsqueeze(dim=1)

    dec = mmgpt.gen_vision_model.decode_code(
        generated_tokens.to(dtype=torch.int),
        shape=[parallel_size, 8, img_size // patch_size, img_size // patch_size],
    )
    dec = dec.to(torch.float32).cpu().numpy().transpose(0, 2, 3, 1)

    dec = np.clip((dec + 1) / 2 * 255, 0, 255)

    visual_img = np.zeros((parallel_size, img_size, img_size, 3), dtype=np.uint8)
    visual_img[:, :, :] = dec

    os.makedirs("generated_samples", exist_ok=True)
    for i in range(parallel_size):
        save_path = os.path.join("generated_samples", "img_{}.jpg".format(i))
        PIL.Image.fromarray(visual_img[i]).save(save_path)


generate(
    vl_gpt,
    vl_chat_processor,
    prompt,
    parallel_size=1, # to generate a single image
)

Here is a diff of the script changes to give you a better sense of what changes were needed to get it running on my Mac:

 vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained(
     model_path, trust_remote_code=True
 )
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()

device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
vl_gpt = vl_gpt.to(torch.bfloat16).to(device).eval()


 conversation = [
     {
         "role": "<|User|>",
        "content": "A stunning princess from kabul in red, white traditional clothing, blue eyes, brown hair",
        "content": "A small brown dog, mix of a shih tzu, poodle and chihuahua",
     },
     {"role": "<|Assistant|>", "content": ""},
 ]
@@ -47,13 +50,17 @@
     input_ids = vl_chat_processor.tokenizer.encode(prompt)
     input_ids = torch.LongTensor(input_ids)

    tokens = torch.zeros((parallel_size*2, len(input_ids)), dtype=torch.int).cuda()
    for i in range(parallel_size*2):
    tokens = torch.zeros((parallel_size * 2, len(input_ids)), dtype=torch.int).to(
        device
    )
    for i in range(parallel_size * 2):
         tokens[i, :] = input_ids
         if i % 2 != 0:
             tokens[i, 1:-1] = vl_chat_processor.pad_id

     inputs_embeds = mmgpt.language_model.get_input_embeddings()(tokens)

    generated_tokens = torch.zeros((parallel_size, image_token_num_per_image), dtype=torch.int).cuda()
    generated_tokens = torch.zeros(
        (parallel_size, image_token_num_per_image), dtype=torch.int
    ).to(device)

@@ -94,4 +109,5 @@
     vl_gpt,
     vl_chat_processor,
     prompt,
    parallel_size=1,
 )

Let’s run it.

Note: prepare to wait for the 15GB of model weights to download if this is your first time pulling the model.

python generate.py

If all goes well, you can find the output image in the generated_samples folder called img_0.jpg.

On my machine, this run took 3:51. I’ve done a few other runs in the ~2 minute range.

Here is the resulting image

Cleanup

If you want back the 15GB of disk space, you can find the model weights and other files in ~/.cache/huggingface/hub.

Thanks for reading via RSS!

`llm` upgrade pip

Sun, 26 Jan 2025 14:47:56 GMT

The llm package uses a plugin architecture to support numerous different language model API providers and frameworks. Per the documentation, these plugins are installed using a version of pip, the popular Python package manager

Use the llm install command (a thin wrapper around pip install) to install plugins in the correct environment: llm install llm-gpt4all

Because this approach makes use of pip occasionally we run into familiar issues like pip being out of date and complaining about it on every use

❯ llm install llm-ollama
Collecting llm-ollama
...
Successfully installed llm-ollama-0.8.2 ollama-0.4.7

[notice] A new release of pip is available: 24.3.1 -> 25.0
[notice] To update, run: /opt/homebrew/Cellar/llm/0.19.1/libexec/bin/python -m pip install --upgrade pip

The instance of pip that needs to be updated here is the pip being used within llm. I couldn’t find explicit instructions on how to address this issue upon some cursory searches, but the solution is pretty intuitive - props to Simon for a nice API design.

To update pip within llm, run

llm install --upgrade pip

The upgrade notice should disappear if all goes well.

Thanks for reading via RSS!

Claude Citations

Thu, 23 Jan 2025 17:45:38 GMT

Today, Anthropic released Citations for Claude.

In the release, Anthropic disclosed the following customer case study:

“With Anthropic’s Citations, we reduced source hallucinations and formatting issues from 10% to 0% and saw a 20% increase in references per response. This removed the need for elaborate prompt engineering around references and improved our accuracy when conducting complex, multi-stage financial research,” said Tarun Amasa, CEO, Endex.

I decided to kick the tires on this feature as I thought it could slot in very nicely with a project I am actively working on. Also, I couldn’t quickly find Python code I could copy and run so I conjured some.

Note: the following document in this example is 100% model generated and chosen by the model and is meant to illustrate how the API can be used in Python.

Here’s the code

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY"))

message = client.messages.create(
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document",
                    "source": {
                        "type": "text",
                        "media_type": "text/plain",
                        "data": "Interview with Sarah Chen, Product Manager at TechCorp\n\nQ: What made you switch from your previous role to product management?\nA: I was working as a software engineer for 5 years, but I kept finding myself drawn to user research and product strategy. I loved coding, but I realized I was more passionate about understanding user problems and designing solutions.\n\nQ: What's been the biggest challenge in the transition?\nA: The context switching. As an engineer, I could focus deeply on technical problems. As a PM, I'm constantly shifting between user research, data analysis, engineering discussions, and stakeholder management.\n\nQ: How do you measure success in your current role?\nA: It's a mix of quantitative and qualitative metrics. We track user engagement and retention numbers, but I also put a lot of weight on user feedback and satisfaction scores.",
                    },
                    "title": "PM Career Transition Interview",
                    "context": "Interview conducted on March 15, 2024 as part of UX research",
                    "citations": {"enabled": True},
                },
                {
                    "type": "text",
                    "text": "What were the main reasons the interviewee transitioned to product management and what challenges did they face?",
                },
            ],
        }
    ],
    model="claude-3-5-sonnet-latest",
)

answer = ""
citations = []

for block in message.content:
    if block.type == "text":
        answer += block.text
        if block.citations:
            citations.extend(block.citations)

print("\nAnswer:")
print(answer)
print("\nCitations:")
for citation in citations:
    source_text = citation.cited_text.strip()
    start_idx = citation.start_char_index
    end_idx = citation.end_char_index
    print(f'- "{source_text}" [{start_idx}:{end_idx}] from {citation.document_title}')

When we run, we get the answer to our question along with the exact slice of the source document from which the information is extracted, which I’ve added in square brackets.

Answer: Based on the interview, here are the key points about Sarah Chen’s transition to product management:

Reasons for the transition: While working as a software engineer for 5 years, she found herself naturally drawn to user research and product strategy. Although she enjoyed coding, her true passion lay in understanding user problems and designing solutions.

Main challenges: The biggest challenge in her transition has been context switching. This was a significant change from her engineering role, where she could focus deeply on technical problems. As a Product Manager, she has to constantly shift between multiple responsibilities including:

User research

Data analysis

Engineering discussions

Stakeholder management

This showcases how the role of a PM requires juggling multiple responsibilities and a different skill set compared to her previous technical role.

Citations:

“A: I was working as a software engineer for 5 years, but I kept finding myself drawn to user research and product strategy. I loved coding, but I realized I was more passionate about understanding user problems and designing solutions.” [127:364] from PM Career Transition Interview

“A: The context switching. As an engineer, I could focus deeply on technical problems. As a PM, I’m constantly shifting between user research, data analysis, engineering discussions, and stakeholder management.” [420:631] from PM Career Transition Interview

This feature gives anyone who knows how to call an API the ability to create Perplexity-like, inline citations from sources. I’ve thus far not had the need or patience to prompt an LLM to do this work myself, so I can’t speak to how difficult it is to do “by hand”, but it’s a useful and satisfying feature to have native to the model API^[1].

And is yet another divergence from the OpenAI API spec ↩

Thanks for reading via RSS!

Zoomable Document

Thu, 23 Jan 2025 01:50:48 GMT

Credit

I can’t actually remember if I got inspiration for this idea from Amelia’s presentation, but watching that again recently, it seems likely so I am crediting her for it here.

The Spark

I’m trying something unusual with this post. I think it’s going to be a lot of fun. I was listening to Nabeel Hyatt and Dan Shipper’s conversation, where they mentioned code has become content. This resonated with me because I’ve lately found, specifically with the projects on this site, that writing and building have started to go hand in hand for me. The prose I write to describe my ideas often becomes the prompts I use to build them and vice versa.

Powerful models and their code generating abilities are incredible synthesizers for ideas. Prior to having a LLM at hand to write most of the code, I wouldn’t have the energy to prototype or attempt building many of them. Most didn’t feel worth the effort or I’d get stuck or frustrated and move on. With that context, I’m going to (attempt to) build this interactive post by writing the post.

Starting from Scratch

As I write now, I have Cursor open and I am writing in a file at src/content/notes/2025/document-?/index.mdx in my Astro project. The folder and post are currently called document-? because I don’t really know what this concept is yet, but I needed to start somewhere. Maybe it’s called something else when you’re reading it. My site has interactive Astro and React components at src/components. I can include these components in my .mdx content which has been a really enjoyable way to play around with ideas.

Understanding Content Structure

For this post, we are building an experience that allows the reader to interact with the content of a post at multiple levels of detail and specificity. Most content has at most just a few levels of content. The first are headings. These include things like titles and section headers often denoted by # tags in markdown. These titles and headers can be summarized into a table of contents to give the reader an overview of what’s in an article or the ability to jump around or permalink to certain parts. A second level of content is the linear prose or the body of the work. This is a bit of a kitchen sink category, including prose, lists, code, images and everything that composes most of the written work. Finally, a third level of content includes things like asides, footnotes and citations that appear next to or after the main prose. Sometimes these are off to the side of content or aggregated all together at the end.

With LLMs, we can create summaries that provide at-a-glance skimmability and arbitrary levels of specificity. Imagine zooming in and out of a document like you do when using an app like Google Maps. If you navigate to New York City and zoom into a particular neighborhood, you can see the roads and the locations of the buildings. This zoomed in view is like reading the original prose of a blog post. At this level of zoom, you are limited in how much you can see. Your browser, phone or field of view is only so large. You can’t look at every road in the city at the same time. Zoom out further and we can see the boroughs or even other states, but then we’d see very few roads. There’s simply too much detail to show when zoomed out at this level. Prose is like this as well but most “maps” of prose are relatively static. In practice, we actually spend relatively limited effort making our prose accessible at different levels of zoom. What would that even look like?

Like a map, we may want to zoom out and view an entire piece of written text from a high level, understanding its broad features. Today, the closest thing to something like this is a table of contents. In the best case, the author has thoughtfully titled their chapters in a way that actually gives us something akin to a high level overview.

Another area that can help is a glossary. With the right vocabulary, we can locate references to specific items of interests. Unfortunately, most glossaries are limited beyond close to perfect string matches. They include things like technical terms, acronyms, and specialized vocabulary. But they don’t allow us to find where certain concepts are mentioned or where certain motifs occur, at least not flexibly. Glossaries rely on the foresight of the author to anticipate how a reader might want to navigate the prose.

Digital documents contain many improvements. We can “find” any word or phrase across an arbitrarily large document. But it’s still fuzzy how you might find all the places in a written work where a character displays “kindness”, for example. In a literary work, the author doesn’t come out and say when a character is kind. They show the character’s kindness through their actions. We can’t just run “find” on the word “kind”. We’ll miss a lot.

Building the Zoomable Document

Let’s return to what a zoomable document looks like. We have our street level view. That is the unmodified document. We see all the words, images, code, asides, footnotes, everything. The raw, unedited text as the author created it.

Let’s zoom out. Now we’re seeing neighborhoods. Each neighborhood represents a chunk of content that’s roughly smaller than a major heading but larger than individual paragraphs.

Markdown supports 6 heading levels. Not everyone uses all 6 levels but some people do. So when we zoom out, we’re zooming out relative to the levels of content the author has provided, but we can’t necessarily rely on these “cliff notes”. In fact, we have to assume the content won’t already contain a “roadmap” for the reader. That is the utility of what we aim to provide with our “zoomable document”.

Technical Implementation

How can we construct this zoomable document? We’ll first need to take a look at the raw content that exists at the street level. From here, we will need to compute summaries for each different level of zoom. When we zoom out, content at the previous level fades away and content at the next level fades into view. As we zoom further and further out, the content is condensed into summaries that capture the essence of the lower levels, giving the reader the ability to quickly scan through these summaries, then zoom into areas of interest, like a dynamic map.

I store zoom levels as separate markdown files in the same directory as the post. This keeps the main post file clean and makes it easier to manage different zoom levels independently. The files are imported directly into the MDX file using Vite’s ?raw suffix, which imports the markdown content as strings.

The file structure looks like this:

src/content/posts/2025/zoomable-document/
├── index.mdx          # Main post file
├── _level_2.md        # Level 2 zoom content
├── _level_3.md        # Level 3 zoom content
├── _level_4.md        # Level 4 zoom content
└── _level_5.md        # Level 5 zoom content

The content is passed to the React component using this schema:

interface ContentLevel {
  level: number;
  content: string;
}

interface ZoomableDocumentProps {
  children: ReactNode;
  levels: ContentLevel[];
}

To generate content for those levels, with this (or any) document as context, I use the following prompt

User

Using markdown (including headers, list and code if useful) given the content at Level 1, write Levels 2-5, maintaining the author’s voice and provide increasingly general levels of specificity as the levels increase. Don’t just summarize. Provide some of the same details as the original text at the lower levels. Do more summarizing at the higher levels. Use your best judgment to determine which details are relevant at each level. Remember, lower levels are like street and neighborhood views on Google Maps where you can see roads and buildings. Higher levels are like county, state and even country views where we get more of the general geography of a space but see fewer details.

The results have been both interesting and challenging to achieve. I created the first version of the ZoomableDocument React component by using an early version of this prose as the prompt and giving an LLM instructions to “build that”. From there, I continued writing more in prompts and here to explain what approaches were working well and refined the prose further. The above prompt allows me to easily use an LLM to regenerate the higher level summaries whenever I make edits to this Level 1 version.

To create a zoomable document:

Write markdown content normally in the .mdx file
Generate zoom levels using a language model with the raw post as context using the prompt above
Save each zoom level as a separate markdown file in the same directory
Import the zoom levels to the MDX file:

import Level2Content from './_level_2.md?raw';
import Level3Content from './_level_3.md?raw';
import Level4Content from './_level_4.md?raw';
import Level5Content from './_level_5.md?raw';

export const zoomLevels = [
  { level: 2, content: Level2Content },
  { level: 3, content: Level3Content },
  { level: 4, content: Level4Content },
  { level: 5, content: Level5Content },
];

Wrap your content in the ZoomableDocument component with the levels prop

<ZoomableDocument client:load levels={zoomLevels}>
  {children}
</ZoomableDocument>

In the future, I could find a way to automatically load the levels and wrap the content in the ZoomableDocument component.

Future work

Given the recent release of Claude Citations, I’ll probably revisit this concept introducing explicit linkages between the summarized content and the source text.

Thanks for reading via RSS!

Taking Browser Screenshots with Deno

Sun, 19 Jan 2025 17:15:05 GMT

Today, I needed to turn SVGs into PNGs. I decided to use Deno to do it. Some cursory searching showed Puppeteer should be up to the task. I also found deno-puppeteer which seemed like it would provide a reasonable way to make this work.

To start, let’s set up a deno project

deno init deno-browser-screenshots
deno-browser-screenshots

Using `puppeteer`

Now, add some code to render an SVG with Chrome via puppeteer.

import puppeteer from 'https://deno.land/x/puppeteer@16.2.0/mod.ts';

const svgString = `
<svg width="512" height="512" xmlns="http://www.w3.org/2000/svg">
  <rect width="100%" height="100%" fill="#87CEEB"/>
  <circle cx="256" cy="256" r="100" fill="#FFD700"/>
  <path d="M 100 400 Q 256 300 412 400" stroke="#1E90FF" stroke-width="20" fill="none"/>
</svg>`;

if (import.meta.main) {
  try {
    const browser = await puppeteer.launch({
      headless: true,
      args: ['--no-sandbox'],
    });

    const page = await browser.newPage();

    await page.setViewport({ width: 512, height: 512 });
    await page.setContent(svgString);

    await page.screenshot({
      path: 'output.png',
      clip: {
        x: 0,
        y: 0,
        width: 512,
        height: 512,
      },
    });

    await browser.close();
  } catch (error) {
    console.error('Error occurred:', error);
    console.error('Make sure Chrome is installed and the path is correct');
    throw error;
  }
}

When we run this code, we get the following error

deno run -A main.ts

Error occurred: Error: Could not find browser revision 1022525. Run "PUPPETEER_PRODUCT=chrome deno run -A --unstable https://deno.land/x/puppeteer@16.2.0/install.ts" to download a supported browser binary.
    at ChromeLauncher.launch (https://deno.land/x/puppeteer@16.2.0/src/deno/Launcher.ts:99:30)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async file:///Users/danielcorin/dev/lab/deno-puppeteer/main.ts:12:21
Make sure Chrome is installed and the path is correct
error: Uncaught (in promise) Error: Could not find browser revision 1022525. Run "PUPPETEER_PRODUCT=chrome deno run -A --unstable https://deno.land/x/puppeteer@16.2.0/install.ts" to download a supported browser binary.
      if (missingText) throw new Error(missingText);
                             ^
    at ChromeLauncher.launch (https://deno.land/x/puppeteer@16.2.0/src/deno/Launcher.ts:99:30)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async file:///Users/danielcorin/dev/lab/deno-puppeteer/main.ts:12:21

Using npm’s puppeteer, we can install Chrome via npx

npx puppeteer browsers install chrome

However, this still did not resolve the error for me. It turns out the npx command install the browser to ~/.cache/puppeteer.

To use it, we need the following modifications to our code to provide an executablePath.

import puppeteer from 'https://deno.land/x/puppeteer@16.2.0/mod.ts';

const svgString = `
<svg width="512" height="512" xmlns="http://www.w3.org/2000/svg">
  <rect width="100%" height="100%" fill="#87CEEB"/>
  <circle cx="256" cy="256" r="100" fill="#FFD700"/>
  <path d="M 100 400 Q 256 300 412 400" stroke="#1E90FF" stroke-width="20" fill="none"/>
</svg>`;

if (import.meta.main) {
  try {
    const browser = await puppeteer.launch({
      headless: true,
      args: ['--no-sandbox'],
      executablePath:
        Deno.env.get('HOME') +
        '/.cache/puppeteer/chrome/mac_arm-131.0.6778.108/chrome-mac-arm64/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing',
    });

    const page = await browser.newPage();

    await page.setViewport({ width: 512, height: 512 });
    await page.setContent(`
      <style>
        body { margin: 0; background: transparent; }
        svg { display: block; }
      </style>
      ${svgString}
    `);

    await page.screenshot({
      path: 'output.png',
      clip: {
        x: 0,
        y: 0,
        width: 512,
        height: 512,
      },
    });

    await browser.close();
  } catch (error) {
    console.error('Error occurred:', error);
    console.error('Make sure Chrome is installed and the path is correct');
    throw error;
  }
}

With that, the code runs successfully and we get output.png in our working directory.

Using `astral`

There is another, Deno-native library called astral (no relation to the Python tooling company). I was curious to see how the DX compared for this simple use case.

The following code worked for me without issue

import { launch } from 'jsr:@astral/astral';

const svgString = `
<svg width="512" height="512" xmlns="http://www.w3.org/2000/svg">
  <rect width="100%" height="100%" fill="#87CEEB"/>
  <circle cx="256" cy="256" r="100" fill="#FFD700"/>
  <path d="M 100 400 Q 256 300 412 400" stroke="#1E90FF" stroke-width="20" fill="none"/>
</svg>`;

if (import.meta.main) {
  try {
    const browser = await launch();
    const page = await browser.newPage();

    await page.setViewportSize({ width: 512, height: 512 });
    await page.setContent(`
      <style>
        body { margin: 0; background: transparent; }
        svg { display: block; }
      </style>
      ${svgString}
    `);

    const screenshot = await page.screenshot();
    Deno.writeFileSync('output_astral.png', screenshot);

    await browser.close();
  } catch (error) {
    console.error('Error occurred:', error);
    throw error;
  }
}

On first run, astral manages the downloading of Chrome for you.

deno run -A main_astral.ts

Downloading chrome 125.0.6400.0                                                    100.00%
Download complete (chrome version 125.0.6400.0)
Inflating /Users/danielcorin/Library/Caches/astral/125.0.6400.0                                        100.00%
Browser saved to /Users/danielcorin/Library/Caches/astral/125.0.6400.0

This code works but unexpectedly outputs a PNG with size 1024 x 1024 and I’m not entirely sure why.

Adding the following not-very-nice-looking code seemed to fix this issue - maybe there is a better way.

await page.unsafelyGetCelestialBindings().Emulation.setDeviceMetricsOverride({
  width: 512,
  height: 512,
  deviceScaleFactor: 1,
  mobile: false,
});

With each approach, there were different rough edges. Both were able to fit my use case with a few tweaks.

Thanks for reading via RSS!

LLM Conversation Branching

Fri, 17 Jan 2025 02:08:37 GMT

tip

The interactive visuals in the post don’t play so nice on small devices. You can still enjoy on your phone but be sure to check them out later on a bigger screen.

I wrote about Conversation Branching on my main blog and I’ve been prototyping an LLM chat app called Delta that has first-class conversation branching. LLM conversations typically follow a linear path, making it unintuitive to explore alternative directions or recover from miscommunications. This post is an interactive introduction to conversation branching but also an exploration into additional UX patterns that using a canvas can enable.

Here are the basic primitives of conversation branching. Click any node on the left. The conversation on the right updates to show the prompts and responses (since we’re modeling an LLM conversation).

What is machine learning?

Machine learning is a branch of AI that enables systems to learn from data

The motivation behind conversation branching is multifold

Keep the context clean

Language models can be sensitive to typos or ambiguous phrases that send the conversation in the wrong direction. Often, if you notice the model is misunderstanding you, it’s more effective to delete the previous conversation turn or update your message. ChatGPT actually provides subtle support for branching with the message “edit” feature.

Explore parallel threads, maintaining conversation history

While editing message and implicit branching helps keep the context clean, making branches visible and navigable adds additional depth. We now can explore multiple lines of thinking from a shared starting premise, while maintaining the integrity of the context, keeping the LLM focused on the thing you care about, in each of the branches.

We’re now exploring multiple lines of thought branching off of the original question “What is machine learning?”. As above, we could navigate any of these branches individually and linearly by clicking on a node, and branch off any node with a new message. However, the canvas has a ton of content, making it difficult to navigate or get an at-a-glance sense of what each node contains.

LLMs can help us here. We can summarize each node based on the prompt and response, then show summaries when zoomed out to make the content more digestible at a high level.

Try zooming in and out on either of the last two visualizations to see how the content adapts based on zoom level.

Semantic clustering and exploration

With concise summaries, we can now add a new perspective to these conversations. If we generate embeddings of the summaries, which gives us a list of floats for each summary, we can then apply Principal Component Analysis to reduce the many dimensions down to just two. Finally, we use those two floats as coordinates to position each of the conversation nodes, clustering them by their semantic similarity.

Here are the results:

By my assessment, there are three main categories

ML and programming languages/development (bottom left)
ML and medicine/society (top left)
PyTorch (top right)

Equidistant from the three clusters is the more generic, start message.

I always find it interesting to explore latent space like this. Doing so with a conversation is a bit like blazing a trail and identifying new terrain, seeing where similar pieces fit together and finding unexplored areas in the negative spaces.

Thanks for reading via RSS!

Local VLMs Have Improved

Thu, 16 Jan 2025 17:44:47 GMT

About 6 months ago, I experimented with running a few different multi-modal (vision) language models on my Macbook. At the time, the results weren’t so great.

An experiment

With a slight modification to the script from that post, I tested out llama3.2-vision 11B (~8GB in size between the model and the projector). Using uv and inline script dependencies, the full script looks like this

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "ollama",
# ]
# ///

import os
import sys
import ollama

PROMPT = "Describe the provided image in a few sentences"


def run_inference(model: str, image_path: str):
    stream = ollama.chat(
        model=model,
        messages=[{"role": "user", "content": PROMPT, "images": [image_path]}],
        stream=True,
    )

    for chunk in stream:
        print(chunk["message"]["content"], end="", flush=True)


def main():
    if len(sys.argv) != 3:
        print("Usage: python run.py <model_name> <image_path>")
        sys.exit(1)

    model_name = sys.argv[1]
    image_path = sys.argv[2]

    if not os.path.exists(image_path):
        print(f"Error: Image file '{image_path}' does not exist.")
        sys.exit(1)

    run_inference(model_name, image_path)


if __name__ == "__main__":
    main()

We run it with

uv run vlm.py llama3.2-vision propose_moves.png

As a reminder, this is the propose_moves.png image

Reading inline script metadata from `vlm.py`
The image shows two game screens side by side, each representing a different position of a game.  The left screen is labeled "Tactic" at the top and has an empty gray box under it with the word "Game started!" followed by "You are placing: X". Under that, there's another label that reads "Lock Moves," but no boxes are present below this label. In the middle of the screen, there is a 3x3 grid filled with white squares and two of them have an 'X' in them. The right screen is also labeled "Tactic" at the top, with the same information as the left screen: "Game started!" followed by "You are placing: O". Under that label, there's another "Lock Moves" label without boxes below it. In the middle of this screen, there is a 3x3 grid filled with white squares and two of them have an 'O' in them.

Both screens appear to be part of a game where players take turns placing their marks on a grid. The game seems to involve strategy and planning ahead since each player has limited moves available before the other can respond.%

I was surprised at how good a description this was for such a small model. I played around a bit with structured extraction but the results weren’t particularly good.

Here’s an incorrect example

ollama run llama3.2-vision

>>> Output the board state of board on the left as JSON. Output JSON only.  /Users/danielcorin/Desktop/propose_moves.png
Added image '/Users/danielcorin/Desktop/propose_moves.png'
{"board": ["X", "O", "", "", "", "", "", "", ""], "player_turn": "X"}

Thanks for reading via RSS!

The Unreasonable Effectiveness of Generating UI with React and Tailwind

Mon, 06 Jan 2025 00:54:48 GMT

If you’ve experimented with Claude Artifacts or v0.dev, maybe you’ve been delighted (as I have) that the language model can conjure a UI for an idea you describe. Most of my experience building software, especially professionally, comes from working on the “backend”. Building frontend (read: user interfaces in a browser) is harder for me (or doesn’t come as easily), because while I have experience writing software, I don’t have as much experience writing this type of software.

The “Magic” Stack

When generating UI, one specific stack has proven to be both popular and effective relative to the technologies I have tried. That stack is React and Tailwind.

Both Claude Artifacts and v0 use these technologies by default and there is a reason why. For the language model, co-locating the component styling with the structural markup is highly effective and steerable by prompting.

Comparing Different Approaches

Given the following prompt, here’s what claude-3-5-sonnet-20241022 generates in Cursor (the components below are real, working React code - try them out!).

note

I made a few minor adjustments post-generation to the components to ensure they render nicely on small screens

User

use react, tailwind and lucide-react to create an interface for a bullet-journal inspired calendar

Bullet Journal

165 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-4 h-4 sm:w-5 sm:h-5 text-gray-600" />;
      case 'done':
        return (
          <CheckCircle2 className="w-4 h-4 sm:w-5 sm:h-5 text-green-600" />
        );
      case 'migrated':
        return <Square className="w-4 h-4 sm:w-5 sm:h-5 text-blue-600" />;
      case 'cancelled':
        return <XCircle className="w-4 h-4 sm:w-5 sm:h-5 text-red-600" />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  return (
    <div className="w-full max-w-2xl mx-auto p-4 sm:p-8 bg-white rounded-lg shadow-md">
      <div className="flex items-center gap-2 sm:gap-4 mb-4 sm:mb-8">
        <CalendarIcon className="w-5 h-5 sm:w-6 sm:h-6 text-gray-600" />
        <h1 className="text-xl sm:text-2xl font-semibold text-gray-600">
          Bullet Journal
        </h1>
      </div>

      <div className="mb-4 sm:mb-8">
        <input
          type="date"
          value={selectedDate.toISOString().split('T')[0]}
          onChange={(e) => setSelectedDate(new Date(e.target.value))}
          className="w-full px-3 sm:px-4 py-2 text-sm sm:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
      </div>

      <div className="flex flex-col sm:flex-row gap-2 sm:gap-4 mb-4 sm:mb-8">
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          className="flex-1 px-3 sm:px-4 py-2 text-sm sm:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
        <button
          onClick={addTask}
          className="px-3 sm:px-4 py-2 text-sm sm:text-base bg-blue-600 text-white rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500"
        >
          Add
        </button>
      </div>

      <div className="space-y-2">
        {currentEntry?.tasks.map((task) => (
          <div
            key={task.id}
            className="flex items-center gap-2 p-2 border border-gray-200 rounded-md"
          >
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              className="focus:outline-none"
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              className={`text-sm sm:text-base ${task.status === 'done' ? 'line-through text-gray-500' : ''}`}
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

165 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-4 h-4 sm:w-5 sm:h-5 text-gray-600" />;
      case 'done':
        return (
          <CheckCircle2 className="w-4 h-4 sm:w-5 sm:h-5 text-green-600" />
        );
      case 'migrated':
        return <Square className="w-4 h-4 sm:w-5 sm:h-5 text-blue-600" />;
      case 'cancelled':
        return <XCircle className="w-4 h-4 sm:w-5 sm:h-5 text-red-600" />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  return (
    <div className="w-full max-w-2xl mx-auto p-4 sm:p-8 bg-white rounded-lg shadow-md">
      <div className="flex items-center gap-2 sm:gap-4 mb-4 sm:mb-8">
        <CalendarIcon className="w-5 h-5 sm:w-6 sm:h-6 text-gray-600" />
        <h1 className="text-xl sm:text-2xl font-semibold text-gray-600">
          Bullet Journal
        </h1>
      </div>

      <div className="mb-4 sm:mb-8">
        <input
          type="date"
          value={selectedDate.toISOString().split('T')[0]}
          onChange={(e) => setSelectedDate(new Date(e.target.value))}
          className="w-full px-3 sm:px-4 py-2 text-sm sm:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
      </div>

      <div className="flex flex-col sm:flex-row gap-2 sm:gap-4 mb-4 sm:mb-8">
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          className="flex-1 px-3 sm:px-4 py-2 text-sm sm:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
        <button
          onClick={addTask}
          className="px-3 sm:px-4 py-2 text-sm sm:text-base bg-blue-600 text-white rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500"
        >
          Add
        </button>
      </div>

      <div className="space-y-2">
        {currentEntry?.tasks.map((task) => (
          <div
            key={task.id}
            className="flex items-center gap-2 p-2 border border-gray-200 rounded-md"
          >
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              className="focus:outline-none"
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              className={`text-sm sm:text-base ${task.status === 'done' ? 'line-through text-gray-500' : ''}`}
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

User

use react and lucide-react to create an interface for a bullet-journal inspired calendar. output the react component with styles and a corresponding css module

Bullet Journal

152 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';
import styles from './ReactCalendarv1.module.css';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className={styles.icon} />;
      case 'done':
        return <CheckCircle2 className={styles.icon} />;
      case 'migrated':
        return <Square className={styles.icon} />;
      case 'cancelled':
        return <XCircle className={styles.icon} />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  return (
    <div className={styles.container}>
      <div className={styles.header}>
        <CalendarIcon className={styles.calendarIcon} />
        <h1>Bullet Journal</h1>
      </div>

      <meta name="viewport" content="width=device-width, initial-scale=1.0" />

      <div className={styles.dateSelector}>
        <input
          type="date"
          value={selectedDate.toISOString().split('T')[0]}
          onChange={(e) => setSelectedDate(new Date(e.target.value))}
        />
      </div>

      <div className={styles.taskInput}>
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
        />
        <button onClick={addTask}>Add</button>
      </div>

      <div className={styles.taskList}>
        {currentEntry?.tasks.map((task) => (
          <div key={task.id} className={styles.task}>
            <button
              className={styles.statusButton}
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
            >
              {getStatusIcon(task.status)}
            </button>
            <span className={task.status === 'done' ? styles.completed : ''}>
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

152 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';
import styles from './ReactCalendarv1.module.css';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className={styles.icon} />;
      case 'done':
        return <CheckCircle2 className={styles.icon} />;
      case 'migrated':
        return <Square className={styles.icon} />;
      case 'cancelled':
        return <XCircle className={styles.icon} />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  return (
    <div className={styles.container}>
      <div className={styles.header}>
        <CalendarIcon className={styles.calendarIcon} />
        <h1>Bullet Journal</h1>
      </div>

      <meta name="viewport" content="width=device-width, initial-scale=1.0" />

      <div className={styles.dateSelector}>
        <input
          type="date"
          value={selectedDate.toISOString().split('T')[0]}
          onChange={(e) => setSelectedDate(new Date(e.target.value))}
        />
      </div>

      <div className={styles.taskInput}>
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
        />
        <button onClick={addTask}>Add</button>
      </div>

      <div className={styles.taskList}>
        {currentEntry?.tasks.map((task) => (
          <div key={task.id} className={styles.task}>
            <button
              className={styles.statusButton}
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
            >
              {getStatusIcon(task.status)}
            </button>
            <span className={task.status === 'done' ? styles.completed : ''}>
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

Not much difference between these two from the user’s perspective. Pretty straightforward LLM UI output with lots of opportunities for improvement.

The former component exists entirely within a single .tsx file. The latter has the markup and style separated into .tsx and .css.module files.

Iterating on the Initial Design

Inevitably, we will want to make changes to the first iteration from the model. Let’s add a calendar component so we can see the number of tasks that have been input on each day.

User

update the component to add a calendar ui element. the calendar should display the count of the number of tasks and their types each day

Bullet Journal

235 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-4 h-4 md:w-5 md:h-5 text-gray-600" />;
      case 'done':
        return (
          <CheckCircle2 className="w-4 h-4 md:w-5 md:h-5 text-green-600" />
        );
      case 'migrated':
        return <Square className="w-4 h-4 md:w-5 md:h-5 text-blue-600" />;
      case 'cancelled':
        return <XCircle className="w-4 h-4 md:w-5 md:h-5 text-red-600" />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  const getDaysInMonth = (date: Date) => {
    const year = date.getFullYear();
    const month = date.getMonth();
    const firstDay = new Date(year, month, 1);
    const lastDay = new Date(year, month + 1, 0);
    const days = [];

    for (let i = 0; i < firstDay.getDay(); i++) {
      days.push(null);
    }

    for (let i = 1; i <= lastDay.getDate(); i++) {
      days.push(new Date(year, month, i));
    }

    return days;
  };

  const getTaskStats = (date: Date) => {
    const entry = entries.find(
      (e) => e.date.toDateString() === date.toDateString()
    );
    if (!entry) return null;

    const stats = {
      todo: 0,
      done: 0,
      migrated: 0,
      cancelled: 0,
    };

    entry.tasks.forEach((task) => {
      stats[task.status]++;
    });

    return stats;
  };

  return (
    <div className="w-full max-w-4xl mx-auto p-4 md:p-8 bg-white rounded-lg shadow-md">
      <div className="flex items-center gap-2 md:gap-4 mb-6 md:mb-8">
        <CalendarIcon className="w-5 h-5 md:w-6 md:h-6 text-gray-600" />
        <h1 className="text-xl md:text-2xl font-semibold text-gray-600">
          Bullet Journal
        </h1>
      </div>

      <div className="grid grid-cols-7 gap-1 mb-6 md:mb-8 text-center text-sm md:text-base">
        {['S', 'M', 'T', 'W', 'T', 'F', 'S'].map((day) => (
          <div key={day} className="font-semibold py-1 md:py-2">
            {day}
          </div>
        ))}
        {getDaysInMonth(selectedDate).map((date, i) => (
          <div
            key={i}
            className={`p-1 md:p-2 min-h-[60px] md:min-h-[100px] border ${date?.toDateString() === selectedDate.toDateString() ? 'border-blue-500' : 'border-gray-200'} rounded-md ${date ? 'cursor-pointer hover:bg-gray-50' : ''}`}
            onClick={() => date && setSelectedDate(date)}
          >
            {date && (
              <>
                <div className="text-right text-gray-600 text-xs md:text-base">
                  {date.getDate()}
                </div>
                {getTaskStats(date) && (
                  <div className="mt-1 text-[10px] md:text-xs space-y-0.5 md:space-y-1">
                    {Object.entries(getTaskStats(date)!).map(
                      ([status, count]) =>
                        count > 0 && (
                          <div
                            key={status}
                            className="flex items-center justify-between"
                          >
                            {getStatusIcon(status as Task['status'])}
                            <span>{count}</span>
                          </div>
                        )
                    )}
                  </div>
                )}
              </>
            )}
          </div>
        ))}
      </div>

      <div className="flex flex-col md:flex-row gap-2 md:gap-4 mb-6 md:mb-8">
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          className="flex-1 px-3 md:px-4 py-2 text-sm md:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
        <button
          onClick={addTask}
          className="px-4 py-2 bg-blue-600 text-white text-sm md:text-base rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500"
        >
          Add
        </button>
      </div>

      <div className="space-y-2">
        {currentEntry?.tasks.map((task) => (
          <div
            key={task.id}
            className="flex items-center gap-2 p-2 border border-gray-200 rounded-md text-sm md:text-base"
          >
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              className="focus:outline-none"
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              className={
                task.status === 'done' ? 'line-through text-gray-500' : ''
              }
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

235 collapsed lines
import {
  Calendar as CalendarIcon,
  CheckCircle2,
  Circle,
  Square,
  XCircle,
} from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

type DayEntry = {
  date: Date;
  tasks: Task[];
};

export default function Calendar() {
  const [entries, setEntries] = useState<DayEntry[]>([]);
  const [selectedDate, setSelectedDate] = useState<Date>(new Date());
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setEntries((prev) => {
      const existingEntry = prev.find(
        (entry) => entry.date.toDateString() === selectedDate.toDateString()
      );

      if (existingEntry) {
        return prev.map((entry) =>
          entry.date.toDateString() === selectedDate.toDateString()
            ? {
                ...entry,
                tasks: [
                  ...entry.tasks,
                  {
                    id: crypto.randomUUID(),
                    text: newTask,
                    status: 'todo',
                  },
                ],
              }
            : entry
        );
      }

      return [
        ...prev,
        {
          date: selectedDate,
          tasks: [
            {
              id: crypto.randomUUID(),
              text: newTask,
              status: 'todo',
            },
          ],
        },
      ];
    });

    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setEntries((prev) =>
      prev.map((entry) => ({
        ...entry,
        tasks: entry.tasks.map((task) =>
          task.id === taskId ? { ...task, status } : task
        ),
      }))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-4 h-4 md:w-5 md:h-5 text-gray-600" />;
      case 'done':
        return (
          <CheckCircle2 className="w-4 h-4 md:w-5 md:h-5 text-green-600" />
        );
      case 'migrated':
        return <Square className="w-4 h-4 md:w-5 md:h-5 text-blue-600" />;
      case 'cancelled':
        return <XCircle className="w-4 h-4 md:w-5 md:h-5 text-red-600" />;
    }
  };

  const currentEntry = entries.find(
    (entry) => entry.date.toDateString() === selectedDate.toDateString()
  );

  const getDaysInMonth = (date: Date) => {
    const year = date.getFullYear();
    const month = date.getMonth();
    const firstDay = new Date(year, month, 1);
    const lastDay = new Date(year, month + 1, 0);
    const days = [];

    for (let i = 0; i < firstDay.getDay(); i++) {
      days.push(null);
    }

    for (let i = 1; i <= lastDay.getDate(); i++) {
      days.push(new Date(year, month, i));
    }

    return days;
  };

  const getTaskStats = (date: Date) => {
    const entry = entries.find(
      (e) => e.date.toDateString() === date.toDateString()
    );
    if (!entry) return null;

    const stats = {
      todo: 0,
      done: 0,
      migrated: 0,
      cancelled: 0,
    };

    entry.tasks.forEach((task) => {
      stats[task.status]++;
    });

    return stats;
  };

  return (
    <div className="w-full max-w-4xl mx-auto p-4 md:p-8 bg-white rounded-lg shadow-md">
      <div className="flex items-center gap-2 md:gap-4 mb-6 md:mb-8">
        <CalendarIcon className="w-5 h-5 md:w-6 md:h-6 text-gray-600" />
        <h1 className="text-xl md:text-2xl font-semibold text-gray-600">
          Bullet Journal
        </h1>
      </div>

      <div className="grid grid-cols-7 gap-1 mb-6 md:mb-8 text-center text-sm md:text-base">
        {['S', 'M', 'T', 'W', 'T', 'F', 'S'].map((day) => (
          <div key={day} className="font-semibold py-1 md:py-2">
            {day}
          </div>
        ))}
        {getDaysInMonth(selectedDate).map((date, i) => (
          <div
            key={i}
            className={`p-1 md:p-2 min-h-[60px] md:min-h-[100px] border ${date?.toDateString() === selectedDate.toDateString() ? 'border-blue-500' : 'border-gray-200'} rounded-md ${date ? 'cursor-pointer hover:bg-gray-50' : ''}`}
            onClick={() => date && setSelectedDate(date)}
          >
            {date && (
              <>
                <div className="text-right text-gray-600 text-xs md:text-base">
                  {date.getDate()}
                </div>
                {getTaskStats(date) && (
                  <div className="mt-1 text-[10px] md:text-xs space-y-0.5 md:space-y-1">
                    {Object.entries(getTaskStats(date)!).map(
                      ([status, count]) =>
                        count > 0 && (
                          <div
                            key={status}
                            className="flex items-center justify-between"
                          >
                            {getStatusIcon(status as Task['status'])}
                            <span>{count}</span>
                          </div>
                        )
                    )}
                  </div>
                )}
              </>
            )}
          </div>
        ))}
      </div>

      <div className="flex flex-col md:flex-row gap-2 md:gap-4 mb-6 md:mb-8">
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          className="flex-1 px-3 md:px-4 py-2 text-sm md:text-base border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
        />
        <button
          onClick={addTask}
          className="px-4 py-2 bg-blue-600 text-white text-sm md:text-base rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500"
        >
          Add
        </button>
      </div>

      <div className="space-y-2">
        {currentEntry?.tasks.map((task) => (
          <div
            key={task.id}
            className="flex items-center gap-2 p-2 border border-gray-200 rounded-md text-sm md:text-base"
          >
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              className="focus:outline-none"
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              className={
                task.status === 'done' ? 'line-through text-gray-500' : ''
              }
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

The Multi-File Challenge

To do the same for our two-file approach, we now need to start thinking about what context we’re going to provide to the model. Ideally, we would generate diffs to the existing files, as this is one of the faster and more efficient ways we could make changes. There are many new tools available that can facilitate multi-file changes but needing to coordinate changes across multiple files adds complexity compared to the single-file approach. For simplicity, let’s send both files to the LLM and make the same ask (I’ll @-ref both files in Cursor in that chat, then send the same prompt from above).

Cursor outputs code like

// ... existing imports ...
import { eachDayOfInterval, endOfMonth, format, startOfMonth, isSameDay } from 'date-fns'

export default function Calendar() {
    // ... existing state ...

and

/* ... existing styles ... */

.calendar {
  display: grid;
  grid-template-columns: repeat(7, 1fr);
  gap: 4px;
  margin-bottom: 2rem;
}

so now I manually need to apply these changes (Cursor Pro has a model that supports one-click application of code changes from chat, but I’m trying to stick to just using the model for now).

Oh, and it looks like the model actually introduced new code using dependencies we don’t have installed

import {
  eachDayOfInterval,
  endOfMonth,
  format,
  isSameDay,
  startOfMonth,
} from 'date-fns';

This type of things (arguably a hallucination of sorts) happens a lot less when making changes in a single file. I hear you, it’s not a fair comparison: Tailwind in-file to CSS with a separate module file. Let’s put all the CSS styles in the same file then compare what it’s like to iterate.

User

use react and lucide-react to create an interface for a bullet-journal inspired calendar. output the react component with css styles all in one file

Bullet Journal

176 collapsed lines
import { CheckCircle2, Circle, Square, XCircle } from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

const styles = {
  container: {
    maxWidth: '800px',
    margin: '1rem auto',
    padding: '1rem',
    width: '95%',
    backgroundColor: 'white',
    borderRadius: '8px',
    boxShadow: '0 2px 4px rgba(0,0,0,0.1)',
    '@media (min-width: 640px)': {
      margin: '2rem auto',
      padding: '2rem',
    },
  },
  header: {
    marginBottom: '2rem',
    textAlign: 'center' as const,
    fontSize: '1.25rem',
    fontWeight: 'bold',
    '@media (min-width: 640px)': {
      fontSize: '1.5rem',
    },
  },
  taskInput: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
    '@media (min-width: 640px)': {
      flexDirection: 'row' as const,
      gap: '1rem',
    },
  },
  input: {
    width: '100%',
    padding: '0.5rem',
    border: '1px solid #ddd',
    borderRadius: '4px',
    fontSize: '1rem',
  },
  button: {
    width: '100%',
    padding: '0.5rem 1rem',
    background: '#333',
    color: 'white',
    border: 'none',
    borderRadius: '4px',
    cursor: 'pointer',
    '@media (min-width: 640px)': {
      width: 'auto',
    },
  },
  taskList: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
  },
  task: {
    display: 'flex',
    alignItems: 'center',
    gap: '0.5rem',
    padding: '0.5rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    wordBreak: 'break-word' as const,
  },
  statusButton: {
    background: 'none',
    border: 'none',
    padding: 0,
    cursor: 'pointer',
    display: 'flex',
    alignItems: 'center',
    flexShrink: 0,
  },
};

export default function Calendar() {
  const [tasks, setTasks] = useState<Task[]>([]);
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setTasks((prev) => [
      ...prev,
      {
        id: crypto.randomUUID(),
        text: newTask,
        status: 'todo',
      },
    ]);
    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setTasks((prev) =>
      prev.map((task) => (task.id === taskId ? { ...task, status } : task))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-5 h-5" style={{ color: '#666' }} />;
      case 'done':
        return (
          <CheckCircle2 className="w-5 h-5" style={{ color: '#22c55e' }} />
        );
      case 'migrated':
        return <Square className="w-5 h-5" style={{ color: '#3b82f6' }} />;
      case 'cancelled':
        return <XCircle className="w-5 h-5" style={{ color: '#ef4444' }} />;
    }
  };

  return (
    <div style={styles.container}>
      <meta name="viewport" content="width=device-width, initial-scale=1.0" />
      <div style={styles.header}>Bullet Journal</div>

      <div style={styles.taskInput}>
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          style={styles.input}
        />
        <button onClick={addTask} style={styles.button}>
          Add
        </button>
      </div>

      <div style={styles.taskList}>
        {tasks.map((task) => (
          <div key={task.id} style={styles.task}>
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              style={styles.statusButton}
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              style={
                task.status === 'done'
                  ? { textDecoration: 'line-through', color: '#999' }
                  : {}
              }
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

176 collapsed lines
import { CheckCircle2, Circle, Square, XCircle } from 'lucide-react';
import { useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
};

const styles = {
  container: {
    maxWidth: '800px',
    margin: '1rem auto',
    padding: '1rem',
    width: '95%',
    backgroundColor: 'white',
    borderRadius: '8px',
    boxShadow: '0 2px 4px rgba(0,0,0,0.1)',
    '@media (min-width: 640px)': {
      margin: '2rem auto',
      padding: '2rem',
    },
  },
  header: {
    marginBottom: '2rem',
    textAlign: 'center' as const,
    fontSize: '1.25rem',
    fontWeight: 'bold',
    '@media (min-width: 640px)': {
      fontSize: '1.5rem',
    },
  },
  taskInput: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
    '@media (min-width: 640px)': {
      flexDirection: 'row' as const,
      gap: '1rem',
    },
  },
  input: {
    width: '100%',
    padding: '0.5rem',
    border: '1px solid #ddd',
    borderRadius: '4px',
    fontSize: '1rem',
  },
  button: {
    width: '100%',
    padding: '0.5rem 1rem',
    background: '#333',
    color: 'white',
    border: 'none',
    borderRadius: '4px',
    cursor: 'pointer',
    '@media (min-width: 640px)': {
      width: 'auto',
    },
  },
  taskList: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
  },
  task: {
    display: 'flex',
    alignItems: 'center',
    gap: '0.5rem',
    padding: '0.5rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    wordBreak: 'break-word' as const,
  },
  statusButton: {
    background: 'none',
    border: 'none',
    padding: 0,
    cursor: 'pointer',
    display: 'flex',
    alignItems: 'center',
    flexShrink: 0,
  },
};

export default function Calendar() {
  const [tasks, setTasks] = useState<Task[]>([]);
  const [newTask, setNewTask] = useState('');

  const addTask = () => {
    if (!newTask.trim()) return;

    setTasks((prev) => [
      ...prev,
      {
        id: crypto.randomUUID(),
        text: newTask,
        status: 'todo',
      },
    ]);
    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setTasks((prev) =>
      prev.map((task) => (task.id === taskId ? { ...task, status } : task))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return <Circle className="w-5 h-5" style={{ color: '#666' }} />;
      case 'done':
        return (
          <CheckCircle2 className="w-5 h-5" style={{ color: '#22c55e' }} />
        );
      case 'migrated':
        return <Square className="w-5 h-5" style={{ color: '#3b82f6' }} />;
      case 'cancelled':
        return <XCircle className="w-5 h-5" style={{ color: '#ef4444' }} />;
    }
  };

  return (
    <div style={styles.container}>
      <meta name="viewport" content="width=device-width, initial-scale=1.0" />
      <div style={styles.header}>Bullet Journal</div>

      <div style={styles.taskInput}>
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          style={styles.input}
        />
        <button onClick={addTask} style={styles.button}>
          Add
        </button>
      </div>

      <div style={styles.taskList}>
        {tasks.map((task) => (
          <div key={task.id} style={styles.task}>
            <button
              onClick={() => {
                const nextStatus: Record<Task['status'], Task['status']> = {
                  todo: 'done',
                  done: 'migrated',
                  migrated: 'cancelled',
                  cancelled: 'todo',
                };
                updateTaskStatus(task.id, nextStatus[task.status]);
              }}
              style={styles.statusButton}
            >
              {getStatusIcon(task.status)}
            </button>
            <span
              style={
                task.status === 'done'
                  ? { textDecoration: 'line-through', color: '#999' }
                  : {}
              }
            >
              {task.text}
            </span>
          </div>
        ))}
      </div>
    </div>
  );
}

Now, let’s transform that single file with the second prompt

User

update the component to add a calendar ui element. the calendar should display the count of the number of tasks and their types each day

Bullet Journal

302 collapsed lines
import { CheckCircle2, Circle, Square, XCircle } from 'lucide-react';
import { useMemo, useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
  date: string;
};

const styles = {
  container: {
    width: '100%',
    maxWidth: '90ch',
    margin: '0 auto',
    padding: '1rem',
    backgroundColor: 'white',
    borderRadius: '8px',
    boxShadow: '0 2px 4px rgba(0,0,0,0.1)',
    '@media (min-width: 640px)': {
      padding: '2rem',
    },
  },
  header: {
    marginBottom: '2rem',
    textAlign: 'center' as const,
    fontSize: '1.25rem',
    fontWeight: 'bold',
    '@media (min-width: 640px)': {
      fontSize: '1.5rem',
    },
  },
  taskInput: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
    '@media (min-width: 640px)': {
      gap: '1rem',
    },
  },
  input: {
    width: '100%',
    padding: '0.5rem',
    border: '1px solid #ddd',
    borderRadius: '4px',
    fontSize: '1rem',
  },
  button: {
    padding: '0.5rem 1rem',
    background: '#333',
    color: 'white',
    border: 'none',
    borderRadius: '4px',
    cursor: 'pointer',
    width: '100%',
    '@media (min-width: 640px)': {
      width: 'auto',
    },
  },
  taskList: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
  },
  task: {
    display: 'flex',
    alignItems: 'center',
    gap: '0.5rem',
    padding: '0.5rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    wordBreak: 'break-word' as const,
  },
  statusButton: {
    background: 'none',
    border: 'none',
    padding: 0,
    cursor: 'pointer',
    display: 'flex',
    alignItems: 'center',
    flexShrink: 0,
  },
  calendar: {
    display: 'grid',
    gridTemplateColumns: 'repeat(7, 1fr)',
    gap: '2px',
    padding: '0.5rem',
    '@media (min-width: 640px)': {
      gap: '4px',
      padding: '1rem',
    },
  },
  calendarDay: {
    padding: '0.25rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    minHeight: '60px',
    cursor: 'pointer',
    '@media (min-width: 640px)': {
      padding: '0.5rem',
      minHeight: '80px',
    },
  },
  dayHeader: {
    fontSize: '0.75rem',
    marginBottom: '0.25rem',
    color: '#666',
    '@media (min-width: 640px)': {
      fontSize: '0.875rem',
      marginBottom: '0.5rem',
    },
  },
  stats: {
    fontSize: '0.625rem',
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '1px',
    '@media (min-width: 640px)': {
      fontSize: '0.75rem',
      gap: '2px',
    },
  },
};

export default function Calendar() {
  const [tasks, setTasks] = useState<Task[]>([]);
  const [newTask, setNewTask] = useState('');
  const [selectedDate, setSelectedDate] = useState(
    new Date().toISOString().split('T')[0]
  );

  const addTask = () => {
    if (!newTask.trim()) return;

    setTasks((prev) => [
      ...prev,
      {
        id: crypto.randomUUID(),
        text: newTask,
        status: 'todo',
        date: selectedDate,
      },
    ]);
    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setTasks((prev) =>
      prev.map((task) => (task.id === taskId ? { ...task, status } : task))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return (
          <Circle className="w-4 h-4 sm:w-5 sm:h-5" style={{ color: '#666' }} />
        );
      case 'done':
        return (
          <CheckCircle2
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#22c55e' }}
          />
        );
      case 'migrated':
        return (
          <Square
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#3b82f6' }}
          />
        );
      case 'cancelled':
        return (
          <XCircle
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#ef4444' }}
          />
        );
    }
  };

  const calendarDays = useMemo(() => {
    const today = new Date();
    const start = new Date(today.getFullYear(), today.getMonth(), 1);
    const end = new Date(today.getFullYear(), today.getMonth() + 1, 0);
    const days = [];

    for (let d = new Date(start); d <= end; d.setDate(d.getDate() + 1)) {
      days.push(new Date(d));
    }

    return days;
  }, []);

  const getTaskStats = (date: Date) => {
    const dateStr = date.toISOString().split('T')[0];
    const dayTasks = tasks.filter((t) => t.date === dateStr);

    return {
      todo: dayTasks.filter((t) => t.status === 'todo').length,
      done: dayTasks.filter((t) => t.status === 'done').length,
      migrated: dayTasks.filter((t) => t.status === 'migrated').length,
      cancelled: dayTasks.filter((t) => t.status === 'cancelled').length,
    };
  };

  return (
    <div style={styles.container}>
      <div style={styles.header}>Bullet Journal</div>

      <div style={styles.taskInput}>
        <input
          type="date"
          value={selectedDate}
          onChange={(e) => setSelectedDate(e.target.value)}
          style={styles.input}
        />
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          style={styles.input}
        />
        <button onClick={addTask} style={styles.button}>
          Add
        </button>
      </div>

      <div style={styles.taskList}>
        {tasks
          .filter((task) => task.date === selectedDate)
          .map((task) => (
            <div key={task.id} style={styles.task}>
              <button
                onClick={() => {
                  const nextStatus: Record<Task['status'], Task['status']> = {
                    todo: 'done',
                    done: 'migrated',
                    migrated: 'cancelled',
                    cancelled: 'todo',
                  };
                  updateTaskStatus(task.id, nextStatus[task.status]);
                }}
                style={styles.statusButton}
              >
                {getStatusIcon(task.status)}
              </button>
              <span
                style={
                  task.status === 'done'
                    ? { textDecoration: 'line-through', color: '#999' }
                    : {}
                }
              >
                {task.text}
              </span>
            </div>
          ))}
      </div>

      <div style={styles.calendar}>
        {calendarDays.map((day) => {
          const stats = getTaskStats(day);
          return (
            <div
              key={day.toISOString()}
              style={{
                ...styles.calendarDay,
                backgroundColor:
                  day.toISOString().split('T')[0] === selectedDate
                    ? '#f0f0f0'
                    : 'white',
              }}
              onClick={() => setSelectedDate(day.toISOString().split('T')[0])}
            >
              <div style={styles.dayHeader}>{day.getDate()}</div>
              <div style={styles.stats}>
                {stats.todo > 0 && (
                  <div style={{ color: '#666' }}>○ {stats.todo}</div>
                )}
                {stats.done > 0 && (
                  <div style={{ color: '#22c55e' }}>✓ {stats.done}</div>
                )}
                {stats.migrated > 0 && (
                  <div style={{ color: '#3b82f6' }}>□ {stats.migrated}</div>
                )}
                {stats.cancelled > 0 && (
                  <div style={{ color: '#ef4444' }}>× {stats.cancelled}</div>
                )}
              </div>
            </div>
          );
        })}
      </div>
    </div>
  );
}

302 collapsed lines
import { CheckCircle2, Circle, Square, XCircle } from 'lucide-react';
import { useMemo, useState } from 'react';

type Task = {
  id: string;
  text: string;
  status: 'todo' | 'done' | 'migrated' | 'cancelled';
  date: string;
};

const styles = {
  container: {
    width: '100%',
    maxWidth: '90ch',
    margin: '0 auto',
    padding: '1rem',
    backgroundColor: 'white',
    borderRadius: '8px',
    boxShadow: '0 2px 4px rgba(0,0,0,0.1)',
    '@media (min-width: 640px)': {
      padding: '2rem',
    },
  },
  header: {
    marginBottom: '2rem',
    textAlign: 'center' as const,
    fontSize: '1.25rem',
    fontWeight: 'bold',
    '@media (min-width: 640px)': {
      fontSize: '1.5rem',
    },
  },
  taskInput: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
    '@media (min-width: 640px)': {
      gap: '1rem',
    },
  },
  input: {
    width: '100%',
    padding: '0.5rem',
    border: '1px solid #ddd',
    borderRadius: '4px',
    fontSize: '1rem',
  },
  button: {
    padding: '0.5rem 1rem',
    background: '#333',
    color: 'white',
    border: 'none',
    borderRadius: '4px',
    cursor: 'pointer',
    width: '100%',
    '@media (min-width: 640px)': {
      width: 'auto',
    },
  },
  taskList: {
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '0.5rem',
    marginBottom: '2rem',
  },
  task: {
    display: 'flex',
    alignItems: 'center',
    gap: '0.5rem',
    padding: '0.5rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    wordBreak: 'break-word' as const,
  },
  statusButton: {
    background: 'none',
    border: 'none',
    padding: 0,
    cursor: 'pointer',
    display: 'flex',
    alignItems: 'center',
    flexShrink: 0,
  },
  calendar: {
    display: 'grid',
    gridTemplateColumns: 'repeat(7, 1fr)',
    gap: '2px',
    padding: '0.5rem',
    '@media (min-width: 640px)': {
      gap: '4px',
      padding: '1rem',
    },
  },
  calendarDay: {
    padding: '0.25rem',
    border: '1px solid #eee',
    borderRadius: '4px',
    minHeight: '60px',
    cursor: 'pointer',
    '@media (min-width: 640px)': {
      padding: '0.5rem',
      minHeight: '80px',
    },
  },
  dayHeader: {
    fontSize: '0.75rem',
    marginBottom: '0.25rem',
    color: '#666',
    '@media (min-width: 640px)': {
      fontSize: '0.875rem',
      marginBottom: '0.5rem',
    },
  },
  stats: {
    fontSize: '0.625rem',
    display: 'flex',
    flexDirection: 'column' as const,
    gap: '1px',
    '@media (min-width: 640px)': {
      fontSize: '0.75rem',
      gap: '2px',
    },
  },
};

export default function Calendar() {
  const [tasks, setTasks] = useState<Task[]>([]);
  const [newTask, setNewTask] = useState('');
  const [selectedDate, setSelectedDate] = useState(
    new Date().toISOString().split('T')[0]
  );

  const addTask = () => {
    if (!newTask.trim()) return;

    setTasks((prev) => [
      ...prev,
      {
        id: crypto.randomUUID(),
        text: newTask,
        status: 'todo',
        date: selectedDate,
      },
    ]);
    setNewTask('');
  };

  const updateTaskStatus = (taskId: string, status: Task['status']) => {
    setTasks((prev) =>
      prev.map((task) => (task.id === taskId ? { ...task, status } : task))
    );
  };

  const getStatusIcon = (status: Task['status']) => {
    switch (status) {
      case 'todo':
        return (
          <Circle className="w-4 h-4 sm:w-5 sm:h-5" style={{ color: '#666' }} />
        );
      case 'done':
        return (
          <CheckCircle2
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#22c55e' }}
          />
        );
      case 'migrated':
        return (
          <Square
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#3b82f6' }}
          />
        );
      case 'cancelled':
        return (
          <XCircle
            className="w-4 h-4 sm:w-5 sm:h-5"
            style={{ color: '#ef4444' }}
          />
        );
    }
  };

  const calendarDays = useMemo(() => {
    const today = new Date();
    const start = new Date(today.getFullYear(), today.getMonth(), 1);
    const end = new Date(today.getFullYear(), today.getMonth() + 1, 0);
    const days = [];

    for (let d = new Date(start); d <= end; d.setDate(d.getDate() + 1)) {
      days.push(new Date(d));
    }

    return days;
  }, []);

  const getTaskStats = (date: Date) => {
    const dateStr = date.toISOString().split('T')[0];
    const dayTasks = tasks.filter((t) => t.date === dateStr);

    return {
      todo: dayTasks.filter((t) => t.status === 'todo').length,
      done: dayTasks.filter((t) => t.status === 'done').length,
      migrated: dayTasks.filter((t) => t.status === 'migrated').length,
      cancelled: dayTasks.filter((t) => t.status === 'cancelled').length,
    };
  };

  return (
    <div style={styles.container}>
      <div style={styles.header}>Bullet Journal</div>

      <div style={styles.taskInput}>
        <input
          type="date"
          value={selectedDate}
          onChange={(e) => setSelectedDate(e.target.value)}
          style={styles.input}
        />
        <input
          type="text"
          value={newTask}
          onChange={(e) => setNewTask(e.target.value)}
          placeholder="Add new task..."
          onKeyDown={(e) => e.key === 'Enter' && addTask()}
          style={styles.input}
        />
        <button onClick={addTask} style={styles.button}>
          Add
        </button>
      </div>

      <div style={styles.taskList}>
        {tasks
          .filter((task) => task.date === selectedDate)
          .map((task) => (
            <div key={task.id} style={styles.task}>
              <button
                onClick={() => {
                  const nextStatus: Record<Task['status'], Task['status']> = {
                    todo: 'done',
                    done: 'migrated',
                    migrated: 'cancelled',
                    cancelled: 'todo',
                  };
                  updateTaskStatus(task.id, nextStatus[task.status]);
                }}
                style={styles.statusButton}
              >
                {getStatusIcon(task.status)}
              </button>
              <span
                style={
                  task.status === 'done'
                    ? { textDecoration: 'line-through', color: '#999' }
                    : {}
                }
              >
                {task.text}
              </span>
            </div>
          ))}
      </div>

      <div style={styles.calendar}>
        {calendarDays.map((day) => {
          const stats = getTaskStats(day);
          return (
            <div
              key={day.toISOString()}
              style={{
                ...styles.calendarDay,
                backgroundColor:
                  day.toISOString().split('T')[0] === selectedDate
                    ? '#f0f0f0'
                    : 'white',
              }}
              onClick={() => setSelectedDate(day.toISOString().split('T')[0])}
            >
              <div style={styles.dayHeader}>{day.getDate()}</div>
              <div style={styles.stats}>
                {stats.todo > 0 && (
                  <div style={{ color: '#666' }}>○ {stats.todo}</div>
                )}
                {stats.done > 0 && (
                  <div style={{ color: '#22c55e' }}>✓ {stats.done}</div>
                )}
                {stats.migrated > 0 && (
                  <div style={{ color: '#3b82f6' }}>□ {stats.migrated}</div>
                )}
                {stats.cancelled > 0 && (
                  <div style={{ color: '#ef4444' }}>× {stats.cancelled}</div>
                )}
              </div>
            </div>
          );
        })}
      </div>
    </div>
  );
}

Not bad! So what am I complaining about?

Using a model to prompt for edits to a React component with styles defined as CSS, even in the same file, pretty much always requires a full-file rewrite.

Why?

The React component with in-file CSS styles has the following structure

import { useMemo, useState } from 'react';

// other imports

const styles = {
  // 1. styles
};

export default function Calendar() {
  // 2. functions and logic

  return (
    <div style={styles.container}>
      {
        // 3. logic and markup
      }
    </div>
  );
}

Comparing diffs

When we prompt to make updates in the UI, we will almost always be making changes in areas 1 and 3 to implement new or modified functionality.

With Tailwind, the styles live right on the React markup as classNames. We can easily highlight and select smaller regions of the code and prompt the model using a tool like Cursor to make changes. This approach is more token-efficient and as a result, is faster and cheaper. It also encourages a more holistic understanding of the code you’re working with as a developer. In my experience using models to generate code, the less thinking you do or the less you understand, the less likely what you’re attempting is going to work.

Here is the diff between the two versions of the component with CSS styles in-file:

diff --git a/src/content/notes/2025/llm-tailwind-react/components/CssCalendar.v1.tsx b/src/content/notes/2025/llm-tailwind-react/components/CssCalendar.v2.tsx
index 2b34248..e177c5a 100644
--- a/src/content/notes/2025/llm-tailwind-react/components/CssCalendar.v1.tsx
+++ b/src/content/notes/2025/llm-tailwind-react/components/CssCalendar.v2.tsx
@@ -1,15 +1,16 @@
 import { Circle, CheckCircle2, Square, XCircle } from 'lucide-react'
 import { useState } from 'react'
 import { useState, useMemo } from 'react'

 type Task = {
     id: string
     text: string
     status: 'todo' | 'done' | 'migrated' | 'cancelled'
     date: string
 }

 const styles = {
     container: {
         maxWidth: '600px',
         maxWidth: '800px',
         margin: '0 auto',
         padding: '2rem',
         backgroundColor: 'white',
@@ -45,7 +46,8 @@ const styles = {
     taskList: {
         display: 'flex',
         flexDirection: 'column' as const,
         gap: '0.5rem'
         gap: '0.5rem',
         marginBottom: '2rem'
     },
     task: {
         display: 'flex',
@@ -62,12 +64,36 @@ const styles = {
         cursor: 'pointer',
         display: 'flex',
         alignItems: 'center'
     },
     calendar: {
         display: 'grid',
         gridTemplateColumns: 'repeat(7, 1fr)',
         gap: '4px',
         padding: '1rem'
     },
     calendarDay: {
         padding: '0.5rem',
         border: '1px solid #eee',
         borderRadius: '4px',
         minHeight: '80px'
     },
     dayHeader: {
         fontSize: '0.875rem',
         marginBottom: '0.5rem',
         color: '#666'
     },
     stats: {
         fontSize: '0.75rem',
         display: 'flex',
         flexDirection: 'column' as const,
         gap: '2px'
     }
 }

 export default function Calendar() {
     const [tasks, setTasks] = useState<Task[]>([])
     const [newTask, setNewTask] = useState('')
     const [selectedDate, setSelectedDate] = useState(new Date().toISOString().split('T')[0])

     const addTask = () => {
         if (!newTask.trim()) return
@@ -75,7 +101,8 @@ export default function Calendar() {
         setTasks(prev => [...prev, {
             id: crypto.randomUUID(),
             text: newTask,
             status: 'todo'
             status: 'todo',
             date: selectedDate
         }])
         setNewTask('')
     }
@@ -97,11 +124,42 @@ export default function Calendar() {
         }
     }

     const calendarDays = useMemo(() => {
         const today = new Date()
         const start = new Date(today.getFullYear(), today.getMonth(), 1)
         const end = new Date(today.getFullYear(), today.getMonth() + 1, 0)
         const days = []

         for (let d = new Date(start); d <= end; d.setDate(d.getDate() + 1)) {
             days.push(new Date(d))
         }

         return days
     }, [])

     const getTaskStats = (date: Date) => {
         const dateStr = date.toISOString().split('T')[0]
         const dayTasks = tasks.filter(t => t.date === dateStr)

         return {
             todo: dayTasks.filter(t => t.status === 'todo').length,
             done: dayTasks.filter(t => t.status === 'done').length,
             migrated: dayTasks.filter(t => t.status === 'migrated').length,
             cancelled: dayTasks.filter(t => t.status === 'cancelled').length
         }
     }

     return (
         <div style={styles.container}>
             <div style={styles.header}>Bullet Journal</div>

             <div style={styles.taskInput}>
                 <input
                     type="date"
                     value={selectedDate}
                     onChange={e => setSelectedDate(e.target.value)}
                     style={{ ...styles.input, flex: '0 0 auto' }}
                 />
                 <input
                     type="text"
                     value={newTask}
@@ -116,27 +174,53 @@ export default function Calendar() {
             </div>

             <div style={styles.taskList}>
                 {tasks.map(task => (
                     <div key={task.id} style={styles.task}>
                         <button
                             onClick={() => {
                                 const nextStatus: Record<Task['status'], Task['status']> = {
                                     todo: 'done',
                                     done: 'migrated',
                                     migrated: 'cancelled',
                                     cancelled: 'todo'
                                 }
                                 updateTaskStatus(task.id, nextStatus[task.status])
                 {tasks
                     .filter(task => task.date === selectedDate)
                     .map(task => (
                         <div key={task.id} style={styles.task}>
                             <button
                                 onClick={() => {
                                     const nextStatus: Record<Task['status'], Task['status']> = {
                                         todo: 'done',
                                         done: 'migrated',
                                         migrated: 'cancelled',
                                         cancelled: 'todo'
                                     }
                                     updateTaskStatus(task.id, nextStatus[task.status])
                                 }}
                                 style={styles.statusButton}
                             >
                                 {getStatusIcon(task.status)}
                             </button>
                             <span style={task.status === 'done' ? { textDecoration: 'line-through', color: '#999' } : {}}>
                                 {task.text}
                             </span>
                         </div>
                     ))}
             </div>

             <div style={styles.calendar}>
                 {calendarDays.map(day => {
                     const stats = getTaskStats(day)
                     return (
                         <div
                             key={day.toISOString()}
                             style={{
                                 ...styles.calendarDay,
                                 backgroundColor: day.toISOString().split('T')[0] === selectedDate ? '#f0f0f0' : 'white'
                             }}
                             style={styles.statusButton}
                             onClick={() => setSelectedDate(day.toISOString().split('T')[0])}
                         >
                             {getStatusIcon(task.status)}
                         </button>
                         <span style={task.status === 'done' ? { textDecoration: 'line-through', color: '#999' } : {}}>
                             {task.text}
                         </span>
                     </div>
                 ))}
                             <div style={styles.dayHeader}>{day.getDate()}</div>
                             <div style={styles.stats}>
                                 {stats.todo > 0 && <div style={{ color: '#666' }}>○ {stats.todo}</div>}
                                 {stats.done > 0 && <div style={{ color: '#22c55e' }}>✓ {stats.done}</div>}
                                 {stats.migrated > 0 && <div style={{ color: '#3b82f6' }}>□ {stats.migrated}</div>}
                                 {stats.cancelled > 0 && <div style={{ color: '#ef4444' }}>× {stats.cancelled}</div>}
                             </div>
                         </div>
                     )
                 })}
             </div>
         </div>
     )

Compare these two distinct areas of change to the changes in the versions using Tailwind:

diff --git a/src/content/notes/2025/llm-tailwind-react/components/TailwindCalendar.v1.tsx b/src/content/notes/2025/llm-tailwind-react/components/TailwindCalendar.v2.tsx
index 16574eb..d4767ff 100644
--- a/src/content/notes/2025/llm-tailwind-react/components/TailwindCalendar.v1.tsx
+++ b/src/content/notes/2025/llm-tailwind-react/components/TailwindCalendar.v2.tsx
@@ -77,20 +77,76 @@ export default function Calendar() {
         entry.date.toDateString() === selectedDate.toDateString()
     )

     const getDaysInMonth = (date: Date) => {
         const year = date.getFullYear()
         const month = date.getMonth()
         const firstDay = new Date(year, month, 1)
         const lastDay = new Date(year, month + 1, 0)
         const days = []

         for (let i = 0; i < firstDay.getDay(); i++) {
             days.push(null)
         }

         for (let i = 1; i <= lastDay.getDate(); i++) {
             days.push(new Date(year, month, i))
          }

         return days
     }

     const getTaskStats = (date: Date) => {
         const entry = entries.find(e => e.date.toDateString() === date.toDateString())
         if (!entry) return null

         const stats = {
             todo: 0,
             done: 0,
             migrated: 0,
             cancelled: 0
         }

         entry.tasks.forEach(task => {
             stats[task.status]++
         })

         return stats
     }

     return (
         <div className="max-w-2xl mx-auto p-8 bg-white rounded-lg shadow-md">
         <div className="max-w-4xl mx-auto p-8 bg-white rounded-lg shadow-md">
             <div className="flex items-center gap-4 mb-8">
                 <CalendarIcon className="w-6 h-6 text-gray-600" />
                 <h1 className="text-2xl font-semibold">Bullet Journal</h1>
             </div>

             <div className="mb-8">
                 <input
                     type="date"
                     value={selectedDate.toISOString().split('T')[0]}
                     onChange={e => setSelectedDate(new Date(e.target.value))}
                     className="px-4 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
                 />
             <div className="grid grid-cols-7 gap-1 mb-8 text-center">
                 {['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'].map(day => (
                     <div key={day} className="font-semibold py-2">{day}</div>
                 ))}
                 {getDaysInMonth(selectedDate).map((date, i) => (
                     <div
                         key={i}
                         className={`p-2 min-h-[100px] border ${date?.toDateString() === selectedDate.toDateString() ? 'border-blue-500' : 'border-gray-200'} rounded-md ${date ? 'cursor-pointer hover:bg-gray-50' : ''}`}
                         onClick={() => date && setSelectedDate(date)}
                     >
                         {date && (
                             <>
                                 <div className="text-right text-gray-600">{date.getDate()}</div>
                                 {getTaskStats(date) && (
                                     <div className="mt-1 text-xs space-y-1">
                                         {Object.entries(getTaskStats(date)!).map(([status, count]) => count > 0 && (
                                             <div key={status} className="flex items-center justify-between">
                                                 {getStatusIcon(status as Task['status'])}
                                                 <span>{count}</span>
                                             </div>
                                         ))}
                                     </div>
                                 )}
                             </>
                         )}
                     </div>
                 ))}
             </div>

             <div className="flex gap-4 mb-8">

The Efficiency of Tailwind with LLMs

A diff of about half the number of lines is needed to make the prompted changes in the Tailwind component. This does not suggest a Tailwind approach is necessarily better than others, but rather that a language model is more effective at following instructions requiring small, more localized modifications. Thus, if you use Tailwind/React, you have an easier time iterating and building with a language partner than several other approaches and project structures. This understanding is implicitly reflected in the approaches taken by default (presumably prompted into) tools like Claude Artifacts and v0. However, not all models and tools take this approach by default - it’s not the only way to quickly build UIs with language models, but it is a highly effective and fast way.

For completeness, here are some tools that don’t seem to use React and Tailwind by default:

Val Town’s Townie
ChatGPT Canvas

Model Biases and Context Influence

It’s also possible that claude-3-5-sonnet-20241022, one of the most popular models for coding, has a propensity for writing React/Tailwind code, which could influence the approaches taken by popular tools. When prompting Claude with similar prompts as above, I usually get React code, even when I don’t ask for it. With other popular models like deepseek-chat or gpt-4, I don’t always get React. Sometimes I just get plain HTML or other frameworks like Vue (when I prompt for everything in a single file).

Part of the behavior we’re seeing is context-specific as well. When you prompt the model to create a component like the one we’ve described, if I am working in Cursor and in a file called <whatever>.tsx, that gets passed to the model as context and influences the code it generates. It would definitely not be what we wanted if we got Vue code in our .tsx file.

As someone who has leaned heavily on LLMs to write UIs for me, I’ve found React and Tailwind to be a particularly potent combination for fast iteration. After diving into the actual code structure above, why that is now makes a bit more sense.

Thanks for reading via RSS!