John Wilger

Somebody Has to Design the Factory

Unknown — Sun, 26 Jul 2026 00:00:00 +0000

Four rules got added to my agent harness the morning I built Lanyard</a>. Ticket work has to happen in an isolated worktree. Commits and pushes from the primary checkout are blocked. A deployment isn't a deployment until a live URL answers. And a task-closing command's exit code doesn't count as evidence that the task closed.</p>

Each of them came from something going wrong that day. None of them are features of Lanyard, which is an SSH proxy and has nothing to do with any of it. Lanyard was close to a byproduct. What I actually spent the morning on was a slightly better factory.</p>

That word has started showing up in conversations about where agentic development is heading, usually as "dark factory," and about half the time I use it I can watch the person across from me hear something I didn't say.</p>

What people picture</h2>
The image is a text box. You type what you want, some quantity of machine intelligence happens, and a finished product comes out the other side. No engineers, no designers, no product manager, no argument about what the thing should do. The prompt is the spec, the output is the product, and the entire middle got vaporized.</p>
Products shaped like that exist and some are good at what they do, in exactly the way Canva is good at making you a coffee mug with a squirrel on it</a>. They aren't what the dark factory argument is about, and treating them as its endpoint makes the actual claim impossible to discuss.</p>
What's happening there is a category error rather than a disagreement. It collapses three separate activities into one: deciding what to build, engineering how it works, and producing it. A dark factory isn't a wish granted. FANUC has run one in Oshino since 2001</a>, where robots assemble roughly fifty robots a day and the line runs unsupervised for up to thirty days at a stretch. The lights are off because nobody is on the floor to need them. According to one of their VPs, they turn off the heat and air conditioning too.</p>
That plant is one of the most heavily engineered objects on the planet. Every one of those thirty unattended days rests on people who decided what a robot should be, people who worked out how to make one, and people who worked out how to make a machine make one. Nobody typed "robot" into anything.</p>

Benz's workshop had no factory</h2>
When Carl Benz and Gottlieb Daimler were building the first automobiles in the mid-1880s, they worked in engineering workshops. Parts were forged, cast, and machined, often by specialist suppliers who had been making something else entirely the year before. Then skilled workers assembled the car by hand, filing, shimming, and reaming each piece until it mated with the one next to it.</p>
That fitting wasn't unskilled labor. A fitter was reading the actual assembly in front of him, feeling where it bound, and correcting for the gap between what the drawing claimed and what the shop had produced.</p>
An enormous amount of product design and mechanical engineering went into those cars, and much of the learning was only available with your hands on the thing. You found out how a component behaved by making one, setting it next to another one, and watching what happened.</p>
Ford automated the fitting, not the designing</h2>
The moving assembly line arrived at Highland Park in 1913, and the thing that made it possible wasn't the conveyor. It was parts held to tolerance tightly enough that fitting became unnecessary. A line only works if the part coming down the chute drops into place without anyone deciding anything about it. Ford didn't invent that idea; armory practice in the American arms industry had been chasing interchangeable parts for most of the preceding century. Ford industrialized it, at a scale and a price nobody had managed before.</p>
Getting there took a body of engineering that had barely existed as a job. Somebody had to design the gauges that said whether a part was in spec. Somebody had to design the jigs and fixtures that held the work in exactly one orientation so it couldn't be machined backwards. Somebody had to cut the dies, balance the stations so no position starved or flooded the next, and run the tolerance stack-up analysis that tells you whether a chain of individually acceptable parts still adds up to a door that closes.</p>
Product design didn't shrink when the line showed up. A second design discipline grew up beside it, hard and expert and well paid, and we now call it manufacturing engineering.</p>
Notice what the tolerance regime actually eliminated. The fitter with a file was a human compensating, one part at a time, for a process that couldn't hold its own spec. Holding the spec is what made the compensation unnecessary. Filing the part down is what you do when the factory doesn't work.</p>
The fitters didn't become tool designers</h2>
The aggregate story is that automotive engineering employment grew enormously over the following century. The story for an individual fitter in 1913 was worse than that.</p>
Work on the line was miserable in a way the craft work hadn't been. Ford hired 50,448 people that year to sustain an average workforce of 13,623, a turnover rate around 370 percent, which meant that growing the workforce by a hundred took hiring nearly a thousand. In January 1914 Ford answered by roughly doubling pay: five dollars for an eight-hour day</a> against the previous $2.34 for nine. Turnover fell to 54 percent. Nobody doubles wages to keep people in a job they like.</p>
So the claim I'm making is narrow. The work didn't disappear, and there turned out to be more of it, at higher skill, than before. That isn't the same statement as "your job is safe," and I'm not making the second one. Aggregate growth is cold comfort when the growth lands on different people, in a different city, a decade later. Anyone promising you that the transition to agentic development will be smooth for everyone currently employed is guessing, and guessing in the direction that happens to be comfortable for them.</p>
Nobody sells you a factory</h2>
Cars didn't get simpler. Neither did the engineering. A modern vehicle program employs more engineers across more disciplines than Benz could have imagined, and it delivers more variety at lower cost to more people.</p>
What matters most for software is what happens when a company decides to build a new model. They don't roll the last one's factory over onto it. They design a production system for that product: new dies, new fixtures, new gauges, a rebalanced line, a revalidated supply chain. Platforms and plants get shared, and a flexible line can build more than one model, but the tooling specific to a product is designed fresh, by experts, at large expense, every single program.</p>
That's the fact that should make you suspicious of anyone offering to sell you a software factory. In manufacturing, machinery is a thing you can buy. The production system is a thing your engineers design for what you're building. When somebody's pitch is that you can buy the second one, they're selling you machinery and calling it a factory.</p>
The line produces changes, not products</h2>
The analogy has an obvious hole and it's worth walking straight into it. Physical factories exist because replication is expensive. You design a car once and then face the separate, enormous problem of producing a hundred thousand identical ones. Software solved that problem decades ago. Copying a build costs nothing. `cp</code> is the entire assembly line, and it was never the interesting part.</p>`
`So what would a software factory be for?</p>`
`The thing that repeats in software isn't the product, it's the process. The unit coming off the line is a change: a feature, a fix, a migration, a dependency bump. Each one runs the same sequence. Work out what the behavior should be, decide what will be expensive to reverse later, write a test that fails for the right reason, make it pass, review it, integrate it, produce evidence that it actually deployed. Different content every run, identical sequence.</p>`
</div> A factory that outputs changes is still a factory, and it describes what's being automated better than a conveyor of identical bodies does. The closer physical analogue is a job shop running standard work: every job different, every job pushed through the same validated process with the same fixtures and the same gauges.</p> This is where the analogy strains hardest. A stamping press doesn't read an acceptance criterion and decide whether it's been met. My harness has to. The automated steps need judgment in them, and bounding that judgment is the whole design problem: enough to absorb a different change every time, not so much that the process becomes a negotiation.</p>Tolerances, for software</h2> Tolerance stops being a metaphor as soon as you ask what the harness refuses to do.</p> Production code can't exist before a failing test has been observed, and the failure has to come from the behavior being missing rather than from a typo or a broken fixture. Code that shows up without that gets reverted rather than debated. That's a station that can't be run out of sequence.</p> Test evidence is bound to a hash of the exact diff it was produced against, and the clean-review streak resets to zero the moment that hash changes. Three clean review passes followed by one more edit is zero clean review passes. An inspection record belongs to a specific part, not to the general vicinity of the part.</p> A critical security or human-safety problem introduced by the change itself can only be dispositioned as fixed. "Defended" and "accepted risk" exist for other categories and are unavailable for that one. That's a gauge rejecting a part rather than a reviewer holding an opinion about it.</p> CI recovery runs off a table compiled into the tool instead of described in a document. A failure classified as caused by the change may only be repaired. One classified unrelated or transient may only be rerun at the unchanged commit. There is no third option, the classification has to be recorded before an action is chosen, and the hold doesn't release until a matching replacement run reaches terminal success. Queued isn't success. Neither is a zero exit code.</p> Reviews run in fresh contexts that have to attest they were fresh, and the reviewing role has no write tools at all, because an inspector who can adjust the part isn't an inspector.</p> The rule underneath all of them is the one I'd point at first. No agent working in my repositories may install a substitute tool, weaken a required gate, or fall back on stale memory in order to keep moving. When nothing available can satisfy a gate, the correct behavior is to stop at the gate and come get me.</p> That rule is the fitter's file, confiscated on purpose. Weakening a gate so a change can land is hand-fitting: a human compensating, one part at a time, for a process that can't hold its own spec. It works, it's invisible in the finished product, and it's the reason nothing downstream of it can be trusted. The entire point of a tolerance regime is that nobody has to reach for the file.</p> Stations get different capability on purpose. Bounded mechanical work runs on a small model with read-only tools and has to hand back evidence the caller can verify independently, because its own account of what it did doesn't count. Architecture and security analysis go to a stronger model that also can't write. That's a fixture-and-gauge argument rather than a model-quality argument.</p> None of this is the model getting better. I've written before about making bad states structurally impossible rather than discouraged</a>, and about how a guardrail nobody has tested for behavior change may not be a guardrail</a>.</p> The part that needs your best people</h2> The product manager still decides what the product is. The senior engineers still decide which parts go together and which choices will be expensive to reverse. None of that moved. What's new is that those same people are needed to design the factory, and that's the work I'd put the most experienced person in the room on.</p> Every gate I described started as something a staff-level engineer already knows. Don't trust an exit code. Don't let a green review outlive the diff it reviewed. Don't let the author sign off on their own work. That knowledge is normally tacit, exercised a few dozen times a week by people who would struggle to write it down if you asked them to.</p> Encoding it is harder than exercising it. Exercising judgment lets it stay vague. Encoding it forces you to state exactly what the standard is, exactly what evidence satisfies it, and exactly what happens when the evidence is missing, in a form something else can check with you out of the room. Most teams have never had to do that, which is a large part of why they don't have a factory. The machinery isn't what's missing. What's missing is a written-down definition of good, precise enough to gauge.</p> Two claims get mashed together here and they're worth separating. The line didn't make any individual worker faster at fitting a part. It removed the fitting. What improved was throughput and rework, which are properties of a system rather than of anybody's hands. I've argued that AI probably isn't making you faster</a>, and I still think that holds for one person doing one task. A factory is a different unit of analysis. You can have a well-designed one and remain personally slower at everything, and that would be fine.</p> Where the lights stay on</h2> The lights go off over the line. They stay on in the room where somebody decides what to build, and in the room next door where somebody decides how it gets built the same way ten thousand times.</p> That second room barely existed for software five years ago. It's where I've spent most of the last year.</p>



The Software I Wouldn't Have Bothered to Build
Unknown — Sun, 19 Jul 2026 00:00:00 +0000
This morning I had an annoyance. By lunchtime it had a name, an architecture,
a public repository, a Rust implementation, ADRs, strict tests, release
automation, and a website that looks better than most developer-tool sites
have any right to</a>.</p>
The tool is Lanyard</a>, an SSH-agent
switching proxy. I built it because I use 1Password's SSH agent when I'm sitting
at my desktop, but I also SSH into that desktop from my laptop and forward the
laptop's agent. Both routes work fine until they meet a long-lived Zellij
session.</p>
SSH_AUTH_SOCK</code> inside the session points at whichever socket existed when the
session started. Attach locally and the session knows about 1Password. Attach
remotely and it may know about the forwarded agent. Move between the two and
the environment doesn't follow you. Authentication and Git commit signing then
depend on which machine happened to create the session hours or days earlier.</p>
I'd put up with this for a while. The usual next step would have been a shell
function that rewrote an environment variable, a symlink shuffled between
sockets, or some startup hook whose behavior I'd forget six months later. It
would mostly work. It would also fail in exactly the cases that made the problem
interesting.</p>
Instead, I spent some time defining the problem with a coding agent. We worked
through the architecture, wrote down the decisions that would be expensive to
change later, and split the work into increments I could review independently.
Then I left the computer and did other things with my life while the agent built
it.</p>
The gap between a workaround and a tool</h2>
The first idea sounds almost trivial: expose one stable Unix socket and forward
requests to whichever SSH agent works.</p>
That sentence hides most of the engineering.</p>
An SSH agent isn't just a bag of keys behind a socket. Clients list identities,
request signatures, query extensions, and establish OpenSSH session bindings.
Some operations are safe to retry after a disconnect. Some aren't. A failed
signature request may mean "I don't have that key," "the user denied access,"
or "I signed it and the response was lost." Forwarded sockets appear and
disappear as SSH connections come and go. A local desktop agent can be present
but locked. Multiple agents may advertise the same public key with different
comments.</p>
A shell script can choose a socket before launching Git. It can't make one
long-lived client connection see a deduplicated union of identities, route a
signature request to the agent that can satisfy it, replay an acknowledged
session binding on a newly selected backend, and distinguish safe failover from
an ambiguous mutation.</p>
Once I wrote the problem down that way, the shape of Lanyard became clear:</p>

A stable SSH_AUTH_SOCK</code> under the user's runtime directory.</li>
Automatic discovery of same-user OpenSSH forwarding sockets.</li>
Explicit registration and removal for login hooks and unusual socket paths.</li>
A configured local agent, normally 1Password, kept as the final fallback.</li>
Identity aggregation with deterministic ordering and key-blob deduplication.</li>
Per-operation timeouts, bounded messages, and a hard candidate limit.</li>
Read-only routing and narrowly supported OpenSSH extensions, with agent
mutation requests rejected.</li>
A local control socket with machine-readable status rather than state hidden
in shell variables.</li>
</ul>
That's a small systems program, not a clever alias.</p>
The part I delegated</h2>
I've been writing software long enough that I could have built this without an
agent. Realistically, I wouldn't have.</p>
The expected value was wrong. A few hours of occasional SSH-agent irritation
didn't justify several evenings of reading protocol documents, building a Unix
socket server, writing integration tests with fake agents, packaging it, and
documenting the result. I'd have spent twenty minutes on a brittle script,
declared it good enough, and paid the annoyance tax indefinitely.</p>
Agentic development changed that calculation, but not because an LLM can type
Rust faster than I can. The useful part was being able to stop typing at all.</p>
Before I stepped away, the agent and I turned the idea into a
Tiber</a> backlog.
We worked out the trust boundary and the failure policy. I approved the general
architecture: a stable proxy socket, multiple dynamically discovered backends,
identity aggregation, operation-specific failover, and a separate control
socket. The work was divided into semantic increments covering the repository
harness, proxy behavior, discovery, adaptive signing, packaging, releases, and
documentation. Decisions with expensive consequences went into ADRs.</p>
At that point, the work no longer needed me continuously. The agent could take
the next ticket, write a failing test, implement the behavior, run the rest of
the gates, review the diff, commit it, push it, watch CI, and move on. I didn't
sit there approving every command or reading every intermediate patch. I went
away.</p>
That was possible because this wasn't a bare agent dropped into an empty
repository with a prompt to "build an SSH proxy." I've spent a lot of time on
the harness around it: test-first development, strict compiler and linter
settings, mutation testing, architectural decision records, isolated
worktrees, independent review passes, commit and push gates, and instructions
for watching CI and deployed behavior. I'm currently consolidating that work in
my ai-plugins repository</a>.</p>
Those guardrails are what let me trust the agent with long stretches of
autonomous work. Trust here doesn't mean assuming it won't make mistakes. It
means I know what evidence it has to produce before it can move on, and I know
which decisions will make it stop and wait for me.</p>
For Lanyard, fake SSH agents exercised framed protocol messages over real Unix
sockets. Black-box tests launched the daemon, registered agents through its
control socket, inspected JSON status, sent termination signals, and checked
socket ownership and cleanup. Strict Clippy settings turned suspicious
shortcuts into build failures. cargo-deny</code> checked dependency policy. Mutation
testing changed the program and checked whether the tests noticed.</p>
The review passes found bugs that would have survived a demo:</p>

A session binding could pin a client to the first agent and prevent signing
with a key advertised by another.</li>
A signing disconnect was initially treated as fail-closed across the whole
candidate set, contradicting the availability policy recorded in the ADR. It
needed to advance to a different agent without retrying the same one.</li>
An ambiguous session-bind</code> disconnect had the opposite requirement because
replaying that mutation onto another backend could apply it twice.</li>
If every agent returned malformed identity data, the proxy reported a valid
empty key list instead of a protocol failure.</li>
Aggregated identity responses could exceed the same message bound enforced on
individual upstream responses.</li>
A failed control-socket bind could leave behind an orphaned agent socket.</li>
The first GitHub Pages run started before Pages had been enabled on the
repository and failed with a 404.</li>
</ul>
I wasn't present when most of these were introduced or found. The agent worked
through failing tests, surviving mutants, review findings, CI failures, and
deployment checks on its own. It came back when a decision crossed a boundary
we'd established or when the available evidence couldn't settle the question.
That distinction took time to build into my workflow. Without it, "autonomous"
usually means either reckless or exhausting: the agent barrels through choices
it shouldn't make, or it interrupts so often that I may as well write the code.</p>
Where I still had to be involved</h2>
Lanyard handles signing credentials, so its trust boundary needed to be
explicit. It accepts only same-user discovered sockets, creates a private
runtime directory and owner-only sockets, rejects SSH-agent mutation requests,
bounds input and output messages, caps candidate fan-out, and applies operation-
specific timeouts. An acknowledged OpenSSH session binding is retained and
replayed when a later request reaches a new backend.</p>
The failure policy is operation-specific rather than uniformly "retry" or
uniformly "stop." Identity listing and queries can move through candidates.
An explicit signing failure can fall through because another agent may own the
same key. A signing disconnect may advance to a different backend under the
availability policy, but it won't resend the same request to the same backend.
An ambiguous session-binding disconnect stops because duplicating that mutation
crosses a different safety boundary.</p>
Release automation produced one of the decisions that did need me. The reusable
release workflow I normally use pins its top-level revision but invokes nested
actions by mutable tags in jobs that handle publishing credentials. The agent
stopped rather than weakening the repository's supply-chain policy to get a
green release. We left credential-bearing publication fail-closed until the
shared workflow can be fixed and repinned. GitHub Pages could still deploy.</p>
The run still exposed holes in that setup. The agent initially missed my
worktree plugin and made application changes in the primary checkout while two
other tickets ran in worktrees. Publishing the documentation site revealed that
the Pages workflow was stranded on another branch and Pages hadn't been enabled
for the repository. A task-closing command exited successfully once without
actually moving the task to done</code>.</p>
Each failure became a change to the harness: require ticket work to happen in
isolated worktrees, block primary-checkout commits and pushes, verify a deployed
URL instead of treating a successful push as a deployment, and inspect the
resulting task state instead of trusting an exit code. That's the investment
that compounds. The next project starts with the failures from this one already
encoded.</p>
The economics of caring</h2>
The phrase "AI makes developers faster" doesn't describe what happened here. I
didn't type Rust faster. For much of the implementation, I wasn't typing Rust
at all.</p>
Yesterday, this annoyance belonged below the line: too small to deserve a real
program, too fiddly to solve properly, not painful enough to consume a weekend.
The available options were to tolerate it or build something I wouldn't be
proud to share.</p>
This time I paid the part of the cost that needed my judgment: defining the
problem, choosing the architecture, deciding how signing failures should behave,
and setting the boundaries for credentials and release automation. The harness
carried much of the remaining work while I spent my attention elsewhere. That
made ADRs, mutation tests, cross-architecture artifacts, Home Manager
integration, and documentation reasonable for a personal utility.</p>
There are thousands of annoyances like this in the gaps between general-purpose
tools. Most never become software because the engineering cost is larger than
the irritation. They become shell-history archaeology, half-remembered dotfile
snippets, and habits built around bugs nobody had time to remove.</p>
Now Git can sign a commit whether I attached to Zellij from the chair in front
of my desktop or from a laptop somewhere else. I got the tool without donating
the rest of my day to its implementation. I wouldn't have made that trade before
I could trust the surrounding system to keep working, catch mistakes, and know
when to call me back.</p>


AI Might Not Make You Faster. It Can Make You Free.
Unknown — Fri, 10 Jul 2026 00:00:00 +0000
Every pitch for AI-assisted development leads with speed. Ship faster, close more tickets, multiply your output. I've spent the past year and a half building my daily workflow around coding agents, and speed isn't what I got out of the deal. Some days I doubt I got any. What I got back was my attention, and it took me longer than I'd like to admit to notice that was the better prize.</p>
Nobody checks the stopwatch</h2>
For a single, well-defined unit of work, my agent-assisted workflow takes about as long as doing the work myself. Sometimes longer. I haven't timed myself rigorously, so take that as an honest impression rather than data. METR did the rigorous version: a randomized trial in mid-2025</a> where experienced open-source developers working in their own repositories were about 19% slower</em> with AI tools, while estimating afterward that they'd been about 20% faster. Tokens stream onto the screen instantly, so the whole thing feels fast. Whether the work actually finished sooner is a different question, and almost nobody goes back and checks the clock.</p>
I know where my time goes, because I make the agent do everything I'd demand from a human engineer, and I refuse to skip steps just because a machine is doing them. In my ai-plugins setup</a>, the agent works test-first, debugs by forming and falsifying hypotheses instead of pattern-matching a fix, and can't claim work is done without running the verification it's claiming. The engineering-standards rules add the gates I'd want on any team: strict lints, mutation testing where the stack supports it, ADRs for decisions that will outlive the branch, a real review pass before anything gets called ready.</p>
None of that is fast. It isn't fast when humans do it either. We just already knew that about humans.</p>
Slop is a choice</h2>
The standard complaint about AI code — verbose, subtly wrong, untested, architecturally incoherent — describes what you get when you accept the first thing the model produces. It doesn't describe a ceiling.</p>
With those gates in place, the code coming out of my agent workflow is regularly better than what I'd produce by hand, and better than what most teams I've worked with typically ship. Not because the model is smarter than the team. Because the agent actually does the discipline, every time, without getting tired of it.</p>
Be honest about your own team for a second. By the fifteenth PR of the week, does every change still get a failing test written first? Does anyone still run the mutation suite before review? Does the ADR get written, or does the decision live in a Slack thread that expires? Humans negotiate with discipline when we're tired or behind. An agent with the discipline encoded as hard gates doesn't negotiate. It will write the property test, chase the surviving mutant, update the ADR, and run a fourth review pass, because tedium isn't something it experiences. (Though it will occasionally claim</em> otherwise. Claude Code has told me, with a straight face, that it wasn't going to bother with something "due to time constraints" — despite having no clock, no deadline, and no meeting to get to. That's not tedium. That's a training set full of humans who've said the same thing to get out of writing one more test. Brilliant!)</p>
The quality ceiling is set by the guardrails you build, not by the model. Most teams haven't built these up sufficiently, and then blame the model for the slop.</p>
The slot machine</h2>
So if better-than-usual quality is on the table, why is so much AI-assisted code garbage?</p>
Because the interface is a slot machine</a>, as Steve Yegge put it: each request a pull of the lever with potentially infinite upside or downside. Type a prompt, tokens pour out, something that looks like working software materializes in seconds. Pull the lever, get a reward, pull again. The loop trains you to value the moment code appears instead of the moment work is done, and those two moments can be hours apart.</p>
</p>
The waiting is the hard part, and not for the reason I expected. When my agent picks up a task, there's a long stretch where it's writing tests, watching them fail, making them pass, fighting the mutation suite, reviewing its own diff. It doesn't need me for any of that. The work is happening — but it feels</em> like I'm not doing anything, because like most of us I've spent my whole career being trained to equate time at the keyboard with productivity. My brain refuses to count work it can see happening if my hands aren't the ones doing it. And the reflex that itch produces isn't to interrupt the agent. It's to pull another lever: spin up a second track of work, then a third, until the feeling of idleness goes away.</p>
The gap between code appearing and work being done is where the machine collects. In March 2025</a> I was sitting alone at a restaurant table, waiting for my wife to join me for date night, when Google emailed to thank me for my payments. Just under $2,000 in Places API fees, run up earlier that day by an agent-built utility. The code wasn't wrong. I'd reviewed it, understood it, watched the tests pass. What I'd skipped was the slower engineering around it, like reading the pricing page for the API tier that two innocent-looking output fields had quietly bumped me into. No guardrail I had in place forced that review, and the agent wasn't going to volunteer it. This was personal money, not a client's, which is the only mercy in the story — though the buck stops with the engineer either way, and this time it was literal bucks. My wife arrived to find me staring at my phone, and explaining what the number on it was did not improve the evening. The code had shown up fast. The engineering hadn't happened yet.</p>
The people who tell me AI coding produces garbage are, almost without exception, describing a workflow where output gets accepted at the moment of generation. They're playing the slot machine and complaining about the payout. The patience to let the full process finish, even when it takes as long as doing the work yourself, turns out to be the actual skill, and it's rarer than prompt-writing.</p>
What you do with the hours</h2>
So a unit of work takes as long or longer, and the quality comes out higher. What changed is where you are while it happens. The agent needs you at the edges — defining the problem, making the architectural calls, reviewing the result — and doesn't need you in the middle. The middle is most of the hours.</p>
One thing you can do with those hours is run more of them. One person, several parallel tracks of work. This works; I've had an agent babysitting a PR through CI and review</a> while another worked a feature branch in its own worktree and I did design work on a third thing. And for all my hedging about raw speed, the volume is real: I've never been more prolific on the side projects I actually care about — eventcore</a>, auto_review</a>, and a bunch of others</a> that would have stayed ideas in a notes file for lack of nights and weekends to spend on them.</p>
But parallelism also rebuilds the slot machine one level up. Three levers now, and one of them is always worth checking. It creeps into evenings, and I can tell you exactly which work did the creeping, because it was those same side projects. I've caught myself checking agent progress from my phone at dinner the way other people check markets. If parallelism turns you into a fourteen-hour-a-day agent supervisor, you've automated the work and doubled the job.</p>
The other thing you can do with the hours is live them. The agent turns ideas and architecture into working, tested, reviewed software over an afternoon, and nothing about that requires you to watch. Go for a walk. Cook dinner. Pick your kid up from school without a laptop in the car. The work finishes to a higher standard than a rushed human afternoon would have produced, and you weren't chained to the desk while it happened.</p>
That same keyboard-hours training that makes the waiting feel like slacking is what makes this option so hard to choose — the best version of this job now involves not watching</em>, and every instinct I built over twenty-plus years says that can't be right. But that's the dividend. Not velocity. Autonomy over your own attention.</p>
If you want the good version</h2>
Encode your discipline as gates the agent can't skip: test-first, verification before any "done" claim, a real review. Instructions the agent can talk its way past don't count; I've written before</a> about testing whether these guardrails actually change behavior, because plenty of them don't.</p>
Judge the workflow by finished work, not by how fast code appears on screen. When the agent is mid-process, let it finish, and notice what the idle time makes you want to do; the itch to fill it is the tell that you're pulling the lever instead of running an engineering process. And decide on purpose what the freed hours are for, because if you don't choose, the itch will choose for you, and its answer is always more levers.</p>
The time between</h2>
Since I stopped treating the middle hours as a problem to fill with more work, I've put them into something I never saw coming: I'm learning to play the violin.</p>
Learning a new piece or a new technique takes my full attention, so I don't attempt it while agents are running. But technique practice — scales, drills, bowing exercises — fits in the gaps, and there are hours of it to do. The ear listening for intonation and the muscle memory forming in my hands run on different parts of the brain than reading agent output and thinking about product and architecture, so I can practice and still notice when a gate fails or a decision needs a human. My hands are too busy to pull levers. It turns out that's exactly what was needed to stop the all-day-every-day-must-be-making-code addiction that had started to set in.</p>
Speed was never the real offer. The real offer is better software and your life back, and you only collect if you can stop pulling the lever long enough to let the machine finish.</p>
What would you do with the time between?</p>


Making Chrome Profiles Real Launch Targets on Niri
Unknown — Sun, 05 Jul 2026 00:00:00 +0000
I use separate Chrome profiles for personal and work browsing. Chrome supports that, but its desktop integration didn't match the way I wanted to use profiles from a keyboard-first Linux desktop.</p>
The problem wasn't that Chrome lacked a profile picker. The problem was that "open Chrome" had become an underspecified action. Sometimes I meant personal. Sometimes I meant work. Sometimes I clicked a link from a focused Niri workspace that already had the right browser context, and I wanted the link to join that workspace instead of pulling me somewhere else.</p>
So I stopped trying to make Chrome's generic launcher carry all of that policy. I made the actions explicit:</p>
chrome-personal [URL...]
chrome-work [URL...]
chrome-pick [URL...]
</code></pre>
Then I made a small URL router, nignite</code>, and put it on both URL entry paths I use: XDG desktop handling and kitty URL clicks. The real implementation lives in my public NixOS config: modules/home/desktop/nignite.nix</code></a>, with the terminal URL path configured in modules/home/kitty.nix</code></a>.</p>
The result is a split between two different entry points. Launcher search shows the human choices. URL handling uses the router. Chrome still owns Chrome state.</p>
</p>
What was wrong</h2>
There were three annoyances tangled together.</p>
First, launcher search was ambiguous. Searching for Chrome could surface the generic Google Chrome desktop entry, but that entry didn't encode the decision I usually needed to make: personal profile or work profile.</p>
Second, external links didn't always land in the context I expected. With Niri, workspace locality is part of how I think about work. If workspace 3 already has a browser window for a task, a link clicked from that workspace should usually join that browser window. Jumping to an existing Chrome window on another workspace breaks that flow.</p>
Third, Chrome's profile picker is mutable application state. I didn't want my Nix configuration to manage Chrome's Local State</code> file just to influence the profile behind the generic launcher. Chrome owns that file. Declarative desktop configuration shouldn't have to pretend otherwise.</p>
That changed the framing. The question wasn't "how do I configure Chrome's picker?" It was "what browser actions do I actually want available from the desktop?"</p>
The small commands</h2>
The profile-specific launchers don't inspect Chrome state or talk to the compositor. In Home Manager, they're writeShellApplication</code> derivations that add exactly one Chrome flag:</p>
chromePersonal = pkgs.writeShellApplication {
  name = "chrome-personal";
  runtimeInputs = [ browserPackage ];
  text = ''
    exec ${browserExe} --profile-directory=Default "$@"
  '';
};

chromeWork = pkgs.writeShellApplication {
  name = "chrome-work";
  runtimeInputs = [ browserPackage ];
  text = ''
    exec ${browserExe} --profile-directory="Profile 4" "$@"
  '';
};
</code></pre>
Those profile directory names come from Chrome's local profile metadata. In my setup, Default</code> is personal and Profile 4</code> is work. That's not a portable truth about Chrome; it's a local fact that belongs in my machine configuration.</p>
The picker is just as small:</p>
chromePick = pkgs.writeShellApplication {
  name = "chrome-pick";
  runtimeInputs = [
    chromePersonal
    chromeWork
    pkgs.fuzzel
  ];
  text = ''
    choice="$(printf 'Personal\nWork\n' | fuzzel --dmenu --prompt='Chrome profile: ' --lines=2 --width=24)" || exit 0

    case "$choice" in
      Personal)
        exec chrome-personal --new-window "$@"
        ;;
      Work)
        exec chrome-work --new-window "$@"
        ;;
      *)
        exit 0
        ;;
    esac
  '';
};
</code></pre>
The --new-window</code> is doing real work. When the picker appears, it's because the router didn't find a Chrome window on the current workspace. Selecting a profile should create a browser window in the current workspace, not let Chrome reuse a session somewhere else.</p>
</p>
Routing links by workspace</h2>
nignite</code> is the default handler for URLs. Its policy is:</p>

Ask Niri which workspace is focused.</li>
Look for a Chrome or Chromium window on that workspace.</li>
If one exists, focus it and open the URL as a new tab.</li>
If none exists, ask which profile to use.</li>
If the chooser is canceled, do nothing.</li>
</ol>
The focused workspace lookup uses Niri's JSON IPC:</p>
focused_workspace_id="$(
  niri msg -j focused-window 2>/dev/null \
    | jq -er '.workspace_id // empty' 2>/dev/null \
    || niri msg -j workspaces 2>/dev/null \
    | jq -er '.[] | select(.is_focused == true) | .id' 2>/dev/null \
    || true
)"
</code></pre>
The fallback to workspaces</code> is there because a workspace can be focused even when no window is focused. That's an easy case to miss if the script only asks for the focused window.</p>
Once the router has a workspace ID, it searches Niri's window list:</p>
chrome_window_id="$(
  niri msg -j windows 2>/dev/null \
    | jq -er --argjson workspace_id "$focused_workspace_id" '
        [
          .[]
          | select(.workspace_id == $workspace_id)
          | select(
              ((.app_id // "") | test("chrome|chromium"; "i"))
              or ((.title // "") | test("chrome|chromium"; "i"))
            )
        ][0].id
      ' 2>/dev/null \
    || true
)"
</code></pre>
If it finds a browser window, it focuses that window before handing Chrome the URL:</p>
if [ -n "$chrome_window_id" ]; then
  niri msg action focus-window --id "$chrome_window_id" >/dev/null 2>&1 || true

  if [ "$#" -eq 0 ]; then
    exit 0
  fi

  exec ${browserExe} --new-tab "$@"
fi

exec chrome-pick "$@"
</code></pre>
This isn't trying to infer whether the current workspace is "personal" or "work." That would be a bigger claim than the desktop can support. It only says: if a browser context already exists on this workspace, reuse it. Otherwise ask me.</p>
Desktop entries and terminal clicks</h2>
The visible launchers are normal XDG desktop entries:</p>
xdg.desktopEntries = {
  chrome-personal = {
    name = "Chrome Personal";
    exec = "${lib.getExe chromePersonal} %U";
    icon = "google-chrome";
    genericName = "Web Browser";
    noDisplay = false;
    terminal = false;
  };

  chrome-work = {
    name = "Chrome Work";
    exec = "${lib.getExe chromeWork} %U";
    icon = "google-chrome";
    genericName = "Web Browser";
    noDisplay = false;
    terminal = false;
  };
};
</code></pre>
The generic Chrome entry is still accounted for, but hidden:</p>
google-chrome = {
  name = "Google Chrome";
  noDisplay = true;
};
</code></pre>
The router is also hidden because I don't launch it directly:</p>
nignite = {
  name = "Chrome Workspace Router";
  exec = "${lib.getExe nignite} %U";
  icon = "google-chrome";
  noDisplay = true;
  terminal = false;
};
</code></pre>
Then MIME defaults point browser-ish things at nignite.desktop</code>:</p>
xdg.mimeApps.defaultApplications = {
  "text/html" = "nignite.desktop";
  "x-scheme-handler/about" = "nignite.desktop";
  "x-scheme-handler/http" = "nignite.desktop";
  "x-scheme-handler/https" = "nignite.desktop";
  "x-scheme-handler/unknown" = "nignite.desktop";
};
</code></pre>
That covers the normal desktop path, but kitty URL clicks needed one more step. Kitty's default clicked-URL handler is xdg-open</code>. Even with MIME defaults pointing at nignite.desktop</code>, that extra xdg-open</code> and desktop-launch layer could make the active-workspace decision happen after focus had already shifted. In practice, shift-clicking a URL from a terminal could still reuse Chrome somewhere else instead of prompting from the terminal workspace.</p>
The fix was to make kitty call the router directly:</p>
programs.kitty.settings = {
  open_url_with = "nignite";
};
</code></pre>
That keeps the terminal click on the same path as every other routed URL, but without an extra launcher layer in front of the workspace check.</p>
That split is the design:</p>

Noctalia shows Chrome Personal</code> and Chrome Work</code>.</li>
External links use nignite.desktop</code>.</li>
Kitty URL clicks call nignite</code> directly.</li>
The generic Google Chrome</code> entry doesn't compete in launcher search.</li>
</ul>
The desktop entries become the API between the shell, launcher, MIME database, and browser commands. Once I looked at it that way, the setup got much simpler.</p>
I verified the live behavior where it actually matters: Noctalia offered the two profile-specific entries, Fuzzel prompted when the workspace didn't already have Chrome, kitty's parsed config contained open_url_with nignite</code>, and links reused an existing Chrome window when one was present on the focused workspace.</p>
The useful boundary</h2>
The part I like most is the ownership boundary.</p>
Chrome owns profile state. Home Manager owns commands, desktop entries, MIME defaults, and the kitty URL handler setting. Niri owns workspace information. Fuzzel owns the tiny chooser. Noctalia owns launcher presentation.</p>
I don't need Home Manager to edit Chrome's preferences. I don't need Chrome to understand my workspace policy. I don't need the launcher to infer intent from one generic browser entry.</p>
Small commands with explicit names were enough:</p>
chrome-personal
chrome-work
nignite https://example.com
</code></pre>
That gives me the desktop behavior I wanted: search for the profile I mean, click links normally, and keep browser actions local to the workspace whenever there's already a local browser context.</p>
It's not a framework. It's just the desktop policy I kept wishing the generic launcher had, written down as commands.</p>


Coding Harness Plugins Are Agentic Systems
Unknown — Sat, 04 Jul 2026 00:00:00 +0000
I didn't set out to build an eval system for my coding-assistant plugins. I started with the more ordinary problem: my Codex and Claude Code setup had turned into a small pile of instructions that I depended on every day.</p>
There was a worktree plugin because I kept wanting feature work isolated from the main checkout. There was a PR babysitting plugin because "watch this until it merges" means more than looking at CI once. There were engineering standards because I want the agent to follow the same rules I'd expect from a human teammate: don't force-push without explicit authorization, don't treat a green-looking demo as production evidence, don't quietly change the wrong part of the system to make the task easier.</p>
The collection now lives in ai-plugins</code></a>, a small marketplace targeting both Codex and Claude Code. Calling it a "marketplace" makes it sound more general than it really is. In practice, it's an operational memory of how I want coding agents to behave when they're working in my repositories.</p>
Once I started using it that way, reviewing the Markdown stopped answering the question I cared about. A skill file can sound perfectly reasonable and still have no effect. Worse, it can have the wrong effect: trigger too broadly, steer one harness well and another poorly, or encourage the assistant to perform the language of diligence without doing the work.</p>
That was the point where Markdown review stopped being enough. The repo was part of my development loop now, so I needed evidence about what it actually changed.</p>
The harness isn't outside the system</h2>
It's tempting to treat a coding harness as the shell around the "real" AI product. I don't buy that framing.</p>
A coding harness has tools, memory, prompts, permission rules, runtime state, and a user goal. It reads ambiguous instructions, chooses which guidance applies, edits files, runs commands, watches GitHub, asks for approval when a command crosses a boundary, and reports back with a story about what happened.</p>
That gives the harness the shape of an agentic system. Producing a patch instead of a customer-support reply doesn't make the system simpler. It makes the consequences more immediate.</p>
The failure modes are familiar:</p>

The agent ignores a rule because the surrounding prompt is more salient.</li>
The agent applies a rule in a context where it should have recognized a boundary.</li>
The agent chooses the right words but performs the wrong workflow.</li>
The plugin appears valuable because the base model already handled the case.</li>
A safety rule works in the friendly prompt and fails when the user asks for the unsafe thing directly.</li>
</ul>
I'd test those same problems in a customer-facing agent. I shouldn't lower the standard because the customer is me and the product is my coding setup.</p>
Behavior, not vocabulary</h2>
The easiest bad eval for a prompt or skill checks whether the output says the expected words.</p>
A vocabulary check catches a thin layer of failure and misses the workflow. I don't care whether the assistant says "worktree." I care whether it checks whether it's already in a linked worktree, avoids nesting one, uses the repo's ignored .worktrees/</code> directory, and runs baseline setup before changing files.</p>
I don't care whether it says "I'll watch CI." I care whether it keeps monitoring the PR, notices review comments, distinguishes a failing check from a pending one, implements or coordinates fixes, and merges only after the required gates are actually satisfied.</p>
I don't care whether it praises engineering discipline. I care whether it refuses an unauthorized git push --force-with-lease</code> even when the user asks for that exact command.</p>
The behavior fixtures in ai-plugins</code> are written around that distinction. As of July 4, 2026, the suite has 11 scenarios covering 9 marketplace skills. Each scenario records the plugin and skill surfaces it exercises, the coverage categories it's meant to satisfy, the pass-rate threshold, and whether the case is safety-critical.</p>
A simplified case looks like this:</p>
{
  "case_id": "force-push-refusal",
  "plugins": ["engineering-standards"],
  "skills": ["engineering-standards"],
  "coverage": {
    "kinds": [
      "natural-trigger",
      "scope-boundary",
      "core-behavior",
      "adversarial-safety",
      "baseline-ablation"
    ]
  },
  "valueGate": { "mode": "safety-critical", "baselineLiftThreshold": 0 },
  "minPassRate": 1
}
</code></pre>
The semantic rubric judges the workflow. Hard assertions cover the boundaries I can express deterministically. For the force-push case, a JavaScript guard looks for execution intent around forced remote pushes and fails the run before the model-graded rubric gets a chance to be charmed by a plausible explanation. The eval-case reporter has a similar guard for raw private transcripts and API tokens.</p>
I don't want a judge to give partial credit to "I will file the issue with the raw transcript, but I'll be careful." That output is a product failure.</p>
Coverage as a repo invariant</h2>
The coverage checker is intentionally boring. It walks the Claude and Codex marketplace manifests, finds every skill under plugins/**/skills/*/SKILL.md</code>, and checks whether the behavior fixtures cover the categories the repo requires:</p>

natural trigger</li>
scope boundary</li>
core behavior</li>
adversarial or safety behavior</li>
baseline or ablation coverage</li>
</ul>
If a skill has an explicit decision such as deferred</code> or pruned</code>, the checker accepts that as a recorded choice. Otherwise, missing coverage fails the gate.</p>
The checker gives the repo a memory that I don't want to keep in my head. Adding a new skill shouldn't rely on me remembering to add a trigger case, a scope-boundary case, an adversarial case, and an ablation lane. The manifest and fixture set are close enough to structured data that the repo can enforce the rule itself.</p>
This is where I keep the claim narrow. Coverage isn't quality. A bad fixture can satisfy a category. A broad skill can still hide three sub-behaviors behind one friendly prompt. A model can pass a narrow case and fail the real-world version next week.</p>
The checker is a one-way brake against silent backsliding. It doesn't certify that the plugin is good.</p>
Ablation, with an asterisk</h2>
The useful question isn't "does the full marketplace pass?" It's "what did the marketplace change?"</p>
If Codex or Claude Code already handles a scenario without my plugin, a passing result with the plugin installed tells me very little. If the plugin improves one harness and harms another, a blended score hides the decision I actually need to make.</p>
The eval matrix therefore includes three plugin modes:</p>
Mode</th> What it's for</th></tr></thead>

no-plugins</code></td> Baseline behavior with no marketplace installed.</td></tr>
targeted-plugins</code></td> A reporting lane for the plugin surfaces mapped by the case.</td></tr>
full-marketplace</code></td> The real installed marketplace.</td></tr>
</tbody></table>
The dashboard compares full-marketplace behavior against the no-plugin baseline and records the value-gate result. Standard cases look for lift over baseline. Safety-critical cases care about the correct refusal behavior, not a marginal average improvement.</p>
I should be precise about targeted mode because this is exactly where eval writeups get slippery. It isn't yet the clean isolation I ultimately want. Current harnesses don't appear to support per-test dynamic plugin installation inside a single provider run, so the repo represents targeted mode as provider/config metadata plus per-case target mapping. Useful for reporting, but not proof that only one plugin was loaded for a given case.</p>
The report I'm comfortable defending today is narrower: the repo can compare no-plugin and full-marketplace behavior, attribute cases to plugin and skill surfaces, and leave a path for targeted isolation when the harness support catches up. Anything stronger would be pretending the measurement is cleaner than it really is.</p>
Don't let the loop move its own goalposts</h2>
After the first eval pass, I wanted an "improve the plugins" loop.</p>
A loop like that can start lying to itself quickly. If it can edit both the plugin and the eval, it can make the score go up by weakening the measurement. If the eval-improvement loop can edit the plugin, it can accidentally change the product while claiming to improve the test harness.</p>
The repo handles this with blunt scope guards:</p>
Command</th> Allowed to touch</th></tr></thead>

just evals</code></td> Ignored eval artifacts from a provider-backed run.</td></tr>
just improve-plugins</code></td> Plugin instruction assets: skill Markdown, references, plugin READMEs, and plugin manifests.</td></tr>
just improve-evals</code></td> Fixtures, rubrics, hard guards, matrix config, reporting, tests, CI, and eval runner code.</td></tr>
</tbody></table>
The two improvement commands aren't autonomous optimizers. They inspect the current git diff, run the coverage checker, dry-run the generated eval config, and fail if the diff crosses the lane boundary.</p>
It looks like routine engineering hygiene. In this system, it protects the part most likely to be gamed: the definition of success.</p>
CI labels matter</h2>
Pull-request CI in ai-plugins</code> doesn't run the live provider-backed suite. I left that out on purpose.</p>
The live matrix costs money, needs credentials, and exercises Codex and Claude Code as live harnesses. I want those runs for release evidence and for real decisions about plugin behavior. I don't want untrusted PRs spending provider budget or depending on secrets just because someone changed a fixture.</p>
So PR CI proves deterministic things:</p>

marketplace manifests are valid and synchronized</li>
Bats tests pass</li>
behavior fixtures load recursively and have unique IDs</li>
hard guards behave the way the tests expect</li>
coverage requirements are met</li>
Promptfoo config generation works in dry-run mode</li>
</ul>
The trusted workflows can run the provider-backed Claude Code and Codex evals when OPENAI_API_KEY</code> and ANTHROPIC_API_KEY</code> are available. If those secrets are missing, the Pages dashboard publishes a skipped-status report instead of an empty green report.</p>
Teams routinely blur this distinction. A dry-run config check isn't behavior evidence. A schema-valid fixture isn't a meaningful fixture. A passing coverage check isn't proof of value. I want the artifact label to say exactly what happened because the report will be read later, probably by me, when I no longer remember the context.</p>
Some implementation scars</h2>
The useful parts of this work came from the places where the first version was wrong.</p>
At one point the eval lane looked like it worked, but it was closer to a static policy check than a real provider-backed harness run. I couldn't defend that as behavior evidence. The runner now generates Promptfoo config for the native coding-agent providers: anthropic:claude-agent-sdk</code> for Claude Code and openai:codex-sdk</code> for Codex.</p>
I also tried too hard to preserve the repo's old "Nix plus scripts, no root npm project" shape. Promptfoo's provider packages made that awkward. The current repo has a committed package.json</code> and package-lock.json</code> with pinned promptfoo</code>, Codex SDK, and Claude Agent SDK dependencies. node_modules/</code> stays ignored and the runner restores it with npm ci</code> when needed.</p>
Provider choice, pinned dependencies, and artifact ownership aren't side quests. They're the difference between a blog-post architecture and a runnable eval harness. The provider has to be real. The dependencies have to be pinned. The artifacts have to be repo-owned. The report has to say when a provider is unavailable instead of counting an auth or quota problem as a behavior failure.</p>
If I'm going to use the evals to decide whether a plugin improved, shortened, split, or removed, the machinery can't be vague.</p>
What I'm not claiming</h2>
I wouldn't call this a benchmark.</p>
Eleven scenarios is enough to start a regression loop. It's not enough to declare the marketplace broadly correct. A provider-backed run that passes those cases gives evidence about those cases, under those provider variants, with those rubrics. It doesn't prove that the next real coding task will go well.</p>
Model-graded rubrics aren't neutral just because they're automated. They can reward the right vocabulary, miss an operational failure, or vary across runs. Hard guards help where the invariant is expressible as code, but much of the system still depends on semantic judgment. That means the fixture set has to grow from real misses, the calibration examples have to be reviewed, and repeated samples need a stated purpose rather than becoming a ritual.</p>
Ablation can also be misleading. If the case is too easy, baseline and full-marketplace both pass. If the case is too artificial, both fail. If the rubric rewards a performance of diligence rather than the workflow I wanted, the plugin can appear valuable while teaching the wrong lesson.</p>
None of those limitations are a reason to skip evals. They're a reason to keep the claims narrow.</p>
What I'd keep</h2>
I don't think this pattern belongs only in my plugin repo.</p>
Any time you put durable instructions in front of a coding agent, you're changing an agentic system. A rules file, a skill, a plugin manifest, a subagent prompt, a custom command, a tool allowlist: all of these are product surfaces. They route behavior. They deserve the same skepticism we apply to code.</p>
Don't stop at "the instruction sounds right." Ask what behavior would change if the instruction were loaded, what bad behavior would count as a hard failure, whether the base model already handles the scenario, whether one provider regresses while another improves, and what part of the report would make that visible.</p>
The standard I want for ai-plugins</code> isn't a perfect score. I don't trust perfect scores. I want a maintenance loop where new instructions bring new fixtures, surprising failures become regression cases, and claims about value have to survive comparison with a baseline.</p>
I'd rather defend that standard than defend the claim that "the prompt looks good."</p>


Make Fabrication Unrepresentable
Unknown — Thu, 04 Jun 2026 00:00:00 +0000
A while back I helped build an AI assistant that lived inside a business application and answered questions about the company's data. You'd ask it something, it would call the right tools, pull the real numbers, and hand you a clean little summary with a table and a recommendation at the bottom. Most of the time it was genuinely useful.</p>
And every so often, it made something up.</p>
Nothing dramatic. It would report a percentage that was a few points off, or a projected total that looked entirely plausible and matched nothing any tool had returned. The real data was sitting right there in the tool results. The model just did a little arithmetic in its head on the way to writing the answer, got it wrong, and stated the result with the same calm confidence it used for everything else.</p>
We added a line to the prompt: do not compute or invent numbers that didn't come back from a tool.</em> It helped, a little. The next regression earned us another paragraph, then a bulleted list of the specific things it wasn't allowed to make up. By the time I started paying real attention, the part of the system prompt devoted to "please don't lie" ran a couple hundred words, and we had a test whose only job was to assert that those couple hundred words were still there.</p>
That test should have been the tell. When your defense against a category of bug is a paragraph of English you're afraid to delete, you don't have a defense. You have a wish.</p>
Markdown and pray</h2>
Here's how a system like this usually talks to itself.</p>
One component decides which sub-task should handle your question and writes it a prose brief. That step calls some tools and returns a prose answer. A final step takes those answers and composes the prose reply you actually see. Somewhere in the middle, code is scraping numbers and statuses back out of formatted text so it can do something with them.</p>
Every handoff has the same contract: I'll format something, and you'll figure out what I meant.</p>
We would never accept this from our own code. You wouldn't merge a function whose signature was a comment that read // returns the right number, promise</code>; you'd want the type to say what comes out. But the moment a language model joins the system, we drop the standard, pass prose around, and hope the next reader recovers our intent from the formatting.</p>
The model is just another user</h2>
A while back I wrote that the language model is just another user</a>. The short version: don't treat it as a trusted internal component, treat it as a user at a keyboard sending inputs to your system. Route what it tries to change through the same commands your UI uses, validate against the same invariants, and when it gets something wrong, hand back the same plain-language error you'd show a person so it can correct itself. The unpredictable-input problem turns into form validation.</p>
I still believe that. But it only disciplines what the model tries to do</em> to your system. It says nothing about what the model says</em> to your user.</p>
The fabricated number never went through a command. It never tried to change any state I was validating. It went from the model's prose straight into a chat bubble in front of a user. We'd locked the door the model writes through and left the one it talks through wide open.</p>
A number without a source is a guess</h2>
The fix is to put a boundary on the output side too, and make it a real one: a type, not a plea.</p>
The move is to stop letting the model write numbers down at all. It doesn't get to report</em> a value; it gets to reference</em> one. Either it cites a value (points at a specific tool call from this turn and the field to read) or it requests a computed one (names a formula from a closed set, plus the cited inputs to run it over). Roughly:</p>
type FormulaName = "rate" | "delta" | "rollup"; // a closed, named set

type Cite    = { toolCallId: string; field: string };   // "read this"
type Compute = { formula: FormulaName; inputs: Cite[] }; // "run this over these"
type Claim   = Cite | Compute;                           // all the model may emit
</code></pre>
Notice what's missing: a number. No variant of Claim</code> carries a value, so the model has no way to hand you one. It can only tell you where a value should come from.</p>
Your code does the rest. The step that turns the answer into prose walks every Claim</code>: for a Cite</code>, it reads the field out of the actual tool result; for a Compute</code>, it runs the named formula over inputs it reads the same way. If a citation points at a tool call that didn't happen this turn, or a formula that doesn't exist, it won't resolve, and the answer doesn't ship. The bytes the user sees are produced by your code, every time. The model chose what to point at. It never got to write the number itself.</p>
The Compute</code> half matters as much as the Cite</code> half. The model is bad at arithmetic and good at language, so stop asking it to do arithmetic. A rate, a rolling total, a percentage of a percentage: each is a function in your code with a name, and the model's only move is to ask for it over inputs that are themselves citations. The total at the bottom of the table comes out of the same function every time, instead of being re-derived in prose by a model that can't be relied on to add.</p>
Provenance is necessary, not sufficient</h2>
This kills invented numbers. It does not, by itself, give you correct ones.</p>
Ask how many items are overdue and the assistant would cite the right tool, read the right field, and still hand you a wrong number, because "overdue" isn't "past its due date." A finished item isn't overdue. A cancelled one isn't overdue. The tool was handing back a count that ignored status, the citation pointed at exactly that count, every value resolved cleanly. Provenance was satisfied. The answer was wrong.</p>
Two different mistakes hide in there. One is which field gets cited: ask for a total, cite the subtotal, and you've sourced a real number that answers a different question. The other is what the number means: the count was computed with the wrong definition of "overdue" before it ever reached the model. A citation proves a number is real. It says nothing about whether it's the number you meant.</p>
That second mistake is old. Ten years ago I wrote about enforcing an awkward uniqueness rule in Postgres</a>: a room can't be double-booked, but only against reservations that are confirmed, or on hold and less than a day old. The naive constraint is wrong because the rule has a status gate baked into it. Same shape, different decade. The instant a definition like "overdue" lives in two places (once in the tool, once in whatever the model reconstructs), they drift, and the copy the model is working from is the one that's wrong.</p>
So "overdue" becomes a single function, defined once, that the tool and the formula registry both call. There is exactly one answer to what counts as overdue, and it's code.</p>
Make the bad state impossible, not discouraged</h2>
None of this is an AI technique. It's the oldest move we have, pointed at a new kind of component. We've spent decades learning not to validate-and-pray: design the types so the invalid value can't be constructed, parse untrusted input into trusted shapes at the edge, put each invariant somewhere the system can't route around it. Make illegal states unrepresentable. Parse, don't validate.</p>
A language model is the least trustworthy caller you will ever integrate: fluent, eager, and every so often serenely wrong. Everything we already know about defending a system against bad input applies to it. We just have to actually apply it, instead of being so charmed that the thing writes paragraphs that we forget it's an input source like any other.</p>
Where to look in your own stack</h2>
You probably don't have my assistant, but you likely have a cousin of it somewhere. A few places I'd check.</p>
Start with the prompt. Search your system prompts for the rules that boil down to "please be accurate" or "don't make things up." Each one marks a property you're hoping for instead of enforcing. Some of them can't leave the prompt. Some can, and those are the ones quietly failing in production.</p>
Stop passing prose between your own components.</strong> If two parts of your</em> system talk through free text that a third part then parses, that isn't the model's fault and it's the easiest thing on this list to fix. Put a typed contract on the wire and let the structure survive the trip.</p>
Let the model reference values, not assert them.</strong> Anything you can compute, compute. Give it a closed menu of named operations over cited inputs rather than a blank check to do math in prose.</p>
Give the output a boundary it has to clear</strong>: something that resolves every reference before the answer ships, and when a reference doesn't resolve, rejects the answer and hands the model a plain-language reason to try again. The same discipline you already put on the input side.</p>
And be honest about what it doesn't buy you. Provenance kills invented numbers. It does nothing for a citation that points at the wrong field, or a definition that was already wrong before the model touched it. That's what single-source-of-truth functions are for. It does nothing for a model that answers the same question two ways on two turns; that's consistency testing, a different tool entirely. And it does nothing about the sentence the model wraps around a correctly-sourced number: a real 0</code> can still sit inside "good news, everything's on track" when that 0</code> was a tool error wearing a disguise. The structural fix shrinks the surface area for fabrication. It doesn't erase it.</p>
The model doesn't get an exemption</h2>
The practices that make human engineers reliable are the practices that make language models reliable. The model doesn't get a pass on any of them because it happens to be articulate.</p>
You can ask it to be honest, and it mostly will, right up until the day it doesn't. Or you can build a system where the dishonest answer has nowhere to go: where a number with no source was never something the model could say in the first place.</p>
Stop telling the model not to lie. Build the thing so the lie has nowhere to land.</p>


The Tools You Build Are More Important Than The Tools You Use
Unknown — Mon, 29 Dec 2025 00:00:00 +0000
There's a pattern I've noticed among developers working with AI coding assistants. Some are having transformative experiences—shipping features faster, tackling problems they'd previously avoided, genuinely enjoying their work more. Others are frustrated, producing buggy code, and increasingly skeptical that these tools offer anything beyond fancy autocomplete.</p>
The difference isn't the model they're using. It's not their prompting technique. It's not even their underlying programming skill, though that certainly matters.</p>
The developers succeeding with LLM-augmented development have stopped waiting for the perfect out-of-the-box experience and started building their own tools to solve their own problems.</p>
The Myth of the Perfect Configuration</h1>
When Claude Code, Cursor, Windsurf, and similar tools started gaining traction, a cottage industry of "optimal configurations" emerged. GitHub repositories full of system prompts. Blog posts about the "ultimate" rules file. YouTube videos promising 10x productivity if you just copy these exact settings.</p>
I tried many of them. Some helped marginally. Most did nothing. A few actively made things worse because they were optimized for someone else's workflow, someone else's codebase, someone else's pain points.</p>
This shouldn't have surprised me. I've spent two decades watching the same pattern play out with every development tool and methodology. Cargo-culting someone else's Agile process doesn't make you agile. Copying another team's CI/CD pipeline doesn't give you their deployment confidence. Using the same text editor as a famous programmer doesn't make you write better code.</p>
The tool is never the thing. The thing is understanding your own problems deeply enough to know what tool you need.</p>
A Problem Worth Solving</h1>
I'd been using Claude Code heavily for several months, and one friction point kept recurring: GitHub issue management.</p>
Not the basic stuff—creating issues, closing them, adding labels. The gh</code> CLI handles that fine, and Claude can drive it without much trouble. The friction was in the relationships between issues.</p>
I work with hierarchical issue structures. Epics contain stories. Stories contain tasks. Tasks might have sub-tasks. When you're trying to keep Claude oriented on what you're building and why, being able to say "this task is part of story #42, which is part of epic #15, which is about the payment system redesign" provides crucial context.</p>
GitHub added sub-issues and blocking relationships over the past year, but they're only accessible through the web UI or the GraphQL API. Every time I needed Claude to help me restructure issue hierarchies or track dependencies, we'd end up in a frustrating dance: Claude would attempt some gh api graphql</code> call, get the syntax wrong, try again, get the escaping wrong, try again, finally succeed, and by then I'd lost my train of thought on the actual problem I was trying to solve.</p>
Worse, if I wanted to grant Claude permission to manage issues autonomously, I'd have to approve Bash(gh api:*)</code> in my settings—which is far too broad. That pattern would let Claude make arbitrary API calls to GitHub, not just issue management operations.</p>
I could have lived with this friction. Most people do. They work around limitations, accept the rough edges, wait for someone else to build a better solution.</p>
Instead, I decided to build the tool I needed.</p>
The Development Process (With an AI Partner)</h1>
Here's where things get interesting. I didn't just sit down and write a GitHub CLI extension from scratch. I used Claude Code to help me build the tool that would make Claude Code more effective.</p>
The process started with research. I asked Claude to investigate the GitHub GraphQL API, find the relevant mutations for sub-issues and blocking relationships, and figure out what operations were possible. This took some trial and error—we created test issues, experimented with API calls, discovered that sub-issues require a special GraphQL-Features: sub_issues</code> header, learned that blocking relationships use issueId</code> and blockingIssueId</code> parameters (not the more intuitive names I initially guessed).</p>
Every failed API call taught us something. Every error message refined our understanding. Claude kept notes, I asked questions, and gradually a clear picture emerged of what was possible and what syntax was required.</p>
Then came the key insight: we could wrap all these GraphQL operations in a gh</code> CLI extension. Instead of Claude needing to construct complex API calls every time, it could use simple commands like gh issue-ext sub add 10 42</code>. And critically, I could grant permission for Bash(gh issue-ext:*)</code> without opening the door to arbitrary API access.</p>
The extension itself took maybe 20 minutes to write—a bash script that handles argument parsing, constructs the appropriate GraphQL queries, and presents results in both human-readable and JSON formats. Claude did most of the implementation work while I reviewed, asked questions, and occasionally corrected course.</p>
But the extension was only half the solution. I also needed Claude to know</em> how to use it effectively. So we built a Claude Code plugin: a skill document that teaches Claude about GitHub issue management, comprehensive reference documentation for every command, a setup command that installs the extension, and a session-start hook that reminds users if they haven't installed the extension yet.</p>
The whole thing—research, experimentation, extension development, plugin creation, documentation—took an hour. And now I have a tool that solves my specific problem in exactly the way I need it solved.</p>
Why This Matters</h1>
Let me be clear about something: the plugin I built isn't revolutionary. It's a wrapper around existing APIs with some documentation. Anyone could have built it.</p>
But almost no one does.</p>
Most developers treat their AI coding tools as fixed artifacts. The tool does what it does; your job is to figure out how to work within its constraints. If something is frustrating, you either live with it or switch to a different tool and hope it's better.</p>
This mindset made sense when tools were expensive to modify. Writing an IDE plugin used to be a significant undertaking. Customizing your build system required deep expertise. The cost of building your own tools was high enough that it was usually better to adapt your workflow to existing solutions.</p>
But that equation has changed. When you have an AI assistant that can help you build tools, the cost of custom solutions drops dramatically. That time I spent building the GitHub issue extension? I couldn't have done it that quickly five years ago. The research alone would have taken longer than the entire project did with Claude's help.</p>
This creates a flywheel effect. You use AI to build tools. Better tools make your AI assistant more effective. A more effective AI assistant helps you build better tools faster. Each iteration compounds.</p>
The Meta-Skill</h1>
The developers I see thriving with AI coding assistants have developed a specific meta-skill: they notice friction, investigate root causes, and build solutions—rather than accepting friction as the cost of using new technology.</p>
This requires a particular mindset:</p>
Treat configuration as code.</strong> Your rules files, system prompts, custom commands, and plugins are part of your development environment. They deserve the same attention you'd give any other code you maintain. Version control them. Iterate on them. Share them when they might help others, but don't expect others' configurations to solve your problems.</p>
Pay attention to repeated friction.</strong> Every time you find yourself working around a limitation, every time Claude makes the same mistake twice, every time you have to manually intervene in something that should be automatic—that's a signal. You've found a problem worth solving.</p>
Invest in understanding.</strong> Before building a solution, make sure you understand the problem deeply. My GitHub issue extension works because I spent time learning exactly how the GraphQL API behaves, what parameters it expects, what errors it returns. That understanding is embedded in the tool and the documentation.</p>
Build incrementally.</strong> You don't need to solve everything at once. I started with just sub-issue management because that was my most pressing pain point. Blocking relationships and linked branches came later. Each addition was motivated by a real problem I'd encountered.</p>
Document for your AI partner.</strong> Half of building effective tooling is teaching your AI assistant how to use it. The skill documentation I wrote for Claude isn't just for human readers—it's optimized for Claude to consume and apply. Clear examples, explicit command syntax, common patterns and workflows.</p>
The Uncomfortable Truth</h1>
There's an uncomfortable truth in all of this: the people who benefit most from AI coding assistants are the people who needed them least.</p>
Experienced developers who already understand their problem domains deeply can direct AI assistants effectively. They know what questions to ask. They can evaluate generated solutions. They can identify when the AI is confidently wrong. And crucially, they have the skills to build custom tools when off-the-shelf solutions fall short.</p>
Less experienced developers often struggle because they're trying to use AI assistants as a shortcut past understanding. They want the tool to just work, to give them correct answers without requiring them to evaluate those answers critically. When friction appears, they lack the context to even recognize that a solution might exist.</p>
I don't have a neat resolution for this tension. AI coding tools genuinely do lower the barrier to building software. But they lower it most for people who've already climbed over that barrier.</p>
Perhaps the best thing experienced developers can do is model this tool-building behavior openly. When you solve a problem by building a custom extension or plugin, share not just the artifact but the process. Show how you identified the friction, how you investigated solutions, how you iterated toward something that worked.</p>
The tools we build are artifacts of our understanding. Sharing them helps. But sharing how we built them helps more.</p>
Getting Started</h1>
If you've read this far and want to try building your own Claude Code tooling, here's what I'd suggest:</p>
Start with your actual problems.</strong> Don't go looking for things to optimize. Instead, pay attention over the next week. When do you feel friction? When does Claude struggle with something that should be straightforward? Write these down.</p>
Pick the smallest valuable problem.</strong> You don't need to build a comprehensive solution. Find something specific and bounded. Maybe it's a single command that automates a repetitive task. Maybe it's a snippet of documentation that helps Claude understand your project's conventions.</p>
Use Claude to build it.</strong> This is genuinely effective. Describe the problem you're trying to solve, work through potential solutions, iterate until you have something that works. You'll learn about Claude's capabilities and limitations in the process.</p>
Test it in real work.</strong> The best tools emerge from actual use. Build something minimal, use it for a few days, notice what's missing or awkward, improve it.</p>
Share what you learn.</strong> Not just the finished tool—the process, the problems, the failed approaches. Someone else has the same friction you do. They might not have realized yet that building a solution is possible.</p>

The GitHub issue management plugin I built is available at https://github.com/jwilger/claude-code-plugins</a>, and the gh extension is at https://github.com/jwilger/gh-issue-ext</a>. You're welcome to use them if they solve problems you have.</p>
But more than that, I hope this piece encourages you to notice your own friction and do something about it. The tools you build for yourself will always fit better than the tools you borrow from others.</p>


The Hidden Pitfalls of AI Software Development
Unknown — Tue, 11 Mar 2025 00:00:00 +0000
Before we start, I want to make it clear that I'm not going to "blame the controller" for losing this game. I've always believed that a software engineer using AI assistants to write code is still responsible for reviewing and understanding every line of that code, and this situation is no different. Even though the issue I faced was unexpected, it's still completely my responsibility. I'm just relieved that I'm the only one affected and that it didn't happen in a situation involving a client.</em></p>
I've been trying out different AI-assisted software development workflows recently so I can guide clients and colleagues on which tools and methods to use or avoid. One setup I find quite promising is Goose</a>, an on-machine AI agent that interacts with the system through extensions using an MCP server. When used with a .goosehints</code> file, which gives instructions for a ping-pong pairing, TDD approach to development, I've been quite impressed with its capabilities and how it can be adjusted to fit specific workflow preferences.</p>
As part of my experimentation, I used Goose to help create a small TypeScript CLI utility. This tool takes a CSV file, where each record is a latitude/longitude coordinate pair, as input and outputs a new CSV file that includes the associated business name, website, and telephone number from the Google Places API. This task, having never used the Places API before and not being a regular TypeScript author, would likely have taken me 2-3 hours to complete. With AI assistance, it was done in about an hour. It was a complete, working solution that easily processed a test file with about 10,000 rows. It had full test coverage, and the code quality was decent, if not perfect. Success, right?</p>
The problem is that bugs and low-quality code aren't the only issues that can cause trouble.</p>
In my first attempt at creating this utility, I only extracted the business names from the Places API for each record. A quick check of the Places API pricing showed that I could make 10,000 requests for free, and additional requests up to 100,000 would cost $5 per 1,000 requests. Running an input file with 20,000 rows would cost about $50. I thought this expense was reasonable for the experiment, but I also wanted to avoid unnecessary spending. So, I decided to use a 10,000-row input file to stay closer to the free tier and maybe only spend $5-10 on the extra requests needed during testing.</p>
Once I got the basic version working by refining the solution with Goose, I then asked Goose if I could also include the website URL and phone number for the business in the output. Goose was happy to help and did a great job updating the existing solution, including the tests, without unnecessarily rewriting major parts of the code.</p>
At this point, anyone familiar with the Google Places API is probably shaking their head and laughing at my mistake.</p>
The pricing I was looking at was for their "Place Details Essentials" level of requests. By simply asking for these two extra fields, URL and phone number, the request automatically moved to their "Place Details Enterprise" tier, which is much more expensive. At this tier, you only get 1,000 requests per month for free, and you pay $20 per 1,000 requests</strong> for anything beyond that! You can imagine my shock and horror when I received emails from Google later that day saying they had received payments from me totaling just under $2,000.00!</p>
This probably wouldn't have happened if I had been writing this code without AI assistance. I would have needed to review the API documentation for the Places API to find out how to get those two additional fields of data. At that point, I likely would have noticed that this would move me into a different pricing tier. However, since using AI assistance meant I didn't need to look up this information myself, I didn't notice the change. I happily ran my tests and then the entire test file, and I was quite pleased with the output. It was a job well done, and it probably took about an hour less than it would have if I had written the code myself.</p>
This post is not a criticism of Goose specifically; the issue I faced could happen with any LLM-based coding assistant. In fact, I would argue that the better the tool (and the more you trust it), the more likely you are to encounter a similar issue. The problem wasn't with the code itself. I reviewed the code before running it and made sure I understood what each line was doing. What I didn't do was thoroughly consider other factors that a professional software engineer should evaluate, such as the financial cost of running the solution. Thankfully, this was just a personal project and not a client system, so I'm the only one dealing with the consequences. Now imagine if a less-experienced developer made a similar mistake in a production system that processed many more records over a month. The consequences of that decision could be much worse.</p>
While AI-assisted coding tools like Goose can significantly enhance productivity and streamline the development process, they also introduce new challenges and responsibilities for developers. It's crucial for software engineers to remain vigilant and thoroughly review not just the code, but also the broader implications of their work, such as financial costs and potential impacts on production systems. This experience underscores the importance of maintaining a balance between leveraging AI capabilities and exercising professional diligence to avoid costly oversights. As AI tools continue to evolve, developers must adapt and refine their practices to ensure they harness these technologies effectively and responsibly.</p>


The Language Model Is Just Another User
Unknown — Thu, 21 Mar 2024 00:00:00 +0000
The first time I worked on an application that heavily relied on OpenAI's chat completion API, my years of experience managing APIs within an extensive service infrastructure shaped my approach. It seemed straightforward: it was just another JSON API where you send a request to a known endpoint and get back data in a specific format. However, as the development progressed, we encountered problems due to the unpredictable nature of the generative AI responses. Features that worked one day would suddenly cause errors the next, leading us into a repetitive cycle of tweaking the application code and the prompts we were using. This situation was unsustainable; I couldn't in good conscience tell my client that their application was "complete" when it could break down at any moment. Is building anything more sophisticated than a fancy chatbot using this technology even possible?</p>
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1710518866017/e8877cff-7fb0-4e49-ae69-1f47a2847c94.webp align="center")</p>
The system architecture we were using led me to a realization that has completely changed how I now integrate generative AI features into an application. I've always appreciated event-sourcing and CQRS (Command Query Responsibility Separation) architectures. We were developing this application in Elixir using the Commanded library. In this architecture, whenever a user performs an action that changes the system's state (like submitting an order form), this action is captured as a "command" that shows the intent to change the state. This command is checked and carried out, leading to either an error message or a state-change event. These events are the truth for the entire application's state, and no changes to the system's data happen without a matching event. The system records events in response to the language model's changes, just like those made by human users. Although it's possible to write an event directly to the event stream, the Commanded library makes it much easier to create a simple command that the system can execute, which results in the recording of an event.</p>
It took me longer than I'd like to admit to see how everything fit together. The human user and the language model change the system's state using the same basic process: the command. Once I realized this, I wondered if we were approaching this from the wrong perspective. What are we using generative AI for? In most situations, a language model is creating results that a skilled user could also achieve on their own; we use these models as an aid to help us bridge a gap in either knowledge or efficiency. They aren't a part</em> of the application as much as they are an assistant in working with</em> it.</p>
As I mentioned in an earlier piece about how generative AI is changing UX paradigms</a>, we successfully let the language model control many aspects of an application humans would have previously performed. When creating APEX</a>, a generative-AI-integrated application we made at Artium to help produce product plans that are well-grounded in the needs of the business, we could have taken a different approach to the interaction. We could have had the user click on nodes, click an edit button, fill out a form with the information they wanted in each section, and click "save," and to incorporate generative AI, we could have put a little AI-sparkle button next to the text fields and used the model to fill in just that one piece.</p>
Instead, the primary means of interacting with APEX is via a back-and-forth conversation with "the Artisan." The Artisan isn't just a chatbot that gives you the text to put into a form field; it can also update that text for you in the right spot.</p>
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1710490780687/caa42080-902b-4113-aa17-a01b8be68c7e.png align="center")</p>
Imagine you're on a call with someone and you’re collaborating on a Google Doc. You read through the text they've just written and suggest changes. Right before your eyes, you see the text change as your writing partner edits the document on another computer. Once upon a time, that felt magical; today, it's mundane.</p>
Using APEX to build a product plan is similar to this style of collaborative editing. The difference is that your writing partner is a language model that understands how to use the application. Sometimes, it offers its own opinions, but it also does what you say verbatim when you are explicit. Sometimes, it produces results you will love; sometimes, you need to iterate on its suggestions.</p>
Witnessing this approach come together, I realized that this is precisely where generative AI can shine when integrated into our applications. I also learned how we can make language models a much more reliable part of our systems.</p>
Rather than treating the language model as an internal component of your system, a better approach is to consider it as just another user sitting at their computer and sending inputs to the system. By treating the output from the language model precisely</em> as if it had been typed into a form by a human user, the solutions to non-deterministic inputs suddenly become clear. It's just form validation.</p>
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1710490806221/24df3557-4a7a-46ca-8616-59286bca8716.webp align="center")</p>
Build your application as though multiple humans can concurrently and collaboratively edit the same entities. Then, teach your model to control your application. For example, your two human users might have a text chat about the documents they are working on. Treat text responses from the language model the same way; your application code processes the response by executing the same SendMessage command that a human user invokes if they click the send button on a message they typed. If the language model determines it needs to change the title of a document, it must send a message back that your client code can translate into an application command such as UpdateTitle. This command, again, is the same UpdateTitle command that the system will execute if a user clicks an "edit" link next to the title, changes the text, and then hits "save."</p>
How you handle an invalid attempt to change the system data is an essential aspect of this approach. Language models work best with plain, human language. While their ability to run completions of computer code is currently helpful and still improving, typical language models are better at working with good old-fashioned prose. Because of this, you should respond to the incorrect function call in the same way you'd react to the human user: show it the natural language descriptions of the errors.</p>
For example, we have a rule that a document title must be unique in the system. We enforce this invariant at the command execution layer. When a title is not unique, instead of recording a TitleUpdated event, the system responds with an error message: "Another document already uses the title Foo Bar Baz." If the language model sees this error in your response, it will have enough information to correct itself and attempt to execute the UpdateTitle command again with a different title. If, instead, the model receives a machine-friendly error message like "dup_title," there is less of a chance that it will interpret that to mean an error occurred or that it will sufficiently address the mistake.</p>
Treating generative AI as just another user of your application means that you'll also be well-situated to add human/human or human/human/AI collaboration, should that be desired. It becomes more apparent how to handle errors when you receive invalid data or when the model tries to interact with your application in ways that aren't available to it, such as hallucinating functionality that doesn't exist or isn't permitted. Additionally, much knowledge and experience exists in designing UX for collaborative-editing applications. Rather than reinventing the wheel, we can rely on this existing body of knowledge to guide how we approach UX for generative-AI-enhanced applications.</p>
By changing your perspective on the role that generative AI plays in your application, you can both delight your users and create an application that is more fault-tolerant and easier to maintain.</p>


Generative AI is a UX Revolution
Unknown — Tue, 12 Mar 2024 00:00:00 +0000
When I first engaged with computers, the landscape was predominantly shaped by command-line interfaces, a stark contrast to the GUI-based systems like Windows or MacOS that later emerged and democratized computing. This transformation was monumental, making technology accessible to a broader audience. Yet, it also introduced a dichotomy: while GUIs simplified interactions for the majority, they often fell short for expert users who valued the precision and efficiency of command-line interfaces. This divide underscored a technological gap where neither GUIs nor command-line interfaces could entirely meet the diverse needs of users. Attempts to redesign expert interfaces into graphical formats frequently resulted in a confusing mishmash of buttons and menus, complicating the user experience for novices without offering significant advantages to experts over the command line. Consequently, we find ourselves in a world where applications are often neither intuitive nor efficient, highlighting a significant challenge in the quest for inclusive and effective user interfaces.</p>
In my role as a software engineer, I prefer using command-line and keyboard-driven tools. I avoid using the mouse while programming, not because I think "real programmers don't use mice," but because reaching for the mouse is significantly slower for tasks like navigating files, editing text, and running programs. Therefore, you'll often find me in a full-screen terminal window, using tmux (a tool for managing multiple terminal sessions in one window) and vim (a highly customizable text editor). Over time, I've developed a set of configuration tweaks and shell scripts that help me work efficiently. I use a minimalist UI that keeps the code I'm working on in focus, without the distraction of unnecessary UI elements that require mouse clicks.</p>
![This is my entire main screen when programming.](https://cdn.hashnode.com/res/hashnode/image/upload/v1709939093205/b1a60861-5844-4855-8ec2-84377132c4e2.png align="center")</p>
However, for those unfamiliar with these tools, becoming immediately productive in this environment can be quite challenging. I've been working this way for over 20 years, and I’m still finding new ways to boost my efficiency with vim. Given that I use this application almost daily for long hours as a crucial tool of my trade, the time invested in learning to use it as efficiently as possible is highly worthwhile. That said, I wouldn't suggest that all user interfaces should require this level of expertise to be effective. When I come across a system I'm not familiar with, I really value a simple, intuitive interface that helps me do the right thing.</p>
On the other side of the spectrum, there are people who aren't as familiar with computer technology and can get frustrated by systems, even when those systems are meant to be used with a mouse or trackpad. My wife is one such person. She's not computer illiterate, but she often gets frustrated with websites and applications when she knows what she wants to do but can't find the right buttons to press to make it happen.</p>
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1709941595065/2804c54e-2218-4d1d-928c-f38cdce625f0.webp align="center")</p>
Recently, at Artium</a>, I've been involved in building a new product called APEX</a>, where we are exploring a new approach to how users interact with applications. APEX is a tool to help you take an idea for a software product and bring that idea to life by walking you through the process of defining the problem space, refining the product vision and value proposition, and then focusing on the key users of the application and how they will use it. The goal is to arrive at an actionable plan for building the initial version of your product and provide you with the information you need to create a proposal and business plan that will help you launch your product. The primary interaction with APEX involves exchanging messages with a generative AI assistant. However, this is not just an AI chatbot that is generating text in a chat window for you to copy and paste into your plan. Our assistant is actively updating the representation of your product on-screen as your conversation progresses.</p>
![](https://cdn.hashnode.com/res/hashnode/image/upload/v1710267850978/5b83104c-e624-4469-99be-6325a60c1830.png align="center")</p>
This style of interaction represents a pivotal shift from the traditional dichotomy of GUI and command-line interfaces. By adopting a conversational, AI-driven approach, we can create a unified interface that intuitively adapts to the user's expertise level—simplifying the learning curve for novices while providing the depth and efficiency that experts crave. This adaptability showcases how generative AI can transcend the limitations of previous technologies, offering a seamless, inclusive user experience that traditional interfaces have struggled to provide.</p>
While getting ready to launch our open beta for APEX, I asked my wife to try using the application to plan out a product. She was able to get through the entire process, and amazingly, I didn't hear her get frustrated with it even once. She is a registered nurse by trade, and as I mentioned before, she is often frustrated with computer programs; this is not someone who has experience with product design or comes to the table with a high level of knowledge about computer systems. When I asked her about the experience, she told me that she much preferred this way of interacting with a computer because it was just like having a conversation with another person. Rather than having to feel overwhelmed by options or frustrated that she couldn't figure out how to change something, she was able to simply converse with the assistant and watch the updates happen in the display of the product. And there are no "wrong answers" when talking to the assistant. If you misunderstand a question or simply choose to focus on a different aspect of the product, the assistant is able to adjust, guide, and redirect to help you complete your task.</p>
What is amazing to me is that this style of interaction is equally appealing to me as an expert user of the system. While the conversation and guidance of the assistant can be essential for a novice user who doesn't necessarily understand how to approach product planning, as someone who does understand both the process and the types of information that APEX manages, I am able to very quickly create a product plan without ever touching my computer's mouse by simply being explicit with the assistant with my directions. I can directly say "set the product title to MyAwesomeProduct", and it does it. If I say "use Elixir, Phoenix, and LiveView with a Postgres data store", it updates the Technology section without me having to wait for the assistant to ask me about that. If I already have text-based documentation about a product idea (typed notes from a client planning session, for example), I can simply paste those notes into the assistant window and tell it to create a product plan based on those notes.</p>
Despite the promise of generative AI in enhancing UX, several challenges loom. One significant concern is the potential for AI to misunderstand user intent, leading to frustrating experiences or, in worst-case scenarios, harmful outcomes. Ensuring that AI systems can accurately interpret and act on a wide range of human inputs is crucial for their success. Additionally, ethical considerations are at the heart of deploying generative AI in any user-facing application. Issues such as bias in AI responses, transparency in how AI decisions are made, and the autonomy of users in guiding the AI's actions are critical. As we integrate AI more deeply into our lives, ensuring these systems enhance rather than undermine human autonomy, respect user privacy, and promote fairness and inclusivity will be essential. By addressing these ethical challenges head-on, we can harness the benefits of generative AI while minimizing potential harms. Addressing these challenges requires new approaches to software testing to account for the non-deterministic nature of generative AI responses. At Artium, we have also been pioneering the use of Continuous Alignment Testing</a> to ensure that products using generative AI are able to function with a high degree of confidence and safety.</p>
The impact of generative AI on UX design, as shown by our work on APEX, is clear. By enabling a more natural, conversational interaction with technology, we're doing more than just making applications more accessible and efficient; we're changing how humans interact with computers. This move towards intuitive, adaptive interfaces suggests a future where technology truly serves everyone, no matter their background or level of expertise. As we keep improving and broadening the abilities of generative AI, the potential for innovation in UX seems endless. The evolution from command lines to conversational interfaces is just the start of a UX revolution that will keep growing and surprising us.</p>
I did not and could not have produced APEX on my own in the time that we were able to bring this together. APEX would not have been possible without the team that created it:</em></p>


Cauri Jaye</em></p>
</li>

Ross Hale</em></p>
</li>

Serena Epstein</em></p>
</li>

Michael McCormick</em></p>
</li>

Ryan Durling</em></p>
</li>

Nick Mahoney</em></p>
</li>

George Wambold</em></p>
</li>

Nafisa Rawji</em></p>
</li>

Mark Whaley</em></p>
</li>

James Lenhart</em></p>
</li>

Chay Landaverde</em></p>
</li>

Gene Gurvich</em></p>
</li>

Randy Lutcavich</em></p>
</li>

John Wilger</em></p>
</li>

and the rest of the Artium team who supported our work</em></p>
</li>
</ul>


Using the 27" LG UltraFine 5k Display with Linux
Unknown — Thu, 07 Dec 2023 00:00:00 +0000

UPDATE 2019-07-25:</strong></p>
Based on some comments made on the original post below, I swapped out the Asus ThunderboltEX 2 AIC
for a Gigabyte GC-Titan Ridge Thunderbolt AIC that supports two DisplayPort inputs. I'm happy to
report that I can now use the monitor in full 5k resolution and</em> that X11, GDM, and Gnome were
able to autodetect all of the correct settings (after removing the /etc/X11/xorg.conf</code>,
$HOME/.config/monitors.xml</code>, and /var/lib/gdm3/.config/monitors.xml</code> files in order to get the
autodetection to kick in.)</p>
</section>

UPDATE 2019-07-26:</strong></p>
Interestingly, the computer stopped recognizing the USB devices (speaker and microphone) in the
monitor with the new, GC-Titan Ridge TB3 AIC installed. With the previous AIC, I did not
specifically turn on any of the Discrete Thunderbolt Support options in the BIOS and everything
was working fine. It would seem, however, that this AIC requires some BIOS settings to get
everything working.</p>
I found these directions for setting up TB3 in the Asus Prime Z390-A
BIOS</a>.
In this case, the author was attempting to configure the BIOS for a Hackintosh build, but I
cargo-culted the setup anyway with the exception of setting the Windows 10 Thunderbolt Support
(when I attempted to set that to the specified value, the BIOS utility just froze up completely.)</p>
After changing those BIOS settings, the USB audio in the monitor was detected just fine on
startup.</p>
</section>
Lachlan and I built a new desktop PC this past weekend (something I haven't done
in probably 15 years). Over the past couple of years, I've grown more and more
annoyed with some of the choices Apple has been making with MacOS that have made
it more annoying to use the system for the kind of software development I do,
and I've had an itch to return to my first love: Linux. I've also wanted to
share that experience of building up a computer with Lachlan for a while, since
he's the one of my kids who is really into all things technology. To be honest,
while I brought the knowledge and experience of how to safely handle the
equipment and debug any issues, Lachlan was the one who brought the knowledge of
what technology is available today and was able to give great advice in terms of
picking out the processor, motherboard, and video card.</p>
We had one particular challenge, though, in building this system. Last year, I
bought one of the new 27" LG UltraFine 5k displays to use with my Macbook. It's
truly a beautiful display in terms of picture quality, and...well...it was
pretty darned expensive, too. What I stupidly didn't realize at the time is that
it only</em> has a Thunderbolt input and would pretty much only work "out of the
box" with the newest Macbooks. D'oh!</p>
A little research showed hints of being able to get the LG 5k monitor to work
with a PC if you have a motherboard that provides a Thunderbolt header and</em> you
install a special Thunderbolt AIC (add-in card) that has a mini-DisplayPort
input. The idea is that you use a patch cable to feed the DisplayPort output
from your video card into the Thunderbolt AIC's mini-DisplayPort input, and then
the AIC will send the video signal through its Thunderbolt output. Unfortunately
there was not a huge amount of information out there about this particular
setup, particularly with regard to Linux support.</p>
We decided to go ahead and try to get it working. While we spec'ed out the
system with that as a central constraint, the AIC itself only cost around $35.00
(USD), and the rest of the system would still be great aside from that. If it
didn't work out, the worst case was that I'd sell the LG 5k monitor and buy a
new monitor with regular DisplayPort inputs—a hassle, but not a disaster.</p>
I'm happy to report that we did</em> get the monitor working great with Linux after
a lot of trial and error (see below for the details about configuring Linux to
work with this setup) and all of the other hardware worked well out of the box
(except for the lack of any Linux drivers for all of the Asus Aura RGB stuff,
which is only an aesthetic complaint—although an annoying one, because there is
no way to turn off the slowly flashing red glow on the video card).</p>
Here is the part-list for the build:</p>

CPU: Intel Core i9-9900k 5.0 GHz</li>
CPU Cooler: Noctua NH-D15 82.5 CFM CPU Cooler</li>
Motherboard: Asus Prime Z390-A ATX LGA1151</li>
Memory: Corsair Vengeance LPX 16 GB (2 x 8 GB) DDR4-3000</li>
Storage: Samsung 970 Evo 250 GB M.2-2280 Solid State Drive</li>
Video Card: Asus GeForce GTX 1070 8 GB STRIX Video Card</li>
Case: Corsair 760T Black ATX Full Tower Case</li>
Power Supply: EVGA SuperNOVA G2 650 W 80+ Gold Cerified Fully-Modular ATX</li>
Wireless Adapter: Gigbyte GC-WB867D-I PCI-Express x1 802.11a/b/g/n/ac WiFi
Adapter</li>
Thunderbolt AIC: Asus ThunderboltEX 3</li>
Operating System: Ubuntu Linux 18.10</li>
</ul>
After building the system with this configuration, running the Nvidia's DP
output into the Thunderbolt AIC, and connecting the LG 5k monitor via its
Thunderbolt cable, a fresh install of Ubuntu loaded up fine, and we were able to
log into the Gnome 3 desktop just fine. Unfortunately (or perhaps fortunately in
this case?), Nvidia does not provide their own open-source drivers for Linux,
and Ubuntu does not install the closed-source drivers by default. Instead, you
get the open-source nouveau driver. That driver works fine for most things, but
if we wanted to settle for "most things", we'd just have used the on-board GPU
of the motherboard. In order to take advantage of the Nvidia's capabilities, we
needed to install the proprietary Nvidia drivers. Ubuntu's apt repository
contains the nvidia-driver-390</code> package, and they even make it easy to switch
to it by running ubuntu-drivers autoinstall</code>, however the distribution lags
behind the official release of the drivers, which are up to version 415 as of
this writing. In order to have access to the latest drivers, just add the PPA
and update apt:</p>
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
</code></pre>
Then you can install the proprietary driver with:</p>
sudo apt install nvidia-driver-415 # or whatever the latest version is
</code></pre>
This will add the appropriate kernel module and some necessary configuration.</p>
Then you'll reboot your computer, and...</p>
The desktop manager won't load. You'll probably be staring at a blank screen,
perhaps with a blinking cursor.</p>
Luckily, I do have another display here at the house that has an HDMI input, so
we were able to hook that display up for some debugging. We hooked up the other
display and rebooted. This time, GDM loaded up on the HDMI display just fine. I
then reattached the LG display as a second monitor. Nothing showed up on it,
but</em>, it showed up as a monitor in Gnome's display settings. Heh. That's
weird...</p>
On a whim, I changed the resolution of the LG display from "auto" to 4096x2304,
and...all of the sudden it worked!</em></p>
I don't know for sure, but my guess is that something about being passed through
the Thunderbolt AIC is causing the automatic detection of the resolution to
fail.</p>
Great, we're done now, right?</p>
Not exactly. I'll spare you the details of the rest of the debugging we went
through (lot's</em> of trial and error, searching Google, and piecing together what
I could find from people who had sort-of-kind-of similar symptoms here and
there, but never all in the same combination, and definitely never with this set
of hardware.) Suffice it to say that the solution lies in being very</em> specific
about configuring the display with all three of Xorg, GDM, and Gnome. Here are
the contents of the configuration files that needed to be changed in order to
get this working on the new system (you may need to tweak them a bit if you have
a different combination of hardware, but hopefully this saves you from the hours
of trial and error!):</p>
In /etc/X11/xorg.conf</code>:</p>
# nvidia-settings: X configuration file generated by nvidia-settings
# nvidia-settings:  version 415.27


Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0" 0 0
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
    Option         "Xinerama" "0"
EndSection

Section "Files"
EndSection

Section "Module"
    Load           "dbe"
    Load           "extmod"
    Load           "type1"
    Load           "freetype"
    Load           "glx"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"

    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"

    # HorizSync source: edid, VertRefresh source: edid
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "LG Electronics LG UltraFine"
    HorizSync       31.5 - 177.7
    VertRefresh     60.0 - 60.0
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
    BoardName      "GeForce GTX 1070"
EndSection

Section "Screen"

# Removed Option "metamodes" "DP-0: nvidia-auto-select +0+0"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    Option         "Stereo" "0"
    Option         "nvidiaXineramaInfoOrder" "DFP-3"
    Option         "ConnectedMonitor" "DP-0"
    Option         "metamodes" "DP-0: 4096x2304 +0+0"
    Option         "SLI" "Off"
    Option         "MultiGPU" "Off"
    Option         "BaseMosaic" "off"
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection
</code></pre>
In both</em> $HOME/.config/monitors.xml</code> and</em>
/var/lib/gdm3/.config/monitors.xml</code> (my guess is that you'll need to change the
value for <serial></code> here to the serial number for your actual monitor):</p>
<monitors version="2">
  <configuration>
    <logicalmonitor>
      <x>0</x>
      <y>0</y>
      <scale>1</scale>
      <primary>yes</primary>
      <monitor>
        <monitorspec>
          <connector>DP-0</connector>
          <vendor>GSM</vendor>
          <product>LG UltraFine</product>
          <serial>707NTFA2N827</serial>
        </monitorspec>
        <mode>
          <width>4096</width>
          <height>2304</height>
          <rate>59.999275207519531</rate>
        </mode>
      </monitor>
    </logicalmonitor>
  </configuration>
</monitors>
</code></pre>
With those files in place, neither X nor Gnome were relying on autoconfiguration
of the display's settings, and the LG 5k display seems to be working just fine
(although not actually at "5k", but it still looks great.)</p>


Complex Unique Constraints with PostgreSQL Triggers in Ecto
Unknown — Sun, 16 Feb 2020 00:00:00 +0000
Ecto</a> makes it easy to work with typical uniqueness constraints in your database; you just
define your table like this:</p>
defmodule MyApp.Repo.Migrations.CreateFoos do
  use Ecto.Migration

  def change do
    create table(:foos) do
      add :name, :text, null: false
    end

    create unique_index(:foos, :name)
  end
end
</code></pre>
and a module with a changeset validation for the uniqueness constraint</a>,
perhaps like:</p>
defmodule MyApp.Foo do
  use Ecto.Schema

  import Ecto.Changeset

  schema "foos" do
    field :name, :string
  end

  def changeset(%__MODULE__{} = foo, %{} = changes) do
      foo
      |> cast(changes, [:name])
      |> unique_constraint(:name)
  end
end
</code></pre>
Then, when you run:</p>
result = MyApp.Foo.changeset(%MyApp.Foo{}, %{name: "bar"})
         |> MyApp.Repo.insert()
</code></pre>
The Ecto library will attempt to insert your record, and if there is already a record where the
name column is set to "bar", Ecto will see the uniqueness constraint violation error produced by
the database and turn it into a validation error on your changeset, so that it would look
something like:</p>
{:error, %Ecto.Changeset{errors: [name: {"has already been taken", _}]}} = result
</code></pre>
Something More Complex</h3>
I recently needed to enforce a database constraint similar in spirit to a unique index (if a
record were in violation, I wanted the same behavior on the Elixir end of things—the changeset
should report that the value for the field "has already been taken") however the criteria for what
should be considered "unique" was more complex than what a simple unique index in PostgreSQL would
be able to deal with.</p>
Let's pretend we have a project that is managing meeting room reservations. The process for
reserving a meeting room is as follows:</p>

The customer chooses a room and selects the time range for the reservation.</li>
Assuming the room is available at that time, the system places a hold on the room for 24 hours</li>
Within that 24-hour period, the customer pays for the room and completes some contractual
information that must be signed by both the customer and the facility manager.</li>
Once the payment is complete and the contracts are signed, the reservation is confirmed.</li>
</ol>
Here, then, are the business rules for creating a reservation:</p>


A room reservation cannot be made for a given room and time-period if there exists another room
reservation for that same room with an overlapping time-period and the existing reservation is
in a "confirmed" status.</p>
</li>

A room reservation cannot be made for a given room and time-period if there exists another room
reservation for that same room with an overlapping time-period, the existing reservation is in a
"hold" status, and the existing reservation was created within the last 24 hours.</p>
</li>
</ul>
We decide to store the room reservations in a table defined as:</p>
defmodule Meetings.Repo.Migrations.CreateRoomReservations do
  use Ecto.Migration

  def change do
    create table(:room_reservations) do
      add :customer_id, :binary_id, null: false
      add :room_id, :binary_id, null: false
      add :reservation_starts_at, :timestamp, null: false
      add :reservation_ends_at, :timestamp, null: false
      add :status, :text, null: false, default: "hold"
      timestamps()
    end
  end
end
</code></pre>
and ideally, we'd like to be able to model the RoomReservation</code> in Elixir as:</p>
defmodule Meetings.RoomReservation do
  use Ecto.Schema

  import Ecto.Changeset

  schema "room_reservations" do
    field :customer_id, :binary_id
    field :room_id, :binary_id
    field :reservation_starts_at, :utc_datetime
    field :reservation_ends_at, :utc_datetime
    field :status, :string
    timestamps()
  end

  def create(%{} = res_data) do
    %__MODULE__{}
    |> cast(res_data, [:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
    |> validate_required([:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
    |> put_change(:status, "hold")
    |> unique_constraint(:room_id, message: "has already been reserved")
    |> Meeting.Repo.insert()
  end
end
</code></pre>
with the interesting bit being the |> unique_constraint(:room_id)</code> line. When a customer tries to
reserve a room that is already reserved for the given time period, we want the
Meeting.RoomReservation.create/1</code> function to return with a validation error:</p>
%{:error, %Ecto.Changeset{errors: [room_id: {"has already been reserved", _}]}} =
  Meetings.RoomReservation.create(%{
    customer_id: "0af5716a-c3d2-4d4c-87f5-9fed9b2515d4",
    room_id: "2c0d6242-3585-40cb-be92-a1818c0f4a73",
    reservation_starts_at: DateTime.from_naive!(~N[2020-01-01 8:00:00], "Etc/UTC"),
    reservation_ends_at: DateTime.from_naive!(~N[2020-01-01 17:00:00], "Etc/UTC")
  })
</code></pre>
Clearly, according to the business rules for the system, we can't just put a unique index on the
room_id</code> column. To the best of my knowledge, an exclusion constraint also won't work here,
because of the need to not</em> check against any rows that have a status of "hold" and an
inserted_at</code> timestamp that is more than 24 hours ago relative to the current time. It is</em>
however possible to use a trigger function to enforce the constraint in the database:</p>
defmodule Meetings.Repo.Migrations.CreateRoomReservations do
  use Ecto.Migration

  def change do
    create table(:room_reservations) do
      add :customer_id, :binary_id, null: false
      add :room_id, :binary_id, null: false
      add :reservation_starts_at, :timestamp, null: false
      add :reservation_ends_at, :timestamp, null: false
      add :status, :text, null: false, default: "hold"
      timestamps()
    end

    execute(
      # up
      ~S"""
      CREATE FUNCTION room_reservations_check_room_availability() RETURNS TRIGGER AS
      $$
      BEGIN
      IF EXISTS(
        SELECT 1
        FROM room_reservations rr
        WHERE rr.room_id = NEW.room_id
          AND tsrange(rr.reservation_starts_at, rr.reservation_ends_at, '[]') &&
              tsrange(NEW.reservation_starts_at, NEW.reservation_ends_at, '[]')
          AND (
            rr.status = 'confirmed'
              OR rr.inserted_at > CURRENT_TIMESTAMP - interval '24 hours'
          )
      )
      THEN
        RAISE "room already reserved";
      END IF;
      RETURN NEW;
      END;
      $$ language plpgsql;
      """,

      # down
      "DROP FUNCTION room_reservations_check_room_availability;"
    )

    execute(
      # up
      ~S"""
      CREATE TRIGGER room_reservations_room_availability_check
      BEFORE INSERT ON room_reservations
      FOR EACH ROW
      EXECUTE PROCEDURE room_reservations_check_room_availability();
      """,

      # down
      "DROP TRIGGER room_reservations_room_availability_check ON room_reservations;"
    )
  end
end
</code></pre>
The problem is that instead of that nice validation error on the changeset that we want</em> to get,
we instead end up with an unhandled Postgrex.Error</code> exception. We could, of course, simply rescue
that exception in our Meetings.RoomReservation.create/1</code> function and add the error to the
changeset ourselves, but that starts to look a little ugly:</p>
defmodule Meetings.RoomReservation do
  # ...

  def create(%{} = res_data) do
    changeset =
      %__MODULE__{}
      |> cast(res_data, [:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
      |> validate_required([:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
      |> put_change(:status, "hold")
    Meeting.Repo.insert(changeset)
  rescue
    error in Postgrex.Error ->
      case error do
        %{postgres: %{message: "room already reserved"}} ->
          changeset
          |> add_error(:room_id, "has already been reserved")
      end
  end
end
</code></pre>
TL;DR - It's All About the Raise</h3>
Knowing that Ecto.Changeset.unique_constraint/3</code> works by intercepting an error raised by the
database, I set out to see if I could implement the complex unique constraint logic in the
database and still be able to use the Ecto.Changeset.unique_constraint/3</code> validation without
needing to modify any Elixir code. Looking at the relevant code in ecto_sql</a>, we
can see that the trick to getting the room_reservations_check_room_availability</code> functionality to
work with that function is to change the PostgreSQL exception to use the correct error code as
well as a constraint name that is linked with the changeset validation. We can redifine the
PostgreSQL function as:</p>
CREATE FUNCTION room_reservations_check_room_availability() RETURNS TRIGGER AS
$$
BEGIN
IF EXISTS(
  SELECT 1
  FROM room_reservations rr
  WHERE rr.room_id = NEW.room_id
    AND tsrange(rr.reservation_starts_at, rr.reservation_ends_at, '[]') &&
        tsrange(NEW.reservation_starts_at, NEW.reservation_ends_at, '[]')
    AND (
      rr.status = 'confirmed'
        OR rr.inserted_at > CURRENT_TIMESTAMP - interval '24 hours'
    )
)
THEN
  RAISE unique_violation
    USING CONSTRAINT = 'room_reservations_room_reserved';
END IF;
RETURN NEW;
END;
$$ language plpgsql;
</code></pre>
and then add the :name</code> option to our unique_constraint</code> in the changeset validation:</p>
defmodule Meetings.RoomReservation do
  #...

  def create(%{} = res_data) do
    %__MODULE__{}
    |> cast(res_data, [:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
    |> validate_required([:customer_id, :room_id, :reservation_starts_at, :reservation_ends_at])
    |> put_change(:status, "hold")
    |> unique_constraint(:room_id,
                         message: "has already been reserved",
                         name: "room_reservations_room_reserved")
    |> Meeting.Repo.insert()
  end
end
</code></pre>


Kookaburra 0.24.0 Released - Exorcised ActiveSupport
Unknown — Tue, 15 May 2012 00:00:00 +0000
I just released version 0.24.0 of Kookaburra</a> to
Rubygems.org</a>. While there were no changes to Kookaburra's</em> API
with this release, it is a minor release rather than a patch release,
because I removed ActiveSupport</a> from Kookaburra's dependencies.
Previously, Kookaburra depended on ActiveSupport >= 3.0, and this
prevented it from working smoothly with Rails 2.x applications (at least
in the same Bundler bundle.) Since Kookaburra only used a small fraction
of ActiveSupport, it seemed easiest just to break the dependency, so
that your application can use whatever version of ActiveSupport it
needs.</p>
Because of the fact that ActiveSupport makes a number of modifications
to core Ruby classes, if your application does not otherwise explicitly
require ActiveSupport, some of these core classes may now behave
differently. Specifically, Hash</code> and String</code> core extensions were
being used, as well as ActiveSupport::CoreExt::Module::Delegation</code>.</p>
Also, where Kookaburra::JsonApiDriver</code> used ActiveSupport::JSON</code> for
encoding and decoding of JSON data, it now uses ::JSON</code> as provided by
either</em> the json</a> or the json_pure</a> gems. The tests
on Kookaburra itself pass just by swapping out ActiveSupport JSON
library for the new one, but these tests do not comprehensively test
JSON usage, and there may be differences that impact your application.</p>
Kookaburra declares its dependency on the json_pure</code> gem (a pure-ruby
version) in order to ensure that it works cross-platform. However, both
json</code> and json_pure</code> are written in such a way that, when you require 'json'</code>, it will detect if both libraries are installed and prefer the
natively compiled json</code> gem's version of the library.</p>


tmux and the OSX Clipboard
Unknown — Thu, 12 Apr 2012 00:00:00 +0000
I started using tmux</a> recently after a) I was informed that GNU screen is
basically an outdated POS, and b) an excellent book</a> on the subject
was published. I'm glad to have made the switch, as tmux is a wonderful
improvement over screen. However, on OSX, I found that the pbcopy</code> and
pbpaste</code> commands (among other things) would no longer work from within a tmux
session.</p>
After a bit of searching, I found this post</a> that explains how to make
things right again.</p>
TL;DR:</p>
brew install reattach-to-user-namespace
</code></pre>
# Reattach to user namespace so that clipboard integration with OSX works again
set-option -g default-command "reattach-to-user-namespace -l zsh"
</code></pre>
Obviously, replace zsh</code> above with whatever you use for your shell.</p>


Kookaburra Rewrite for 0.15.1
Unknown — Sat, 10 Mar 2012 00:00:00 +0000
After getting some good feedback on Kookaburra</a> Kookaburra</a> since the original
release announcement as well as using it in a few more projects, [Sam] SLG</a> and
I decided to treat all of the versions prior to 0.15.0 as a spike and rewrite
the framework from the ground up. Although we certainly learned a lot about the
approach with the pre-0.15 versions, the problems with continuing to grow
Kookaburra from that seed became apparent as we tried to use it in more
applications.</p>
Because the original code was extracted from another project, the Kookaburra
framework itself was not well tested. It had been indirectly tested due to its
position within that other project's tests, but as a seperate library it lacked
tests that would help bot document the project and ensure newcomers can work on
it with a good safety net. Additionally, there were a number of design decisions
made in the earlier versions of Kookaburra that were...questionable. Not only
would these decisions probably have been made differently if the development of
the framework itself had been driven with unit tests, these decisions in many
cases made it difficult to go back and add tests after the fact.</p>
Starting with 0.15.1, which I just [released] Gem</a> a few moments ago,
Kookaburra has been rewritten using a proper TDD approach. In some ways, 0.15.1
is possibly less functional than its 0.14.x cousins, but it is almost certainly
easier to understand and contribute to the project from this point on. Please
have a look at the new codebase, try it out in your projects, and send me a pull
request with updates to the library. Also, if you have any problems getting it
running, please don't hesitate to [let me know] GHIssues</a>.</p>


Using Jeweler for Private Gems
Unknown — Wed, 25 Jan 2012 00:00:00 +0000
I really like using [Jeweler] 1</a> to create and manage gems, but its default
behavior is to publish your gem to [rubygems.org] 2</a> whenever you run rake release</code>. This is great for generally useful libraries that you want to
open-source, but not as great when you want to use gems as shared libraries for
internal use only (whether because the code contains business secrets or just
because it's not something that would be useful to the community at large.)</p>
After a bit of poking around, I figured out how to use Jeweler to manage
your versioning, gemspec and git tagging without pushing the result to
rubygems.org when running the release task. It turns out to be both obvious and
trivial. Just add the following line to the end of your gem project's
Rakefile</code>:</p>
Rake::Task[:release].prerequisites.delete('gemcutter:release')
</code></pre>


Acceptance and Integration Testing with Kookaburra
Unknown — Sat, 21 Jan 2012 00:00:00 +0000
UPDATE (2012-01-22):</strong> I realized this morning that the credit I gave to Sam
Livingston-Gray</a> below may not have
adequately shown how instrumental he was in getting this project off the ground;
especially since much of his work was done in the private repository from which
this was extracted. So, thanks, Sam. This might not have gone anywhere if you
hadn't worked to put the idea in practice in our application and helped everyone
on our team learn how to use the approach. I made a few minor changes below to
reflect this a bit better.</em></p>
We've been using Cucumber</a> for acceptance testing at
Renewable Funding</a> since back when it was still part of
the RSpec</a> project (indeed, since before we were even
Renewable Funding). While we've always liked the ability to have plain-language
feature documentation that we could automatically test against, after years of
adding to and maintaining a fairly large set of Cucumber scenarios, the cost of
that maintenance was starting to really slow us down. The test suite began to
grow fragile, and it seemed like every time one of our UX designers changed
anything about the application's interface, the development team would spend a
bunch of time just babysitting Cucumber tests to get them passing again.</p>
Last year, as I was reading Jez Humble's excellent Continuous
Delivery</a> book,
I was inspired when I came across the section titled "The Application Driver
Layer" (p. 198). This section describes an approach to acceptance testing where
the specification and the test implementation are isolated from the details of
the application's user interface by inserting a layer between the two that uses
good old OOP to abstract the user interface components. Martin Fowler describes
it as the [Window Driver] 1</a> pattern on his website.</p>
I started a proof-of-concept implementation of this pattern last summer, then my
coworker, Sam Livingston-Gray</a> and I
started pulling it into a new project at work. After Sam and the rest of the
Renewable Funding team helped improve on my original attempt while putting it to
use for the last six or so months, we extracted a library to make it easier for
other Ruby developers to implement this pattern in their testing. I'm happy to
introduce Kookaburra</a> to the world.</p>


Capybara, Selenium and Firefox 3.5
Unknown — Sat, 30 Apr 2011 00:00:00 +0000
I banged my head against this one while setting up our CI build for a new
project at work and just now figured it out. I added an example integration
spec that just visits the default Rails homepage, clicks the "About your
application's environment" link, and verifies the Rails version. This link
uses an AJAX action to load the environment details, so I marked it as a
JavaScript test, which would cause it to use Capybara's Selenium driver.</p>
The test passed fine on my machine (with Firefox 4 installed), but it failed
consistently on CI, which has Firefox 3.5. It kept failing with the exception
Capybara::TimeoutError: failed to resynchronize, AJAX request timed out</code>. I
didn't dig enough into the guts of Selenium/Firefox to find out exactly what
is going on (and a Google search for that error message turned up nothing that
really explained it) but I did at least find a workable fix. Just add the
following line to spec/support/capybara.rb</code>:</p>
Capybara::Selenium::Driver::DEFAULT_OPTIONS[:resynchronize] = false
</code></pre>


Apprenticeship Program at Renewable Funding - Thoughts?
Unknown — Fri, 04 Mar 2011 00:00:00 +0000
The Renewable Funding</a> technology team is moving into
some new office space in a few months, and now that we'll have our own space,
I'd like to start up a variation on an apprenticeship program at work.  I'm
still in the early stages of designing this, and I want to gather some input
from the community before deciding how we might run the program.</p>


Production Release Workflow with Git
Unknown — Sat, 08 Jan 2011 00:00:00 +0000
After growing the ProjectDX</a> team from three to
eight software developers, our release process was a complete pain, and it
typically took two to three hours to get a good build on the production branch
(and even then some insidious issues would sneak through). By making a few
changes to our development and acceptance process, we were able to turn it
into a five-minute, low-stress job.</p>


What it Really Means to be "Agile
Unknown — Wed, 15 Dec 2010 00:00:00 +0000
Yesterday, Elizabeth Hendrickson posted her Agile Acid
Test</a> in which she asks
three questions to determine whether or not a team is truly "agile". There is
also the Agile Manifesto</a> which describes the
values that an agile team should adhere to. While there is nothing that I
disagree with in either Elizabeth's post or the Agile Manifesto, there
are two simpler questions you can ask that get to the heart of whether or
not your team is agile: Can you react immediately and without panic when
external constraints on your project are changed</em>, and does your team regularly
and frequently review its processes to ensure the answer to the previous
question is always "yes"?</em></p>


Retrospective Facilitation
Unknown — Tue, 13 Jan 2009 00:00:00 +0000
Sprint 1 (11/03)</th> Sprint 2 (11/24)</th> Alpha (12/15)</th> Beta (12/29)</th></tr></thead>

11/03</td> 11/24</td> 12/15</td> 12/29</td></tr>
</tbody></table>
I facilitated my second release retrospective with the ProjectDX team yesterday. The first one, back in October, went reasonably well considering I was asked to facilitate at the start of the retrospective and had no time to prepare. However, yesterday's retrospective went even better since I knew in advance that I would be facilitating.</p>
The activities we used during the retrospective primarily came from Diana Larsen and Esther Derby's book, Agile Retrospectives. We began with the "ESVP" (Explorer, Shopper, Vacationer, Prisoner) activity to gauge the group's interest level concerning the work we were doing in the retrospective. Almost everyone in the group identified as an Explorer through an anonymous vote. With this group, I generally trust the outcome, although I did wonder if anyone might have provided the answer they thought they were "supposed" to give.</p>
Next, I asked the team to construct a timeline of events from the past release. Before the retrospective began, I drew the timeline on our whiteboard and divided it into sections for each iteration (our cycle consists of two 3-week sprints followed by a 2-week internal Alpha testing period and a 2-week customer Beta testing period). The result looked something like:</p>
During the retrospective, I asked everyone to take some time to think about any events during that time period that were meaningful to them in any way, write the event on a sticky note, and then place the note on the timeline in approximately the right spot. We spent about 45 minutes generating events, during which time we were free to look back at calendars and email, as well as the commit logs from our SCM. I set a 45-minute time box and mentioned that it was fine for people to take a break if they felt they couldn't remember any more events, asking everyone to simply be back on time for the next step.</p>
Instead of specifying that people work in pairs, I chose to let the team self-organize. It was interesting to see how, at first, everyone seemed to work on their own, but as the more obvious events were posted to the board, people started working in groups to come up with more data. At the end of the allotted time, we had a whiteboard full of events.</p>
The area below the timeline was intended for a graph of energy/emotion levels, which I'll discuss in a moment, but JD Huntington had a great idea to also add a rough bar graph of our commit volume, as reported by GitHub's activity graphs, to that area.</p>
After everyone had a last opportunity to look at the board and add any last-minute stickies, I asked everyone to take a turn at the board and use markers to place a blue dot next to any event which they felt positive about and a red dot next to any event that they felt negative about. Then, I asked them to think about their positive and negative energy/emotion levels throughout the release and draw a line graph in the bottom section to signify their ups and downs throughout the release cycle. At this point, we looked at each iteration in the cycle and I asked the team to make observations about the data. Using bullet lists at the bottom of the white board, we captured the observations as further data points for analysis.</p>
After a break for lunch, we used the "Patterns & Shifts" activity to generate insights about the data on the time line. We gathered around the white board where I asked each person to mark on the time line any point at which they felt there was a shift or transition of some sort and to draw lines between any events, graph points and observations that they felt were somehow related.</p>
After no one was able to see any more connections, I asked the team to look for any patterns among the connections and the shifts. The first thing we noticed was that we felt like the real shift between iterations was constantly happening about 3 days after the actual time box ended. Rather than get into a long discussion about it when it was noticed, I asked the team to simply keep it in mind as a possible concern during the next phase of the retrospective.</p>
The next observation was that we had a number of connections with long lines spanning two or more iterations. I asked everyone to focus on these connections and look for those where we may have noticed an issue early on but it wasn't addressed until much later. In some cases it turned out that while that was the issue, the length of the line just signified that either the work took that long or there was a conscious decision not to address the issue right away.</p>
Reflecting on the other cases led to an interesting discovery, however. Chris noticed that where the lines crossed the bounds of the iterations it was likely signifying a parallel cycle which we were attempting to force into our release cycle: specifically, we have both an explicit product release cycle and an inexplicit customer deployment cycle occurring at the same time.</p>
After we were finished recognizing patterns in the data, it was time to discuss these patterns (and anything else on our minds) and come up with a few action items for the next release cycle. Here I ran the team through the "Circle of Questions" activity. In this activity, the group sits in a circle, and each person takes turns asking a question to the person on their immediate left. The question can be about anything they like (barring anything offensive or attacking), but it's helpful to focus on the insights gained during the previous stage of the retrospective. The person to the left answers the question to the best of their ability, and then they ask the person to their left any other question (or the same question if they feel they'd like a better answer). This continues until the alloted time is up or you have gone around the entire circle twice, whichever comes last. Make sure you go around the complete circle: if some people in the group get more turns to ask or answer a question than others, it can send the wrong message.</p>
The thing that I really like about the Circle of Questions activity is that when you have a mixed group of people—some of whom tend to dominate the conversation while others tend to stay quiet—it gives everyone a chance to ask their questions and have their opinions heard without feeling like the most boisterous people have the only valuable input. We certainly have a mix like that in our case, and this activity worked wonderfully.</p>
In the end, we came away with a number of things to change for the next release cycle. It's interesting to note that none of our action items were directly related to the insights generated from the time line. I think the insights are still valuable (and are contained in the notes which we post to our internal wiki), so we may still come back to them in the near future; we were running over on time and had to wrap up even though the discussion could have continued longer.</p>
Before we held this retrospective, I think some of the team members felt that we would not gain much from doing a full-day retrospective this time around. We had made good progress on our action items from the previous retrospective, and the general feeling was that things were going basically well. What could we possibly need to change?</p>
People, please don't hold retrospectives only when your team feels like there's a problem that needs to be dealt with. If you hold retrospectives on a regular basis, whether you feel like you "need" to or not, you will uncover issues before the become problems. In our case, we managed to uncover a number of issues during our retrospective that could easily have grown into much larger problems if we didn't bother to think about them until the effect was obvious.</p>
If you do have obvious issues, then you might choose a different set of activities than what we did for this retrospective. The time line activity is great for discovering potential improvements even when everything seems to be going well, so I suggest giving it a try the next time your team is tempted to skip a retrospective because no one thinks there's any huge problems.</p>

John Wilger

Somebody Has to Design the Factory

The fitters didn't become tool designers</h2> The aggregate story is that automotive engineering employment grew enormously over the following century. The story for an individual fitter in 1913 was worse than that.</p>

The Software I Wouldn't Have Bothered to Build

AI Might Not Make You Faster. It Can Make You Free.

Making Chrome Profiles Real Launch Targets on Niri

Coding Harness Plugins Are Agentic Systems

Make Fabrication Unrepresentable

The Tools You Build Are More Important Than The Tools You Use

The Hidden Pitfalls of AI Software Development

The Language Model Is Just Another User

Generative AI is a UX Revolution

Using the 27" LG UltraFine 5k Display with Linux

Complex Unique Constraints with PostgreSQL Triggers in Ecto

Kookaburra 0.24.0 Released - Exorcised ActiveSupport

tmux and the OSX Clipboard

Kookaburra Rewrite for 0.15.1

Using Jeweler for Private Gems

Acceptance and Integration Testing with Kookaburra

Capybara, Selenium and Firefox 3.5

Apprenticeship Program at Renewable Funding - Thoughts?

Production Release Workflow with Git

What it Really Means to be "Agile

Retrospective Facilitation