There’s a joke in consulting. A client comes to you and says they have “detailed specs,” then hands you the title of their project or a sparse bulleted list. Lately this joke has gotten a little less funny, though, because that’s exactly what people are trying to hand an AI as a prompt, expecting quality software back.
Since 1952, when Grace Hopper created the first compiler, A-0 for the UNIVAC I, developers have increasingly been able to write more human-readable code, letting the compiler worry about turning it into something a machine can run. Over time those languages got more and more readable, and developers moved up abstraction layers. With AI, we’ve moved up yet another layer. We can now write down our ideas and have an AI “compile” them into code for us.
Wire enough agents together and you can add testing and review steps, in what people have started calling a “software factory.” Ideas go in one end, and features come out the other. You now have an “idea compiler.” In this blog, we’ll look at what it means to move up an abstraction layer, and how we can use AI to give us much of what the compiler gave us one layer down.
When to Stop Reading the Output
Let’s look at C# and .NET as an example, since .NET uses a just-in-time (JIT) compiler. You build a C# project and the compiler turns your code into Intermediate Language (IL). When you run it, the JIT turns that IL into machine code. Very few .NET developers ever read that IL, and nobody asks to review it in a pull request, because we trust it.
New abstraction layers in the stack have to earn that trust the same way. When the first compilers showed up, plenty of programmers didn’t believe a machine could write code as compact and efficient as theirs, so they checked the output manually. Eventually, compilers got good enough that checking the machine code was a waste of time. When AI-generated code gets that good (if it hasn’t already), we’ll probably stop reading it too. Teams running “dark factories” (where no human writes or reviews the code) are betting we’re there already.
We can now write down our ideas and have an AI “compile” them into code for us.
The Compiler Says ‘No’
If you write two overloads for the same method that both match a particular call, the C# compiler doesn’t just pick one for you. It stops with an error: “The call is ambiguous between the following methods or properties.” Misspell a variable name and the build fails. A big part of programming has always been turning a fuzzy idea into something precise enough that a very literal machine doesn’t complain about any remaining vagueness.
The idea compiler of AI, though, rarely says no to things. Ask a coding agent to “add discounts to checkout” and you won’t get an error back, even if it’s a bad idea. The agent decides on its own whether discounts stack, whether they apply before or after tax, and what happens when one expires while it’s already in the cart. Then it writes the tests:
[Fact]
public void Discounts_Stack_When_Multiple_Codes_Applied()
Nobody in the business explicitly decided that discounts should stack. The agent did it all by itself, and then it wrote a test to make sure nobody changes it in the future. To be fair, human developers guess all the time too, but they usually guess out loud. They ask in Slack, bring it up at standup, or leave a pull request comment that says “assuming these stack, let me know if not.” With a vague prompt, an agent makes the same guess, but without telling anyone.
Catching the Compilation Errors
If ideas are the new source code, our human attention belongs on the ambiguity, the places where AI makes an assumption but a compiler would have thrown an error. Here are a few ways to use deterministic tools, AI, and your own experts as a check on your AI “idea compiler.”
1. Review the Specs Like Code
If AI is going to “compile” a spec, the spec deserves a pull request too. Have reviewers look for the assumptions and vague spots an agent would fill in on its own. A spec line like “Send customers a reminder before their appointment” should come back with review comments: How long before? In whose time zone? What happens if the appointment gets rescheduled after the reminder goes out? Answering “24 hours before, in the customer’s time zone” at that stage shifts the decision left, before anyone has written code around the wrong answer.
2. Make the Agent Ask
Put something like this in your AGENTS.md or CLAUDE.md:
## Before writing code
List every assumption you're making about the requirements and every
question the current spec doesn't answer. Stop and wait for answers
before implementing. Don't write tests that lock in an assumption a
person never confirmed.
Now “add password reset” comes back with something like this before any code gets written:
1. How long should the reset link stay valid?
2. Should resetting a password log the user out on other devices?
3. Should the form say whether an email address has an account?
A list of open questions is the closest thing an idea compiler has to a compile error. Tools are starting to build this in, too. GitHub’s Spec Kit has a /speckit-clarify command that asks up to five targeted questions about your spec and writes the answers back into it before the agent plans anything.
3. Write Your Own Compile Errors
Nothing stops an AI in its tracks more than an actual compiler error. Did you know that you can create your own compiler warnings and errors in .NET? Start by turning on TreatWarningsAsErrors, so every warning stops the build and the agent can’t leave one behind for someone else to find.
Then turn the rules your team cares about into Roslyn analyzers. For example, imagine you want to flag the use of ToList() on an IQueryable with no Take():
// ❌ bounded by time, not volume; a busy day loads every row
return db.Orders
.Where(o => o.CreatedAtUtc >= since)
.ToList();
Here’s the heart of the analyzer that catches it:
// e.g. "ToList"
var name = memberAccess.Name.Identifier.Text;
if (!MaterializingMethods.Contains(name)) return;
// Only database queries
var type = ctx.SemanticModel.GetTypeInfo(memberAccess.Expression).Type;
if (!IsQueryable(type, queryable)) return;
// Already has a Take()
if (ChainContainsTake(memberAccess.Expression)) return;
ctx.ReportDiagnostic(Diagnostic.Create(
UnboundedQuery, memberAccess.Name.GetLocation(), name));
Set its severity to error in .editorconfig, and the build fails with “‘ToList’ loads every matching row; add .Take(n) before materializing.” The agent reads that message and fixes the code before you ever see it.
For simple bans, like “domain code never calls DateTime.Now,” Microsoft.CodeAnalysis.BannedApiAnalyzers needs one line in a text file. Architecture tests with NetArchTest do the same for layering, and other platforms have ESLint rules, Error Prone, and Semgrep.
Analyzers have limits. A while (retries < maxRetries) loop looks bounded to an analyzer even when someone forgot retries++. They catch the patterns you can describe, and the rest still needs tests and people.
4. Write the Edge Cases as Tests
You want to be careful about allowing an agent to grade its own homework. Instead, before the coding agent starts, have another agent write the edge cases as end-to-end tests in something like Playwright. One might double-click Submit on the booking form and expect one appointment, not two. Property-based tests go even further. State a rule such as “converting any appointment time to UTC and back gives the same time,” and tools like FsCheck or CsCheck can throw hundreds of generated dates and time zones at it. A Claude Code PreToolUse hook that rejects writes under tests/e2e/ also keeps the agent from “fixing” a failure by editing your tests.
5. Test Your Tests
Agents can sometimes write tests that run through your code without checking much of anything. A test like this one passes no matter what the shipping math does:
[Fact]
public void CalculateShipping_Works()
{
var cost = calculator.CalculateShipping(
weightKg: 2.5m, zip: "49503");
Assert.True(cost >= 0);
}
Mutation testing tools like Stryker.NET change your code, for example flipping a > to a < or deleting a line, then rerun your tests. If nothing fails, those tests weren’t checking the code very well. Run mutation testing on just the changed lines so it stays fast, and fail the build when a change survives.
6. Put a Different Model on the Red Team
Having the agent that wrote the code also test it is as bad as asking a developer to test their own code. So borrow from security best practices and have one model on the blue team building the feature, and a model from a different vendor on the red team attacking it. The red team might find that changing the order ID in the export URL returns another customer’s data, a hole the blue team never thought to test.
7. Pay Down Comprehension Debt
A term making the rounds right now is “comprehension debt.” It’s technical debt that lives in your team’s heads instead of in the codebase. The system works, but nobody can explain why. A software factory that compiles your vague ideas into code can pile up comprehension debt quickly, especially when no human ever saw many of the decisions it made along the way. The bill for this kind of debt can be impossible to see, and it might not come due for months or years.
You may not read every line of code any more than you read IL. But when something goes wrong, you want someone on the team who can drop down a layer and understand what’s happening there. Say a customer gets charged twice for one order, and every test passes. Someone has to open the code the agent wrote, trace the checkout flow, and find the retry that runs the payment a second time. Tracing it is a lot harder in a codebase nobody on the team has read, so keep people in the habit, for example by rotating who reviews agent pull requests each sprint.
Who’s the Compiler?
I do think compiling ideas into code is the future of development, but at least for now, it takes both human and AI steps to play the role of the compiler.
Think back to the client who hands you the title of their project. Good developers don’t build from that; they go back to the client with questions: who uses this, what happens when a payment fails, do discounts stack? Asking those questions has always been the most valuable thing a developer brings to a project, and our current idea compilers can’t do it on their own yet. Until they can, the developer’s job is to be the part of the idea compiler that throws the errors.
At Trailhead, we build with AI coding agents every day, and the analyzers, review agents, and tests in this post are how we keep them honest on our own projects and our clients’.
If your team is building its own software factory, or wondering whether it should, let’s talk. We can help add the compiler features your agents are missing, working next to your developers or taking the project on for you.



