Post 4 ended on the part of the build I was most pleased with, which is a fine way to end a post and a slightly dishonest way to end a series. So here is the other side of it.
Some context first. This was never meant to be a product. It started as an experiment to find out how hard it actually is, in 2026, for someone who audits financial statements for a living to build a working AI tool. The answer turned out to be two answers. Getting something that runs is much easier than I expected. Getting something you would put your name on is much harder.
The tool has a lot of problems. Below are the ones worth knowing about if you ever want to build something like it, kept general on purpose, because the specific bugs will be fixed by the time most people read this.
The input has to be shaped a particular way
The tool expects two Excel files, named a specific way, each with a ledger sheet and a chart of accounts sheet, each with a specific set of columns. Costs negative, revenue positive. If your files don't look like that, it doesn't run.
That sounds like a small thing and it isn't. The chart of accounts has to already carry a hierarchy, because that hierarchy is what the materiality rules key off. If your categories are named differently to mine, the tool doesn't fail loudly. It quietly falls back to a default threshold and applies the wrong sensitivity to a line you cared about. There are also a few values sitting in the script that are calibrated to my sample data, including the year it filters on, which anyone running it on their own ledger would have to change by hand.
This is what a template looks like as opposed to a product. The difference between the two is mostly the work of handling every way real data can arrive, and that work is enormous, unglamorous, and where most of the effort in any finance system actually goes.
It can only be as specific as your journal narrations
When the tool builds its prompt, the entire evidence base for why something moved is the free-text description field on the underlying journals, grouped and ranked by size. That's it.
So if your journals carry narrations like "Q3 commission accrual, revised rate card", the commentary comes back specific and useful. If they carry "JE 4471", or nothing at all, the model has a number and no story, and it does the honest thing: it says the cause can't be determined and moves on.
I find that clarifying. People ask what data quality means for AI in finance and usually get an abstract answer about governance. Here it is concrete and slightly boring. It means whoever posts the journal writes a sentence a stranger could follow.
It compares to last year, not to plan
The tool does actuals against prior year. There is no budget in it, and no forecast.
Which means that for most finance teams it answers the second most interesting question. Nobody's board asks why costs moved against last year. They ask why costs moved against the number the team committed to in October.
That's not an AI limitation, and I want to be precise about it. The ledger holds actuals. Budget lives somewhere else, in a planning tool or a model or a spreadsheet somebody owns. Getting the two into one comparison is a plumbing problem. The AI part would barely change.
The reason usually isn't in the ledger
This is the limit underneath all the others, and the only one I don't think you can engineer your way out of.
The ledger records that receivables went up 22%. It does not record that you agreed 60-day terms with three key customers in March. It records that staff costs rose. It does not record that two of those hires were replacements and one was a new territory. It records a spike in entertainment costs in September. It does not record that it was one client event, approved and budgeted months earlier.
Every one of those explanations exists somewhere. In an email, a contract, a set of board minutes, or somebody's head. None of it is in the file. A model that produces confident reasons from a ledger that doesn't contain them isn't being clever, it's guessing with better grammar.
So the tool treats that gap as the deliverable instead of trying to close it. You run it, read the five questions it ends on, write the answers in plain English into a small context file, and run it again. The second pass carries what you know into the commentary. Two runs, and maybe fifteen minutes of your time in between.
That's less impressive than "AI writes your commentary". It's also the only version I'd defend. The tool is a very fast, very literal junior who has read every transaction and understands none of the business.
It still gets things wrong
Post 4 was about a prompt pattern that cut down on invented explanations by giving the model somewhere useful to put its uncertainty. It worked. It did not work completely.
The output still occasionally reaches past the data it was given, and the more structured and finance-shaped the output looks, the harder that is to spot. Percentages get calculated in places where a percentage doesn't mean anything, like a balance that crossed from negative to positive. Edge cases in the materiality rules let things through, or hold things back, in ways I only found by going looking.
None of that is unusual, and none of it is a reason to give up on the idea. It is a reason to keep a person between the output and the audit committee, which is roughly where I started.
Why I'm not that bothered
Every problem above falls into one of two buckets, and both of them are getting smaller.
The first bucket is my code. Rigid inputs, hardcoded values, edge cases in the rules. That's ordinary iteration. Each of those took an afternoon to find and ten minutes to fix, and the list gets shorter every time I sit down with it. Nothing there is hard, it's just work.
The second bucket is the model, and that one improves without me doing anything at all. The version of this tool I started in April used a model that has since been replaced twice. Things that needed three paragraphs of prompt scaffolding a year ago now work on the first try. If I rebuilt this from scratch today it would be shorter, better, and take a fraction of the time. That trend has been reliable for three years and I have no particular reason to think it stops.
Which is the actual finding from the experiment, and the reason I'd tell any finance person to try building something. Not that the tool is good. It's fine. But the distance between "I have an idea" and "I have something that runs on real data" has collapsed, and it keeps collapsing.
What the series was actually about
If you've read all five of these, the honest summary is smaller than the one I'd have written in April.
A well-built AI workflow will read your ledger, apply a materiality policy you can inspect, write a structured first draft in under a minute, and tell you the handful of things it doesn't know. That's real, and a couple of years ago it wasn't.
It won't tell you why. It doesn't have the email.
The judgment sits exactly where it always sat. What changed is that you now start from a draft and a list of questions rather than a blank text box at 6pm, which is the thing I said I wanted to fix in Post 1.
That's the series. If you've built something similar, or you're a finance person who has been meaning to and hasn't started, I'd like to hear about it. Find me on LinkedIn.
