Cleveland, Ohio · Software engineering
Somewhere in your company an AI pilot is still a pilot. Someone reads every answer before it goes out, and nobody is counting their hours. The vendor’s number was 95 percent, of something. Nobody wrote down what “working” means, so nobody can say whether it does.
We build AI into business software, carefully. The first thing we ask is what the current state is costing you.
Where it hurts
A stall is quieter than a failure and costs more, because nobody adds it up. These are the ones we hear about most. See if any of them are yours.
The demo was in the spring. It is still “in pilot.” Nobody wrote down what done looks like, so nobody can call it finished, and nobody can kill it either.
How many months has it been?
Someone reads every answer before it goes out. That is the right call. It is also a job nobody budgeted, done by someone who already has one.
What is that costing you a month?
The vendor quoted a number. Ask what was measured, on how many examples, from whose data. If the answer takes longer than a sentence, there isn’t one.
Would you bet a customer on it?
One wrong reply reached a customer and the whole program froze. The idea was fine. There was no plan for the day it went wrong, so the day it went wrong became the plan.
Who decided to stop, and what would let them start again?
The team turns work around quicker now. The error rate didn’t move. The thing you were trying to change is exactly where it was.
Which number were you trying to move?
The provider updated the model. Nobody told you. The prompts that worked in June don’t anymore, and there is no test that would prove it.
How would you know today?
The feature needs to write to software with no API, no tests, and one person who understands it. That person is on vacation.
What happens when they leave?
A customer’s lawyer, or yours, asks why a document is in your system and what license let it in. The honest answer was a shrug.
What would you show them?
If two of these are yours, it is worth thirty minutes. If none are, you probably don’t need us yet, and we will tell you so.
The first call
We ask what is stuck, how long it has been that way, what you have tried, and what it is costing you. We don’t bring a deck. We don’t send a proposal afterward unless you ask for one.
You should leave the call knowing which kind of problem you have: the model, the data, the software around it, or the fact that nobody wrote down what “working” means. It is usually the last one, and it is the cheapest to fix.
At the end we both decide whether there is a next step. “No” is a fine answer. It is faster than “let me think about it,” and we will say it too if we are not the right people. Where we can, we will tell you who is.
Work
We are a new practice, so most of what we do is in the open: a complete AI application built in public, the plan we write before we build, and the software we built before this that is still running.
Open source · In progress
A complete AI assistant for the order desk of a fictional industrial distributor, built in public and written up part by part, including the parts that go wrong. Elixir and Phoenix. Every decision, test, and number published.
Follow the build →Sample deliverable
The document we write before we build. This one is for Order Desk: why AI, what “working” means, how it is evaluated, the architecture, security, cost, launch stages, and risks. Twelve sections. Every threshold is a number. It is what a Plan engagement produces.
Read the plan →Before Cleveland Software, Nicholas co-founded Treetop Interactive and led its engineering for more than fifteen years. Treetop has since closed, so the projects live here. Some of them are still running, which matters more to us than how they looked at launch. A few:
A suspicious-activity reporting app for iPhone and Android that routes reports from citizens and first responders straight to state intelligence centers. Adopted by New York, Pennsylvania, Ohio, Virginia, Colorado, and Louisiana, and by the Ohio Department of Public Safety as A Safer Ohio. Still in service today.
Mobile apps · Backend · Agency integrationsA web application that manages and displays data on foreign direct investment into the United States: interactive maps, sortable tables, and an admin backend so the department could update the data itself.
Web application · Data · MapsProduct configuration and pricing software for a manufacturer of fluid and gas pumping equipment. Customers configure the equipment they need and get a quote without waiting on a rep. An order desk problem, fifteen years before Order Desk.
Web application · Configuration · PricingAn online certification and reporting platform where municipalities get certified for their sustainability practices and residents can see how their community is doing.
Web application · Certification · ReportingChild sponsorship software. Supporters browse children by country and age, choose one to sponsor, and the system manages the recurring donations.
Web application · Subscriptions · PaymentsiOS and Android apps that let veterans record and share stories of their service, with a companion website. WordPress is the backend, reached through its REST API for accounts, stories, and search.
Mobile apps · WordPress APIA web app that collects, interprets, and charts infant sleep data from a portable USB monitor so parents and clinicians can see the patterns.
Web application · Device dataA web application where companies run branded bracket contests around their own products, with payments and social sign-in. After launch the client was accepted into AlphaLab, a Pittsburgh startup accelerator.
Web application · PaymentsHow we work
The same steps every time, in the same order. Chip Huyen’s book AI Engineering lays them out well, and the Order Desk series walks through each one with code. None of them are secret. Most of them get skipped.
What the feature does, who is responsible for its output, and what it must never do.
Acceptance rate, error rate, latency, cost per use. A minimum to ship and a target for six months out.
A model with a simple prompt and no help. Everything after has to beat it.
Examples, then retrieval, then tuning. Each step is earned by the measured failures of the last one.
Mask private data before it leaves. Check every number and identifier that comes back.
Prompt version, model version, what was retrieved, what came back, and what the person did with it.
Automation gets more authority only when the numbers earn it. Some steps never should.
Pinned versions, a second provider, a nightly check, and a rollback you have practiced.
Services
Almost every engagement starts with a plan. The prices are here so you can decide before we talk.
Fixed fee · Three weeks
For when nobody can say what “working” means.
We write the plan before anything is built: what the feature is for, what “working” means in numbers, how we will test it, what it will cost to run, and what it must never do. Your engineers can build from it. Your lawyer can read it.
If the honest answer is “not yet,” the plan says so, and says what would change the answer.
$7,500 · See a finished plan
Fixed scope per feature
For when the plan says yes.
We build the feature into the system you already run, or build the system if there is not one. Retrieval, guardrails, an evaluation suite that runs on every change, logs a person can read, and a review step for whoever owns the result.
Most first features take eight to sixteen weeks. Your team works alongside us in your repositories, and everything we make is yours.
Quoted from the plan
Monthly · Cancel any time
For when the model changes on a Tuesday.
After launch we run the evaluations, watch the numbers, handle model changes and provider upgrades, and keep the runbook current.
When a model you depend on changes its behavior, you hear it from us, not from a customer.
From $1,500 a month
For when the feature has nowhere to live.
If the system the feature needs to live in has no API, no tests, and one person who understands it, we fix that first. Same care, no AI required. We have done this inside PHP, .NET, and Java, and we build new on Elixir and Phoenix.
For when someone asks where a document came from.
Generating text is cheap now. Proving where a source came from is not, and it is the part a lawyer asks about. For teams that ingest licensed content, we preserve the copyright and license information that arrives with each source and produce a per-source record of why it was allowed in, in a form anyone can check without asking us. See a record. Starts with a Plan.
Order Desk
Most AI examples stop where the hard part starts. We are building one all the way to the numbers a business would accept, in the open, and writing up every part, including the ones that go wrong.
Halstead Valve & Fitting is a fictional 600-person distributor. Its order desk gets about 900 emails a day asking for quotes, lead times, and order status. Order Desk reads each email, looks up the catalog and the ERP, and drafts a reply and a proposed action for a rep to approve, edit, or discard. Nothing is sent and nothing is written to the ERP without a person.
Production-grade here means: measured against written thresholds, guarded on input and output, logged and traced, reviewed by a person, and runnable by someone other than the author.
Open source under the MIT license. Elixir, Phoenix, and Ash. The fake ERP, catalog, and email corpus are generated and included, so anyone can run the whole thing. The repository opens with Part 1.
| # | Part | Status |
|---|---|---|
| 0 | The plan | Published |
| 1 | The world: a synthetic catalog, ERP, and inbox | In progress |
| 2 | Baseline and the golden set | Next |
| 3 | The evaluation guideline | Planned |
| 4 | The judge | Planned |
| 5 | Prompts and structured output | Planned |
| 6 | Retrieval | Planned |
| 7 | Fuzzy part descriptions | Planned |
| 8 | Guardrails | Planned |
| 9 | Router and gateway | Planned |
| 10 | The rep console | Planned |
| 11 | Observability | Planned |
| 12 | Security | Planned |
| 13 | Cost and latency | Planned |
| 14 | The finetuning decision | Planned |
| 15 | Launch stages | Planned |
About
Cleveland Software is Nicholas Zographos. I have built software for businesses for twenty-five years: custom web applications, mobile apps, and the unglamorous systems in between. Before this I co-founded Treetop Interactive, where we built for state agencies, nonprofits, manufacturers, and startups. Some of that software is still running, which I care about more than how it looked at launch.
I like software that can be checked. A number over an adjective. A record over a promise. Most of what I build on my own time is the same idea in different clothes: public data that shows its sources, records that can be verified without trusting whoever made them. Order Desk is that habit applied to the thing everyone is now selling.
I work in Elixir and Phoenix by preference and in whatever you already run by necessity. When a project needs more hands, I bring in people I have worked with before. I take a small number of engagements at a time so each one gets senior attention.
I live and work in Cleveland, Ohio.
Contact
One email. The system it lives in, the part that is stuck, and how long it has been that way. We reply within two business days and tell you honestly whether we are a fit.
scoping@clevelandsoftware.com