How to Add AI to Your Existing App (Without Rebuilding It)

You can add AI to an existing business application without rebuilding it, and in almost every case you should. The three features that reliably pay off — search that understands meaning, summarisation, and classification — all attach to software you already run: one new endpoint, one API call, and one column or table added to your existing database. No framework change, no migration, no rewrite. A first useful feature is typically two to three weeks of work, and at the volumes a small business actually processes it costs single-digit dollars a month to run. This guide covers which features are worth building, what they cost, and how to prove one in a fortnight.

Key takeaways: Start with search, summarisation, or classification — they map onto data you already store and their failure modes are cheap. Bolt on to your current stack rather than rebuilding; the model is an API call, not an architecture. Price the pilot in tokens before you commit, put a human in the loop wherever an error costs money, and settle the data questions before the first request leaves your network. Costs below are list prices as of August 2026 and planning estimates, not a quote.

The three places AI actually pays off

Most AI projects that stall did not fail technically. They were pointed at a problem where being right 90% of the time is not useful. The three below work because a wrong answer is visible, cheap, and recoverable.

Search that understands meaning. Conventional search matches words; users search with intent. A customer typing "item hasn't arrived" finds nothing when your records say "delivery delayed", and a sales rep looking for "bulk pricing for restaurants" misses the document titled "HORECA volume rate card". Semantic search closes exactly that gap, and it is the highest-return first feature for any application holding more than a few thousand records.

Summarisation. Long threads, call notes, support histories, vendor documents — anything someone currently reads in full before acting. A three-line summary at the top of a record saves minutes on every open, and if it is wrong the full text is one click below it.

Classification and routing. Tagging an incoming ticket by type and urgency, sorting invoices by vendor, flagging which enquiries look like real buying intent. The least glamorous of the three and usually the most valuable, because it replaces a decision a person makes hundreds of times a week with no thought and some inconsistency.

What these share: the output is a suggestion attached to data you already have, a person sees it in the normal flow of work, and being wrong costs a correction rather than money.

Where AI does not pay off

• Anything that must be exactly right every time. Tax calculations, payroll, stock arithmetic, invoice totals. These are what your database is for
• Decisions with a real cost of error and no human in the loop. Auto-approving refunds, auto-cancelling orders, auto-emailing customers without review. Ship it as a recommendation first, and automate only the paths that have earned it
• Volumes too small to matter. If a person handles it in twenty minutes a week, automating it saves twenty minutes a week and adds a system to maintain
• A chatbot on your homepage, when nobody asked for one. The most-requested AI feature and the least-used one. Ask what your team currently reads or sorts by hand — that is where the value is

Bolt-on vs rebuild: why you almost never need the rebuild

The instinct that adding AI means rebuilding comes from thinking of AI as a platform. It is not. It is a request your existing backend makes to an API — the same shape as the payment gateway or SMS provider you already call.

The honest exception: if your data is scattered across spreadsheets, three disconnected tools, and someone's inbox, the work is not an AI project — it is a data project, and no model will paper over it. That is the same build-versus-buy decision covered in our guide to custom software for small business.

The other reason bolt-on wins is reversibility. An AI feature added as one endpoint behind a flag can be switched off in a minute if it misbehaves — worth more in the first six months than any amount of architectural elegance.

A worked example: semantic search on data you already have

Say you run a B2B platform with 20,000 product and document records, and sales staff cannot find things unless they know the exact internal name. This is the flow we build on an existing application, and it is deliberately unremarkable.

Step 1 — add a vector column beside the data you already store. No new database. Postgres does this with the pgvector extension; MySQL 9 has a native VECTOR type; if you are on something older, a managed vector service works and is still not a rebuild.

-- Postgres, using pgvector
ALTER TABLE products ADD COLUMN embedding vector(1024);

CREATE INDEX ON products
  USING hnsw (embedding vector_cosine_ops);

Step 2 — generate an embedding for each record, once. An embedding is a numeric representation of meaning. You run your 20,000 records through an embedding model in a background job. This is a one-off backfill measured in minutes and a few dollars, plus a small hook so new records are embedded on write.

Step 3 — at query time, embed the search text and ask the database for nearest neighbours. One extra query on a table you already own:

SELECT id, name, description
FROM products
ORDER BY embedding <=> $1   -- $1 = the query's embedding
LIMIT 10;

Step 4 — optionally, let a model write the answer. For a support or knowledge use case, pass those ten results to a model and ask it to answer using only what was retrieved, with links back to the source records. This is what people mean by RAG, and the important part is not the model — it is step 3. If retrieval returns the wrong ten rows, no model saves the answer.

That is the entire feature: one column, one background job, one extra query, one optional API call. Nothing in the architecture changed, and every existing keyword search still works — run both and merge the results, which outperforms either alone. This is the shape of work we do on OffyBox, the multi-tenant B2B platform we build and operate: capability added into a live product customers are already using, not a parallel system built alongside it.

What it actually costs

AI features are billed per token — roughly ¾ of a word — split into input (what you send) and output (what comes back). Published list prices vary by model tier. Anthropic's Claude family, as an example of current pricing:

Prices are Anthropic list rates as published in August 2026 and will change; other providers price in comparable tiers. Check current rates before you budget.

The reason those numbers surprise founders is that real workloads are small. Summarising 5,000 support tickets a month at roughly 800 input and 150 output tokens each is 4M input and 0.75M output tokens — about $8 a month on a small model. Classification is cheaper still, because the output is one word.

Two mechanics cut it further, and both are worth knowing before you are quoted a number:

• Prompt caching. If every request repeats the same instructions or reference document, the repeated prefix can be cached and re-read at roughly a tenth of the input price. On a high-volume classifier this is the difference between a rounding error and a line item
• Batch processing. Work that does not need an answer this second — overnight tagging, backfills, reports — typically runs at around half price on a batch endpoint

Build ranges are INFOCRUD planning figures reviewed in August 2026 for a single feature on an existing, working application; scope, data quality, and how much review workflow the output needs will move them. The pattern holds across projects though: the build is the cost, and the model usage is rounding. Anyone quoting a large monthly AI platform fee at these volumes is selling you a platform, not a feature. For how we size build estimates generally, our MVP cost guide uses the same method.

Data and privacy questions to settle before you start

Settle these before the first request leaves your network, not after legal asks:

• What data leaves, and does it need to? Most features work on a fraction of a record. Send the description, not the customer's phone number. Redacting identifiers is cheap at the start and awkward to retrofit
• Is your provider training on your inputs? On the major business APIs the default is no, with short documented retention. Read the terms for the specific product and keep a copy of what they said the day you signed
• Where does inference run? If you have data-residency obligations, several providers let you pin a region — establish this before you build, because it constrains which models you can use
• What does the DPDP Act require of you? For personal data of people in India, a new processor means a new disclosure and a new contract term. Paperwork, not a blocker, and easier before launch
• What do you log? Prompts and outputs are useful for debugging and are also a copy of whatever you sent. Set a retention period on that log deliberately

The practical rule: an AI provider is a sub-processor, exactly like your payment gateway or email service. You already have a process for adding one of those.

How to pilot in two weeks

A pilot should answer one question — is this better than what the team does now — and it should be cheap enough that a "no" is fine.

1. Phase 01Days 1–3 — pick one job and write down what "better" means. One workflow, one team, one measurable claim: "support staff find the right record in under 30 seconds instead of two minutes", or "90% of tickets arrive correctly tagged". If you cannot state the number, the pilot has no result, only opinions.
2. Phase 02Days 4–9 — build the thinnest version on real data. Real records, not a sample; production is where the messy data lives. Ship it behind a feature flag to a handful of internal users, with the AI output shown as a suggestion beside the existing flow rather than replacing it.
3. Phase 03Days 10–14 — measure, then decide honestly. Compare against the number from day 3, count the corrections users made, and read the ten worst outputs — those tell you more than the averages. Then either roll it out, tune the prompt and retrieval and re-run, or switch off one endpoint and stop.

Note what is not in that plan: no model training, no fine-tuning, no data science hire. Fine-tuning is a real technique and almost never where you start — well-retrieved context and a clear prompt beats a fine-tuned model on a vague one, at a fraction of the effort.

Mistakes that turn a feature into a rewrite

• Starting with the model instead of the job. "We should use AI" produces demos; "our team retypes this into a spreadsheet every morning" produces features
• Skipping retrieval quality. When answers are wrong, teams reach for a bigger model. Nine times in ten the problem is that the wrong records were retrieved, and no model fixes that
• Shipping it as automation on day one. Suggest first, automate later, and only on the paths where you have watched it be right for weeks
• Building a new system beside the old one. Two sources of truth is a worse problem than the one you started with
• No evaluation set. Twenty real examples with known-good answers, checked whenever you change the prompt, is the whole discipline. Without it, every prompt change is a guess and regressions ship silently

Frequently asked questions

Do we need our own model, or is an API enough?

An API is enough. Training or hosting your own model is a serious undertaking that makes sense when you have a genuinely unusual data domain, strict isolation requirements, or volume high enough that per-token pricing stops being rounding. None of those apply to a first feature.

How accurate is it, honestly?

For classification into a small set of well-defined categories on clear inputs, high enough that the corrections are rare. For summarisation, good enough to save a read, not good enough to be the only record. The right question is not "how accurate" but "what happens when it is wrong" — design the feature so that answer is boring, and accuracy stops being the deciding factor.

Will this work on our old application?

Usually yes. The integration is an HTTP call and a database column, both of which almost any stack from the last fifteen years supports. What actually blocks projects is not framework age — it is data that is unstructured, scattered, or duplicated. That is worth assessing before anything else.

Which model should we use?

Start on a small, cheap one and only move up if the quality is not there. Teams routinely default to the largest available model and pay five times as much for a classification task that a small model does correctly. Test both on your twenty-example evaluation set — the answer takes an afternoon and it decides your running cost for the next year.

What does it cost to keep running?

At small-business volumes, less than most SaaS subscriptions — usually $5–50 a month in model usage, plus a few hours of attention. The cost that matters is the build, and after that the ongoing shape is the same as any other feature you own.

The bottom line

Adding AI to an existing application is a feature-sized project, not a platform decision. Pick one job your team does by hand — finding things, reading things, sorting things — attach a suggestion to it inside the flow they already use, measure it against what they do today, and keep it switchable. Two to three weeks, single-digit dollars a month to run, and one endpoint you can turn off. Do that once and you will know far more about where AI helps your business than any strategy exercise will tell you.

If you are weighing an AI feature and want to know whether it is a two-week bolt-on or a data problem in disguise, that is worth an outside opinion before you budget for it. You can explore AI and modernization to see how we scope this work, or book a free 30-minute technical review and we will tell you which of your workflows is actually worth automating — and which ones are not.