5 things to check before you scale an AI-built app

By Anna-Maria Radkevych, Project Manager at NERDZ LAB · 14 Oct 2026 · 7 min read

In this article:

• 1. Code quality and tests
• 2. Time and business rules
• 3. Security
• 4. Third-party services
• 5. Process and communication
• Our approach: fix or rewrite
• Wrapping up

With AI, you can build a first working app in just a few weeks. It looks great on the demo, and that’s a real win. But more and more products come to us that work on the demo and still aren’t ready for real users.

We recently took over exactly that kind of project: an online therapy platform where clients book sessions with therapists, pay for them, and talk over video, audio, and chat. The apps were already live in the stores, and people were using them. But under the hood, a lot of things were holding the product back.

We chose to stabilize the product instead of rebuilding it from scratch. Here are five things we check on every project like this, and that you should check too before you scale an app built with AI.

1. Code quality and tests

The first thing we noticed was how uneven the code was. The style changed from file to file, there were lots of copy-pasted blocks and old code left commented out, and some features were only half done. That’s a typical sign of AI: it writes each piece separately, for one specific request, and doesn’t see the product as a whole. So the same problem gets solved in different ways in different places, and things that look finished actually aren’t.

There were no automated tests at all. That means every change has to be checked by hand, and fixing one thing can easily break something next to it. AI rarely writes tests unless you ask it to directly, and when a product is built fast, nobody usually asks.

2. Time and business rules

AI writes code that looks right, but it doesn’t know how your business works. If nobody tells it that a therapist and a client might live in different time zones, it won’t think about it. And when it needs a specific number, it just puts that number straight into the code. This is where we found most of the hidden problems.

The app didn’t take the user’s local time into account, and that issue ran through the whole product, from the therapist’s schedule to bookings. Within the same flow, time was shown in 24-hour format in one place and 12-hour format in another.

The pricing logic for different session lengths was confusing and partly baked right into the code. Session slots were fixed too: 30 minutes of session plus a 30-minute break, no matter what the therapist had set up.

3. Security

We made security our first phase of work, before any new features. For a product that stores mental health data and takes payments, that’s not up for debate.

AI aims for “make it work,” not “make it safe.” The fastest way to connect a service is to paste its access key right into the code, and AI often does exactly that. So we found the usual set: access keys stored in the code, some data reachable from outside without proper protection, and some important actions with no limits at all. One of these gaps was already costing the product owner real money every day.

Here’s what that looked like. To log in, a user gets a confirmation code by SMS, and every SMS costs money. Bots started requesting codes in bulk, and the SMS bill kept growing every day, even though no real new users were signing up.

The fix turned out to be simple: we limited how often codes can be requested, both on the server and in the app itself. But until someone took a fresh look, the product was paying for someone else’s attack every single day.

4. Third-party services

The product was made up of four mobile apps, an admin panel, a server, and a separate server just for chat. On top of that, it used about a dozen outside services for calls, chat, notifications, SMS, payments, and email. For calls and chat, it was running two different providers side by side. That happens when features are added one AI request at a time: each time, the AI suggests whatever service suits it, without checking if something similar is already in place.

The spots where these services connect were exactly where the most important things broke for users. Notifications didn’t reach therapists, calls didn’t connect, and chat only opened at the time of the session, even though therapists often need to message a client before that.

Some problems had nothing to do with code at all. For example, therapists in one country couldn’t make calls because local mobile carriers block internet calls. You won’t see that in the code or in a demo.

When chat and notifications are unreliable, the first idea that comes to mind is to switch providers. The client was considering that too. We suggested finding the real cause first. It turned out the problem wasn’t the service itself, but how the product was using it. So instead of an expensive migration, we brought everything onto one working setup and fixed the logic on our side.

And some “bugs” were really just missing scenarios. Say a client books and pays for a session today, but the session is two weeks away. Chat only opened on the day of the session, so the therapist couldn’t even let the client know about a reschedule. We changed it so chat opens right after payment.

5. Process and communication

When a team takes over someone else’s code, the big temptation is to rewrite everything right away. We started differently: by figuring out how the system actually works. On the server side, we began with the database structure and a map of who talks to the server and why. The mobile team picked up a specific task from the client first and noted other problem areas along the way, discussing them with the rest of the team.

A lot at the start depends on access and context: how certain parts are supposed to work, where the servers are set up, which keys are still valid. Some of that simply isn’t in the code, so it’s worth planning time to collect access and answers right from the start.

The PM’s job here is to keep priorities straight and explain to the client what we’re fixing now and what can wait. We split the work into phases by importance: security first, then reliable calls, chat, and notifications, then smaller fixes, and only after that new features. Where we could, we also simplified the scope together with the client. For example, instead of building a separate screen for top-rated therapists, we just sorted the existing list by rating.

As a PM, I’ll be honest: the hardest part of projects like this isn’t the tech, it’s expectations. The client sees an app that’s already in the stores and being used. So it’s totally fair to ask why spend weeks on “stabilizing” when you could build new features right away. I’ve learned to answer that not with a list of bugs, but with what it means for the business: how much money goes to attacks, how many sessions fall through because of calls, how many clients never got a message from their therapist. Once the conversation is about impact, agreeing on priorities gets a lot easier.

Our approach: fix or rewrite

Our rule is simple: small things we fix on the spot, but if the problem is in the approach itself, we rewrite it using solutions we’ve already tested on other projects. The team makes each of these calls together, after talking it through.

Two examples:

• The profile in the client app. This part was so broken that fixing it would have taken longer than building a new version. We rewrote it from scratch using SwiftUI, Apple’s modern tool for building app screens.
• Notifications. The data needed for push notifications was stored together with other user info. We moved it into its own place, and notifications became more reliable and easier to maintain.

Rewriting the profile had another benefit. As soon as we started rebuilding the screen, we could see that some problems weren’t in the app at all, but in the admin panel: photos and passwords weren’t saving, some sections were empty, and session length ignored the therapist’s settings. The app was just showing what it received. That’s why we always check the whole chain, from the admin panel and server all the way to the user’s screen.

Wrapping up

AI really does speed up the start: the first version of a product shows up in weeks, not months. But none of the problems above show up in a demo. They appear once real users, real money, and real markets come into play. That’s when you need a team that understands both the code and the business.

If you have a product built with AI, run through this quick checklist before you scale:

Before you scale: checklist

☐ The code is consistent, with no copy-paste or half-done features, and key flows are covered by tests

☐ Time, prices, and booking rules work correctly for every user and aren’t baked into the code

☐ Access keys are protected, payments are verified, and important actions have limits

☐ Outside services don’t overlap, and notifications and calls work reliably in all your markets

☐ Your team has the access, documentation, and priorities agreed with you