Introduction
AI in education has become shorthand for one thing: a chat window bolted onto an existing course or app. Ask it a question, get an explanation, move on. That is the version almost every EdTech product shipped in the last two years, and it is also why most of them saw no real change in completion rates or learning outcomes.
Pedals Up is an AI native product engineering company that builds web, mobile, SaaS, and AI products for startups and scale ups across the US, UK, and UAE. We have sat in enough of these product reviews to notice the pattern: the chatbot works fine in a demo. It does nothing for retention six months later.
The reason isn’t the model. It’s what the model is connected to, or more often, what it isn’t.
Why the chatbot layer never fixes the outcome problem
A tutoring chatbot answers the question a student typed. It has no memory of what that student got wrong last week, no signal on which concept they are actually stuck on, and no way to adjust the next lesson based on that pattern.
That’s not a limitation of the language model. It’s a limitation of the product around it. Personalization requires a feedback loop: capture what a learner does, score it against a mastery model, and feed that back into what they see next. A chat interface sitting on top of static course content skips all three steps.
Duolingo’s approach is instructive here, not because of any specific number, but because of where the AI actually sits. Their adaptive engine adjusts difficulty and review timing based on a learner’s answer history, and the generative AI features sit on top of that engine rather than replacing it. The chat feature is the visible layer. The scoring and spacing logic underneath is the actual product.
What is AI native learning infrastructure
AI native learning infrastructure means the assessment, feedback, and content sequencing decisions in a learning product are driven by a live data pipeline, not a static curriculum with a chatbot attached. The model reads learner performance data continuously and the product changes what it shows next based on that data, rather than the learner having to ask for help.
Building this requires three things most EdTech roadmaps underestimate: a structured way to capture performance signals at the question or task level, a model or rules engine that turns those signals into a mastery estimate, and a content system flexible enough to serve different sequences to different learners. Skipping any one of the three leaves you with a chatbot wrapper.
This is also where a lot of the real engineering effort in our AI integration work actually goes. Clients come in asking for “an AI tutor” and the conversation quickly moves to what data the platform is currently capturing, because that answer determines whether an AI feature can be personalized at all or just generic.
How does AI actually improve learning outcomes?
AI improves learning outcomes when it changes what a learner sees next based on what they just did wrong, not when it simply answers a question they ask. The improvement comes from a tight loop between assessment and content delivery, where the system narrows in on a learner’s specific gap rather than repeating the same explanation to everyone. Without that loop, an AI feature functions as a search box with better phrasing, and search boxes do not move completion rates.
The compliance problem nobody budgets for
Student data is some of the most protected data category in software. In the US, FERPA governs education records and COPPA adds another layer for anyone under 13. Most AI vendors were not built with either in mind, and plugging a general purpose model into a K-12 or higher ed product without checking how that vendor stores and trains on data is how a compliance review turns into a six month delay.
The fix is not avoiding AI. It’s architecting the data flow so that personally identifiable student information never reaches a model that retains or trains on it, and logging exactly what data left your system and when. That is infrastructure work, and it needs to happen before the tutoring feature ships, not after a district asks for a data processing agreement you cannot answer.
The U.S. Department of Education’s Office of Educational Technology addressed this directly in its 2023 report on AI in teaching and learning, warning that AI tools deployed without clear data governance create risk that outweighs the instructional benefit. That warning still holds. The full report is available through the Department’s technology office.
What is the biggest risk of using AI in education products?
The biggest risk is deploying an AI feature that handles student data without a clear answer for where that data goes, who can access it, and whether the vendor trains future models on it. This is not a hypothetical concern. Institutional buyers ask this question directly during procurement, and a vague answer stalls or kills the deal regardless of how good the AI feature itself is. Getting the data governance answer right before launch is faster than retrofitting it after a district or university flags it.
A pattern worth paying attention to
Chegg’s own investor disclosures in 2023 named ChatGPT directly as a factor in slowing new customer growth, and the company’s stock dropped sharply the day that was announced publicly. The lesson there isn’t that AI killed a homework help business. It’s that a static, search based product got outcompeted by something that could actually explain a concept in the moment a student needed it.
Khan Academy took the opposite path and built Khanmigo as a tutor that stays inside guardrails set by educators, refusing to give direct answers and instead pushing students toward the reasoning. That design choice, not the underlying model, is what separates a tool teachers trust from one they ban.
Both outcomes came from the same technology. The difference was in the product decisions around it.
Not every completion problem needs an AI layer
Everything above assumes AI is the right lever for the outcome problem. It isn’t always, and it’s worth saying that plainly in a post about AI in education.
Pedals Up designed and developed Avyapp, a game based learning platform built for enterprise and course providers, and it doesn’t lean on AI at all. The engagement model runs on classic gamification: progress mechanics, a rewards system, and a marketplace where learners redeem points for real benefits. Completion, in this model, comes from incentive design, not personalization.
Avyapp’s own materials project completion rates moving from around 5 percent, typical for unmotivated self paced digital courses, to over 85 percent under this model. That’s their projection, not an independent study, but the mechanism behind it is straightforward and well understood in behavioral design: people finish things they’re rewarded for finishing, regardless of how personalized the content is.
That’s worth sitting with before greenlighting an AI roadmap. If the real problem is that learners start a course and never come back, a rewards and gamification layer can move that number without touching a model at all. AI earns its place when the gap is about relevance and pacing, not motivation.
The two problems get confused constantly, and it’s an expensive mistake. Teams build a sophisticated AI tutor when what their drop off data actually shows is that nobody had a reason to open the app on day three.
What founders should actually build first
Before writing a single prompt for a tutoring feature, map what performance data your platform already captures at the task level. If the answer is “not much,” that gap is the actual project, and the AI layer is secondary.
Then decide, explicitly, what data can and cannot leave your system, and get that answer in writing before a procurement conversation forces it. Retrofitting compliance under deadline pressure is where AI education products lose months.
Only after those two are settled does the model selection and prompt design conversation matter, and by that point it is usually the easiest part of the build.
The takeaway
AI in education is not a feature you add. It’s an infrastructure decision about what data your product captures and how that data changes what a learner sees next. Teams that treat it as a chatbot launch get a demo that impresses and a retention curve that doesn’t move. Teams that treat it as a data and workflow problem end up with something an institution can actually trust and a student actually returns to.
If your team is scoping an AI feature for a learning product and the conversation keeps circling back to what data you can actually use, that’s usually the sign you’re asking the right question. We work through exactly that kind of scoping with EdTech teams building their AI roadmap, from the Quick-Start assessment through the actual build.
Frequently Asked Questions
Does adding a chatbot to my EdTech platform count as AI personalization?
No. A chatbot that answers questions on demand is a support feature, not personalization. Real personalization requires a feedback loop that scores learner performance and changes what content is shown next, which a standalone chat interface does not do on its own.
Is FERPA a concern if my AI tutor is for adult learners, not K-12?
FERPA applies to education records held by institutions that receive federal funding, which includes many colleges and universities, not just K-12 schools. If your platform sells into higher ed, the same data governance questions apply regardless of learner age.
Do I need to build my own AI model to personalize learning content?
No. Most education products use an existing foundation model for language tasks like explanation or feedback generation, paired with a custom scoring or mastery model built on the platform’s own performance data. The custom part is usually smaller than founders expect, but it’s the part that actually drives personalization.
What should an EdTech founder budget for first when adding AI features?
Budget for the data capture and governance work before the AI feature itself. A tutoring chatbot is often a few weeks of integration work. Getting your platform to capture usable performance signals and documenting where student data goes typically takes longer and determines whether the AI feature can personalize anything at all.
