In 2024 the customer retention team brought to the product team's attention that we were 27% over the projected churn rate, and that there had been a pattern of feedback from leaving customers that specifically the awarding stage of the award management workflow was a significant part of why they were leaving. It was categorized into two sections.
- The functionality and design of the app was poor. The app did not update in a timely manner and information required to make the awarding decisions was fragmented.
- The amount of time the admins had to spend each year leading up to the awarding process training their co-workers on how to use our app was untenable in the long run.
- Difficulty-ranked scholarship queue: surfaces the hardest-to-place scholarships first, so admins tackle what's most at risk of going unawarded.
- Funding-and-need-ranked applicant queue: surfaces students with the most funding on the table and fewest applications in, so the highest-stakes students get seen first.
- Cross-filtering selection: clicking a scholarship or a student filters the other side automatically, replacing manual cross-referencing with one click.
- Single recommended applicant: Accept/Edit actions only appear on the top-ranked candidate, removing decision paralysis from a long qualified list.
- Qualifications breakdown table: shows GPA, Major Fit, Financial Need, and Essay scores so the recommendation is auditable, not a black box.
- Award action moves scholarships to an Awarded list: keeps awarded and pending scholarships from ever being mixed up or re-awarded.
- Live budget and cycle dashboard: shows budget, amount awarded, and time remaining at a glance, no separate report needed.
- Multi-cycle switcher: lets admins reference past cycles without disturbing the current cycle's in-progress data.
Given a mandate to figure out what was happening and why, I set up meetings with the premium support team, customers, and the dev team. Here is what I learned:
- The app updated slowly because of a data architecture issue that was hard and expensive to fix.
- It was hard to train the whole team because they only used the app for about two weeks a year to complete their portion of the work, and it was so complicated and unintuitive that they would forget how to use it each new year.
Our awarding workflow was so fragmented it required opening 3+ tabs simultaneously, making it look outdated during sales pitches. This friction drove customer churn and lost sales to competitors. A traditional UI overhaul to fix this would require a minimum of 6 months of dev time.
Admins don't want a prettier UI; they want the heavy lifting of data gathering automated while remaining the final decision-makers.
Instead of a full redesign, code a new AI-packaged feature that automates the universal rubric schools use to award scholarships.
Overcome admin fears of AI hallucinations by applying media literacy best practices (clear sourcing and verifiable logic) to build trust.
Working with support, we diagrammed all the logic used to make an awarding decision so that there was no AI black-box scenario where the AI chooses a recipient but no one is exactly sure how that choice was made.
To ensure the AI does not hallucinate, we built parallel code whose sole job was to run the application data (directly from the database, unmediated by AI) through the scholarship award equation, then compare that result to the AI's result. If they match, great. If not, the AI reruns its process until they do.
If you're asking yourself why we're even using AI if this can be done programmatically with code alone: I would point you to significant evidence that announcing a new feature built using AI can positively affect stock prices of public companies.
Shows the reasoning the AI used to arrive at its recommendation
AI draws data directly from the database, no interpretation of the data
Presents the most likely alternative so the admin can compare
Asks permission before taking action
Opening 3+ tabs to manually aggregate, cross-reference, and analyze student data.
Reviewing automated recommendations on a single screen with instant, auditable data citations.
Condensed the process so a single person can handle the entire workflow.
- Enabled admins to instantly verify data and justify their awarding choices, making the product defensible in sales conversations.
- Saved 6+ months of dev time by avoiding a full UI overhaul.
- Neutralized a key competitor advantage and eliminated a primary churn driver.
- 6 months after deployment, churn reduced by 15%.
- Sales anecdotally communicated that the auto awarding feature drove 22% of follow-up questions (specifically around data governance and logistics), indicating serious purchase intent.