AI Writing Assistant

Introduction

Babbel is a paid, subscription-based language learning app for web and mobile. To support users across their entire learning journey, the content design team worked across 4 different agile clusters: Self-study, Live, Engagement, Growth.

As Gen AI tools became part of our daily work, I led a project to answer a practical question: Could we use AI to save time and write consistently in Babbel's voice?

The problem

The content design team faced 2 recurring challenges:

• Consistency — keeping one voice across a large, fast-moving product.
• Efficiency — making the best use of a small team's time and skills.

Generic prompting didn't sound like Babbel. Output was often too long, missing the context of the actual UI component.

To mitigate that, we started curating a library of prompts. This however required a lot of time, and the prompts themselves had to be saved and handed off so teammates could reuse them.

From prompts to a purpose-built assistant

While the prompt library worked for a while, I decided to build a dedicated AI assistant specifically for content design.

An assistant could hold its purpose in a way that one-off prompts couldn't:

• ✓Scoped specifically to content design work.
• ✓Grounded in Babbel's UX content guidelines.
• ✓Fewer hallucinations, shorter prompts, more context.
• ✓Reusable by the whole team.

At the time, we had access to Langdock, a model-agnostic AI playground that allowed creating "assistants" similar to custom GPTs.

I started with writing instructions, encoding best practices like keeping copy short, and returning multiple options to choose from. This shaped the output, but to make it sound like Babbel, I uploaded our UX content guidelines as attached knowledge.

The assistant should now know what any content designer on the team knows, and need far less fine-tuning.

Building a way to test the assistant

I ran a short workshop with the team to define a fixed set of test prompts: questions we'd re-ask every time we changed the assistant. They fell into 2 groups:

• General questions — e.g. "What should I keep in mind when writing a title?"
• Daily-task questions — e.g. "Rewrite this within 50 characters, following Babbel's voice"

The team helped generate prompts and evaluate output; I owned the framework, recording the output, and analysing it.

Choosing a model

We ended up with 10 prompts, spanning from testing content for specific UI components to text expansion and translation length. I ran them across 4 models, and rated each on quality, accuracy, and instructions compliance.

In this example, the goal was to provide additional context through an image, and ask the assistant to rewrite its message according to Babbel's content guidelines.

After analysing all 40 outputs, a few patterns emerged:

• ChatGPT 4o mini — very fast, decent copy, but frequently ignored instructions.
• ChatGPT 4o — solid overall, but also missed instructions.
• Claude 3 Sonnet — reliable copy that respected the brief; occasional long answers.
• Claude 3.5 Sonnet — followed instructions, but copy and answers were often weaker and long.

My pick was Claude 3 Sonnet ✔️

This model provided the best balance of instruction-following and on-brand copy. After presenting the findings to the team, we decided to start using it, and to document findings and feedback. I also planned to keep testing new models as they released.

Impact

The assistant was adopted by the content design team as a daily writing partner for first drafts and idea generation, based on Babbel's UX content guidelines.

On average, creating 5 versions of a message and deciding which ones to discard would take around 10 minutes. With the assistant, it takes 2 minutes.

“With the team project work, I was impressed with the framework you created to test the AI assistant; your work was central to the project, and helped move the whole thing forward.”

Next steps

In this project, I mostly focused on choosing a model. Additional factors that could be tested are:

• File format of the attached knowledge
• System prompt
• Temperature setting of the model

Furthermore, since LLMs were bad at counting individual characters, I could switch to counting words for better accuracy.