AI-driven experience decreases user workload by 70%
WEB RESPONSIVE AND APP
Company: Estante Mágica (B2B2C Edtech)
Role: UX researcher | UX/UI designer
Team: along with 6 devs
Overview
Estante Mágica brings classroom stories to life. Teachers, our users, implement Estante Mágica project in their classrooms, allowing each child to create their own stories. They upload the children's contents in our platform. Estante Mágica transforms them into various products. The main one is the printed book, acquired by parents and delivered in school for a Book Signing Event.
Problem
The workload of editing, and reviewing each child's content was the main reason for churn.
Goals
Simplify the editing and reviewing of students' stories for teachers.
Results
AI integration resulted in:
1) 70% reduction in workload.
2) Antecipated 45% decrease in churn.
The Book Signing Event is a proud moment.
But to get there, teachers had a lot of previous work.
Process
Discovery
“This is so much work!! I lost my entire weekend typing each child's text!”
Eliana, teacher
We contact each user who experiences churn to uncover the underlying reasons. The top 3 factors contributing to churn are:
71.2%
Time and effort constraints
Users’ Taskflow
Marcia, school manager
The scholars’ peak of joy and pride arrives during the book delivery at a book signing event. However, the journey leading up to it is filled with hard work and stress, often leaving users feeling frustrated.
For the majority of teachers who completed our project, as well as those who left, ensuring a perfect book was a laborious process.
User Research
Churn Report
“I had to ask my family to help me review the books. Yet some errors slip through.”
The book is a representation of teachers' work in the classroom.
The book is a source
of pride
for teachers.
Departures due to resignations, health, etc
The Book Signing event has a
comercial purpose for schools
Consequently:
1- Teachers express concerns about how schools and parents might react to any errors in the book.
2- Teachers often find themselves working overtime to complete typing and reviewing tasks.
3- Teachers often seek external assistance to ensure the book is error-free
Secondary Research:
Engagement with other schools projects
Interview and Survey
As part of our ongoing commitment to discovery, we maintain regular contact with current users, with different profile:
from private and public schools
new and old users
from high touch and low touch experience
Key Takeaways
US$
25,150
annual salary
15.1%
UP TO
525
students/teacher
5.1%
Teachers in Brazil are undervalued and overworked.
UP TO
60
weekly work hours
How to ensure an error-free book with minimal teacher effort?
Design
AI Democratization
In late 2022, advanced AI models were widely adopted, enhancing content editing and reviewing. We crafted an AI-powered experience to reduce teachers' workloads on our platform.
Instead of typing each child story
Instead asking for help to review
Objectives:
User behavior concerning the new features: speech-to-text and AI-review
Comprehensibility of the workflow
Gauge the perceived value generated by these additions
AI Testing and Analysis
We conducted various tests and created different prompts to assess how OpenAI would behave in different usage contexts
Speech-to-text
Utilized in noisy environments.
Children dictating stories.
Proficiency in diverse languages.
Key takeaways
Record or type.
Listen to your own recording.
Accept or discard text/recordings.
Key takeaways:
They could use Open-AI supported speech-to-text
Open-AI would review the children story
AI-powered review
Provided appropriate suggestion.
Ensured syntax and grammar align with the chosen language.
Cost
Simulated usage for monitoring recording and/or review request quantities.
Calculator to estimate book creation cost, prompts to train AI, and speech-to-text testing environment
Achieving 100% accuracy proved unattainable.
Speech-to-text accuracy approached nearly 100%,
Reviewing process presented specific challenges.
Nonetheless, it reached a level of satisfaction that enhanced our overall confidence in the experience.
Design Process
Speech-to-text for Usability test
Design for Usability Test
Grammarly
Offers correction suggestions
Allows accept or discard suggestions.
Provides an option for giving feedback.
The speech-to-text feature was clearly a tool to ease teachers’ workload.
But the AI-supported review brings some considerations, such as:
The typing process already includes an autocorrector.
Teachers should distinguish between autocorrector in typing and AI review.
I aimed to design an experience that would be both user-friendly and familiar to teachers. As a result, I drew inspiration from WhatsApp for the speech-to-text feature and from Grammarly for the review process.
Inspiration
autocorrector in typing experience
AI-review suggestion cards
It's essential that teachers take ownership of the editing process.
Teachers must have the autonomy to determine what needs correction.
Popup pre-review, explaining the one-time use
Usability test
The usage behavior (where and when to click) was confirmed during validation.
Speech-to-text was universally understood and well-received.
Clear understanding of the review environment
At first, they were concerned about how much AI would change the child style
But they had a positive response to it being a suggestion rather than a replacement.
Initial oversight of the trash icon.
Because of our expected cost, we first limit the AI-review use by 1 time.
Delivery
Final version improvements and insights
AI review
Cost monitoring
Initially, limiting usage to one attempt in the test wasn't a problem. However, we opted to grant teachers greater flexibility by offering unlimited AI reviews.
We accomplished this through a request tracking dashboard for both speech-to-text and AI review.
Speech-to-text
AI Accuracy and User Transparency
Our AI may not be 100% accurate, so we've added a learning notice and a feedback feature. Additionally, we removed 'accept all suggestions' to encourage users to review each suggestion for accuracy.
Encouraging New Feature Adoption
Users often approach new AI experiences with fear. We introduced pop-up notifications highlighting our latest speech-to-text and AI review features
Emphasizing the AI work in review process
We've introduced a pre-review loading screen to underscore the active AI assistance throughout the review process. It also serves as a visual indicator, ensuring users are aware of AI assistance before proceeding with their review.
Results
Through testing and feedback, we can confidently state that there has been a substantial 70% reduction in workload.
Consequently, we are anticipating a 45% decrease in the churn rate.
Numbers so far*
Despite the high level of usage, our request volumes and associated costs remain below our initial projections.
This underscores the seamless and dependable nature of the experience.
Speech-to-text
115,737
Stories
dictated
750,619
Requests
so far
AI review
131,205
AI-reviewed books
More CONFIDENT
teachers
The AI assistance was the best
thing this year!
I used to ask for help to review the text.
Now, I did all by myself!
It made my life easier!
3,751,687
Suggestions
*from August to October, 2023
Qualitative feedback
More TIME
for teachers
“It used to take me 5 days to type the texts.
This time, I locked
myself in my office and dictated
all the students' texts in 1 hour!
It was so fast!”
We were pleasantly astonished by the positive outcomes resulting from our AI utilization:
More INCLUSION
within children
”I have students who can't write,
so they dictated their own stories. They felt actively engaged
in the project.
It was so rewarding to
empower them!”