Increasing engagement and completion rates by 320% for Airtribe’s Daily Challenges Feature - UI/UX Case Study

Increasing engagement and completion rates by 320% for Airtribe’s Daily Challenges Feature - UI/UX Case Study

Project overview

Project overview

Timeline
Timeline

Feb 2024 - Mar 2024

Feb 2024 - Mar 2024

App type
App type

Web app

Web app

Project type
Project type

Internship

Internship

Tool(s)
Tool(s)

Figma

Figma

Key skills
Key skills

Product Design and User Research

Product Design and User Research

Stakeholders
Stakeholders

Lead Product Designer, Product Engineers and Co-founder

Lead Product Designer, Product Engineers and Co-founder

Team member(s)
Team member(s)

Grace Lal

Grace Lal

My role
My role

User researcher, Product Designer

User researcher, Product Designer

Project summary

Project summary

This project aimed to improve the user experience of the Daily Challenges feature by optimizing the user flow. The feature had a low engagement and participation rate and a high abandonment rate during the challenge. Through user research, I identified the learner's pain points, such as the inability to view and attempt previous questions.

I collaborated with the community manager to interview learners, created a revised user flow, conducted competition analysis, and redesigned the feature. The final redesign simplified the overall user experience and after the launch, it led to a 320% increase in engagement and participant rates.

This project aimed to improve the user experience of the Daily Challenges feature by optimizing the user flow. The feature had a low engagement and participation rate and a high abandonment rate during the challenge. Through user research, I identified the learner's pain points, such as the inability to view and attempt previous questions.

I collaborated with the community manager to interview learners, created a revised user flow, conducted competition analysis, and redesigned the feature. The final redesign simplified the overall user experience and after the launch, it led to a 320% increase in engagement and participant rates.

About the company

About the company

Airtribe is an ed-tech startup offering courses in Product Building space - including engineering and product management. Airtribe has a new-age upskilling platform focused on the intersection of expert-led education and community-based learning. The courses encourage active learning instead of passive lecture-watching. They are live programs with workshop-style sessions, direct access to expert instructors, access to the community with peers, and regular feedback.

The instructors and mentors hail from top tech unicorns and decacorns like Gojek, Razorpay, Uber, Coinbase, Microsoft, Meta, Google, and more. Airtribe’s target audience consists of professionals with a couple of years of experience looking to upskill in there or make a career transition. Airtribe has helped 1000+ learners in their upskilling journey through their courses.

Airtribe is an ed-tech startup offering courses in Product Building space - including engineering and product management. Airtribe has a new-age upskilling platform focused on the intersection of expert-led education and community-based learning. The courses encourage active learning instead of passive lecture-watching. They are live programs with workshop-style sessions, direct access to expert instructors, access to the community with peers, and regular feedback.

The instructors and mentors hail from top tech unicorns and decacorns like Gojek, Razorpay, Uber, Coinbase, Microsoft, Meta, Google, and more. Airtribe’s target audience consists of professionals with a couple of years of experience looking to upskill in there or make a career transition. Airtribe has helped 1000+ learners in their upskilling journey through their courses.

Business opportunity

Business opportunity

The daily challenges are open to our paid and free community. By improving the user experience of the feature, the free community will engage more efficiently. This might encourage them to take up our paid courses and turn them into one of our learners which we contribute to our revenue and sales goals.

The daily challenges are open to our paid and free community. By improving the user experience of the feature, the free community will engage more efficiently. This might encourage them to take up our paid courses and turn them into one of our learners which we contribute to our revenue and sales goals.

Problem

Problem

Airtribe frequently organizes daily challenges (number of days varies like 7 to 21 days), to engage the enrolled learners and free community in continuous learning by practicing and testing their skills. We noticed that although learners initially participated enthusiastically, their participation rate decreases as the challenge progress. We need to redesign the Airtribe’s daily challenges user experience to retain the learners in the challenge and increase the participation rate even more.

Airtribe frequently organizes daily challenges (number of days varies like 7 to 21 days), to engage the enrolled learners and free community in continuous learning by practicing and testing their skills. We noticed that although learners initially participated enthusiastically, their participation rate decreases as the challenge progress. We need to redesign the Airtribe’s daily challenges user experience to retain the learners in the challenge and increase the participation rate even more.

Goals

Goals

Increase the adoption, engagement, and retention rates of the feature by redesigning the challenges to make them more engaging and reduce drop-off.

Increase the adoption, engagement, and retention rates of the feature by redesigning the challenges to make them more engaging and reduce drop-off.

Impact

Impact

I designed an experience to ensure that users found the challenges valuable. Firstly, Introduced a gamified experience to encourage learners to keep participating constantly and keep up the streak. Second, Introduced a new way of redemption if a learner misses the streak. The redesigned solution, increased learner participation by 47% and retention rate from 12.5% to 47.17%.

I designed an experience to ensure that users found the challenges valuable. Firstly, Introduced a gamified experience to encourage learners to keep participating constantly and keep up the streak. Second, Introduced a new way of redemption if a learner misses the streak. The redesigned solution, increased learner participation by 47% and retention rate from 12.5% to 47.17%.

Research

Research

Problem discovery

I looked at our data and found that a lot of the learners started the challenge, however, less number of learners were able to complete the challenge or even complete more than 75% of the challenge. To understand better, our community manager interviewed the participants to understand why the learners would drop off in the middle of the challenge. Here, learners are working professionals (The target group).

Following were the problems identified:

  • Learners don’t have time and motivation to maintain the streak of participation.

  • Learners are not motivated to spend their holidays (weekends or national holidays) to attempt the challenge.

  • Since the learners are working professionals, they want to enjoy their time off work with family.

  • Learners plan to practice the challenge later but can’t find the questions.


The current daily challenge process

In the current flow,

  • We announce the challenge and schedule a kick-off call.

  • In the call, the instructions are informed, and the challenge commences from the next day.

  • Every day at 9 am IST a new challenge is published, and the link is shared in the discord community.

  • The learners go to the challenge page through this link and after logging in, they upload their solutions.

  • After 15 days into the challenge, we provide the learners a redemption attempt and ask them to solve their missed last 15 questions and upload them on the discord thread.


Issues in the current daily challenge process and user flow

The current process is manual. We have to manually take up the submission during the redemption period and evaluate whether to mark it as complete or not.

  • As mentioned by the learners, they can’t visit the past questions. This discourages the learner from participating further and leaving the challenge.

  • The only source of motivation is the leaderboard which might become stagnant. The learners who are consistent will always be on the top and the learners who might have missed few questions may feel demotivated as it would be difficult to come to the top.

  • The redemption period is manual and requires a lot of effort by the community team to keep the track and record the challenge progress.

Competitive analysis

At this point, I started to explore some of the direct/indirect competitors on how they conduct their challenges. I also looked at some other challenge platforms like Duolingo, Hackerrank etc to understand how their experience helps to retain their users.

Following are the insights:

  • Some platforms use a streak-based approach to motivate learners. If the learner misses one day, the streak decreases.

  • Some platforms show the unsolved questions so that the learner can easily go and practice them later.

  • Some platforms use the concept of “Lives”, where after you get a question wrong your lives are decreased indicating that you have less number of attempts left.

Problem discovery

I looked at our data and found that a lot of the learners started the challenge, however, less number of learners were able to complete the challenge or even complete more than 75% of the challenge. To understand better, our community manager interviewed the participants to understand why the learners would drop off in the middle of the challenge. Here, learners are working professionals (The target group).

Following were the problems identified:

  • Learners don’t have time and motivation to maintain the streak of participation.

  • Learners are not motivated to spend their holidays (weekends or national holidays) to attempt the challenge.

  • Since the learners are working professionals, they want to enjoy their time off work with family.

  • Learners plan to practice the challenge later but can’t find the questions.


The current daily challenge process

In the current flow,

  • We announce the challenge and schedule a kick-off call.

  • In the call, the instructions are informed, and the challenge commences from the next day.

  • Every day at 9 am IST a new challenge is published, and the link is shared in the discord community.

  • The learners go to the challenge page through this link and after logging in, they upload their solutions.

  • After 15 days into the challenge, we provide the learners a redemption attempt and ask them to solve their missed last 15 questions and upload them on the discord thread.


Issues in the current daily challenge process and user flow

The current process is manual. We have to manually take up the submission during the redemption period and evaluate whether to mark it as complete or not.

  • As mentioned by the learners, they can’t visit the past questions. This discourages the learner from participating further and leaving the challenge.

  • The only source of motivation is the leaderboard which might become stagnant. The learners who are consistent will always be on the top and the learners who might have missed few questions may feel demotivated as it would be difficult to come to the top.

  • The redemption period is manual and requires a lot of effort by the community team to keep the track and record the challenge progress.

Competitive analysis

At this point, I started to explore some of the direct/indirect competitors on how they conduct their challenges. I also looked at some other challenge platforms like Duolingo, Hackerrank etc to understand how their experience helps to retain their users.

Following are the insights:

  • Some platforms use a streak-based approach to motivate learners. If the learner misses one day, the streak decreases.

  • Some platforms show the unsolved questions so that the learner can easily go and practice them later.

  • Some platforms use the concept of “Lives”, where after you get a question wrong your lives are decreased indicating that you have less number of attempts left.

Ideation & Brainstorm

Ideation & Brainstorm

After completing the problem discovery and competitive research/analysis, I started with ideation. I came up with a bunch of ideas. After coming up with the ideas, I had a meeting with the stakeholders.

Constraints

The project had a low budget and less workforce which led to a constraint that we cannot include a solution where the learner can solve the question on our platform directly. To overcome this, we decided that we would let the learners attempt the question elsewhere and submit the submission proof.

Features/Improvements:

After having a discussion, we came to the following conclusion:

  1. Introduction of Gems: To remove the manual intervention of the community team for the redemption round, we came up with Gems. Learners can earn a gem for every 4 consecutive submissions. Using these gems, learners can unlock a missed question and attempt it for which the submission will be considered. However, we also realized that if learners have more gems, they would give up the challenge in between and appear whenever they want. To maintain discipline and be fair to everyone, we decided that a learner could earn a maximum of 3 gems.

  2. Visibility of past questions: After a lot of discussion, we concluded that the learner must be provided with all the questions. But to keep the challenge fair and unbiased for everyone, if the learner does not have any gems, then he cannot submit the answer. The questions will be visible all the time.

  3. Leaderboard: To keep the motivation, we decided to keep the leaderboard. However, in the fist fold the leaderboard will only show the top 5 learners. On choosing to view more, only the top 10 learners would be shown. This is made sure that no one was demotivated by their rank even if they didn’t perform well.

  4. Certificate of completion: To keep the challenge authentic, we have made sure to provide the certificate of completion to only those who complete all the challenges. This is to ensure they stay on track and the seriousness of the challenge. Even if the learner falls short of one question, he won’t be awarded the certificate.

The revised user flow:

After finalizing the features and improvements, I made the new daily challenge process and user flow.

In the revised daily challenge process, everything is similar to the previous process up to publishing the new challenge. In the new user flow, I have allowed the users to view the past missed questions to ensure that they practice it on their own time, however, the submission will be locked. I discarded and replaced the manual intervention with using gems for the redemption process. After the learner attempts 4 consecutive challenges, he will be awarded a gem. This gem will redeem the learner’s one of the past missed questions.

After completing the problem discovery and competitive research/analysis, I started with ideation. I came up with a bunch of ideas. After coming up with the ideas, I had a meeting with the stakeholders.

Constraints

The project had a low budget and less workforce which led to a constraint that we cannot include a solution where the learner can solve the question on our platform directly. To overcome this, we decided that we would let the learners attempt the question elsewhere and submit the submission proof.

Features/Improvements:

After having a discussion, we came to the following conclusion:

  1. Introduction of Gems: To remove the manual intervention of the community team for the redemption round, we came up with Gems. Learners can earn a gem for every 4 consecutive submissions. Using these gems, learners can unlock a missed question and attempt it for which the submission will be considered. However, we also realized that if learners have more gems, they would give up the challenge in between and appear whenever they want. To maintain discipline and be fair to everyone, we decided that a learner could earn a maximum of 3 gems.

  2. Visibility of past questions: After a lot of discussion, we concluded that the learner must be provided with all the questions. But to keep the challenge fair and unbiased for everyone, if the learner does not have any gems, then he cannot submit the answer. The questions will be visible all the time.

  3. Leaderboard: To keep the motivation, we decided to keep the leaderboard. However, in the fist fold the leaderboard will only show the top 5 learners. On choosing to view more, only the top 10 learners would be shown. This is made sure that no one was demotivated by their rank even if they didn’t perform well.

  4. Certificate of completion: To keep the challenge authentic, we have made sure to provide the certificate of completion to only those who complete all the challenges. This is to ensure they stay on track and the seriousness of the challenge. Even if the learner falls short of one question, he won’t be awarded the certificate.

The revised user flow:

After finalizing the features and improvements, I made the new daily challenge process and user flow.

In the revised daily challenge process, everything is similar to the previous process up to publishing the new challenge. In the new user flow, I have allowed the users to view the past missed questions to ensure that they practice it on their own time, however, the submission will be locked. I discarded and replaced the manual intervention with using gems for the redemption process. After the learner attempts 4 consecutive challenges, he will be awarded a gem. This gem will redeem the learner’s one of the past missed questions.

Design

Design

To begin with the design, I started thinking about the edge cases that might occur and the state of the challenge. For the overall design challenge, the states would be:

  • Challenge onboarding

  • Challenge is live

  • Challenge is over

For each of these states, I then laid out the edge cases that might occur.

State I: Challenge onboarding

When the challenge commences, we need to onboard the learner on the platform to familiarize them with it. Although we have a kick-off call, many of the learners might not attend it and appear for the challenge directly. For such a scenario, onboarding is necessary.

For the onboarding screen, I have designed the instructions/information carousel to inform learners about the guidelines of the challenge.

  • These instructions/information will be visible for the first time on the first day of the challenge only.

  • Since the most important thing the learner comes for challenge of the day or the question of the day, I made sure that it is the first thing that the learner comes across.

  • To ensure that the instructions/information are clear, I have designed each instruction/information card to include a title for the instruction/information, a brief description, and the icons used for the features and instructions/information across the platform to familiarize the learner with the platform.

  • The instructions/information are as follows:

    • Challenge tracker: Learners can keep track of the challenge.

    • Leaderboard: Learners can see where they stand in the challenge.

    • Recovery Gems: Learners can earn gems and use them to unlock previously missed questions/challenges.

    • Certificate of Completion: Learners can earn a sharable certificate of completion after completing all the questions/challenges.

State II: Challenge is live

When the challenge is live, the learner can access the gems collector, challenge tracker, leaderboard, past questions, past question page, and Today’s question. There were many edge cases to consider for the design of each of the elements.

Gems collector

The gems can be collected by the learner after 4 consecutive submissions. The collector would have four states as shown in the image below.

Challenge tracker

Learners can track their challenge progress using the challenge tracker. The challenge tracker consists of the total number of challenges/questions for that challenge. The challenge tracker has 5 states:

  • Challenge of the day: The state on the day of the challenge/question.

  • Selecting the attempted challenge: This is highlighted by a green circle to indicate that the particular challenge/question was submitted on the day expected.

  • Selecting the missed challenge: This is highlighted by a red circle to indicate that the particular challenge/question was missed on the day expected.

  • Selecting the redeemed challenge: This is highlighted by a light purple circle to indicate that the particular challenge/question was submitted using the gem.

  • Completion of the challenge: The state on the day when the challenge concludes.

Leaderboard

Learners can track where they stand among other learners for the given challenge duration. The leaderboard has 3 states as below:

  • Leaderboard inactive: The state where the learner has not attempted the challenge and hence the leaderboard is locked or in an empty state.

  • First-fold leaderboard: Once the learner submits a solution, he can see the leaderboard. The leaderboard only shows 5 top learners in the first fold. This is done to make sure that we optimize the space.

  • Expanded leaderboard: The state where the leaderboard is expanded. In this state, learners can see the top 10 learners in the form of a scrollable list and their position on the leaderboard.

I have designed the leaderboard such that the positions of all of the learners are not visible to everyone. For someone whose rank is low, it might be discouraging that others can see it.

Today’s question

When a learner comes on the platform, he has the option to view either today’s question or past questions. When the learner opts to view today’s question, he can see one of the following 4 states:

  • Pending challenge: A pending question state if the learner has not attempted the question/challenge.

  • Submission in-progress: An in-progress submission state if the learner has uploaded the solution.

  • Submitting the question: A successful solution uploaded state when the learner gets the option to submit it.

  • Question submitted: A successful submission page where the learner is shown an uplifting confirmation and a list of past questions. If the learner wants to view more of the past questions, he will be redirected to the past question’s tab.

Past questions

When the learner opts to view the past questions, he can see one of the following 4 states:

  • First day of the challenge: If it’s the first day of the entire challenge, there will be an empty state.

  • Past all questions: The learner can view all the past questions.

  • Past missed questions: The learner can filter to view missed questions.

  • Past submitted questions: The learner can filter to view submitted questions.

When the learner hovers over the question’s row, he can see an arrow. When he clicks on it, he is redirected to the past question’s page.

Past question page

When the learner opts to view one of the past questions, he will encounter one of the following 6 states:

  • Attempting missed challenge: If the learner has a gem, then he can see an option to upload the code.

  • Missed challenge submission in progress: If the learner uploads the solution, then during the duration of uploading this state will be visible.

  • Submitting the question: Once the solution is uploaded successfully, the learner can see an option to submit the solution using a gem

  • Confirmation before submitting using a gem: A confirmation pop-up opens up to inform that the gem will be used to submit the solution.

  • Question submitted: The successful submission page where the learner is shown an uplifting confirmation and a list of past questions. If the learner wants to view more of the past questions, he will be redirected to the past question’s tab.

  • Unsuccessful attempt to unlock the challenge: If the learner does not have sufficient gems, he will encounter this state stating to earn a gem to unlock this particular question.

State III: Challenge is over

When the challenge is over, the learner must be able to see his progress and the certificate (if qualified for one). As mentioned in the features/improvement section, the certificate of completion will be visible only when the learner has attempted all the challenges. There can be two states for the challenge completion screen and one state for the question.

Challenge completion state:

  • When the learner has attempted all the challenges, show the certificate.

  • When the learner has not attempted all the challenges, show the progress throughout the challenge.

Question state:

  • Since the challenge will be over, we still need the learner to view the question for future reference.

  • So for this, I designed a state where there is no option for submission but the learner can see the question.

To begin with the design, I started thinking about the edge cases that might occur and the state of the challenge. For the overall design challenge, the states would be:

  • Challenge onboarding

  • Challenge is live

  • Challenge is over

For each of these states, I then laid out the edge cases that might occur.

State I: Challenge onboarding

When the challenge commences, we need to onboard the learner on the platform to familiarize them with it. Although we have a kick-off call, many of the learners might not attend it and appear for the challenge directly. For such a scenario, onboarding is necessary.

For the onboarding screen, I have designed the instructions/information carousel to inform learners about the guidelines of the challenge.

  • These instructions/information will be visible for the first time on the first day of the challenge only.

  • Since the most important thing the learner comes for challenge of the day or the question of the day, I made sure that it is the first thing that the learner comes across.

  • To ensure that the instructions/information are clear, I have designed each instruction/information card to include a title for the instruction/information, a brief description, and the icons used for the features and instructions/information across the platform to familiarize the learner with the platform.

  • The instructions/information are as follows:

    • Challenge tracker: Learners can keep track of the challenge.

    • Leaderboard: Learners can see where they stand in the challenge.

    • Recovery Gems: Learners can earn gems and use them to unlock previously missed questions/challenges.

    • Certificate of Completion: Learners can earn a sharable certificate of completion after completing all the questions/challenges.

State II: Challenge is live

When the challenge is live, the learner can access the gems collector, challenge tracker, leaderboard, past questions, past question page, and Today’s question. There were many edge cases to consider for the design of each of the elements.

Gems collector

The gems can be collected by the learner after 4 consecutive submissions. The collector would have four states as shown in the image below.

Challenge tracker

Learners can track their challenge progress using the challenge tracker. The challenge tracker consists of the total number of challenges/questions for that challenge. The challenge tracker has 5 states:

  • Challenge of the day: The state on the day of the challenge/question.

  • Selecting the attempted challenge: This is highlighted by a green circle to indicate that the particular challenge/question was submitted on the day expected.

  • Selecting the missed challenge: This is highlighted by a red circle to indicate that the particular challenge/question was missed on the day expected.

  • Selecting the redeemed challenge: This is highlighted by a light purple circle to indicate that the particular challenge/question was submitted using the gem.

  • Completion of the challenge: The state on the day when the challenge concludes.

Leaderboard

Learners can track where they stand among other learners for the given challenge duration. The leaderboard has 3 states as below:

  • Leaderboard inactive: The state where the learner has not attempted the challenge and hence the leaderboard is locked or in an empty state.

  • First-fold leaderboard: Once the learner submits a solution, he can see the leaderboard. The leaderboard only shows 5 top learners in the first fold. This is done to make sure that we optimize the space.

  • Expanded leaderboard: The state where the leaderboard is expanded. In this state, learners can see the top 10 learners in the form of a scrollable list and their position on the leaderboard.

I have designed the leaderboard such that the positions of all of the learners are not visible to everyone. For someone whose rank is low, it might be discouraging that others can see it.

Today’s question

When a learner comes on the platform, he has the option to view either today’s question or past questions. When the learner opts to view today’s question, he can see one of the following 4 states:

  • Pending challenge: A pending question state if the learner has not attempted the question/challenge.

  • Submission in-progress: An in-progress submission state if the learner has uploaded the solution.

  • Submitting the question: A successful solution uploaded state when the learner gets the option to submit it.

  • Question submitted: A successful submission page where the learner is shown an uplifting confirmation and a list of past questions. If the learner wants to view more of the past questions, he will be redirected to the past question’s tab.

Past questions

When the learner opts to view the past questions, he can see one of the following 4 states:

  • First day of the challenge: If it’s the first day of the entire challenge, there will be an empty state.

  • Past all questions: The learner can view all the past questions.

  • Past missed questions: The learner can filter to view missed questions.

  • Past submitted questions: The learner can filter to view submitted questions.

When the learner hovers over the question’s row, he can see an arrow. When he clicks on it, he is redirected to the past question’s page.

Past question page

When the learner opts to view one of the past questions, he will encounter one of the following 6 states:

  • Attempting missed challenge: If the learner has a gem, then he can see an option to upload the code.

  • Missed challenge submission in progress: If the learner uploads the solution, then during the duration of uploading this state will be visible.

  • Submitting the question: Once the solution is uploaded successfully, the learner can see an option to submit the solution using a gem

  • Confirmation before submitting using a gem: A confirmation pop-up opens up to inform that the gem will be used to submit the solution.

  • Question submitted: The successful submission page where the learner is shown an uplifting confirmation and a list of past questions. If the learner wants to view more of the past questions, he will be redirected to the past question’s tab.

  • Unsuccessful attempt to unlock the challenge: If the learner does not have sufficient gems, he will encounter this state stating to earn a gem to unlock this particular question.

State III: Challenge is over

When the challenge is over, the learner must be able to see his progress and the certificate (if qualified for one). As mentioned in the features/improvement section, the certificate of completion will be visible only when the learner has attempted all the challenges. There can be two states for the challenge completion screen and one state for the question.

Challenge completion state:

  • When the learner has attempted all the challenges, show the certificate.

  • When the learner has not attempted all the challenges, show the progress throughout the challenge.

Question state:

  • Since the challenge will be over, we still need the learner to view the question for future reference.

  • So for this, I designed a state where there is no option for submission but the learner can see the question.

Results

Results

After the shipment of this project, learners were able to participate effortlessly and efficiently. They were no longer facing the issue of missing out on previous questions and attempts for them. This led to an increase in the participation rate by 47% and a retention rate of 47.7%. The developed and redesigned feature was well received by the learners and there was less manual intervention.

After the shipment of this project, learners were able to participate effortlessly and efficiently. They were no longer facing the issue of missing out on previous questions and attempts for them. This led to an increase in the participation rate by 47% and a retention rate of 47.7%. The developed and redesigned feature was well received by the learners and there was less manual intervention.

Key takeaways

Key takeaways

This was the first project of my internship. In this project, I learned how to account for edge cases and solve them. This project was also the first time where I prepared and did a handoff to the developers.

This was the first project of my internship. In this project, I learned how to account for edge cases and solve them. This project was also the first time where I prepared and did a handoff to the developers.