58:62 JT

83:34 ZV

38:89 NO

Pluralsight Mobile Onboarding

Pluralsight Mobile Onboarding

Pluralsight Mobile Onboarding

an onboarding experience for Pluralsight mobile apps

an onboarding experience for Pluralsight mobile apps

an onboarding experience for Pluralsight mobile apps

Background

What is Pluralsight?

Pluralsight is a subscription-based online learning platform that offers some of the best-in-class courses to help users skill up in tech.

My role at Pluralsight

I led the Native Apps team of 14 stakeholders, which included 3 software development teams (iOS, Android, API), overseeing 7 native apps (iOS, Android, MacOS, Windows, AppleTV, Fire TV, Roku).

Most of our time was spent on the mobile app, and this onboarding experience was one of the most major and successful projects that I led.

In this project, my team and I set out to understand what caused a sharp increase in user churn and designed a solution for it.

Background

What is Pluralsight?

Pluralsight is a subscription-based online learning platform that offers some of the best-in-class courses to help users skill up in tech.

My role at Pluralsight

I led the Native Apps team of 14 stakeholders, which included 3 software development teams (iOS, Android, API), overseeing 7 native apps (iOS, Android, MacOS, Windows, AppleTV, Fire TV, Roku).

Most of our time was spent on the mobile app, and this onboarding experience was one of the most major and successful projects that I led.

In this project, my team and I set out to understand what caused a sharp increase in user churn and designed a solution for it.

Background

What is Pluralsight?

Pluralsight is a subscription-based online learning platform that offers some of the best-in-class courses to help users skill up in tech.

My role at Pluralsight

I led the Native Apps team of 14 stakeholders, which included 3 software development teams (iOS, Android, API), overseeing 7 native apps (iOS, Android, MacOS, Windows, AppleTV, Fire TV, Roku).

Most of our time was spent on the mobile app, and this onboarding experience was one of the most major and successful projects that I led.

In this project, my team and I set out to understand what caused a sharp increase in user churn and designed a solution for it.

Overview

After COVID hit in late 2019 and most tech employers implemented a work from home policy, the behavior of many users on our mobile apps started changing. Prior to COVID, our mobile apps were primarily used by our learners during their commute to and back from work. Since many tech workers no longer were commuting, the usage of our mobile apps was drastically affected.

While reviewing metrics for the 2021 OKR planning, we noticed that there has been a steep increase in first-time app user churn (i.e. more first-time mobile app users would use our app only one time and never return back in the first 28 days). It was quite sharp that in just the span of a few months, we saw user churn increase from 31.8% to 38%. Since this metric was strongly correlated with our company-wide north star metric, Monthly Active Users (MAU), we set out to find a solution to this problem.

Overview

After COVID hit in late 2019 and most tech employers implemented a work from home policy, the behavior of many users on our mobile apps started changing. Prior to COVID, our mobile apps were primarily used by our learners during their commute to and back from work. Since many tech workers no longer were commuting, the usage of our mobile apps was drastically affected.

While reviewing metrics for the 2021 OKR planning, we noticed that there has been a steep increase in first-time app user churn (i.e. more first-time mobile app users would use our app only one time and never return back in the first 28 days). It was quite sharp that in just the span of a few months, we saw user churn increase from 31.8% to 38%. Since this metric was strongly correlated with our company-wide north star metric, Monthly Active Users (MAU), we set out to find a solution to this problem.

Overview

After COVID hit in late 2019 and most tech employers implemented a work from home policy, the behavior of many users on our mobile apps started changing. Prior to COVID, our mobile apps were primarily used by our learners during their commute to and back from work. Since many tech workers no longer were commuting, the usage of our mobile apps was drastically affected.

While reviewing metrics for the 2021 OKR planning, we noticed that there has been a steep increase in first-time app user churn (i.e. more first-time mobile app users would use our app only one time and never return back in the first 28 days). It was quite sharp that in just the span of a few months, we saw user churn increase from 31.8% to 38%. Since this metric was strongly correlated with our company-wide north star metric, Monthly Active Users (MAU), we set out to find a solution to this problem.

Goal

Our goal was to reduce B2B One-and-Dones on mobile from 38% to 30%. (A one-and-done happens when a new mobile user does not return to the mobile app 28 days after their first visit.)

At the time, our apps did not have an onboarding experience. We saw an opportunity in creating an onboarding experience for our first-time mobile app users. Our hypothesis was that by creating a solid onboarding experience for first-time Pluralsight users, we will have a better chance in retaining them for the long run.

With that said, onboarding can be a large time investment and can take so many different directions if not intentionally scoped. Therefore, we spent the early stages of this project scoping down the problem.

It was then time to jump into research to understand the user pain points to scope down the problem and make sure that our solution would give them value and not just solve a business problem.

Goal

Our goal was to reduce B2B One-and-Dones on mobile from 38% to 30%. (A one-and-done happens when a new mobile user does not return to the mobile app 28 days after their first visit.)

At the time, our apps did not have an onboarding experience. We saw an opportunity in creating an onboarding experience for our first-time mobile app users. Our hypothesis was that by creating a solid onboarding experience for first-time Pluralsight users, we will have a better chance in retaining them for the long run.

With that said, onboarding can be a large time investment and can take so many different directions if not intentionally scoped. Therefore, we spent the early stages of this project scoping down the problem.

It was then time to jump into research to understand the user pain points to scope down the problem and make sure that our solution would give them value and not just solve a business problem.

Goal

Our goal was to reduce B2B One-and-Dones on mobile from 38% to 30%. (A one-and-done happens when a new mobile user does not return to the mobile app 28 days after their first visit.)

At the time, our apps did not have an onboarding experience. We saw an opportunity in creating an onboarding experience for our first-time mobile app users. Our hypothesis was that by creating a solid onboarding experience for first-time Pluralsight users, we will have a better chance in retaining them for the long run.

With that said, onboarding can be a large time investment and can take so many different directions if not intentionally scoped. Therefore, we spent the early stages of this project scoping down the problem.

It was then time to jump into research to understand the user pain points to scope down the problem and make sure that our solution would give them value and not just solve a business problem.

Research

1-1 Semi-Structured Interviews

To scope down the problem, we wanted to first understand the experience that new B2B mobile learners have on Pluralsight. So we did 1-1 interviews with that user segment. In our research, we wanted to understand the following questions:

  1. Do learners know what Pluralsight is?

    Since many companies now subscribe to a plethora of different services and provide them to their employees through an access management service (e.g. Okta), users sometimes don't even know what services their company has signed them up for and why. So sometimes the reason users bounce and never return is because of that. The purpose behind this question was to rule this out as a confound.

    It's worth noting, however, that most users who would sign up to speak with us—especially since we didn't incentivize our research studies—would probably know what Pluralsight is. So we proceeded with caution.

  2. Do learners know the value proposition of Pluralsight mobile apps?

    We asked this question to understand whether users know what our mobile app offered (the value proposition) that they wouldn't find on the web platform. Among these value propositions were offline downloads, portability, audio-only player, and a fun quiz experience.

    We wanted to get a sense of whether users need to be onboarded to these features. If that were the case, then we were going to include an introduction to these features during our onboarding flow.

  3. Do learners find the app usable?

    Finally, we wanted to test the user experience of the app. Are app features confusing? Do users find it difficult to download courses? Are some of our highly-retaining mobile features not discoverable enough?

    Poor user experience on the apps could be one of the major driving factors for users leaving the app. So we also wanted to understand whether users found our app to be usable.

Competitive Landscape Review

I conducted a competitive landscape review of how other companies onboard users to get a sense of what the competition is doing.

click here for a full-size image

Coursera

click here for a full-size image

Udemy

click here for a full-size image

Udemy for Business (blocked by subscription wall)

click here for a full-size image

EdX

click here for a full-size image

LinkedIn Learning

click here for a full-size image

Why do a Competitive Landscape in the first place?

The primary reason behind going through this exercise was to get a better sense and understanding of learners’ mental models. Since there is a chance that users who use Pluralsight might have also used other online learning platforms, I wanted to make sure that we design our onboarding experience with users’ mental models in mind and not steer away too much from what they’re used to experiencing with similar services.

Problem
Space

1-1 Semi-Structured Interviews

To scope down the problem, we wanted to first understand the experience that new B2B mobile learners have on Pluralsight. So we did 1-1 interviews with that user segment. In our research, we wanted to understand the following questions:

  1. Do learners know what Pluralsight is?

    Since many companies now subscribe to a plethora of different services and provide them to their employees through an access management service (e.g. Okta), users sometimes don't even know what services their company has signed them up for and why. So sometimes the reason users bounce and never return is because of that. The purpose behind this question was to rule this out as a confound.

    It's worth noting, however, that most users who would sign up to speak with us—especially since we didn't incentivize our research studies—would probably know what Pluralsight is. So we proceeded with caution.

  2. Do learners know the value proposition of Pluralsight mobile apps?

    We asked this question to understand whether users know what our mobile app offered (the value proposition) that they wouldn't find on the web platform. Among these value propositions were offline downloads, portability, audio-only player, and a fun quiz experience.

    We wanted to get a sense of whether users need to be onboarded to these features. If that were the case, then we were going to include an introduction to these features during our onboarding flow.

  3. Do learners find the app usable?

    Finally, we wanted to test the user experience of the app. Are app features confusing? Do users find it difficult to download courses? Are some of our highly-retaining mobile features not discoverable enough?

    Poor user experience on the apps could be one of the major driving factors for users leaving the app. So we also wanted to understand whether users found our app to be usable.

Competitive Landscape Review

I conducted a competitive landscape review of how other companies onboard users to get a sense of what the competition is doing.

click here for a full-size image

Coursera

click here for a full-size image

Udemy

click here for a full-size image

Udemy for Business (blocked by subscription wall)

click here for a full-size image

EdX

click here for a full-size image

LinkedIn Learning

click here for a full-size image

Why do a Competitive Landscape in the first place?

The primary reason behind going through this exercise was to get a better sense and understanding of learners’ mental models. Since there is a chance that users who use Pluralsight might have also used other online learning platforms, I wanted to make sure that we design our onboarding experience with users’ mental models in mind and not steer away too much from what they’re used to experiencing with similar services.

Research

1-1 Semi-Structured Interviews

To scope down the problem, we wanted to first understand the experience that new B2B mobile learners have on Pluralsight. So we did 1-1 interviews with that user segment. In our research, we wanted to understand the following questions:

  1. Do learners know what Pluralsight is?

    Since many companies now subscribe to a plethora of different services and provide them to their employees through an access management service (e.g. Okta), users sometimes don't even know what services their company has signed them up for and why. So sometimes the reason users bounce and never return is because of that. The purpose behind this question was to rule this out as a confound.

    It's worth noting, however, that most users who would sign up to speak with us—especially since we didn't incentivize our research studies—would probably know what Pluralsight is. So we proceeded with caution.

  2. Do learners know the value proposition of Pluralsight mobile apps?

    We asked this question to understand whether users know what our mobile app offered (the value proposition) that they wouldn't find on the web platform. Among these value propositions were offline downloads, portability, audio-only player, and a fun quiz experience.

    We wanted to get a sense of whether users need to be onboarded to these features. If that were the case, then we were going to include an introduction to these features during our onboarding flow.

  3. Do learners find the app usable?

    Finally, we wanted to test the user experience of the app. Are app features confusing? Do users find it difficult to download courses? Are some of our highly-retaining mobile features not discoverable enough?

    Poor user experience on the apps could be one of the major driving factors for users leaving the app. So we also wanted to understand whether users found our app to be usable.

Competitive Landscape Review

I conducted a competitive landscape review of how other companies onboard users to get a sense of what the competition is doing.

click here for a full-size image

Coursera

click here for a full-size image

Udemy

click here for a full-size image

Udemy for Business (blocked by subscription wall)

click here for a full-size image

EdX

click here for a full-size image

LinkedIn Learning

click here for a full-size image

Why do a Competitive Landscape in the first place?

The primary reason behind going through this exercise was to get a better sense and understanding of learners’ mental models. Since there is a chance that users who use Pluralsight might have also used other online learning platforms, I wanted to make sure that we design our onboarding experience with users’ mental models in mind and not steer away too much from what they’re used to experiencing with similar services.

Solution
Space

Compass Design

I worked closely with our product designer to craft the experience.

  1. Compass Discovery

Users will be able to access Compass through the following:

A banner promoting it on the Home Screen

A banner promoting it on the Browse

  1. Compass Content Feed Experience

The compass experience will feature two tabs: Latest and Starred

Latest

Starred

Starred (Empty State)

  1. Compass Features

Supported Content Sources

Author Spotlight

An author or lead spotlight is a place where we can feature Pluralsight content creators so learners can follow and view some of their courses

Conference Recap

A conference recap is a summary of a conference that is hosted on the Pluralsight platform

Featured Pluralsight Course

A featured Pluralsight course is a course hosted on Pluralsight that we promote

Featured Article

A featured article is an external article shared on Pluralsight through Compass

Medium Article

Tweet

YouTube Video

Vimeo Video

Actions Users Can Take

  • Star content

  • Share content

  • Opt into push notifications for new content updates

  • Feedback buttons (thumbs up/down)

  1. Content Hydration

“Content Hydration” is an internal term we used to reference how content would be added to the Compass experience. Since Compass was still an experimental feature that we decided to validate before getting authors’ and leads’ buy-in, we decided to only support two topics:

Google Cloud Platform (GCP) and Android.

One of our Android engineers, who is quite well-versed in these two topics, took on the responsibility for adding new content for users during the experimentation period allowing us to move faster.

Solution
Space

Compass Design

I worked closely with our product designer to craft the experience.

  1. Compass Discovery

Users will be able to access Compass through the following:

A banner promoting it on the Home Screen

A banner promoting it on the Browse

  1. Compass Content Feed Experience

The compass experience will feature two tabs: Latest and Starred

Latest

Starred

Starred (Empty State)

  1. Compass Features

Supported Content Sources

Author Spotlight

An author or lead spotlight is a place where we can feature Pluralsight content creators so learners can follow and view some of their courses

Conference Recap

A conference recap is a summary of a conference that is hosted on the Pluralsight platform

Featured Pluralsight Course

A featured Pluralsight course is a course hosted on Pluralsight that we promote

Featured Article

A featured article is an external article shared on Pluralsight through Compass

Medium Article

Tweet

YouTube Video

Vimeo Video

Actions Users Can Take

  • Star content

  • Share content

  • Opt into push notifications for new content updates

  • Feedback buttons (thumbs up/down)

  1. Content Hydration

“Content Hydration” is an internal term we used to reference how content would be added to the Compass experience. Since Compass was still an experimental feature that we decided to validate before getting authors’ and leads’ buy-in, we decided to only support two topics:

Google Cloud Platform (GCP) and Android.

One of our Android engineers, who is quite well-versed in these two topics, took on the responsibility for adding new content for users during the experimentation period allowing us to move faster.

Solution
Space

Compass Design

I worked closely with our product designer to craft the experience.

  1. Compass Discovery

Users will be able to access Compass through the following:

A banner promoting it on the Home Screen

A banner promoting it on the Browse

  1. Compass Content Feed Experience

The compass experience will feature two tabs: Latest and Starred

Latest

Starred

Starred (Empty State)

  1. Compass Features

Supported Content Sources

Author Spotlight

An author or lead spotlight is a place where we can feature Pluralsight content creators so learners can follow and view some of their courses

Conference Recap

A conference recap is a summary of a conference that is hosted on the Pluralsight platform

Featured Pluralsight Course

A featured Pluralsight course is a course hosted on Pluralsight that we promote

Featured Article

A featured article is an external article shared on Pluralsight through Compass

Medium Article

Tweet

YouTube Video

Vimeo Video

Actions Users Can Take

  • Star content

  • Share content

  • Opt into push notifications for new content updates

  • Feedback buttons (thumbs up/down)

  1. Content Hydration

“Content Hydration” is an internal term we used to reference how content would be added to the Compass experience. Since Compass was still an experimental feature that we decided to validate before getting authors’ and leads’ buy-in, we decided to only support two topics:

Google Cloud Platform (GCP) and Android.

One of our Android engineers, who is quite well-versed in these two topics, took on the responsibility for adding new content for users during the experimentation period allowing us to move faster.

Metrics +
Analytics

Measuring Experiment Success

Below are metrics we Identified to measure success for Compass:

  • Thumbs up to thumbs down ratio

  • Percentage of starred content/user

  • Percentage of content clicks/user

  • Notifications opt-in rate

I worked directly with our data analyst providing him with a fully-fledged event tracking document to measure Compass’ success.

Metrics +
Analytics

Measuring Experiment Success

Below are metrics we Identified to measure success for Compass:

  • Thumbs up to thumbs down ratio

  • Percentage of starred content/user

  • Percentage of content clicks/user

  • Notifications opt-in rate

I worked directly with our data analyst providing him with a fully-fledged event tracking document to measure Compass’ success.

Metrics +
Analytics

Measuring Experiment Success

Below are metrics we Identified to measure success for Compass:

  • Thumbs up to thumbs down ratio

  • Percentage of starred content/user

  • Percentage of content clicks/user

  • Notifications opt-in rate

I worked directly with our data analyst providing him with a fully-fledged event tracking document to measure Compass’ success.

Results

Results

Metrics +
Analytics

Next Steps

Compass was an experimental 0-1 product.

We allowed users to interact with Compass while we collected quantitative data. We saw promising quantitative results. Had I stayed longer at the company, I would have liked to dig in deeper by conducting user interviews with learners who were a part of this experiment to better understand their experience qualitatively.

By analyzing the quantitative and qualitative data collected, I would have created a roadmap of new feature iterations that will make this experience even better.

Ultimately, the goal was to replace the current static home screen to with Compass, by attracting as many authors and leads as possible to be the driving engine for this experience, which in turn will help them promote their work.

Next Steps

Compass was an experimental 0-1 product.

We allowed users to interact with Compass while we collected quantitative data. We saw promising quantitative results. Had I stayed longer at the company, I would have liked to dig in deeper by conducting user interviews with learners who were a part of this experiment to better understand their experience qualitatively.

By analyzing the quantitative and qualitative data collected, I would have created a roadmap of new feature iterations that will make this experience even better.

Ultimately, the goal was to replace the current static home screen to with Compass, by attracting as many authors and leads as possible to be the driving engine for this experience, which in turn will help them promote their work.

Next
Steps

Compass was an experimental 0-1 product.

We allowed users to interact with Compass while we collected quantitative data. We saw promising quantitative results. Had I stayed longer at the company, I would have liked to dig in deeper by conducting user interviews with learners who were a part of this experiment to better understand their experience qualitatively.

By analyzing the quantitative and qualitative data collected, I would have created a roadmap of new feature iterations that will make this experience even better.

Ultimately, the goal was to replace the current static home screen to with Compass, by attracting as many authors and leads as possible to be the driving engine for this experience, which in turn will help them promote their work.

Reflections
+ Learnings

Working on Compass was quite rewarding since it took us one step closer to completely overhauling the mobile home screen, a milestone that the team has been thinking about for years. This would have marked a major evolution of our mobile apps.

One learning in particular that I would love to shed light on is working on smaller chunks and rapidly experimenting with those to be able to iterate faster on the experience. One of the mistakes I did during this project is adding more features as we went into development. Not only did this cause us to move slower, it also caused some frustration for developers. This also led to an over-bloated experience that was meant to be experimental, to begin with.

A much better approach to building a complex product like Compass was by implementing smaller features, testing those out, and then iterating rapidly. This would have allowed us to move at a more steady, healthy, and sustainable pace.

I am proud of the value my team have delivered to our select users, and I have learned invaluable lessons along the way.

Reflections
+ Learnings

Working on Compass was quite rewarding since it took us one step closer to completely overhauling the mobile home screen, a milestone that the team has been thinking about for years. This would have marked a major evolution of our mobile apps.

One learning in particular that I would love to shed light on is working on smaller chunks and rapidly experimenting with those to be able to iterate faster on the experience. One of the mistakes I did during this project is adding more features as we went into development. Not only did this cause us to move slower, it also caused some frustration for developers. This also led to an over-bloated experience that was meant to be experimental, to begin with.

A much better approach to building a complex product like Compass was by implementing smaller features, testing those out, and then iterating rapidly. This would have allowed us to move at a more steady, healthy, and sustainable pace.

I am proud of the value my team have delivered to our select users, and I have learned invaluable lessons along the way.

Reflections
+ Learnings

Working on Compass was quite rewarding since it took us one step closer to completely overhauling the mobile home screen, a milestone that the team has been thinking about for years. This would have marked a major evolution of our mobile apps.

One learning in particular that I would love to shed light on is working on smaller chunks and rapidly experimenting with those to be able to iterate faster on the experience. One of the mistakes I did during this project is adding more features as we went into development. Not only did this cause us to move slower, it also caused some frustration for developers. This also led to an over-bloated experience that was meant to be experimental, to begin with.

A much better approach to building a complex product like Compass was by implementing smaller features, testing those out, and then iterating rapidly. This would have allowed us to move at a more steady, healthy, and sustainable pace.

I am proud of the value my team have delivered to our select users, and I have learned invaluable lessons along the way.

© 2024 — Omar El-Etr

Home

1

Home

1