My Role
UX Designer, UX Researcher on a team of 5 members

Methods
Research-related: competitive analysis, domain research, user interviews, remote usability testing
Design-related: user journey mapping, persona, wireframing, prototyping

Duration
May 2020 - Aug 2020

Capstone Project @IBM

As part of the graduate curriculum from the Master's of Digital Experience Innovation program, we partnered with IBM Cognos Analytics in a team of 6 to complete a capstone project that was focused on improving the onboarding experience for B2B products in the support of AI technology. We followed the agile workflow and met with the IBM mentors on a weekly basis, updating sprint deliverables and improving the research and design process based on the mentors' feedback.

Problem

B2B products are normally complicated with a wide variety of capabilities. They often require technical or business knowledge in order for their customers to become active, productive users of the product. The journey from Discover, Try, and Buy to active usage is therefore complex, time consuming, and varies by user types. If the users are not able to easily use a given product or service during the onboarding process, they will be likely to give up early and cannot unlock the product's potential benefits.

How might we improve user onboarding journey with AI technology that allows customization to different skill levels and learning styles?

Our Goal

Our goal was to research the best practices and leverage Augmented Intelligence (AI) technology to help improve the onboarding experience and to attract and retain new users in a Business to Business (B2B) context.

Determine the Domain

In this project, we chose eCommerce as our focused domain to explore the problems as this project was not limited to IBM technologies/products or their competitors. After discussing with the team, we all agreed that eCommerce would be a good B2B fileld to focus on as due to Covid-19, many people are looking to digitalize their business and setting up an online store would be in a huge demand. With more people became interested in the field, the more likely we would be able to recruit them for research purposes.

My Contributions

Design Solution

Gamification that Drives User Incentives

Users can earn reward points during the onboarding process that could be used to extend trials or purchase apps upon activation of plan. This feature will help drive users' incentives and encourage users to complete essential onboarding steps.

Personalized Customer Service and Voice User Interface

Our AI assistant Shoppie will monitor users' movement based on onscreen activities to quickly understand users' needs and attempt to solve the problem in an automated way. Users can also switch to voice mode to directly talk with Shoppie and let it take control of store setup process.

Data Collection for Optimized Customization Result

Users answer a series of simple questions to obtain the most suitable template that works best for their goals and business needs.

Design Process

Secondary Research

Domain Research

Once we identified the domain, we focused on asking the below questions to help understand the industry landscape and formulate interview questions.

Comparative Analysis

In addition, we also used comparative analysis to learn about existing onboarding flows across eCommerce platforms. This practice allowed us to build interview questions that are specific and targeted to the industry, as well as learning from the current successful onboarding tactics.

Primary Research

Interviews

In order to build empathy and stay in touch with users' needs, we interviewed 15 sponsored users to gain feedback about their onboarding journey using different eCommerce platforms.

Why Interviews?
  • A quick and easy way to collect direct feedback from users.
  • To understand user pain points in a contextual inquiry which helps define the problem statement.
  • Prepare initial user data for synthesis and to inform Persona and Journey Map.
Participant Recruitment

I first created a screener to help identify our targeted users. We aimed to look for participants who have experience of using at least one eCommerce platform. The participants were mainly recruited from the professional network such as LinkedIn and social media platforms such as Facebook and Slack groups. These participants represented a diverse demographics, from small business owners to individuals with side hustles. This was to make sure the user data we collected was less biased with high reliability.

The interview took 45 minutes to complete and covered questions from why and how users choose a specific eCommerce platform to what benefits and challenges they discovered during their onboarding process. Below were some example interview questions that we asked participants. For full details of the interview questions, please click here.

Analysis

Affinity Diagram - Categorized Information in Themes

We used coding method to extract key information from interview transcripts and categorized users' problems under different themes.

We learned that the problems were mainly coming from the two themes: management & operations and customer experience. We focused on these two categories and delved deeper into the affinity diagram by gathering some quantitative data to understand the statistical distribution over the sample. Below are some of our key findings:

Based on the interview, we concluded the following insights:

Persona - Understanding the Targeted Users
To reflect the differences in skill and experience from the users, we made a Primary Persona to represent users with limited experience and a Secondary Persona for users with intermediate to advanced level of knowledge.

We decided to focus on Primary User Melody throughout our design process because the majority of our users were novice to intermediate users and if a design solution works for them, it would likely work for a more experienced user with similar needs as well.
Left: Primary Persona. Right: Secondary Persona
Why Persona?
  • To showcase a representation of targeted users with similar behavior patterns and goals.
  • To make sure the assumptions and knowledge about user needs are aligned between team members and stakeholders.
Journey Map
A user journey map was also created to demonstrate the pain points through different stages of onboarding process. We identified six essential stages: Discover, Register, Setup, Trial, Evaluate and Buy. This method helped us consolidate qualitative data thoroughly and find behavior patterns and common goals across different scenarios.
Why Journey Map?
  • Provide a strong and visual narrative that outlines targeted user's behavior at each stage of onboarding process.
  • Identify key areas where users encounter negative emotions to inform new desin opportunities.
  • An efficient way to assign ownership for team members to work on specific touchpoints.

Ideation

Facilitating Various Design Workshops
Partnered with IBM Cognos Analytics team, we ran 5 design workshops to brainstorm ideas and solutions that were tied to user needs.

Design Opportunities

Dot-voting
Based on the research insights and brainstorming results, our team came up with a variety of ideas and wegrouped them into themes again. We then used dot-voting method to determine the best potential design ideas to move forward with.
In the end, we decided to focus on 4 major areas to improve the onboarding experience, which are customization capability, chatbot service, voice AI and gamification tactics.

Prototype

Sketches

I created the below sketches to visualize one of the user flows in registration and customization process. I also drew a landing page that showcases the entry-point of onboarding.

Medium-Fidelity Prototype

After discussion with the team, I iterated on my sketches and created medium-fidelity prototype that covers the steps from registering and choosing a theme during the onboarding process. To let users choose whether they would like to go over the personalization questions at their own pace, we let users build a sense of control of their own onboarding while providing tailored experience.

High-Fidelity Prototype

Evaluation

Usability Testing
To validate our design, we conducted unmoderated usability testing sessions with 10 users to validate our high fidelity prototype on InVision. We obtained participants' consent before conducting the testing and provided instructions for 5 activities that they would perform. These activities allow the users to go through each stage that we mapped out in the user journey map.

The users completed a survey after the test so that we collected qualitative feedback to identify general problems on the usability of our design. After the test, we used the System Usability Scale (SUS) to measure how usable our design solution is and obtained an average SUS score of 73, meaning that our product is usable by most users. We realized that the score was also greatly affected by one outlier with an extremely low score due to some technical problems of running the prototype on his computer. Besides that, all the other users were positive about using the product.

Another constraint of our usability testing was that we were not able to testing the Voice User Interface (VUI) function due to the nature of technology. Therefore, we received a mixed reactions to the voice feature. 40% of users gave a positive feedback but 60% were average to negative. The feedback was that voice has to be accurate in order to be effective. However, one participant stated that he has carpal tunnel and thought the voice feature is a huge benefit.
Design Iterations
Below were some improvements that we made to improve our design based on the user feedback.

Final Design

Demo Video
We attempted to make a compelling story by using a video to showcase the design solutions. The demo video below walks through all the main user flows that have been tested and improved.
Concept Video - Voice User Interface (VUI)
In addition, we also made a concept video for the Voice User Interface (VUI) solution. We believe that asking simple questions and receiving instant feedback are very valuable, especially for users with disability as well as limited skills and knowledge. Having intuitive and interactive support from VUI will help save users' time on simple tasks and increase overall accessibility.

Reflections

Designing B2B product onboarding experience could be challenging because unlike B2C products, B2B products are normally more complicated in interface design with more functionalities. If time were permitted, we would like to run a second usability test to measure the changes in users' satisfaction and retention rates before and after we made iterations. In addition, since we only explored eCommerce as our focused research domain, some user problems identified in the study could be less relevant than that in another industry. We would also like to do more lateral research to explore onboarding best practices across different industries in the future.

Onward to next project >

One Hour