IBM Casptone Project

UX Design, UX Research, Web Design

Problem

B2B products are normally complicated with a wide variety of capabilities. They often require technical or business knowledge in order for their customers to become active, productive users of the product. The journey from Discover, Try, and Buy to active usage is therefore complex, time consuming, and varies by user types. If the users are not able to easily use a given product or service during the onboarding process, they will be likely to give up early and cannot unlock the product's potential benefits.

Our Goal

Our goal was to research the best practices and leverage Augmented Intelligence (AI) technology to help improve the onboarding experience and to attract and retain new users in a Business to Business (B2B) context. In this project, we chose eCommerce as our focused domain to explore the problems as this project was not limited to IBM technologies/products or their competitors.

  • Duration
  • May 2020 - July 2020
  • Collaborators
  • Derron Lee

    Gary Li

    Holly Kwan

    Jason Luong

    Nedu Achebe
  • Methods
  • Domain Research

    Interviews

    Persona

    Journey Map

    Wireframing

    Prototyping

    Usability Testing
  • Tools
  • Mural

    Sketch

    Figma

    InVision

    Pen & Pencil

My Contributions

Design Solution

Gamification that Drives User Incentives

Users can earn reward points during the onboarding process that could be used to extend trials or purchase apps upon activation of plan. This feature will help drive users' incentives and encourage users to complete essential onboarding steps.

Personalized Customer Service and Voice User Interface

Our AI assistant Shoppie will monitor users' movement based on onscreen activities to quickly understand users' needs and attempt to solve the problem in an automated way. Users can also switch to voice mode to directly talk with Shoppie and let it take control of store setup process.

Data Collection for Optimized Customization Result

Users answer a series of simple questions to obtain the most suitable template that works best for their goals and business needs.

Design Process

Secondary Research

Domain Research

To get started, we conducted domain research to decide which domain to focus on. As the project was not limited to IBM technologies or products, we explored a variety of industries such as health care, finance, insurance, and decided to chose eCommerce based on team consensus. We chose eCommerce as a research focus because: 1) the industry is growing rapidly especially during the global pandemic time due to Covid-19, 2) Many of our team members have acquaintances in eCommerce which would help make our future participant recruitment easier.

Once we identified the domain, we focused on asking the below questions to help understand the industry landscape and formulate interview questions.

Comparative Analysis

In addition, we also used comparative analysis to learn about existing onboarding flows across eCommerce platforms. This practice allowed us to build interview questions that are specific and targeted to the industry, as well as learning from the current successful onboarding tactics.

Primary Research

Interviews

In order to build empathy and stay in touch with users' needs, we interviewed 15 sponsored users to gain feedback about their onboarding journey using different eCommerce platforms.

Why Interviews?
  • A quick and easy way to collect direct feedback from users.
  • To understand user pain points in a contextual inquiry which helps define the problem statement.
  • Prepare initial user data for synthesis and to inform Persona and Journey Map.
Participant Recruitment

I first created a screener to help identify our targeted users. We aimed to look for participants who have experience of using at least one eCommerce platform. The participants were mainly recruited from the professional network such as LinkedIn and social media platforms such as Facebook and Slack groups. These participants represented a diverse demographics, from small business owners to individuals with side hustles. This was to make sure the user data we collected was less biased with high reliability.

The interview took 45 minutes to complete and covered questions from why and how users choose a specific eCommerce platform to what benefits and challenges they discovered during their onboarding process. Below were some example interview questions that we asked participants. For full details of the interview questions, please click here.

Analysis

Affinity Diagram - Categorized Information in Themes

We used coding method to extract key information from interview transcripts and categorized users' problems under different themes.

We learned that the problems were mainly coming from the two themes: management & operations and customer experience. We focused on these two categories and delved deeper into the affinity diagram by gathering some quantitative data to understand the statistical distribution over the sample. Below are some of our key findings:

Based on the interview, we concluded the following insights:

Persona - Understanding the Targeted Users
To reflect the differences in skill and experience from the users, we made a Primary Persona to represent users with limited experience and a Secondary Persona for users with intermediate to advanced level of knowledge.

We decided to focus on Primary User Melody throughout our design process because the majority of our users were novice to intermediate users and if a design solution works for them, it would likely work for a more experienced user with similar needs as well.
Left: Primary Persona. Right: Secondary Persona
Why Persona?
  • To showcase a representation of targeted users with similar behavior patterns and goals.
  • To make sure the assumptions and knowledge about user needs are aligned between team members and stakeholders.
Journey Map
A user journey map was also created to demonstrate the pain points through different stages of onboarding process. We identified six essential stages: Discover, Register, Setup, Trial, Evaluate and Buy. This method helped us consolidate qualitative data thoroughly and find behavior patterns and common goals across different scenarios.
Why Journey Map?
  • Provide a strong and visual narrative that outlines targeted user's behavior at each stage of onboarding process.
  • Identify key areas where users encounter negative emotions to inform new desin opportunities.
  • An efficient way to assign ownership for team members to work on specific touchpoints.
Refining Problem Statement

How might we improve user onboarding journey using Argumented Technology that allows customization to different skill levels and learning styles for eCommerce platforms?

Ideation

Facilitating Various Design Workshops
Partnered with IBM Cognos Analytics team, we ran 5 design workshops to brainstorm ideas and solutions that were tied to user needs.

Design Opportunities

Dot-voting
Based on the research insights and brainstorming results, our team came up with a variety of ideas and wegrouped them into themes again. We then used dot-voting method to determine the best potential design ideas to move forward with.
In the end, we decided to focus on 4 major areas to improve the onboarding experience, which are customization capability, chatbot service, voice AI and gamification tactics.
User Flow
A user flow was created to demonstrate the checkpoints from the beginning of setup to create an online store with an eCommerce platform. Users will be asked to complete some questions for personalization during onboarding. This action creates an endowment effect that helps users build a sense of ownership with the value of product.

Prototype

Sketches

I created the below sketches to visualize one of the user flows in registration and customization process. I also drew a landing page that showcases the entry-point of onboarding.

Medium-Fidelity Prototype

After discussion with the team, I iterated on my sketches and created medium-fidelity prototype that covers the steps from registering and choosing a theme during the onboarding process. To let users choose whether they would like to go over the personalization questions at their own pace, we let users build a sense of control of their own onboarding while providing tailored experience.

High-Fidelity Prototype

Evaluation

Usability Testing
To validate our design, we conducted unmoderated usability testing sessions with 10 users to validate our high fidelity prototype on InVision. We obtained participants' consent before conducting the testing and provided instructions for 5 activities that they would perform. These activities allow the users to go through each stage that we mapped out in the user journey map.

The users completed a survey after the test so that we collected qualitative feedback to identify general problems on the usability of our design. After the test, we used the System Usability Scale (SUS) to measure how usable our design solution is and obtained an average SUS score of 73, meaning that our product is usable by most users. We realized that the score was also greatly affected by one outlier with an extremely low score due to some technical problems of running the prototype on his computer. Besides that, all the other users were positive about using the product.

Another constraint of our usability testing was that we were not able to testing the Voice User Interface (VUI) function due to the nature of technology. Therefore, we received a mixed reactions to the voice feature. 40% of users gave a positive feedback but 60% were average to negative. The feedback was that voice has to be accurate in order to be effective. However, one participant stated that he has carpal tunnel and thought the voice feature is a huge benefit.
Design Iterations
Below were some improvements that we made to improve our design based on the user feedback.

Final Design

Demo Video
We attempted to make a compelling story by using a video to showcase the design solutions. The demo video below walks through all the main user flows that have been tested and improved.
Concept Video - Voice User Interface (VUI)
In addition, we also made a concept video for the Voice User Interface (VUI) solution. We believe that asking simple questions and receiving instant feedback are very valuable, especially for users with disability as well as limited skills and knowledge. Having intuitive and interactive support from VUI will help save users' time on simple tasks and increase overall accessibility.

Reflections

Designing B2B product onboarding experience could be challenging because unlike B2C products, B2B products are normally more complicated in interface design with more functionalities. If time were permitted, we would like to run a second usability test to measure the changes in users' satisfaction and retention rates before and after we made iterations.

In addition, since we only explored eCommerce as our focused research domain, some user problems identified in the study could be less relevant than that in another industry. We would also like to do more lateral research to explore onboarding best practices across different industries in the future.

*AI character design credit to Freepik.

Onward to next project >

One Hour