A/B testing: Navigation
Product: ERP Solution
Note: This is a live project, hence all sensitive information has been replaced by placeholder data
Testing Objectives:
The testing centred on A/B testing two navigation styles within the ERP, with the primary goal of enhancing the overall user experience.
Test Plan:
Targeted six participants: three representing a user from senior management, and three representing associates. The participants were working in energy and construction sector, which is the primary industries where our ERP is to be used.
The testing, conducted remotely via Google Meet, involved two prototypes: Scenario 1 and Scenario 2.
Two prototypes are referred to as Scenario 1 and Scenario 2.
In Scenario 1, the list of headings takes up the major section of the screen. When you click on any heading, that major section of the screen is replaced by the detail page of that heading. To go to another detail page, you need to return to this list of headings page.
In Scenario 2, the list of headings is always visible, though it makes the screen slightly crowded. The main body stays empty unless we select a heading from the list. Once a heading is selected, the details open on the same page in the allocated empty space.
Execution:
Moderated sessions included tasks designed to evaluate the efficiency of the two navigation styles.
Since we were testing A/B for two navigation styles, there were two types of tasks that participants were supposed to complete on both prototypes one after the other in a random order:
Task 1: The first type of task was to "spot the difference" between "Associations section" across three different documents, requiring participants to navigate in and out.
Note: test subjects were not aware that we were testing the navigation.
Task 2: The other type of task was where they needed to stay on the detail page after entering it via the navigation and complete a tasks on that page itself: Adding a new Task to that document as shown below.
(Contd) Then go to the next detail page using the navigation again and do the same, and so on. The difference between task 1 and 2 was that they didn't need to go back and forth quickly this time, like in the "spot the difference" task.
Results and Findings:
Despite our concerns about a crowded interface, Scenario 2 consistently outperformed Scenario 1. Participants navigated more efficiently in Scenario 2, particularly evident in the "spot the difference" task.
Iterations and Design Changes:
The decision was made to adopt the Scenario 2 design, prioritizing its efficiency. Further focus was directed toward ensuring the scalability of the UI pattern across the entire ERP application.
Final Design and Outcomes:
The conclusive design choice was Scenario 2, recognized for its superior usability and efficiency in the context of enterprise project management.
