USABILITY TESTING 2013
Team: Andrew Elliot, Tamara Grigsby, Ann Huang, Zulka Ramirez
Tools: Morae
Methods: Questionnaires, Mobile Testing, Observation, Cognitive Walkthrough
Role: User Researcher
Final Deliverable: Test kit, testing results, and recommendations
Squarehub Family Sharing & Planning is a private social networking application that allows families to connect and coordinate in a fun and functional way. The Squarehub team requested a test of their new user registration process and feedback on other tension points within their application.
Our team recruited persona-appropriate participants and conducted ten in-depth usability tests, delivering our test kit along with a list of recommendations. This writeup summarizes our process; recommendations are private for customer confidentiality.
Based on discussions with Squarehub personnel, our research questions were:
- Which registration process causes fewer errors for first-time application users?
- Which processes do users prefer?
- What obstacles will a first-time user encounter during the process of creating an event?
Identifying and Recruiting the Target Audience
After meeting with two members of the Squarehub team, we narrowed down our target audience:
- a mother or father in a family of 3+
- in a household with at least two iOS devices
- have at least one child above the age of 8
- have at least one working parent
We built our recruitment emails and questionnaire based on these requirements. After filtering the responses, we found ten participants within our target audience.
Building and Running Comparison Registration Tests
Squarehub's test application had just introduced a new user registration process and asked for an evaluation of the newly designed process. Using their previous design as a baseline, we scripted a comparison test and used a post-task questionnaire (below right) to gauge preferences. In order to ensure that we also captured problem points not specifically called out by participants, we also used Morae (below left) to remotely take metrics on in-app user behavior. To decrease order bias, we started half of the participants on the old registration process and started the other half on the new registration process.
Event Creation Cognitive Walkthrough
As a family networking application, one important aspect of Squarehub is its ability to deal with the addition of calendar events. In order to capture insights on this process, we created a simple event and asked our participants to add the event to their newly created profile. While they did this, we asked them to do a cognitive walkthrough with us by engaging in the "think aloud" protocol so that we could hear their thoughts throughout the process. At the same time, we had a researcher remotely watch the user's gestures and facial expressions using Morae in order to capture moments of hesitation or confusion that were not indicated through vocal feedback.
Debrief Questionnaire
We also wanted to capture miscellaneous parent thoughts on family networking applications such as Squarehub. Despite our short two-month timeframe, we asked users to name their top three thoughts/ideas in three general categories that could effect Squarehub's direction.
Report Out
Our report to Squarehub used the Single Usability Metric to highlight the preferred user registration process and Morae logging to highlight the most troublesome aspects of each interface. We found that six out of ten users preferred the newer registration process despite the fact that the old process's average ease of use (4.4/5) was slightly higher than the newer one's (4.1/5). Our Morae results helped explain this discrepancy; though most of the newer process went seamlessly, one particular change from the original version caused considerable user confusion. In our report, we recommended reverting that change to its original implementation while keeping the rest of the newer process.
The event creation task led to a significant amount of information through our "think aloud" process. Users felt that although they completed the named task, they had not succeeded in conveying all the necessary information through the application. Their post-study questionnaires reflected this confusion and showed that a well-designed calendar would be essential to their usage of the application. We listed our participants' suggestions for new features by frequency and type of request.
New applications are difficult to launch, so our group reported a prioritized list of potential user interface and user experience improvements for Squarehub to deal with. With this, we included our full test kit and the data behind our recommendations.
