During my last semester at Cal, I took a graduate course called Needs and Usability Assessment in the School of Information. The class was a whirlwind tour of UX research methods, how to conduct each type of study, and where they best fit in the product development process. The course was taught by a Director of UX at Salesforce.com, and was one of the most industry-oriented course I took at Cal – a huge change from the very academic neuroscience and psychology classes I had become accustomed to. The instructor taught through stories and personal experience with each research method, highlighting the pitfalls, strengths, and applications he had observed. I loved this class, and a lot of my personal philosophy about UX comes from his anecdotal advice.

The methods and topics covered in this class included:

• Ethics and recruiting practices
• Reporting and designing user research
• Usability studies
• Field studies and observations
• Ethnographic research
• Interviews
• Diary studies
• Focus groups
• Contextual inquiries
• Expert review
• Competitive analyses
• Surveys
• Heuristic evaluations
• Card sorting tasks
• Usability and accessibility

For many of the methods that we covered, we had corresponding assignments where we had the chance to conduct those studies ourselves. As part of our final project for the semester, we had to pick at least three of these methods as part of an entire usability assessment.

For this final project, I was in a group with three other students who I had worked with on  field study and heuristic evaluation exercises before. When choosing a project topic, we wanted to work with something that many people had experience with and had access to. We picked the Bay Area Rapid Transit (BART) system, a train system that runs through East Bay and Peninsula regions of the San Francisco Bay Area. All UC Berkeley students are given a Clipper card with their student IDs, which you can charge and swipe to pay for rides on most public transit in the Bay Area. This made finding participants (and experiencing the subject matter ourselves) much easier than some of the past projects or assignments we had in the class (i.e. contextual inquiries for Adobe Illustrator).

Personally, I enjoyed working with something as widely used as public transportation, as it meant my subjects would come from a variety of ages and educational, cultural, and economic backgrounds. Variance along these dimensions makes for complex data (as we saw in our results for this project), but also makes for an interesting design challenge that pushes my problem solving skills. While there was no design component to this assignment, it foreshadowed the challenge of finding a best solution for such a conglomerate of users. Because of this assignment, I would like to, one day, do usability and UX work in the public sector.

Studies

When my team first sat down to discuss this project, we came up with a fairly large and general list of questions we wanted to address. These questions ranged from ticketing, to the actual trains, to station quality, and looked at the entire BART riding experience. We decided to start the assessment by getting a feel for the users and how they interacted with stations and trains physically. The first study we ran was an observational field study, where me and group mates stood in the Downtown Berkeley BART station near the entrances to the trains for one hour at 5 p.m. From here, we each wrote observations following the AEIOU framework, making note of riders’ time at the ticketing machines, whether they used Clipper cards, luggage, traffic, etc.

After looking at our notes collectively, we realized that the scope of our project was far too big, and we would not be able to tell one cohesive story with the time and resources we had; we were asking too many questions about too many different parts of the BART riding experience. We went back to our observations and decided to focus on only the ticketing aspect. It was a step in the BART experience that riders spent a lot of time on, and had more autonomy over. They could pick how to pay for their ride, as well as what type of ticket they got, making this step more active and participatory on the riders’ part.

To further focus our study, we chose to look at riders who did not use Clipper cards. While this contradicted one of our initial reasons for choosing BART, we realized that Clipper card users had a simpler experience that did not require them to interact with the ticketing machines as much as with paper tickets. Since part of the assignment was to run three different types of studies, we wanted to make sure we could still find substantial results and insight into the ticketing process which was unique to BART.

Using our results from the field study and changes to our initial research questions, we came up with a formal description for our project, with BART as our client.

• Client Goal: Provide reliable and fast public transport for the general public
• Problem: Outdated ticketing machines
• Research Goals: 
  • Evaluate current machine usage and experience
  • Find potential improvements to make ticket buying faster and more intuitive
• Target Users: Riders who buy paper tickets to ride BART

Keeping this project specification in mind, we moved on to our second study – cognitive walkthrough. We asked four subjects to walk us through their ticket buying process, looking for what actions seemed memorable, intuitive, or problematic. At each step, we asked follow up questions, and realized that subjects seemed to only notice the options they were interested in, and did not pay attention to other features offered by the machines’ UI. We also started noticing patterns on payment methods and ride frequency/habits, which we explored further in our next study.

For our final study, we chose to interview riders on their riding experiences and habits, why they chose to buy paper tickets, payment methods, and what issues they encountered. We designed a 12 question interview protocol and interviewed six subjects, two of which were from outside the Bay Area and had not used BART before. These interviews were a chance for us to delve deeper into behaviors we saw in the field study and cognitive walkthrough, and to answer any questions we still had.

Key Findings

The first trend we saw across our data was a distribution between frequency of use and payment method. Paper tickets can be paid for with either cash or card, and can be used multiple times by recharging (much like a Clipper card). This resulted in a 2×2 grid that we could categorize users into.

These behaviors can partly be attributed to the fact that once a rider has bought a ticket, there is no way to get that money back, and riders must use any remaining balance towards their next ride. If they do not have enough for their next ride, they need to add more. While subjects liked having both payment options, many complained that paying the exact amount for rides on the machines was a hassle, because those amounts were uneven and the machines only give change in coins.

In analyzing our observations, we tried to group our data into the two metrics that we wanted to improve based on our second research goal: speed and ease of use. With respect to speed and time spent at the machines, many of our subjects were unhappy with the fact that when using card to pay, the machines default to adding $20. If a rider wants to change that, they can only increment the amount $1 or $0.50 at a time. So if a rider wants to pay for a single ride of $4, they would have to press a button 16 times.

When asked about ease of use, many subjects complained about how disconnected the ticketing process was from train routes and stops. The machines are not always located near a map of the BART system, and while there is a chart that lists costs to the various stops, it lists them alphabetically rather than geographically. This is especially problematic to riders from outside the Bay Area, who are likely to plan trips based on cities or sites, rather than specific BART stops.

Suggestions and Possible Improvements

To address the concerns raised by our subjects, we next compiled a list of improvements that BART authorities could implement:

• Allow riders to purchase based on destination, instead of asking them to find and add specific amounts
• Integrate route maps into the UI at the machines
• Make exact fares easier to pay by making using whole, half, and quarter dollar amounts
• Change the $20 default charge when using card, and allow riders to either increment with amounts more than $1, or enter values manually
• Allow machines to give change in bills and not just coins