DESIGN

Overview

Based on the main user research insights, the team began to brainstorm ideas and solutions. After initial brainstorming, the team validated the main concepts with PT. With the initial concepts decided, prototyping and user testing began. With each round of user testing, ideas were refined further and generally the prototype fidelity level increased. Later stages of prototyping were focused less on concept validation and more on fine-tuning the visual design and interaction to facilitate quick, easy task completion.

Participants

We were able to refine our design based on the feedback of 17 participants. These participants were telecom managers of small, medium and large companies in both Madeira and Lisbon.

Overview of Participants

We began the design process by considering user needs and business needs. PT needs to promote a unified and consistent view of itself, to reduce the amount of calls that the call center receives, and to increase transparency about PT processes. Customers want to avoid calling the call center, prevent surprise billing charges, report technical issues quickly, and to know what is happening with requests they have made. We identified two common needs between PT and customers: both want to reduce calls to the call center, and both value transparency about processes.

Business and User Needs

After analyzing business and user needs, we began to brainstorm ideas for how to best to meet these needs, support the most important user tasks, and solve the breakdowns we identified in user research. Initially we did not dismiss any ideas and we generated as many ideas as possible. We narrowed these ideas into final concepts, which we depicted using storyboards. We presented these concepts to PT at our initial kickoff meeting.

With feedback gathered about the concepts, the team thought again about the most important tasks sequences to include in the Portal. The team came up with task flows that detail the screen sequence that a user navigates to achieve a goal. Based on the task flows, the team sketched low-fidelity paper screens to test with users.

Testing Methods

Interviews

We spoke with users before testing to find out more about their daily activities as a telecom manager, as well as figure out how comfortable they were with technology. The goal was to understand how a user's work context and personal characteristics might influence they way they interact with and understand the prototype. We also spoke with users after the test to validate and confirm our observations.

Think-Aloud Method

In all user testing sessions, we used the think-aloud method in combination with all fidelities of prototypes. We walked through a number of pre-defined tasks with users and asked them to verbalize their thought process as they navigated the prototype. If a user became silent, they were asked to "please keep talking." This method helps us understand why a user makes decisions on the interface, and helped us understand the discrepancies between our interface’s flow and the user's mental model of the task.

Heuristic Evaluation

The team conducted heuristic evaluation on various stages of our prototype.The goal of heuristic evaluation is to find the usability problems in a design so that they can be attended to as part of an iterative design process. In a heuristic evaluation, a small set of evaluators examines the interface and judges its compliance with recognized usability principles (the "heuristics").

User Testing

Paper Prototyping

The team employed low-fidelity sketches in this round. Low-fidelity sketches are rough sketches of screens with minimal detail. They are easy and quick to produce and elicit comments on the big picture of functionality rather than design details.

In the first round of user tests, users were presented with paper prototypes of four different navigations and asked to perform several tasks. The goal was to discover the user's mental model for organizing services and completing tasks, as well as their preferred navigation style. Based on the test results, the team merged the best qualities of each of the four different navigations and moved the fidelity to a higher level.

User Testing Round 1 - Paper Prototypes

Interactive PDFs

In the second round of user testing, Keynote was used to make interactive PDFs of the application screens, which were loaded onto the iPad. This level of fidelity allowed the team to evaluate the usability of the prototype on the iPad without translating the visual interface into code. At the same time, back-end development of the basic architecture of a functional prototype in iOS began.

The goal of the second round was to get more feedback on how users accomplish management tasks. A total of six users were tested on several tasks on the interactive PDFs. The team anticipated that the results would inspire further refinement of the navigation structure and further insight into the users' mental models of tasks. The user feedback from this round prompted us to make substantial changes in the organization of the interface.

Functional Prototype

The final round of user testing marked the official roll-out of the functional prototype. For this round, all of the visual design and the entire functionality of the application was coded in iOS and loaded onto the iPad.

Strict think-aloud methods were used in order to find usability issues with the PT Portal, now on a working iPad app prototype. The team sought specific feedback about the usability, interaction, and design details of the app. At this point, the features of the application were fairly set. The main goal was to refine the look and feel of the PT Portal.

User Test Round 3 - Functional Prototype

Findings

The final features and the tasks supported by the PT Portal have been validated and shaped by user feedback. We arrived at the following findings by analyzing the data from three rounds of user testing.

  • 1. A home page helps orient and update users.

  • 2. Each telecom manager categorizes services differently.

  • 3. Clients must be able to monitor charges.

  • 4. The repair request process must be quick, simple, and intuitive.

  • 5. Choosing from contracted plans saves time.

  • 6. Ticket statuses offer a new level of transparency.

  • 7. A usage overview helps clients manage resources.

  • 8. Communication about service requests is key.

  • 9. Search is preferred for navigating large amounts of data.

User Test Round 2 - Interactive PDF

Next Steps

Based on our findings, we have identified several steps to take to make the PT Portal an even better, more usable product:

  1. Permissions

    Allow for different levels of access depending on a user’s role in a company.

  2. My Services

    Managers can customize the service folders on the home page to fit their service profile. New services folder can be added, grouped, and custom-labeled.

  3. Batch Operations

    Requests to change usage allowances, plan details or any other aspect of services can be batched and executed in bulk for groups of employees. These bulk requests would be batched and condensed into one ticket status, instead of being displayed as separate tickets.