Project
My Role
Akbank is one of the largest banks in Turkey. Akbank also achieved significant success by ranking as the 126th most valuable banking brand in the report that comprises the most valuable global banking brands.
​
They were going through a major revamp of the Internet Banking Mobile App and for the transaction screens, wanted to test two different flows with a well-conducted User testing and A/B Test.
​
​
​
My responsibilities as a UX Designer and Usability Specialist were creating working prototypes, recruiting users that fit our personas, planning, conducting, and analysis of the A/B Test and Usability Test and prepare a heuristic report, and present it to the client.
Brief
Challanges
As we were comparing a flow that users were quite accustomed to a flow that is totally new and unfamiliar, the main challenge was to analyze results with taking into consideration that users might be selecting the familiar one because they were accustomed to the flow, and not because the second design is not successful in terms of the experience it provides. To distinguish the difference between two cases, and present it to the client was the biggest challenge.
The brief was to find out the more useful, easy-to-learn, and efficient flow and interface within the two different designs. For transaction screens in the mobile app, they wanted to create two different flows, as the first one was more similar to the old money transfer flow, and the second one was a completely different flow for users, as before selecting any kind of transection option, asking for the amount to be transected.
Personas
We started the process by creating different personas for the mobile app and designated the ones that would be using the transection feature the most.
Planning the A/B Test
Recruitment
4 Days
4 Days
2 Days
0,5 Day
5 Days
8 Days
Completion
Ø1
Ø2
Ø3
Ø4
Ø5
Ø6
Ø7
Presenting to Client
Analysis & Documentation
Test Setup
User Tests
Pilot Test
Prototyping
Ø1 Prototyping
For two different flows, the design team of Akbank created two interfaces. By using Proto.io, I turned those designs into working prototypes.
Ø2 Recruitment
For each persona group, we decided to test 8 users representing the persona features. So, for each design we recruited 32 users, meaning 64 users in total. I didn't work with a recruitment agency for the project. All of the users were selected from our user database from former tests and our acquaintances.
Ø3 Test Setup
Test Setup phase was separated into 3 sub-phases as:
1. Defining The Scope & Preparing the Test Content
2. Equipment Setup
3. Scheduling & Arranging The Venue
1. Defining The Scope & Preparing the Test Content
We started prioritizing the features to be focused more during tests. I wrote a detailed A/B Test & Usability test plan, usability test script inculding usability test tasks will be given to users during test.
2. Equipment Setup
I specified and provided the pieces of equipment needed in tests such as iPhones, Android phones, GoPros, etc. I downloaded the prototypes I have designed for each mobile phone to be used in the tests and checked if all of them work correctly. Incentives are prepared in the name of each user.
3. Scheduling & Arranging The Venue
We wanted to test users in their natural environment. With all of the users, we scheduled sessions in an optimum to arrange sessions as much as possible in a day considering the transportation. For the users who wouldn't have a convenient space to take the test, we arranged the Usability lab.
Ø4 Pilot Tests
To check if all of the test content was clear to understand, pieces of equipment and prototypes were working well, and estimate the average time to prepare and conduct the test, pilot tests with 4 people were completed.
Ø5 User Tests
I moderated A/B Tests that took full five days with 64 users. My colleague helped me with setting up the pieces of equipment, checking if they were working well, and taking notes. Focusing on the transaction process, we observed any kind of problems they faced, gathered all the insights and comments from users.
Ø6 Analysis & Documentation
I started analyzing the test by estimating the average task completion time for each design. Below are the results calculated.
Design A
Design B
01:08 min
02:13 min
I prepared a detailed heuristic report pointing out the usability problems detected during tests, pain points, unclear parts, and wordings for both interfaces. Also, explained in detail why the task completion time for Design B was way higher than Design A. Problems detected were prioritized according to the frequency of occurrence and the importance of completing the task. For all detected usability problems, design solutions were presented in the report. Below, is an example from the heuristic report prepared.
