IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?
IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?
08 February 2017

INTRODUCTION

Having delivered dozens of projects in the last 5+ years, here at Bamboo Apps, we do believe that there are no boundaries to perfection: every good app can be turned into an amazing one. While working at Škoda LittleDriver app we were very well aware that a well-designed App Store page (icon, screenshots, app description) would generate a good amount of organic installs. But when your designers present several screenshots variants, how can you be sure which one will perform the best? That’s the case where A/B testing steps in and Bamboo Apps team decided to test drive this modern approach to see how it works. For this experiment, we partnered with SplitMetrics team - a platform that creates App Store pages and tracks user activity to tell you who’s the winner.

HYPOTHESIS

By doing ŠKODA LittleDriver split testing experiment, our team wanted to find out if changing the order and the background of App Store screenshots has any impact on the number of downloads. The primary goal was to determine the combination that provides the highest conversion figures.

THE TEST

To conduct A/B testing for a page of ŠKODA LittleDriver on App Store we used SplitMetrics service. Since it’s a self-service the only thing we had to take care of is screenshots sets. Our design team created three variations for testing the hypothesis. The first one implied the reordering of existing App Store screenshots by making Drive mode the main screenshot. The second option differed with an alternative background: app screenshots were displayed on devices held by users’ hands. For the third set, we used a picture of the road as the background to draw user’s attention to the app’s capability to simulate every component of a vehicle in real-time.

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

SplitMetrics automatically imported the data from ŠKODA LittleDriver store page, we added new combinations and got an experiment link. Then we launched a targeted ad campaign on Facebook to make sure we get only target audience clicking through the alternatives. Since the page looks similar to the App Store we assume that user behavior is similar and we could use the results for the real environment. After finishing all designs, we directed part of our advertising traffic to the landing pages and began to collect stats.

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

RESULTS

The experiment had been running for 30 days, we brought slightly over 1200 visitors distributed between the alternatives. Here are the results we got from SplitMetrics.

Summary

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

The first alternative (re-ordering) with Drive mode as the first screenshot saw 6.4% increase in app installs as compared to the control variant. The third alternative outperformed the original variant as well - 4.4% increase in app installs. The second alternative was an outsider with the lowest conversion rate of 33.3%. The first variation recorded a robust 16,6% increase in conversion (clicks on ‘Install button’) from the second alternative (lifestyle) and a 10.3% increase in clicks through other screenshots. The difference between the winning and losing alternatives made us think that the order and the background of App Store screenshots are quite influential for customers’ decision on whether to install an app or not.

The percentage of users that clicked the ‘Install’ button.

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

Why did the first variation (re-ordering) win? Last research shows that by default we don’t have much time to impress the user: analysts say that user attention span has reduced to eight seconds. The user doesn’t want to look through multiple screenshots or read long descriptions, he wants to see ‘what your app does’ captured at a glance. It looks like this simple rule applies to our case: put your most important screenshot that reflects the functionality and the purpose of the app first. As evident from the figures above, custom background and complicated design elements can distract user attention and even irritate him: alternatives #2 and #3 got fewer installs.

The percentage of users that scrolled screenshots.

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

Let’s have a look at the screenshots scroll rate. On average it’s 17% across all alternatives. It means that only every sixth visitor wanted to explore more than one screenshot when visited ŠKODA LittleDriver page. This fact does highlight the importance of placing screenshots in the right order.

As A/B test results proved, the name of the app and description is secondary, and most of the time not even taken into account: only one user read the full app description on the App Store (SplitMetrics says they see only 1% of users reading the description across all experiments).

CONCLUSION

On the results of ŠKODA LittleDriver experiment, we’re thoroughly confident that changing the order and the background of App Store screenshots had a robust impact on conversion rate. Our recommendations for ŠKODA Auto a.s.:

  • Use the following screenshots order on the app page in App Store: Drive mode, then ‘My car’ view, then Garage view screen, then Learning mode.
  • Avoid complicated backgrounds and structures in screenshots design.

Following these recommendations may lead up to a 16.6% increase in ŠKODA LittleDriver installs.

The winning alternative.

IS A/B TESTING A GOOD IDEA FOR A BRANDED iPAD APP?

Just keep in mind that there’s always room for many new discoveries in the conversion optimization space. Keep testing, measure and get the most out of your ad campaigns, App Store featuring and organic traffic.

This may interest you

Meet Bamboo Apps at ConCarExpo 2018
Meet Bamboo Apps at ConCarExpo 2018

For the second year in a row Bamboo Apps is excited to attend ConCarExpo, the largest European trade show for automated driving, connected car, and mobility solutions, held on 27-28 June, 2018 in Berlin, Germany.

Read more
How to build product-thinking mentality within your team
How to build product-thinking mentality within your team

This January managers and designers who have recently joined Bamboo Apps are attending the intensive course “UX & UI Design. Product-centric Approach” held at the British Higher School of Art and Design.

Read more