BACKGROUNDHaving delivered dozens of projects in the last 4+ years, here at BAMBOO GROUP, we do believe that there are no boundaries to perfection: every good app can be turned into an amazing one. While working at Škoda Little Driver app we were very well aware that well-designed App Store page (icon, screenshots, app description) would generate a good amount of organic installs. But when your designers present several screenshots variants, how can you be sure which one will perform the best? That’s the case where A/B testing steps in and BAMBOO GROUP team decided to test drive this modern approach to see how it works. Fot this experiment we partnered with SplitMetrics’ team - platform that creates App Store pages and tracks user activity to tell you who’s the winner.
HYPOTHESISBy doing ŠKODA Little Driver split testing experiment, our team wanted to find out if changing the order and the background of App Store screenshots has any impact on the number of downloads. The primary goal was to determine the combination that provides the highest conversion figures.
THE TESTTo conduct A/B testing for a page of ŠKODA Little Driver on App Store we used SplitMetrics service. Since it’s a self-service the only thing we had to take care of is screenshots sets. Our design team created three variations for testing the hypothesis. The first one implied the reordering of existing App Store screenshots by making Drive mode the main screenshot. The second option differed with an alternative background: app screenshots were displayed on devices held by users’ hands. For the third set we used a picture of the road as the background to draw user’s attention to the app’s capability to simulate every component of a vehicle in real-time. SplitMetrics automatically imported the data from ŠKODA Little Driver store page, we added new combinations and got an experiment link. Then we launched a targeted ad campaign on Facebook to make sure we get only target audience clicking through the alternatives. Since the page looks similar to the App Store we assume that user behaviour is similar and we could use the results for the real environment. After finishing all designs, we directed part of our advertising traffic to the landing pages and began to collect stats.
RESULTSThe experiment had been running for 30 days, we brought slightly over 1200 visitors distributed between the alternatives. Here’s the results we got from SplitMetrics.
Summary The first alternative (re-ordering) with Drive mode as the first screenshot saw 6.4% increase in app installs as compared to the control variant. The third alternative outperformed the original variant as well - 4.4% increase in app installs. The second alternative was an outsider with the lowest conversion rate of 33.3%. The first variation recorded a robust 16,6% increase in conversion (clicks on ‘Install button’) from the second alternative (lifestyle), and a 10.3% increase in clicks through other screenshots. The difference between the winning and losing alternatives made us think that the order and the background of App Store screenshots are quite influential for customers’ decision on whether to install an app or not.
The percentage of users that clicked the ‘Install’ button.
Why did the first variation (re-ordering) win? Last researches show that by default we don’t have much time to impress the user: analysts say that user attention span has reduced to eight seconds. The user doesn’t want to look through multiple screenshots or read long descriptions, he wants to see ‘what your app does’ captured at a glance. It looks like this simple rule applies to our case: put your most important screenshot that reflects the functionality and the purpose of the app first. As evident from the figures above, custom background and complicated design elements can distract user attention and even irritate him: alternatives #2 and #3 got less installs.
The percentage of users that scrolled screenshots.
Let’s have a look at the screnshots scroll rate. On average it’s 17% across all alternatives. It means that only every sixth visitor wanted to explore more than one screenshot when visited ŠKODA Little Driver page. This fact does highlight the importance of placing screenshots in the right order.
As A/B test results proved, the name of the app and description is secondary, and most of the time not even taken into account: only one user read the full app description on the App Store (SplitMetrics says they see only 1% of users reading the description across all experiments).
CONCLUSIONOn the results of ŠKODA Little Driver experiment, we’re throughly confident that changing the order and the background of App Store screenshots had robust impact on conversion rate.
Our recommendations for ŠKODA Auto a.s.:
1. Use the following screenshots order on app page in App Store: Drive mode, then ‘My car’ view, then Garage view screen, then Learning mode.
2. Avoid complicated backgrounds and structures in screenshots design.
Following these recommendations may lead up to a 16.6% increase in ŠKODA Little Driver installs.
The winning altenative Just keep in mind that there’s always room for many new discoveries in the conversion optimization space. Keep testing, measure and get the most out of your ad campaigns, App Store featuring and organic traffic.