Cocktail Recommendation: Part 4 - the results

Wow did it work or what! The cocktail recommendation engine I produced earlier this year has been a fabulous success. I set up an AB test of the cocktail recommendation by randomly assigning my visitors to either see the cocktail recommendation engine or not and the main target was to reduce the bounce rates of visitors to my cocktail recipe pages (both buckets contain c. 1million page views).

The above graph shows you the overall impact of the test across all my cocktails with 100days of data. This impact is diluted because of the volume of cocktails for which I don't have enough data to produce cocktail recommendations so below is a list broken out by cocktail id (I have removed the names since work people see this blog and some are naughty). The cocktails in the list below are essentially the cocktails found as the first 10 here (you are warned... that page contains rude words) under the tab "Graph of cocktail recipes in order".

On average for the top 10 cocktails the recommendation engine reduced the bounce rate by 16% with some cases (esp. slippery nipple) having an impact as high as a 31% reduction in bounce rate for the page.

Comments

[Keith Mander]

Nice job Alex! I wonder if it would be possible to next measure the $$$ impact of having stickier content. Are people who spend more time on the site more likely to click on the ad? On the one hand they might be more engaged in the content and find the ads more relevant. On the other hand, they may have their needs satisfied by your site and be then less likely to click elsewhere.

In the long run, it's surely better to have more engaging content so that users come back, recommend the site, link to it etc. - but I'm curious as to whether there is a short term financial loss.

Thoughts?