Alex Schultz

Author: Alex Schultz

  • Cocktail Recommentation Engine: Part 3 – The First Results

    The following table is a subset of the output results from the cocktail recommendation engine (be warned some rude words in some of the cocktail names). I have tried to drill the list down to just the best matches so bear in mind there are probably 30% more matches thrown out which appear to have statistical significance but I can find no reason for the relationship (beyond sometimes they are next to each other in the large cocktail A-Z list).

    Cocktail One – the cocktail someone is on, Cocktail Two – the cocktail I recommend, Probability – the probability that if someone is on cocktail one and visits another cocktail two will be that cocktail. There had to be at least 10 relationships in total recorded between cocktail one and another cocktail in my db for a cocktail to make this table.

    I am hoping that as I get more data this table will grow more accurate and I will have fewer “wtf?” moments looking at the list. I also want to add two more probabilities to the calculation:

    1. Probability if they visit 1 they will like 2
    2. Probability if I display 2 on 1 they will click on it

    Anyway for now enjoy looking at the cocktail relationships and expect more data soon.

    Get my marketing cheat sheets at Click Here

  • Huge steaming piles of data (Project Cocktail Part 2)

    Today has been quite challenging in project cocktail. The main issue is working out how much information to store and what granularity to summarize it on. Data is pouring in rapidly, I have defined over 5000 directional cocktail relationships and probably 10% of my cocktails within one day have at least one relationship I would define as relevant (I shall work on statistical significance fairly heavily later on).

    The issue is that every row in my database is currently taking up 46bytes and I am adding c. 7.5k rows a day. The index on the database is then adding another 25% to this and there is little I can do here since I am required to create a primary key. Therefore daily I am creating 0.4MB of data in unsummarized form. The machine I am using has 100MB of storage and so I could store up to 250 days of raw (unsummarized) data or 2.17million rows (handy cos right now my cocktail DB could theoretically create 2.17million different directional relationships with different likelihood factors).

    In the short term I am going to do nothing more than summarize on a month basis and depreciate the value of prior months over time but in the long term I want to know if recommendations should differ significantly by various user related variables and hence I want to store those variables so that in the future I can have a significant dataset to query and work out the impact of those variables. My target is to make this engine slightly scary at predicting what cocktail you might want to see next and to hit the scary threshold will take a little more than the #1 most likely relationship to deliver. So all things considered I have 0.5TB of data storage going spare right now in 5 mySQL databases… let’s go for the big data 🙂 and see what comes out!!!

    Tomorrow I will be helping move 100 rowing boats so that my new boatclub can undergo an awesome renovation over the next 3wks which means I will have to take a break (and let the data gather into an even higher volume). Hopefully Sunday I will be able to do some analysis and maybe even build a mock up that shows what the output could start to look like. For now goodnight!

    Get my marketing cheat sheets at Click Here

  • Recommendation Engine: Project Cocktail Part 1

    Ok so how does one build a recommendation engine?

    For the last year I have been tweaking improving and revising my cocktail recipes website with the prime goal of driving more traffic to the site. I have certainly acheived some of that with >1500 visitors daily on most days at the moment (still not huge but really massive growth). What is noticeable though compared to my paper airplanes site is a distinct lack of loyalty in my user base. With paperairplanes.co.uk ~32% of the visits daily are from returning users but with cocktailmaking.co.uk only ~9% are.

    I need to make the site better and so my first attempt is to produce a cocktail recommendation engine. Phase one of the project just released to the site and I wanted to chat about that a little bit here.

    I am logging every chain of two cocktails viewed by a visitor to my site within 2hrs. My thinking is that time spent and number of clicks browsing both degrade the correlation between cocktails in a user’s browse path and so two cocktails viewed back to back in a short time period are likely to be very closely related. If I observe this interaction over millions of such pairs (data I can gather in just a few months) the most correlated cocktail combos should stand out from the mix at least for the top 10% of cocktails viewed on my site. That is my thinking and I will start to share the results on here as the data gathering proceeds.

    The wonders of having a website with enough traffic to make analysis interesting 🙂

    Get my marketing cheat sheets at Click Here

  • SELECT … CASE expressions rock!!!

    CASE expressions are a huge timesaving SQL feature that I have just recently fallen in love with. The specific example was trying to find how the activity of a class of users has declined over a period of twelve months. Without CASE this would have required writing and executing 12 different queries and then joining them into a table either by joining all those queries together or by pasting them into (in this case) excel. That’s hard enough but what about 24hrs in a day for day parting analysis across 100000 keywords (far too big for excel <2007 that one ;) )

    CASE selects each of those different queries I described into a column using a clause something like:

    SELECT sum( CASE month WHEN 2006M10 THEN revenue ELSE 0 END) as oct06, sum( CASE month WHEN 2006M11 THEN revenue ELSE 0 END) as nov06, sum( CASE month WHEN 2006M12 THEN revenue ELSE 0 END) as dec06 FROM tbl_userrevenue WHERE userid IN (‘alex’, ‘beth’)

    Which in my mind is a pretty cool tool for analyzing anything where you want to look at discrete time intervals or users classes or almost anything else individually.

    Get my marketing cheat sheets at Click Here

  • Make Your Users Feel Awesome by Looking Good

    The last month for me has been heaven and hell. I have seen my mum and dad, seen a couple of friends from home, visited a geek fest in Austin, raced in San Diego, seen all my eBay friends and even been whale watching in Big Sur having driven the highway 1 down from Santa Cruz. On the other hand my workload has been high (by my standards), I have been on at least 7 flights in 3wks – 4 of them long haul and am entering a huge, high stress period of uncertainty at work. All of this has led me to

    For me it’s really clear that the experiences that make me feel awesome are those that either make me look good or do something that really pleases me and makes me feel good. For this post I am gonna focus on the looking good!

    Google maps is the example I want to use for looking good. Just a few weeks ago I installed it on my blackberry, I use it rarely due to the high bandwidth costs but on two occassions it has been awesome! The most recent occassion was while at a regatta and searching for a particular mexican restaurant in San Diego called the californian coyote. We knew the name but had no idea where it was and neither did anyone close by. Under a minute later using a combination of searching Google and using Google maps I had the address and a route there. When at the restaurant everyone celebrated the fact I got them there and I felt awesome.

    Google maps made me feel awesome by looking good and I am gonna keep using them because of that! Now how can I take that learning to my websites and quickly!!!

    Get my marketing cheat sheets at Click Here

  • Does mybloglog reduce adsense earnings?

    I have grown to really appreciate mybloglog, I think it’s a totally awesome tool. Also as an internet marketing professional I think that Yahoo! got mybloglog for a song. With the distribution this widget gets I am sure there are some quite incredible opportunities for mybloglog to earn millions a year in revenue. However 2wks ago I added mybloglog to my paper airplanes site to see if I could create a community around that site which would be totally awesome. With c. 150 000 unique visitors a month it seemed like a good idea.

    The above graph shows my daily CPM for my left hand adsense skyscraper since the start of 2007 (although I have removed the absolute amounts from the left axis). On adding the my blog log widget (directly below the adsense ad) the CPM dropped dramatically and interestingly as the below graph shows the CTR % stayed the same through that period so effectively for some reason my average CPC dropped.

    I don’t know if it is google detecting there are images close to the adsense ad, I don’t know if it’s mybloglog changing Google’s opinion of my page quality for the adsense crawl or what (it certainly isn’t people clicking on mybloglog and joining my community 🙁 – it didn’t work for paper airplanes, that’s why I took it off) but somehow there was a directly correlated change both when I added my blog log and when I removed it on my CPMs through adsense. I’d love to hear if anyone else has seen the same or if I am just a screw up 🙂

    Get my marketing cheat sheets at Click Here

  • Give up completely on CPM ad networks… NOW

    On my sites video advertising revenue is the big growth area. The following graph shows how my advertising revenue sources (% wise) have changed over the past two years (yellow – video, reddish purple – adsense, blue – cpm ads). I have made no changes to my use of “display” or CPM advertising (the blue area) but I have added adsense and video in completely new placements on my sites.

    You can see an almost perfect inverted ‘S’ curve as the revenue driven by display advertising has fallen as a proportion of my site revenue to only 20% from 100% in under a year. Video is now showing similar growth to Adsense and the overall pie is continuing to grow very nicely (up ~300% since Jan ’05). The question that comes to me now is… how worthwhile is it to show video ads, contextual ads AND banner ads?

    Digging a little deeper the average click on a CPM banner effectively earns me 1/3 of the average click on a contextual ad and something like 1/30 of a video ad plus I get approx 50% of the clicks I get on contextual ads on my CPM ads. So if these users were to click on adsense (at only the CPM click through rate) instead I could make 3x more revenue from those same impressions. It isn’t likely to be that great but even if they were 1/2 as likely to click on the adsense ads as the CPM ads the revenue would be 50% higher for those same visitors.

    A final point is this. Taxes are getting very important to my little business now that it’s getting to be a large chunk of my income and having a multitude of advertising partners is becoming painful on my tax returns so I want to consolidate my revenue streams and hence minimize my accounting costs. Things are looking bad for the lifespan of CPM ads on my site. With a bit of luck they will be gone completely by April and I will let you know the results of banishing them.

    Get my marketing cheat sheets at Click Here

  • Skype in Facebook contact details

    Yes I am partizan but facebook has finally put more than just AIM into it’s instant messenger list. As someone who had 12 skype windows open when I saw this information and who logs into facebook at least 10 times a day this is huge news… HAPPY DAY (oh yeah and sorry for the cheesey photo below, training camp, seville, 2005)!

    Get my marketing cheat sheets at Click Here

  • Sometimes I feel unworthy to be on the web

    Browsing YouTube and a few of my favorite blogs today I was quite simly blown away by what I was seeing and I wanted to share that experience here, 2 YouTube videos and a blog post. The first YouTube video really started it all off and it is called Web 2.0… The Machine is Us. For someone in the trade that’s a pretty obvious statement but I have never seen it expressed this well before… see below:

    The second really exciting video for me was a piece I saw in the comments of the previous video called black button. It reminded me a lot of the short plays the drama groups used to put on at school and really engaged me for the full 7minutes of the piece with just two actors talking. It also brought back memories of the play Copenhagen which I saw in london with my Physics class in my final year of school (and feel all scientists and engineers should be forced to see). Powerful moral content, strong acting and a simple set. Sheer brilliance:

    The final piece I wanted to share is humourous but simply amazing and that is a post from the blog indexed. On Indexed Jessica Hagy draws little management consultant like graphs to explain everything. She does it with humour and her tongue firmly in cheek but she really often hits the nail BANG on the mark. This post is about the relationship between accessibility and desirability but read the whole blog, go back MONTHS, you will be glad you did.

    So these three items… what do they have in common. Two things, 1. I loved them, really truly enjoyed watching them and thinking about them. 2. My web offerings don’t equal the talent of these individuals and yet I am positive earn me more money. That’s totally not fair and I hope that YouTube paying users a share of their revenue will rectify that, cos these guys deserve it!

    Get my marketing cheat sheets at Click Here

  • eBay Search URL Structure Explained

    One of the things I have found very hard as an eBay API developer (with two certified applications) is the eBay search results page URL structure. My latest project requires me to build more complex eBay search URL structures on the fly. Below I explain how to create eBay URLs on the fly based on my understanding.

    The search will return BIN only items, items available to certain countries, search title and desc, select a category, include a price range and show the eBay “welcome new user banner”. There are of course more variables possible but this is a good start.

    The link above is a search for a “4gb ipod nano product red” the query section of the search should be simple enough to spot. You replace the spaces with “-” and append the search string to the search.ebay.com.

    If you wish to then start exploring more complex variables you append _W0 to tell eBay search they are coming and then each variable is preceeded by QQ. The order of the variables appears not to matter.

    So there you have an outline of some of the main variables in the eBay advanced search URLs, this definitely prooved really useful for me to create my little application, I hope it’s useful for you.

    Get my marketing cheat sheets at Click Here