APIs are a topic dear to my heart and part of the fun of moving to California and silicon valley is that I have been able to really start to see and understand the actions of some of the bigger internet companies a little better. Something that I have never understood is why a company would build an API, only allow developers a couple of thousand calls a day (which believe me is pathetically small) and then leave those API's out there relatively unsupported for a significant amount of time.
A quick analysis shows all pure natural search APIs are highly restricted... eg. Google and Yahoo with only a few thousand calls a day each. Why is this?
As I see it the question is one of upside for the search engine. What can we do if you or me have access to their algorithmic data on a large scale. A non-exhaustive list of negative suggestions for the search engines include back engineering their algorithm or tracking SEO performance by seeing which sites perform well for which terms and what variables are true for those sites, producing copernicus and dogpile tools (speaking of which why does dogpile still exist???) which don't monetize with their adverts and finally competitors finding ways to compare and improve their rankings. These all carry significant risk. Conversely I don't see many uses which give benefits to the search engines: syndication onto people's sites in conjunction with contextual adverts is certainly one, cool analytical tools which attract positive PR in general and in their recruitment pool is another and reducing the incidence of scraping of their results is another example.
The search engines do not support their natural search APIs well at all and since there is significant risk and little opportunity to exposing them I don't see this changing in the near future. The same cannot be said for their revenue driving tools. Panama by Yahoo is a really significant tool that they say they have spent years developing and which will have major support. It plugs an API straight into their paid search advertising... this drives revenue... this is good. SImilar can be true for the adwords API and adsense API.
I think it is clear that search engines promote and support the APIs that offer them more revenue however there are certainly areas where this is not true... look at the unseemly battle over maps, look at geocoding and more. In the next post I will explore why companies would offer these random Web 2.0, bandwidth eating, non-revenue generating tools to grow successfully.