Twitter's Value Proposition

Figure 1: Top-20 query set similarity, or, what fraction of the top-20 queries, separated by a time lag (X-axis) are the same. (Click figure to enlarge).

Figure 2: Frequency of occurrence of a query name vs. the number of unique query names in the top-20 (hourly) query sets over 6 months. About half the queries appear twice or once (Click figure to enlarge)

Twitter has become the latest darling Silicon Valley start-up. What started off as a seemingly incremental idea ("micro-blogging" in 140 characters or less) seems to have caught on big time - more people are twittering than ever before. Twitter has shown its prowess in everything from influencing the American presidential election to challenging Iranian theocracy. Its popularity makes it a very compelling service, but how can it make money for its founders, Evan Williams and Biz Stone, and its promoters?

Needless to say, many Twitter posts (tweets) are inane (A dog bit me) instead of newsworthy (I bit a dog). Many users underestimate the difficulty in producing a constant stream of interesting 140-character long information texts from their everyday lives and experiences. Fortunately, Twitter comes with a search engine optimized to index tweets in real-time. So users can query Twitter for useful information within the deluge of tweets being posted every moment. This makes Twitter a real-time application and a perfect vehicle for propagating news on the Internet. Product releases, reviews, security bugs and vulnerabilities, company press releases, executives' and analysts' statements, etc. previously had to wait for a search engine to crawl and index them on the Internet (a lag of weeks sometimes). But because the tweets first come to Twitter directly from users, these announcements are instantly indexed and available for search via Twitter's search engine.

Twitter provides a great API to study queries returned by Twitter over a time period. The API allowed me to download information about the top-20 most-popular queries submitted to Twitter in every 1-hour interval over 6 months. Parsing this information sheds light on what users search for when they go online looking for time-sensitive information. It also suggests ways of monetizing this vast treasure of users' mind-space - what they are think, search, and find as time goes by.

Figure 1 shows top-20 query name set similarity, or, what fraction of the top-20 queries, separated by a time lag (X-axis) are the same. Twitter reported the set of 20 most-popular query names in each hour. The figure is plotted by finding the fraction of common elements between any two such sets separated by a certain lag (X-axis). The similarity quickly dies within 48 hours, and after 14 days, it settles at about 0.1 (meaning only 2 of 20 query names remain the same between the compared query name sets). There is also a noticeable drop in similarity at 24 and 48 hours (probably due to periodicity effects). Also note the diurnal bumps. Why does that happen?

Figure 2 plots the frequency of occurrence of a query name vs. the number of unique query names in the top-20 (hourly) query sets over 3 months. The majority of top-20 queries captured users' interest for a very few hours. Only about 20% of the queries remained in the top-20 lists for more than 10 hours in the 6 month period.

But what does all this mean for making money via Twitter. Well, here are my 2 cents:

1. Marketing benchmarks: Companies can use Twitter query-popularity to measure their marketing success. Say, how much has an advertising campaign been able to enter a customers mind measured in terms of query frequency (say as compared to their competitors). Twitter can develop and sell analytic tools for companies to measure such stuff. In my opinion, looking for this information in user queries is more effective than looking in the tweets themselves because the latter can be gamed (spam tweets) and because tweeting users are still a minority of people passionate about posting messages (as compared to the silent majority that does not post). Monitoring product mentions in query names is also a great way to keep tabs on marketing success. For example, in Figures 2 and 3, a product name occurring frequently is good news, but if the rate of occurrence decreases over time then its time to launch more marketing efforts or to improve the product's visibility in some way.

2. Real-time Customer feedback: Product groups can use Twitter query information to pinpoint product bugs, fast. There is a certain cost for a user to go on the Internet and search Twitter for say, "iPhone screen blank". If such a query bubbles up in query popularity (say, a top-XX query), then the bug is most certainly a widespread issue. Twitter's real-time feature highlights problems very quickly and efficiently. Selling such product specific query information to companies may create a nice revenue stream for Twitter.

3. Keyword Analytics: The occurrence of a product name with another query word may signal a selling opportunity. For example, the query "iPhone anti-virus" may point to market demand for Apple to sell anti-virus software with its iPhone. Keyword analytics can also be used to help in choosing keywords for online advertising.

4. Risk Management: Twitter quickly captures the viral spread of information on the Internet. This could allow a company to react to, say, a malicious video posted about its product on You-tube. Twitter is your fast-response Internet guardian. For example, Twitter can offer a service to subscribed companies that notifies them about any information (positive or negative) gaining traction on Twitter.

And finally, here is a list of Twitter search strings that were in the 20-most popular search lists for 100 hours or more (over a 6 month period). Funny how Apple dominates the top-3 slots, and then there is AT&T on the 4th slot (probably due to selling the iPhone in the USA). Hats off to Apple's marketing to have captured so much of users' mind-space. Or are they gaming Twitter? Or are Twitter users disproportionately Apple fans? Or is Twitter the newest Apple rumor spreading mechanism?

iphone	3266
ipod	1715
apple	1711
at&t	1253
itunes	981
goodnight	975
tweetdeck	881
vegas	680
bbc	674
new york	645
texas	643
obama	529
star trek	503
gaza	491
lakers	485
susan boyle	484
swine flu	460
sxsw	452
slumdog millionaire	448
watchmen	437
iranelection	427
h1n1	398
dollhouse	394
american idol	382
tgif	364
musicmonday	359
easter	354
wolverine	331
kobe	330
adam lambert	320
windows 7	320
valentine's day	281
aig	275
tehran	271
swineflu	267
bsg	259
sydney	256
super bowl	251
starbucks	242
palm pre	224
twilight	223
christmas	223
austin	217
mexico	216
coraline	212
california	212
cavs	206
nba	205
oprah	204
ces	198
ncaa	197
follow friday	195
spring	194
iran	192
wii	190
transformers 2	188
	188
miami	186
bush	183
paris	179
mousavi	177
snl	172
social media	171
spotify	170
superbowl	164
jimmy fallon	164
diddy	164
imax	160
hamas	156
inauguration	154
heroes	154
twestival	153
blackberry	149
brazil	148
macworld	147
earth hour	145
florida	144
lebron	143
happy new year	143
chuck	142
safari 4	138
snow	138
mj's	137
benjamin button	136
bing	136
chelsea	133
chris brown	133
gran torino	130
ellen	127
kris allen	126
nascar	125
teaparty	125
grey's anatomy	124
wimbledon	124
president obama	123
canucks	122
uksnow	122
ted	122
fridays	121
therescue	119
michael	118
michael jackson	118
conan	117
oscars	116
liverpool	116
celtics	115
angels & demons	113
march madness	111
denver	111
blackout	110
jay-z	109
google wave	109
dallas	107
memorial day	106
xbox live	104
gmail	102
player snapshot	102
north korea	102
mardi gras	102
melbourne	101
french	101
father's day	100

Sun setting over the German countryside

Took this one while sitting in the ICE from Cologne to Berlin, between Wolfsburg and Berlin Spandau railway stations. 01.01.2009, 21:01 CET.

The hope of green power ?

Is that Sun driving the wind driving the power-generating windmills producing electricity to power this ICE train cruising at 180 kmph? Please let this be the future for all our sakes.

Is P2P dead?

There have been some significant setbacks for P2P in the past year or so.

1. Pirate Bay's founders are in jail in Sweden for abetting illegal file sharing on their website.
2. Joost, the much lauded P2PTV service, is no longer P2P but is instead a CDN-type streaming service.
3. Another P2P darling, Skype, seems to be adrift with Ebay wanting to get rid of it through a sale or a spinoff.
4. There is a sustained reduction in CDN costs that is making the P2P cost reduction less attractive.
5. Websites like You-tube won the video streaming battle against P2P video streaming long ago, now websites like Rapidshare are leaving P2P file sharing behind as well.
6. Mobile P2P (near-network P2P on mobile devices over Bluetooth etc.) just didn't happen. These devices are more client-serverish than wired devices because upload bandwidth needed for P2P is too pricey.
   

So the key question is whether P2P is dead.

In my opinion, the answer is No. Here are some reasons

1. P2P lacks a business model but has proven to be a remarkably resilient and cost-effective technology. The problem is getting legal content on to P2P networks. Content companies are not going to let users take control of content delivery.
2. But if one is to look at where the biggest growth in broadband usage is going to be, one looks toward China and India. The legal protection for content is significantly weaker in these countries. Moreover, there is a large amount of reasonably priced content (e.g. regional and Bollywood content in India) that will perfectly ride P2P networks.
3. P2P has proven itself for voip (Skype has 400m users). Skype is the established voip leader and it will remain that way for a long time.
4. CDNs do not scale with video quality. That is why Youtube won't do HD - they'll go broke paying for CDN (server) bandwidth. P2P on the other hand can scale up to the extent the access networks allow.

What we may see is an amalgamation of CDNs and P2P technologies. For example, using CDNs for paid content and P2P for promoting the paid content (for free). Cache every recent movie's first 10 minutes on a user's computer using P2P, and then stream the content the user selects via a CDN. Although content pricing can recover CDN distribution costs, the monetary transaction only happens after a user selects to watch. Prior marketing can be P2P.

Pinchpoints in non-linear growth businesses

Non-linear growth business is best explained via examples. Twitter is a non-linear growth business because the number of users doubles every few months. Cellular service is a non-linear growth business because the number of cellular phone subscribers worldwide has doubled every year. Computer hardware is a non-linear growth business because the number of computers has doubled every few years and Moore's law has doubled the number of gates in a microprocessor every 2 years.

Naturally, such non-linear growth in a business attracts competition. For example, there are plenty of social networks vying for user attention and competing with Twitter. There is healthy competition in cellular service providers in most countries. Similarly, IBM has been supplanted in the PC hardware business by HP, Lenovo, Dell, etc. So the profit in a non-linear business can quickly evaporate due to competition. The pie is divided up. Capitalism and market forces 101.

But there are companies that build effective moats around their products and keep the whole pie. The moat is usually a combination of two components. One component is a standards and IPR protection for products. By creating a standardized and widely adopted product (e.g. MS Windows) companies can effectively keep others out. The second component is the high entry bar. A modern CISCO router ruitinely runs a 15-20m line operating system, making it incredibly hard for a start-up to join the fray.

CISCO operates on a pinchpoint, at the bottom of an inverted pyramid. At the top are all Web 2.0, Internet applications, and data centers. All the growth and innovation at the top makes money for CISCO. Intel is another pinchpoint and all PC manufactures compete at the top of the inverted pyramid. Truly smart are the business models that perch companies on the pinchpoints... the pie just keeps getting bigger!

RIMM and blackberries doing just fine

I remember thinking a couple of years ago that Apple iPhones were going to knock the Blackberry off the smart-phone market. But nothing like that has happened. RIMM released some very convincing quarterly results last week. Apparently RIMM seems to be doing just fine, never mind the slew of smart-phone entrants. On top of protecting its corporate user turf RIMM is fighting back with the Blackberry Pearl and Curve, taking the battle to the consumer smart-phone segment. Look out Apple (and Nokia), here come the smart-phone mounties!

RIMM has taken the Apple device threat very seriously. Their device line-up has been continuously updated and has kept pace with the latest cellular technology and device features - 3G, application-store, GPS, cameras - you name it and there is a Blackberry model covering the area. Even user interface, Blackberry's Achilles heel, seems to be getting a lot of attention on the newer models. Its a good thing RIMM is looking beyond push email and into actually building general purpose smart-phones. Smart-phones are good for the company's bottom line because the lions share of RIMM revenue comes from device sales. Users need to see glitzy, application laden devices to consider upgrading frequently. After all email by itself is only a text-based application (remember those monochrome Blackberries that handled email just fine). Fortunately, RIMM has not fallen into the Polaroid-like trap of living off one single success and has instead covered broad market segments with multiple device offerings and continious innovation.

Fortunately for RIMM, Apple has built a deep moat around its iPhone through aggressive pricing and exclusive deals with Telcos that keeps many corporate buyers in Blackberry's stable. Example: Small corporation X, 2000 employees, per unit iPhone cost minus per unit Blackberry cost = $250. Choose the Blackberry and you've saved half a million upfront, avoided iTunes (consumer software) being downloaded and installed on all corporate PCs, and escaped the clutches of very expensive iPhone data plans.

Blackberries are great texting/messaging devices with physical keyboards unlike iPhone's glass tapping virtual keyboard. This qualifies blackberries as the serious corporate tool as compared to the iPhone that has yet to fit into the corporate IT comfort zone. And this brings me to the greatest strength of RIMM - integration with corporate IT systems.

The key Blackberry value is effortless outsourcing of wireless/mobile email for corporate IT departments. No extra servers to run and no expensive data-plans with Telcos. Just a per-month per-account fee and you've got wireless email for employees. Since every other corporation (and Barrack Obama) trusts the Blackberry service, so can corporation X. Outsource email with confidence.

One of the greatest shortcomings of corporate PC email has been the culture of creating and maintaining individual email systems per corporation. This system could have been much cheaper, much better managed and less buggy had there been a uber-email provider like RIMM is for corporate wireless email. And no I don't buy the argument that corporate email needs to be kept inside the company's intranet for data security reasons. If this is so important then please forbid employee blackberries which otherwise bounce every email off a server in Canada.

Fortunately wireless email has remained concentrated in the hands of Blackberry due to its early dominance in this technology and this has made RIMM the single largest wireless push email provider. Everyone benefits, even competing smart-phone manufacturers who can simply buy the Blackberry service for their devices. And thats why RIMM is a buy.

Why bonuses should not be curtailed in banks

There is a flurry of law-making activity in the US House of Representatives and in the US senate to impose upto 90% taxes on large bonus payouts to individuals. AIG's ill-timed bonuses have outraged the entire nation and have led lawmakers to to use this blunt blow to root out bonuses in all sick companies supported by taxpayer money. But the move is a populist one, and one that may have far deeper consequences than recovering a few hundred million in bonus money. I am afraid there is a lot to lose with this last minute decision of making a law limiting bonuses. The taxpayer will actually end up loosing much more in the long run.

In many companies bonuses are given across the organization to all employees. A bonus for a good quarter, a bonus for Christmas, etc. But in knowledge-intensive fields bonuses are disproportionately distributed and given a small subset of employees who create exceptional (significantly over the average) value for the company. Knowledge-intensive fields include finance and banking where a talented person can create a lot more value than another lesser talented person. The distinction between employees is easily measurable in banks (earned profits) and therefore banks can easily reward better-performing employees. The reason for the reward is the hope is that a better performing employee can be retained in the company if she is paid a bonus.

A modern bank's success is almost wholly dependent on its employees. Banks are no longer armies of clerks following a rule book and doing the same thing every single day. Instead, banks are participating in a very sophisticated knowledge-based global game where bankers make decisions on how to invest their depositor's money. They employ some of the brightest minds to compete with other bright minds of other financial institutions, all of them trying to maximize returns on invested money. Quite naturally, the financial institution with the most creative, innovative, smart, and industrious people will take it all away.

Not all minds are equal. Hence all bankers cannot be paid the same income. Its a mind game, and intellect is not equal across the board in every employee (perhaps unfortunately, but certainly by natural design). Bankers are men and women like the rest of us. They will gravitate to financial institutions which pay better. By removing the bonus incentive for bright people in ailing financial institutions we are clearly reducing the chances of these institutions recovering taxpayer money as the brightest will leave or be less motivated.

Bonuses have been wrongly maligned in this whole saga. They are one of the best tools to improve worker productivity, commitment, and a company's bottom line.

East Berlin

And a not-so sunny day, this time the view of East Berlin.

Firefox: Finally, a credible threat to Google?

Today there is an extremely interesting article in WSJ about Firefox's threat to Google. No this is not about Firefox vs. Google's Chrome browser. It is about a small feature in recent Firefox browsers that stores web-pages accessed via the browser off-line and presents this locally-stored content as search results to users as they type words into the Firefox browser's address bar. Such local caching is not a new concept, but the nice interface that Firefox provides is certainly a huge improvement. The article says that local caching of data will eventually erode the high number of search requests going to Google, thereby reducing the value of its search engine. Local caching makes sense for users who get their (locally available) content faster, can access cached content when off-line, and do not have to pay for bandwidth. It makes sense for the content providers that do not have to serve the same content multiple times to the same user, and it makes sense for ISPs whose (flat-rate) bandwidth is spared.

But before content can be cached locally, it needs to be searched and found (at least once) on the Internet...and there is a good chance Google will be employed to do this first search. Still, from my experience I will say that I often search the Internet for the same information. With a convenient locally accessible cache, I would cut the number of times I go to Google. Moreover, there is talk in academic circles about "leaf computing", the opposite of cloud computing. The idea of leaf computing is to build distributed computing platforms. For example, a bunch of like-minded individuals can self-organize their computers into a distributed, specialized web-crawler that goes and searches the Internet for relevant information to index, or syncs up the saved book-marks/local Firefox caches of several users. If there is enough bandwidth and storage on the 'leaf' computers, then why not move away from cloud computing and into a distributed leaf computing architecture?

From a Google Ad-sense point of view, I think Google may modify its search engine usage terms to have the right to serve ads even on locally cached content if this content was initially found using Google. Others may argue that all the value is delivered to a user the first time search results are used (like buying a song: you pay the same no matter how many times you listen), and so Google cannot serve ads for locally cached content. Does Google have the right to serve ads on locally stored content that it initially found? This is analogous to the TIVO advertisement issue of whether advertisement slots bought for a live show can be removed and other advertisements inserted for later recorded playback.

Sunshine in West Berlin

This is why I love today!

View from the TEL building. 1 minute ago.

Wiring up my cellphone to my Web 2.0

I have done 3 fun things over the past 2 months. First, I bought a very capable text.phone - the Nokia E71. Second, I connected my phone to my blog so I can send an email to my blog and it gets posted. And third, I set up a Twitter account where I can post messages via SMS.

Now this changes things. Suddenly I am untethered like never before. The ability to capture and publish information in real-time has moved from professional TV camera crews riding satellite TV vans to anyone with a phone that can send SMS.

That's change for the better. And one more example where tech and gadgets leave the nerd domain and change mainstream society. Information flowing freely, leaking through the porous gaps made possible through such emerging technology will loosen the grip of those who seek to control information, and by extension, the free will of people.

Blackberry's application store foray

Blackberry is joining the app-store party by launching its own application store for the the Blackberry platform. There is a nice developer website and lots of buzz around RIM's latest move to take on the growing popularity of iTunes's app-store and the upcoming Nokia Ovi app-store. It is going to be an uphill battle catching up with the iPhone for all these me-too app-store ; but the Blackberry platform is a slightly different beast and its app-store positioning is quite different for some reasons.

First, most Blackberries are corporate property with significant organizational IT control over the devices. RIM may be positioning the developer tools and app-store for corporate IT development rather than for encouraging college-kid hackers who are trying to make a fast buck or two by writing a small game etc. This may explain why the Blackberry developer license ($200) is more expensive than the iTunes developer license ($99). In my opinion Blackberry isn't nearly looking to equal the number of iPhone applications. Its looking for serious business centric stuff on its app-store.

Second, Blackberries are not fun devices. When I see a Blackberry I start thinking of my consultant friends, complete with black suits and polished leather shoes! The Blackberry application consumer is going to be very unlike a 14-year old teen touting yet-another facebook widget on her iPhone. Instead, its going to be a corporate IT vice-president who likes an activity logging tool for the Blackberry to keep tabs on employees. Or an executive who downloads an extension of pocket-Excel for say, better readability. Bottomline: serious business applications.

Third, Blackberry apps will have to "work" much harder to gain the trust of potential downloaders. Blackberries carry confidential data and compromising this data could put the owners business (and/or job) at risk. Imagine the consequences of a software trojan that opens a connection to a server, dumps the contents of the Blackberry, and then blackmails the user or her organization? Certified and branded software applications have a much better chance of acceptance in the Blackberry user space.

Lastly, the Blackberry back-end is a unique add-on for developers. The "push" technology back-end of RIM can be used to create innovative applications on the Blackberry that may not be possible on other platforms. Question is, what is the (other) killer application for RIM push technology?

Social Network Regulation

Telecommunication companies unbundled the local loop in the past decade to comply with government regulation designed to spur competition in broadband services. The regulation was enacted for allowing competitors to use the last-mile copper, which belongs to incumbent telecommunication companies, in return for a fixed (small) fee. For example, Germany has dozens of broadband providers that use Deutsche Telekom's infrastructure to reach all customers that DT reaches. A customer can switch from DT to another provider if she prefers. Therefore, the governments motive of creating competition in the marketplace has succeeded.

Now lets fast-forward to 2015. Social networking has become ubiquitious. The inevitable shakeout, M&A and standardization forces have linked up all social networks into one huge giant network. People communicate primarily over social networks. The search engine has become second to this the human Gaia - the social uber network. A collective virtual intelligence, call it society 2.0, overlaid over the society we know. Last time a technology, voice telephony, became the flagship mode of communication, government stepped in to regulate it, control it, and overhear if needed. Before last time another technology, snail mail post, became the flagship mode of communication, government stepped in to regulate it, control it, and read it if needed. And so it shall be with be with social networks.

How and when will social network regulation come about?

Reinventing the software development business for the mobile world

Here is Pinch Media's presentation "AppStore Secrets", a data analysis of about 30m downloads of from the Apple iPhone application store. There were several insights in the presentation. The first thing that struck me was the relatively short lifespan of mobile applications compared to PC based applications.

Consider this: According to the presentation, most mobile applications seemed to "die-out" within days. They lie unused, or worse, they are uninstalled within a few days. Not even months. Compare that with Microsoft's MS Office, which usually lives on a PC for its entire lifetime. Usually a new MS Office user increases her usage over the months and years after purchasing it. While MS Office is expensive relative to mobile applications, the former's "per-run" cost may actually be lower than app-store mobile applications.

Fortunately the number of man-hours going into simple mobile applications for things like storing recipes and writing notes is quite small and most applications seem to be coming out of garage startups. I think this will change and we will see the emergence of a conglomeration of developers (e.g. a large mobile development company or even a developers' cooperative) who can amortize the risk and costs of mobile development. This may curtail rags-to-riches stories of 'lone star developers' as success will also be amortized. Still, the synergies of a large developer group working together, sharing code and risk make this a valuable preposition.

High level, reusable and modular tools that can be used to compose applications easily should emerge as the winners for mobile application prototyping and development. Developing everything in C++ from the ground up will not be economically feasible. Instead development tools like Python for the Symbian platform and .NET tools for Windows Mobile will become popular with developers. Given the variety of platforms, a cross-platform tool will have a huge advantage. In my opinion, Adobe's mobile development tools like Flash Lite are well positioned for such high-level and cross-platform mobile development.

iPhone AppStore Secrets - Pinch Media

Sirius XM satellite radio almost went bankrupt. A lesson for paid cellular mobile video?

Satellite radio seemed like a good idea. Crystal clear, ad-free, broadcast distance-agnostic , high-quality audio content for those long commutes, all for about $10 per month. But apparently Sirius-XM, the 800-pound gorilla of US satellite radio, is in dire straits. It avoided Chapter 11 bankruptcy after Direct TV propped it up with funding, probably at the price of being taken over by Direct TV.

But what does this say about users' appetite to pay for content while they are on the move? If paid radio did not fly, then will paid mobile video not fly either? We have a simple analogy here

free FM : paid Satellite radio :: free Digital TV (e.g. DVB-H) : paid cellular mobile video

?

Its tempting to write up the requiem for paid cellular mobile video based on the satellite radio example. But here are some things going for cellular mobile video

1. Mobile VoD allows users to select what they want to see, unlike satellite radio, which is a broadcast medium.
   
2. Cellular video is truly untethered because it is consumed on the mobile device. On the other hand, satellite radio is not carried in the pocket - it is usually built into car entertainment systems. No extra equipment is needed for cellular mobile video unlike satellite radio.
3. Video is inherently a richer and more engaging medium than radio.
   
4. Telecom companies are going to sink a lot of marketing effort into making mobile video a success. Paid mobile video service is one of the most significant use-case for high speed wireless networks. And perhaps, the best escape pod from the dumb-bit-pipe scenario that telecom companies want to avoid.

Still, users' tepid response to satellite radio should worry cellular mobile video hopefuls.