Real Real-time Search will be Socially Transforming

ML Feb 19, 2007

It seems my most popular blog posts are the ones that step outside HitTail’s normal topics. For example, my Web 2.0 post about Sun’s dream that “the network is the computer” has been quoted many times. My last post about the culture war is being copied and commented upon.

The current Internet buzz about HitTail shows that it’s being viewed as a blog topic suggestion engine. That’s fine. Nestled between the two familiar concepts of blogs and the long tail is the easiest way to introduce HitTail to the world. But satisfying that need to view log files and provide real-time analytics-like features is turning out to be so much more. Stephen Colbert called the USA Today the USA Yesterday, which made me think that’s pretty much how Analytics works today. You post. You wait. You see a few interesting details about web traffic yesterday.

But in today’s real-time world, where you have to see the opportunity and seize it, the next day may be too late. And it certainly doesn’t satisfy the need for instant gratification. Bloggers live in the moment. Bloggers want to see the results of their good work within moments of their post. Waiting a day is too long. We in the search optimization business are increasingly aware that real-time search engines, such as Digg, Technorati, Sphere and anything that constructs on-the-fly newspapers from he state-of-the-Internet at the moment are a trend almost as important and influential as search engines themselves. This is causing change.


People using social news sites like Digg are living in the moment. They are flocking from story to story, and It is changing expectations. Google’s default search results are feeling a little like the USA Today. The Web moves ideas so fast, that yesterday’s news is pass√© to the newsorati. They derive their sense of eliteness from being tuned into the pulse–the Internet zeitgeist, as it were. I’m one of these geeks, getting picture-free news-stream through RSS feeds on my phone. So it’s ironic that the home of the acknowledged zeitgeist is last week’s news.

Imagine if you will performing a search in Google. You get the comforting top-10 list that you’re used to. But under the “more” link, it offers you “real-time”. If you answer yes, you could be taken to a color-coded relief map representing everything that’s being published on that topic right at that moment. You would see the landscape change literally the way the Internet landscape itself is changing. If something new gets posted while you’re reading, then something new appears here while you’re watching.

Some day in the near future, this will take the form of a virtual world, but today while we’re still in flatland, it’s will appear as an ebb and flow of articles that are being published, right now at this moment on the topic. Literally, as fast as someone types the blog is as fast as it appears in the “results”. In fact, for those who have keyworded their posts ahead of time, you may even be able to see them as they type, and ask to interject and perhaps do a few instant message exchanges to see if you can’t check a fact for them or give them a more rounded view. In this way, real-time dynamics allows journalistic integrity and accountability to creep back into the public discussion. I fee that the time-delay of today’s crawl-and-index model has actually contributed to some social problems, because bad information can “set in” like a stain.

Such a new system has problems, of course. But technologies like OpenID will tear away the veil of anonymity and unaccountability, where the most important blogs have genuine identities behind them. Mobile phones with full-fledged video cameras will help us all chip away at injustice. And real-time ability to see things being published and respond will raise the bar for fact-checking.

Today, with HitTail, someone with an opinion, fiercely determined to state it and get the word out, cannot rely on the real-time-ish blog search tools such as Google Blogsearch, Technorati or Sphere, because they have just not hit mainstream yet. When I say mainstream, I mean having the same reach as Google default search. Instead, they have to live with the time-delay.

But if you only knew what headline to start with to tap into all the search traffic that is already occurring on those terms… and if you could pick ones that are no so competitive that you shouldn’t try, but can still reach your audience… well, that would be a different story.

One response
  1. Avatar
    Jeff James

    Monetizing this new distribution is going to become a challenge. From Google’s perspective, when the day arrives where people are turning to more real time and disagregated sources of information from more “independent sources.”

    I think the final analysis will prove though that Google will manage well if every blogger with traffic enough chooses to publish on their blog. But who knows, when we reach the point where information is so broken up that perhaps the top 5 distribution networks will have less than 50% of the search volume.

    In any event, ranking for all words, not just larger ones will be of paramount importance…personalized search has begun.

Leave a Reply

Your email address will not be published. Required fields are marked *