The Real Time Search & Social Search Landscape (from SMX East)
Monday, October 4th, 2010The following is a recap from a session at Search Marketing Expo (SMX) East. Follow the conference and session on Twitter.
Realtime Search, from Google
Deep investment in search technology
- Instant – brand new technology that finishes your search query as you’re typing
- Caffeine – new indexing technology
- Social – public content authored by people you know
- Realtime – scrolling search of any updates online (Twitter, Facebook, MySpace, etc.). To realtime search, go here or click the updates option in Google:
- and hundreds of changes you probably didn’t notice

History of realtime
- Launched in December 2009
- Launched Replay in April 2009 – search back in time across Twitter
- Currently pull from Twitter, FriendFeed, Facebook, and more
Realtime Search options
- Time (Any time or Latest)
- Location (Anywhere, Nearby, or Custom Location)
- Images (All updates or Updates with Images)
Jeremy Hylton, Software Engineer, Google
Realtime Search, from Twitter
Overview of Search at Twitter
- Infrastructure
- A highly efficient inverted index (based on Lucene)
- Over 1,000 tweets inserted per second
- Over 12,000 queries per secnd (>1B queries per day)
- Used to power
- Twitter search API (mobile, apps, widgets, etc.)
- Search on Twitter.com
- Numerous services on Twitter.com (ex. related tweets)
Examples of types of Twitter searches
- Follow an ‘event’ in realtime
- Address a hyperlocal questions
- Communicate with a realtime community
- Vanity searches (@mentions)
- Standing search around an interest
- Gadgets integrated on a website (e.g. mentions of an article)
Different relationship between search engine and information
- Traditional search = routing
- Realtime search = filtering
Pothole theory: it’s all about context
- Current focus
- Location
- Communities
- Trends
- Events
What does ranking mean in the realtime world?
- Impact of ranking is different because of time factor in consumption
- Dynamic and relative concept of credibility
- In traditional search, credibility is relatively static and uniform
- In realtime, people’s interests are expressed naturally
- Time effect is critical/micro-trends
- Context, context, context
What in PageRank’s sake does this mean for SEO?*
- Traditional SEO: ‘Hacking’ the search algorithm
- Realtime SEO: ‘Hacking’ human interest
- Realtime traffic perhaps closer to gamin dynamics than search optimization?
Perhaps think of objectives differently?
- For websites, traffic is a proxy for goodness – action is on the website
- In realtime world, engagement may be better proxy – action is more immediate
- Is SEO for realtime more about creating engagement?
*Isn’t PR synonymous to ‘God’ in search world?
Potential metrics that matter
- Follower counts
- List inclusions
- Retweets
- Trending topic
- Clicks/pageviews
Othman Laraki, Director, Twitter
Realtime Search, from Microsoft Bing
Microsoft has done a lot of research around searching vs. asking your network and the implications of each. Below are questions people actually ask, based on type, topics, and motivation.
Questions: Types
Questions: Topics
Questions: Motivation
Answer comparison: search engines vs. Facebook Bing and Ping Social Networks & Search engines Search engine “friends” offering answers to questions
Meredith Ringel Morris, Microsoft Research
Realtime Search, from Yahoo
Social/Realtime Activity
- Yahoo! users are linking their Facebook/Twitter accounts in record numbers
- Twitter integrate into Y! search
Industry Trends – MS-DOS lives on Twitter brand is mass-market, but content often isn’t… Use the “Mom” test. (Does your tweet make sense to the average person – the Mom?) Massmarketing consumption patterns are emerging:
- “Trending now” aggregation
- Algorithmic curation of linked content
- Easily identifiable and filterable service auto-tweets and retweets
- “XSLT for Twitter” ought to be the norm, not the exception
Industry Trends: Rivers are hard to tame Most realtime search experiences target ‘professionals’ Changing consumer behavior is a barrier for mass adoption
- Only limited user value when looking through a pinhole
- Thematic clustering and time-series analytics provide a good proxy
What have we learned?
- Celebrity tweets have higher user engagement than News
- Needs to be easier to find/coverage expanded in-SRP
- ‘Influencers’ rule!
- Signal/noise is sub-optimal and ML ranking is hard
Leading vs lagging indicators of realtime “buzz”
- Realtime “buzz” detection can be derived from multiple signal sources
- Realtime “buzz” detection needs to include a local event baseline
Tweets only an advertiser would love
- Challenge: filter out a handful of intensively positive tweets from 100s of millions in realtime with ultra-high precision
- Solution: Y! Deep science over a deep basket of tweets with many sentiments with filtering




