Data Cloud. Yury Lifshits Yahoo! Research http://yury.name. My Beliefs. The key challenge in web search is structured search Part 1: What is structured search? The key challenge in structured search is collecting data Part 2: Data distribution & idea of Data Cloud
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
The key challenge in web search is structured search
Part 1: What is structured search?
The key challenge in structured search is collecting data
Part 2: Data distribution & idea of Data Cloud
Part 3: Demo: numeric data distribution
The key challenge in collecting data is incentive design
Part 4: Economics of data distribution
Data = data of entities + data of contentData
“what's the value of property X of object Y“
Structured object search
"all concerts this weekend in SF under 20$ sorted by popularity"
Structured content search
"all videos with Tom Brady"
“all comments and blog posts about Bing"
Reality stream, sensors
Market graph & signals
Web indexSearch as a Platform
How to collect all structured data in one place?
Data distributor is any technical solution to accumulate, organize and provide access to structured and semi-structured data
Data publisher: the original distributor of some data
Data retailer: a consumer-facing distributor of some dataData Distributors
Data Cloud is a centralized fully-functional data distribution service
Success metric for data cloud strategy = the total “value” of data on the cloud
Joint work with Paul Tarjan
URL + XPath + regex
Anyone can create a numbr
Joint work with Ravi Kumar and Andrew Tomkins
Two sided market = every product serves consumers of two types A and B
Cross-side network effect: the more type-A users product X has, the more attractive it is for type-B consumers and vice versa
Examples: operating systems, credit cards, e-commerce marketplaces
Two-sided network effects: A theory of information product design
G. Parker, M.W. Van Alstyne, N. Bulkley, M. Van AlstyneNetwork Effect in Two-Sided Markets
Market shares will stabilize
With super-liner preference rule
one of distributors will tip
With sub-liner preference rule
market shares will flatten
Preference rule with external factor:
If all market shares are below 1/sqrt(k)
coalition (sharing data) is profitable for
Coalitions are not monotone
Example: 5 : 4 : 1 : 1
Challenges types A and B
Thanks! types A and B
Follow my research: