Mining the Thin Air-for Understanding of Urban Society.

We explore the potential of crowd-sourced information on human mobility and activities in an urban population drawn from a significant fraction of smartphones in the Los Angeles basin during February-May 2015. The raw dataset was collected by WeFi, a smartphone app provider. The dataset is noisy, irregular, and lean; however, it is large scale (over a billion events), cheap to collect, and arguably unbiased. We employ the state-of-the-art Big Data techniques to turn this structurally thin dataset into semantically rich insights on commuting, overworking, recreational traveling, shopping, and fast food consumption of the Greater LA population. For example, we reveal that Greater LA residents commute substantially longer than what is reported in the US census data. Also, we show that younger individuals dine at McDonald's significantly more than the older population does. Our results have implications for public health, inequality, urban traffic, and other research areas in social sciences. The large number of phones participating in our "crowd" makes it possible to obtain those results without the risk of compromising individual privacy.

Evolution of language: An empirical study at eBay Big Data Lab.

The evolutionary theory of language predicts that a language will tend towards fewer synonyms for a given object. We subject this and related predictions to empirical tests, using data from the eBay Big Data Lab which let us access all records of the words used by eBay vendors in their item titles, and by consumers in their searches. We find support for the predictions of the evolutionary theory of language. In particular, the mapping from object to words sharpens over time on both sides of the market, i.e. among consumers and among vendors. In addition, the word mappings used on the two sides of the market become more similar over time. Our research contributes to the literature on language evolution by reporting results of a truly unique large-scale empirical study.

Quantifying the web browser ecosystem.

Contrary to the assumption that web browsers are designed to support the user, an examination of a 900,000 distinct PCs shows that web browsers comprise a complex ecosystem with millions of addons collaborating and competing with each other. It is possible for addons to "sneak in" through third party installations or to get "kicked out" by their competitors without user involvement. This study examines that ecosystem quantitatively by constructing a large-scale graph with nodes corresponding to users, addons, and words (terms) that describe addon functionality. Analyzing addon interactions at user level using the Personalized PageRank (PPR) random walk measure shows that the graph demonstrates ecological resilience. Adapting the PPR model to analyzing the browser ecosystem at the level of addon manufacturer, the study shows that some addon companies are in symbiosis and others clash with each other as shown by analyzing the behavior of 18 prominent addon manufacturers. Results may herald insight on how other evolving internet ecosystems may behave, and suggest a methodology for measuring this behavior. Specifically, applying such a methodology could transform the addon market.

A Data Scientist's Guide to Start-Ups.

In August 2013, we held a panel discussion at the KDD 2013 conference in Chicago on the subject of data science, data scientists, and start-ups. KDD is the premier conference on data science research and practice. The panel discussed the pros and cons for top-notch data scientists of the hot data science start-up scene. In this article, we first present background on our panelists. Our four panelists have unquestionable pedigrees in data science and substantial experience with start-ups from multiple perspectives (founders, employees, chief scientists, venture capitalists). For the casual reader, we next present a brief summary of the experts' opinions on eight of the issues the panel discussed. The rest of the article presents a lightly edited transcription of the entire panel discussion.