Made with ScholarlyTurn your own files into documents like this
Reviewer-Style Summary: "The Unreasonable Effectiveness of Data"
Authors: Alon Halevy, Peter Norvig, Fernando Pereira (Google) · Venue: IEEE Intelligent Systems, "Expert Opinion," March/April 2009, pp. 8–9 · Type: Position/opinion essay (no new experiments)
Summary of the Claim
Simple learning methods trained on very large, mostly unlabeled Web-scale data can outperform elaborate, hand-built models. The authors argue the field should stop chasing elegant theories and instead embrace complexity and use the "best ally we have": large data.
Problem Addressed
Human-centered language tasks resist elegant mathematics, unlike physics. Hand-annotated corpora for tasks such as document classification, part-of-speech tagging, named-entity recognition, and parsing are slow, costly, and hard for experts to agree on.
Key Arguments (Lessons)
Lesson 1 – Use available large-scale data instead of waiting for annotated data. Useful semantic relations can be learned from search-query statistics and from Web text patterns and formatted tables, with no manual annotation.
Lesson 2 – Memorization beats elaborate rules when data is abundant. Simple n-gram models and linear classifiers on millions of features outperform models that try to discover general rules.
Threshold effect: Scene completion performed poorly with thousands of photos but well with millions.
Rare events matter: Discarding rare events is almost always a bad idea, because much Web data is individually rare but collectively frequent.
Use words, not abstractions: Latent clusters without linguistic counterparts are hard to learn or validate and lose information. Word co-occurrence statistics scale with data and parallelize easily.
Three orthogonal problems: representation language, model encoding, and inference are separate design choices, not a binary "deep vs. statistical" choice.
Evidence Offered
Mostly prior published results and observations, not new controlled experiments:
Google's 2006 trillion-word corpus with counts for sequences up to five words (about a million times the Brown Corpus, which has one million words).
Statistical speech recognition and machine translation, where large memorized phrase tables dominate.
Hays and Efros's scene-completion result across corpus sizes.
Cited work on WebTables, Web-derived query and snippet statistics, and Semantic Web interoperability (references 4–6, 11, 14).
Strengths
Clear, memorable thesis with concrete examples across several applications.
Reframes a real trade-off: data-driven scale versus theory-driven elegance.
Separates three design decisions that the "deep vs. statistical" debate conflated.
Weaknesses and Limitations
No new experiments or quantitative benchmarks in this piece; claims rely on cited work and anecdote.
Selective evidence: Success stories (speech, translation, scene completion) are tasks with naturally abundant data; the claim may not hold where data is scarce or annotation is the only option.
Noisy data is downplayed: The authors acknowledge the Web corpus has spelling and grammatical errors and no hand-corrected tags, but argue size outweighs this without quantifying the trade-off.
Threshold is vague: "A billion or so examples" is asserted, not measured.
Scale-dependence: Advice favors organizations with access to Web-scale data, a practical constraint the essay does not address.
Reproducibility concerns: Results depend on proprietary-scale corpora.
Context and Significance
Written by Google researchers, it argues against the 1980s-style hand-coded grammars and ontologies, and tempers the Semantic Web's promise: standards aid interoperability but do not resolve ambiguity in language or entity names. It remains a widely cited statement of data-driven NLP philosophy.
Key Takeaways for Review
Scale and simple models can beat complex rules when data is abundant.
Rare events and raw word statistics should not be discarded.
Separate representation, encoding, and inference when designing a system.
Treat the claim as an opinion supported by prior evidence; check it against task-specific data availability and noise.