logo image

ATD Blog

Big Data: Should You Trust an Algorithm?


Tue Jan 22 2013


In “Big Data: The Management Revolution” by Andrew McAfee and Erik Brynjolfsson in the Harvard Business Review, the authors argue that big data is far more powerful than the analytics of the past. Executives can measure and therefore manage more precisely than ever before. They can make better predictions and smarter decisions. They can target more effective interventions in areas that so far have been dominated by gut and intuition rather than by data and rigor.

Big Data: Should You Trust an Algorithm?-670f9d51a78c7aeb0d90491196c10b98edc6c438e133e586475fc6e5aeaa1648

While this may be true, there are still some who believe that intuition is indispensable to really good decisions based on large amounts of information. In December 2012, The New York Times Science section profiled Dr. Gurpreet Dhaliwal, a master diagnostician who impresses his peers at medical conferences by taking bits of information – symptoms, lab results, imaging – and homing in on a diagnosis. He occasionally checks himself using a diagnostic checklist program called Isabel. The program has yet to offer a diagnosis that he has missed.


His goal in demonstrating his talent for diagnosis is to “elevate the stature of thinking” in medical care. “Because in medicine, thinking is our most important procedure.” He relies on intuition to separate the signal from the noise in the information about a patient.

Steve Lohr, who writes about technology, economics, and innovation for The New York Times, shared his qualms about big data in a December 2012 article “Sure, Big Data is Great. But So Is Intuition.”

The enthusiasm for quantitative methods, he writes, began with Frederick Taylor’s “scientific management” more than a decade ago. ”The problem is that a math model, like a metaphor, is a simplification. …In so many Big Data applications, a math model attaches a crisp number to human behavior, interests, and preferences. “ Take note, those of you who are using big data for workforce planning and talent management.

Lohr cites recent books for those seeking more insights into big data.

  • Models Behaving Badly, by Emanuel Derman

  • Keeping Up With the Quants, by Thomas H. Davenport (not yet released)

  • The Filter Bubble: What the Internet is Hiding From You,  by Eli Pariser

You've Reached ATD Member-only Content

Become an ATD member to continue

Already a member?Sign In


Copyright © 2024 ATD

ASTD changed its name to ATD to meet the growing needs of a dynamic, global profession.

Terms of UsePrivacy NoticeCookie Policy