Domain experts are subject matter experts who are an authority in a particular area or topic. These experts are everywhere in the business world, agriculture, and everyday life. Through experience and thoughtful reflection, they make decisions that others don’t have the capabilities to make.
One of the books for my graduate course on data ethics (DS 760) had a section titled “The demise of the expert” (Big Data by Kenneth Cukier and Viktor Mayer-Schonberger (2013) ). The book pulls its primary example from MoneyBall, the well-known book and movie that tells the story of how the analytics based Oakland A’s baseball front office outperformed its bigger rivals. Along with other examples, the authors make the case that gut based decisions made by experts using latent knowledge in a small data world can be replaced with analytics based decisions in a big data world. Put simply, these domain expert jobs are at risk in today’s world.
In contrast, I like to think about “the rise of the data informed experts”. While I agree that there are situations where enough data combined with an unwillingness or inability for a domain expert to change will results in lost jobs, there are is even more opportunity for experts to make even better decisions. The goal is a better together scenario which combines data and subject matter expertise.
You don’t have to look any further than baseball to find examples of the rise of the data informed expert. There’s no doubt that all teams now have analytics departments of significant size. But teams are finding that applying analytics alongside intuition gained from years of baseball experience is optimal compared to just analytics or just experience. For example, one of 2018’s surprise teams, the Atlanta Braves, combined “old school feel and new school analytics” to drive improvements (link).
Another change in baseball is the type of people that are hired for leadership positions in the organizations. For example, baseball managers are expected to understand and embrace analytics in addition to the having an in depth understanding of the game and strong communication skills. Two of the league’s leading organizations, the New York Yankees and Boston Red Sox, hired managers for the 2018 season with no previous manager experience due to the combination of these skills. These data informed experts led their teams to 100 win seasons and a world series championship for the Red Sox.
The rise the data informed experts require two critical elements – analytics that are interpretable and experts that are data fluent. Yet another case from baseball is former major league player, Sam Fuld. Fuld was hired to take the information created by the Philadelphia Phillies analytics department and make it consumable for players (link).
While I’ve pulled the examples from baseball, there are examples of data informed experts in all industries. Put simply, data informed experts need to become as common as domain experts. While it will take time for analytics to become more interpretable and experts to become more data fluent, better together scenarios are our best opportunity for making great decisions.
Picture details: Lida Sunrise, 12/10/2018, Canon SD4000, F4.5, 1/400, ISO-1600