Science Shows Something Hilarious About How Much the “Experts” Really Know About Wine

Impact

Knowing the difference between a good wine and a bad wine feels a lot like a sign of maturity. And while some people just like what they like, others turn to wine experts and sommeliers for recommendations. But do these "experts" even know what they're talking about?

According to the best available data: no, at least not nearly as much as they'd probably like you to think. Though countless competitions and publications turn to highly trained experts to sift through the millions of gallons of wine bottled every year and rate each one, statistically speaking, they're no better at picking the best from the worst than the rest of us. 

For years, many in the wine world believed that certain wines were simply always better; that certain vineyards, or even countries, would always produce a better product. However, in 1976, a wine competition pitted classic French vineyards against a few California wines — which were, at the time, generally considered sub-par — and to everyone's shock, California came out on top. That historic day has since become known as the Judgement of Paris and began to plant the seeds of distrust in the general consensus of the global wine community (it was also immortalized in the 2008 movie Bottle Shock).

But fast forward about 30 years and you'll find some of the first real scientific examinations of this question. Around 2005, a retired oceanographer-turned-wine enthusiast named Robert Hodgson began taking a closer, more statistical look at some of the results of his local wine competition and quickly noticed an unusual trend: Only about 10% of wine judges in a field were able to consistently rate the same wine with the same score. Hodgson published his first study in 2008; followed by another in spring 2009 that took a broader sample and demonstrated wines rated highly in one competition could just as easily be rated terribly in another; and followed by another in winter 2009 that examined what a constituted an "expert" wine judge and found that less than 30% of judges could be considered "expert." Taken as a whole, Hodgon's deceptively simple statistical analysis paints an interesting picture. 

"The results are disturbing," Hodgson told the Guardian. "Only about 10% of judges are consistent and those judges who were consistent one year were ordinary the next year. ... Chance has a great deal to do with the awards that wines win."

Of course, experts in the wine industry weren't exactly pleased with Hodgson's findings — "they say I'm full of bullshit but that's OK" — but a number of other studies and analyses have arrived at similar conclusions. 

In 2001, French academic Frédéric Brochet attempted to determine whether a wine's bottle could affect the perception and taste of the wine. In his study, Brochet found that when unwittingly presented with the same wine in two different bottles (one extravagant, one plain), tasters across the board greatly preferred the nicer bottle and described the "two wines" very differently. In that same study, Brochet served experts one red wine and one white wine and asked for their impressions. What the "experts" didn't know was that the "red" wine was just the same white wine with added flavorless food coloring, but that didn't stop them from describing the two wines very differently. 

In 2008, American food and wine critic published a study in the Journal of Wine Economics and found that in 6,000 blind tastings, individuals "who are unaware of the price do not derive more enjoyment from more expensive wine," meaning that expensive wines are not necessarily better. In fact, some tasters enjoyed the more expensive wines slightly less. It was only those with exceptional training who could detect any differences.

In a similar 2011 study, Professor Richard Wiseman of Hertfordshire University found that in a study of 578 people, individuals could only tell the different between a wine costing under $8 and one costing over $16 about 53% of the time for whites and only 47% of the time for reds. You're just as likely to pick something you like by flipping a coin.

And even when wine experts are brought in and a situation is carefully created, the results are just as wild. To recreate a bit of the magic of 1976's Judgement of Paris, a similar Judgement of Princeton was held in New Jersey in 2012. Unfortunately for America, several of the French wines came out on top, however, they only narrowly beat the New Jersey wines and some judges rated certain wines best that others rated worst. On his blog, Marginal Revolution, economist Tyler Cowen points out that based on the analysis of the Princeton professor Richard Quandt, all the wines were "statistically indistinguishable" and if the blind tasting were held again, a Jersey wine could just as easily win. 

All this is to say that wine basically tastes as good as you think it does. Considering the complex psychological factors of price and enjoyment can be tricky, but when wine is involved, it's better just to pick something that's simple, not too expensive and delicious, regardless of what the "experts tell you."

Of course, that won't stop some people from spending their entire lives devoted to this deliciously fermented fruit juice.