Food & Wine Thursdays: Confirmation Bias

Every once in a while I re-subscribe to Wine Spectator for the sole purpose of making sure my United Airlines miles don’t expire. The venerable Speculator has been saying some interesting things in its editorials recently. Their columnists, principally James Laube and Matt Kramer, write like they’re just on the verge of acknowledging that the 2013 wine market is radically different than that of the 1990s only to pull the rug out at the last minute.

In the most recent issue, for instance, Kramer rightfully acknowledges that a more affluent Generation X and the more wine-oriented Millennials are both good things for the wine market but seems convinced that these buyers are going to eventually turn to triple digit trophy wines in the same way as their overcompensating parents. Will they turn away from cheap wine in favor of fine wine? Sure. But those fine wines will be those amazing $30, $40, $50 bottles, not $500 bottle Bordeaux. Kramer either sloppily conflates or deftly sidesteps that distinction.

But enough about Kramer who is, I believe, the only writer at Wine Spectator who is fighting the good fight, and lets turn instead to their resident silver fox, James Laube.

Hey girl. You want to hop on my Harley and take a trip up to a Vermont B&B?

A couple issues back, Laube wrote an editorial about his magazine’s self-described vaunted blind-tasting protocol. How Wine Spectator rates wine is inherently superior to most other publications because all of their tasting is done without knowing a wine’s producer and it is down in a controlled tasting environment in their offices rather than on the grounds of the winery. According to Laube, this prevents “confirmation bias,” which he then describes (in a way that suggests he just looked it up himself) as the tendency of people to favor information that confirms their beliefs.

I’ve always been confused by Wine Spectator’s blind tasting system and how a true blind tasting could routinely result in the same wine regions–Napa, Sonoma, Bordeaux, Burgundy, Tuscany, and Piedmont–as the only regions whose wines routinely are awarded point scores in the 95+ point upper echelon. There are, of course, exceptions, but they are rare.

But in his article, Laube peels back the curtain just a little bit. Wine Spectator‘s tastings are blind only inasmuch as the producers, and virtually nothing else, are unknown. Region, vintage, varietals, are all known to the tasters. In short, they are just as easily susceptible to confirmation bias, only on a broader level.

Obviously you can’t have a fully blind tasting. Blind tasting Cabernet Franc against Pinot Noir or Riesling against Sauvignon Blanc is pointless, but why not blind taste all the Pinots against each other, regardless of region? Where is the harm in that? It was that type of mostly blind tasting in Paris in 1976 that first thrust California wine into the international spotlight. How soon we forget.

The inherent confirmation bias in Wine Spectator’s methodology is apparent in another Laube-penned article, his profile of Mendocino County’s Anderson Valley. Anderson Valley has long been one of my favorite wine regions in California, having discovered the region not because of its wine but because of its beer, with the wine as a pleasant surprise. After what was a very good, thorough, and engaging article on the history of this important region, Laube was overall dismissive of the region’s viability as a world-class wine region, with exception of its Pinot Noir, and the scores reflected that perception.  His reasons? Not many stated other than the region’s climate variability.

So if Laube is going into a blind tasting of Anderson Valley wines with the belief that the Anderson Valley doesn’t produce world class wines other than Pinot Noir, how are we to believe that his scoring isn’t also tainted by confirmation bias?

I suppose you could argue that it was his thorough tasting of the wines from the Valley for this feature that formed that opinion, but if after 30 years writing about wine this was his first foray into a comprehensive tasting of Anderson Valley wines, that would make him a pretty shitty wine journalist.

And even I’ll give a Wine Spectator writer more credit than that and just chalk it up to confirmation bias.

I’d provide links to the articles cited, but Wine Spectator keeps almost everything behind a pay wall and makes you pay for an online subscription even if you have a print subscription. If you want to read the articles, you might check the restroom at your friendly neighborhood wine shop.

About David D.

I'm a wine professional. Like a real one who makes most of his living in wine and have for most of my adult life. I also write, but you can see that.
This entry was posted in I'm Just Sayin, Wine & Cheese and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.