The Judgment of Beaune According to Borda

Just over three years after The Judgment of Paris changed the fortunes of California wines when several bested some of France’s best in 1976, Gallic pride on matters vinous suffered another blow. 

On October 17, 1979, Terry Robards reported in the New York Times, about the Olympiades Gault-Millau du Vin: “[u]nder conditions of deep secrecy last summer, some 300 wines from 33 countries were tasted and analyzed by 62 experts in the vast cellars of Nicolas, a French wine company, in Charenton, a Paris suburb… [H]avoc will erupt anew in the wine market when it becomes known how well some California wines apparently fared.”

But it wasn’t just California that did well.  Among the top ranked Pinot Noirs was the 1975 South Block Reserve from The Eyrie Vineyard in Oregon.

Incensed by the outcome, Robert Drouhin, owner of the renowned Maison Joseph Drouhin in Beaune, proposed a rematch in a letter to Messrs. Gault and Millau dated 12 November 1979: “As a French person, Burgundian, winegrower, cultivator of great wines, I was passionately interested in the ‘Wine Olympics’ that you organized, and terribly disappointed by the results…I am ready to challenge the winners of your Olympics in the Pinot Noir and Chardonnay categories with a selection of Burgundy wines from Maison Joseph Drouhin.”

The story continues in an article entitled “After the Gault-Millau Wine Olympics:  For the Honor of Burgundy, Joseph Drouhin takes up the Gauntlet”  that appeared in the Gault-Millau Magazine dated February 1980, issue no. 130 (translation by Chelsea Janzen): “This courage certainly deserved the response published in the letter from readers of our December 1979 issue…’Your idea is excellent and tempestuous – a taste for competition which we can only applaud. This showdown between our wines and those who have the best places at the Olympics must, of course, take place blind and under the supervision of a bailiff.'”

And so, on January 8, 1980, the magazine reported, Drouhin welcomed the tasting jury “in the Salle du Treuil [the old winch room where the wine press is housed] of the Parliament of the Dukes of Burgundy…opposite the collegiate church, in the heart of Beaune’s historical center…” There were two panels that included winemakers, restauranteurs, Masters of Wine, sommeliers, writers, and importers from several European countries and the United States.  The whites, mostly Chardonnays, were evaluated by Jacques Puisais, Jean Hugel, Max Leglise, J.-P. Morot-Gaudry, Georges Pertuiset, Steven Spurrier – who had organized the Judgment of Paris – Christopher Tatham, MW, Harry Waugh, MW, and Jon Winroth. The reds, mostly Pinot Noirs, were judged by Louis-Régis Affre, Robert Barton-Clegg, Phillippe Bourguignon, Georges Duboeuf, Odette Kahn – who had been on the panel for the Judgment of Paris – Franz Keller, Jean Lameloise, Piero Sattanino, Serge Tonneau, and Rebecca Wassermann.

Each panel tasted six wines from Joseph Drouhin and six from outside of France which had placed at the top of the rankings.  Each judge assigned a score from 0 to 100 to each wine as follows: “A rating of 1 for visual impressions (intensity, tonality, charm of color), a rating of 3 for olfactory impressions (intensity, finesse, frankness of aromas or of the bouquet), a rating of 5 for the impressions in the mouth (balance of flavors and sensations, tactile, fullness, finesse, elegance, purity, persistence in the mouth), a rating of 1 for the overall harmony of the wine,” the article explains. The scores for each wine were averaged and ranked, and therein lies the problem.

This widely used method of obtaining a consensus ranking of the wines suffers from at least one major flaw; it gives disproportional weight to easy graders, those who tend to assign scores at the higher end of the permitted range. This will be quantified below for the two panels. There is a better, more mathematically defensible way to get to the overall ranking, the Borda Count. In this method, points are assigned to candidates based on their ranking; 1 point for last choice, 2 points for second-to-last choice, and so on. The point values for all ballots are totaled, and the candidate with the largest point total is the winner.

Irrepressible nerd that I am, I simply had to see what Borda would yield. In order to do so, I needed the individual scores for each of the wines of each of the judges, which were not given in the magazine article.  I am grateful to Robert Drouhin who supplied them to me anonymized, presumably to protect the judges who soiled the honor of France by rating the foreign wines higher. 

To use Borda, the points given to each of the wines were converted to rankings for each judge. Since there were 12 wines in each category, the top-ranked wine was assigned a Borda score of 11, the second 10, and so on down to the wine ranked last which received 0. In the case of a tie, each wine received the average of the scores assigned to the rankings the group occupies. So, for example, if three wines out of 12 were tied and occupied the third through fifth positions, each received a score of (9+8+7)/3 = 8.  The consensus ranking of the wines was obtained by summing the Borda scores for each of the wines.

Below are the results.  The details and the arithmetic behind them are available upon request.

The Judgment of Borda:

Unlike the outcome of the Judgment of Paris According to Borda where the 1973 Stag’s Leap Wine Cellars Cabernet Sauvignon, which had ranked first by points dropped to second behind the 1970 Château Haut-Brion, the top two wines ranked the same by points and by Borda. So, the 1975 Pinot Noir from The Eyrie Vineyard remained in second place. There are, however, some differences between the results of the two methods just below the two top wines as shown in red. Australia’s 1976 Tyrrell’s Pinot Noir, which Robards reported came in first in the Olympiades, dropped to sixth place in the Borda ranking and was tied for fifth by points.

© iStock
| Applying different weighting to the scores resulted in some surprises.

The story is different for the whites. Here, Borda reverses the second and third ranked wines, resulting in one of Drouhin’s entries moving down. In addition, the bottom two ranked wines reversed order.

The Judgment of Borda:

Borda enforces equal spacing between the scores it assigns to ranking to ensure that each judge’s influence on the overall ranking is exactly the same. Averaging points assigned does not.

The next two tables present what I call the distortion analysis, the first for the reds and the second for the whites. Judge 3R had the greatest influence on the rankings by points of the reds, 1.74 times greater than Judge 1R.  Though the dispersion of points assigned is less for the whites, Judge 2W contributed 1.25 times more to the average than Judge 5W.

As reported in the magazine article, the organizer sent mixed signals as to how he might react to the outcome. “Before knowing the results, Mr Drouhin expressed himself thus: ‘It would certainly be lying to say that the results of this tasting do not matter to me; and yet, the results do not matter.'”

The reaction to the upstart from Oregon was a blend of second guessing, data manipulation, and Gallic vinsplaining: “It might seem surprising to see the 1961 Chambertin Clos de Beze third in the ranking after a young 1975 Pinot Noir from Oregon. In fact, if we ‘neutralize’ [cancel out] the jurors whose notes diverge a lot from the middle notes, the six other jurors agree to put this Clos de Beze in the lead, followed by Beaune Clos des Mouches 1978, then Chambolle-Musigny 1959 then from Oregon Pinot Noir Eyrie Vineyards 1975. Who to believe? The Belgian journalist Serge Tonneau, of Clos de Beze and of Chambolle-Musigny, wrote: ‘superb purity and extraordinary elegance’.”

Robert Drouhin concluded:  “Such tasting showdowns are harmful, they are misinterpreted by the consumer – however they are a benchmark to determine wines of quality; foreign producers must now seek out and focus on the specific characteristics of their region and we, the French, must better still optimize the heritage of our ancestors.” 

In 1987, he adopted the “if I can’t lick ’em, I’ll join ’em” approach and acquired land on the same hill as The Eyrie Vineyard.

Neal D. Hulkower is an applied mathematician and freelance writer living in McMinnville, Oregon.