I need help: Cupping versus "Sensory Assessment"

[Jim Schulman]

(I've promised a paper for James's symposium, and being philosophically insecure, I need to see if I have this right, or if I'm way off base)

In preparation for writing the paper, I've been reading about Sensory Assessment, a collection of techniques, along with its experts, and even ISO standards, used by mass consumer products industries for assessing the taste and feel of foods, perfumes and cosmetics, among other things. This basically involves adding safeguards borrowed from statistical sampling and quality control to make tasting more reliable.

Some of these statistical safeguards may apply to cupping, many do not; but that is a minor point. Much more major is the difference in conception of what tasting is about in the first place.

To the sensory evaluator, a product is a bundle of attributes, say sweet, salty, flowery, and spicy, and a quantity measuring the level of each attribute. The evaluator surveys the marketplace to see which bundles and levels will best sell; then he or she test different formulations to find the one that delivers said bundle at the lowest cost.

This sounds a bit like the way coffees are scored, but the more I read, the more different it became. As far as the evaluators are concerned, there is no product, no coffee, or tea or whatever, just an anonymous substrate which delivers this bundle of tastes. The underlying philosophy is simple: consumers don't want to taste what is really there, they just want it to taste good.

This seems reasonable for a few seconds. But now apply it to your sense of sight or hearing. Whoa! If you just want to hear and see nice stuff, you're a junkie. Why should taste or feel be different?

For things people care about; it isn't. We want to taste the coffee, the care that went into it, etc. etc. For us, it's the coffee itself that is good or bad, not just its taste. We have theories about what makes it so: some emphasize the bean itself, and want the rest of the supply chain to not subtract from that; others also emphasize the positve contribtuions made down the line. But all of us are talking about the coffee, not just its taste.

Wine lovers, food lovers, etc etc are the same. None of them would say "we don't care what it is, as long as it tastes great."

And neither do mass consumers. Whatever else fair trade and organic is; it's partially a need to taste what's real, a gathering rejection of the food processors' "bundles of attributes on an industrial substrate" approach.

Am I way off base, or am I close here?

[barry]

I think there's a threshold of "caring", above which one sits up and takes notice of whatever ills and evils may be involved in the production & distribution of any given product, but below which one is content with enjoying whatever the product may be for whatever enjoyment may be had.

In other words, there is just too much to keep track of, so most just enjoy the ride rather than worrying about every little "wrong".

[Brent]

And neither do mass consumers. Whatever else fair trade and organic is; it's partially a need to taste what's real, a gathering rejection of the food processors' "bundles of attributes on an industrial substrate" approach.

Am I way off base, or am I close here?

I interpret the above paragraph as all other things being equal the fair trade organic will win....

That I think is a fairly accurate interpretation, and the reasoning probably on the mark.

[Jim Schulman]

...
In other words, there is just too much to keep track of, so most just enjoy the ride rather than worrying about every little "wrong".

You're right about picking ones fights and interests. Mass production delivers the stuff we don't much care about cheaply enough so that we can all afford to go deluxe or ethical on the stuff we do.

But the mass and afficionado ends of any market are hugely different. At the mass end, the product is a bundle of appearances with surveys and panel studies establishing their wide appeal. At the afficionado end, surveys don't even work, since everyone is talking to everyone else, and the responses are not independent. Furthermore what they are talking about is the product itself. How it's made is important, not just how it tastes or otherwise appears.

But in the real world; it's just one market, with many people being somewhere in the middle. Since this is my field of expertise, I know for certain that the math and statistics needed to model this middle sector don't exist; and neither do the concepts which allow consumers themselves to deal with it practically. My gut feel is that most consumers are in this middle zone. They are equally uncomfortable with an afficionado market where every detail is hugely important and a "pure" consumer market where only appearance counts, and the making or make-up of the product itself is irrelevent.

My guess is that this hard to fathom discomfort underlies the consumer support for things like organic foods and fair trade, along with the earlier popular efforts for food labelling. These concerns are general, and bypass the detailed knowledge an afficionado market brings to its product; but they also reject the "consumers don;t want to know what's in it" axiom that underlies the practice of sensory evaluation.

So I think Brent may be right; this middle stretch of any market will assert itself more and more, and use certifications as the vehicle.

[Marshall]

This sounds a bit like the way coffees are scored, but the more I read, the more different it became. As far as the evaluators are concerned, there is no product, no coffee, or tea or whatever, just an anonymous substrate which delivers this bundle of tastes. The underlying philosophy is simple: consumers don't want to taste what is really there, they just want it to taste good.

...

Am I way off base, or am I close here?

Only partly off base, Jim.

I think you are confusing the taste evaluator's important, but limited, role in product development with the overall business strategy for the product, which is what I take you to mean by "underlying philosophy." I also think you are over-estimating the differences between aficianados and mass consumers in the extent to which they respond to non-taste attributes (although they may respond to different attributes).

Neither the aficionado nor the mass consumer selects a product based purely on taste, no matter what he or she may believe. Both are influenced by packaging, color, appeals to social status, guilt, cultural symbols, sex and a hundred other elements that keep the marketing departments and their consultants busy.

[Brent]

Neither the aficionado nor the mass consumer selects a product based purely on taste, no matter what he or she may believe. Both are influenced by packaging, color, appeals to social status, guilt, cultural symbols, sex and a hundred other elements that keep the marketing departments and their consultants busy.

Just look at those sugary crappy tasting energy drinks that everyone enjoys...

[Duck]

I think that what you are starting to see more and more of (particularly with the emergence of third wave/quality-minded coffeehouses and the roasters/rest of the supply chain that supports them) is a medium within the medium. My cafe for example, does sell the type of drinks that make most of the contributors here cringe at the thought of them (just the sheer off-kilterness of the coffee to milk ratio, if nothing else), but we ARE making headway and are educating the mass market by constantly serving what the aficinados crave. For example, I have three customers that have been with be for years, going back to my Starbucks days. Then, the sucked down 2000 calorie+ white mocha frappuccinos with extra brownie chips and caramle drizzle with the best of them, flash forward a bit and they now drink traditional (6 oz) capps and are almost giddy when we have a Guest Espresso once a month. It's a progression to be sure, but eventually, it can pay off. Today, we sell more 6, 8 and 12 oz sizes than most other shops I know af across the country. Again, most of the present company that visits here excepted. We host public cuppings, serve SO shots, compete on the USBC circuit, and the whole she-bang! We pride ourselves on our coffee flavored coffee but at the same time we can't take so hard of a line that we drive off the masses as it is exactly those people that we wanrt to invest our energy into, trying to move them ever closer to the aficionado side.

The non-taste attributes are something different and many cafes simply do not have the resources (or how-to knowledge, in many cases) to create quite the same magic outside the cup like we do INside the cup. That's both our biggest strength and biggest liability. You can usually get a better cup if you look loing enough, but you usually don't get to be as trendy as your cubemate while you do it. The added valuse there comes from the intangibles that bigger chains in any industry rely so heavily upon because they know that stripped down to its products core and taking "The Pepsi Challenge" blindfolded, they would lose to the caring and highly skilled artisan every time.

Additionally, many of the certifications that so many retailers focus on and strive to achieve have nothing to do with quality, yet they are promoted to the mass customer base as though it did (Think Fair Trade). Again, with these new coffeehouses and other quality minded venues/outlets starting to emerge, you are starting to see examples of instances that focus more on quality at all costs, which usually results in the farmers receiveing a much high price that $1.26/lb because the quality is there to back them up. How's that for social reponsibilty? Think of the "Meet the Producers" event that Stumptown hosted a couple of months ago. Transparancy like that will shorten the distance between the mass base and the aficionado base, but it will take time and a staying the course by this emerging culture of quality.

Wow. I wrote far more than I meant to. Now my coffee is cold. Better freshen that up...



-- Duck

[bz]

warning: serious g.k. chesterton ripoff to follow.

i'm a believer that people, in general, don't know what they want until they experience it. this is not at all meant to be condescending. it simply acknowledges that people -- consumers -- can't ever wrap their brains around comprehensiveness, and thus can't ever make a totally informed opinion about what it is they want. indeed, their opinions tend to stem from an extremely narrow experiential set. the broader our access to information becomes, the less we truly know what to make of it. it's all random.

the unprecedentedly detailed media coverage of our most recent war is a prime example. more total info, less bottom-line meaning.

what we desire, as consumers of anything, is to make sense of it all.

example: i work in the newspaper industry where, in an attempt to change with the times, managers love to chronically develop new products (free and hip publications for gen y-ers) per the specs of a focus group. old-timers loathe this approach because, they argue, a reader doesn't know that he wants an investigative series about outrageous abuses of minors in the juvenile justice system until he sees it. when he reads about state employees assaulting 16-year-olds in a corrupt, taxpayer funded system, well ... that's the kind of highly meaningful product you can't EVER get a focus group to order up.

and the argument goes something like this: an editor's gut instinct is always a better tool. which sounds totally elitist and gatekeeper-y.

but what i think people really mean when they say that is this: trying to row your way across an infinite sea is exhausting and pointless. you cannot get people to order up a perfect wish list no matter how many focus groups you form. they can't traverse all the possibilities. one is much better off floating amiably in that infinite sea, taking in the wonders as they come.

think about it. philosophically, all major systems of religious thought seek to answer this key human tension: we want comfort, but we also want adventure. we want to taste familiar and wonderful things, but we also want to be wowed by tasteful surprises. we want to be at home and to explore.

this is called "romanticism" because ancient rome was the archetype of this balance. coincidence that this country gave us espresso? you decide!

jim wrote about the following philosophy:

The underlying philosophy is simple: consumers don't want to taste what is really there, they just want it to taste good.

and my point is this: consumers want NEITHER a comprehensive analysis of "what is there," NOR just a feel-good experience. what they want is a tension of both. it's a basic human need.

because, again, trying to traverse the heavens is impossible. it will drive you mad, like pure logic drives chess players and mathematicians mad. but being content to stick one's head into the heavens is pure bliss. poets rarely go mad (despite the stereotype).

basically, you're immersed in "data" without trying to circumnavigate it. there's still time to linger over good things. and what you get is both what's "there" (taste substrates) and what tastes good.

this is the most sensical thing to the human brain.

and this is coffee.

[nick]

Ben, kick-ass post.

Jim, a couple of thoughts on your original post... not necessarily vitally relevant, but since you asked for feedback:

This seems reasonable for a few seconds. But now apply it to your sense of sight or hearing. Whoa! If you just want to hear and see nice stuff, you're a junkie. Why should taste or feel be different?

They're VERY different. It's easy to see and hear Heidi Klum. It's an ENTIRELY different thing to taste or feel her. I'm not kidding. Tactile and taste sensations are much more intimate by nature than sight or sound. Smell is sorta in-between. See, hear, smell, touch, taste, f*ck. From less to more intimate and intense.

I'm not kidding about the "f*ck" either. One of my common spiels is that next to sex, giving someone something to eat or drink is the most intimate of human interactions. That's why I'm so obsessed with making our coffee as great as it can be.

I do agree that there are many things to add into the "consumption-equation," if you don't mind me coining a term based on your original post. Not just the sensual experience, but more knowledge (or lack thereof) about WHAT it is (recipes, ingredients), HOW it came to be (techniques, labor-intensiveness, source of raw materials, history), WHO made it (trappist monks? sweatshop kids? german engineers?), WHEN (vintage year? is it fresh? is it historic?), WHERE (Champagne? Made in the USA?).

However, these are still all statements of fact, and then there's the elusive "WHY" question, which takes many forms: Why are they selling this? Do they just want my money? Do they actually care about my well-being? How much does this barista care about making this drink good? Do I like this company? Are they ethical? Are they cool? What does this product make me look like to other people? Does this product reflect my values? Does this thing make me happy?

Now as Barry has mentioned, different people will care about each part of the equation more or less than than do others. A liberal-arts college student in New York City is much more likely to care about ethics in manufacturing or production than does a 45-year-old-mother-of-two in Overland Park, KS (sorry Sandy).

Just a jumble of thoughts.

Ben, that tension that you speak of... I heard a fascinating segment on NPR (ATC) one day that analyzed the diminishment of that "discovery-and-wonder" stuff as you age into your late 30's and into your 40's. Even in lab animal tests, subjects older than a particular age demonstrated an aversion to "newness" and an affinity for familiarity. Interestingly, the study showed that there was a positive correlation between this diminishment, and a perceived sense of success in one field or career for an extended number of years. So if you wanna keep things fresh and new, and not become all stodgy, then don't hold the same job for too long, and don't get successful.

[Jim Schulman]

Aaron, Ben, Nick; thanks for these though through replies; it's given me bunch to think about, and it'll be a few days before I can reply properly.

I was setting up extreme stereotypes that thanks to your posts I see they really don't cover how people taste. They don't even cover how I taste coffee: The last few days Bob and I have been cupping a shipped coffee of which we also had tasted the preship version. Getting analytical, it's quite clear the original sample had been mixed with another, less powerful and less well prepped lot. As a pure taste experience, it's a lovely coffee, not as powerful and star quality as the original, but a gentle wonderful cup nonetheless. As a "taste the reality" exercise, it's a let down. Chances are Bob and I will be scoring it quite differently, and that I'll be the "idiot consumer."

[Sandy]

.....45-year-old-mother-of-two in Overland Park, KS (sorry Sandy).

To clarify on my behalf that Nick is not talking about me but the geographic location of which i work.

I am not 45 nor do i have two children.


thankyouverymuch.
hmph.

[Jeff Givens]

One of my common spiels is that next to sex, giving someone something to eat or drink is the most intimate of human interactions.

I totally agree with that statement. I remember the time that I made a cappuccino for a friend and he said, "this is the best coffee I've ever had in my life." I almost got teary-eyed when he said that.

Along with procreation, eating and drinking are the most fundamental things of our existence. We are hard-wired for them.

[Peter G]

One thing we must remember when considering coffee tasting...

roasted and brewed coffee is an incredibly complex taste experience, its dominant feature a tension between bitterness, acidity, and subtle sweetness. This is not immediately pleasurable (ask most 10 year olds who try coffee for the first time). We learn to appreciate coffee flavor for its association with the drug effect, its pleasant counterpoint to other foods (like milk and sweets), and for its intriguing complexity. This last trait triggers our innate desire to experience diverse and new foods, the "neophilia" in eating that Pollan discusses in "the Omnivore's Dilemma." Humans, unlike most animals, actively seek out new and unusual taste experiences, even when they can be toxic (alkaloids like caffeine)...

What does this offer to this discussion? Well, thinking about "preference" as most food tests are designed probably does not capture what many people love about coffee drinking. "drug" drinkers are using coffee primarily for the pick-me-up, and may choose high-quality coffee simply because it is less offensive than the cheap stuff. These consumers will likely favor very clean coffee tastes. "coffee flavor" drinkers like the simple flavor of coffee itself, and in particular the way it interacts with milk. These are the folks who love coffee ice cream, coffee milkshakes, Starbucks, etc. The last group are the "explorers" who are tasting coffee the same way folks taste exotic cheese, unusual beer, spirits, etc; as a learning exercise, as a way to explore the culinary world.

The first 2 categories fit snugly into what is contemplated by the market researcher's sensory assesment protocol. If you can identify your target market, you can probably find out which coffees and roasts appeal to these folks, through taste tests and sensory analysis. The third category relies on communication, trust, and all the other senses (including the sense of curiosity). It is a much more wholistic approach to tasting, and it conforms to your "tasting what is there" idea. In reality, most people occupy different categories at different times, sometimes in the same day!

Anyway, just some thoughts....

pg

[nick]

Great post Petey-Gee. To add, I think you'd agree that people are generally some combination of those three types you mention.

Something came to mind that is relevant in some convoluted way: I'm still not really sure I like beer. I drink beer, and part of me can identify a good beer from a lesser-quality brew, but if I stop and really ask myself if I like it or not, I realize that I kinda don't. I don't drink for the "effects" either... not if I'm not in the vicinity of someone from Intelligentsia* .

Peter's post also makes me think of the closing commentary from Anthony Bourdain from the Food Network's "Decoding Ferran Adria**" that Tonx sent me a while ago. After his culinary adventure at El Bulli, Bourdain asks aloud, "Was that good?" referring to the food and the totally new experience that Adria's menu imparts. If I remember correctly, he comes to the conclusion that, in fact, it was not "good" in the traditional sense of "good food." But it was an experience that he loved and cherished and was fascinated by (my paraphrase... not his words). Pick up the DVD if you can find it. It rox.

Appendix
* To be social... not because the experience is otherwise unbearable
** to those less familiar, Ferran Adria has been name-dropped a bunch of times here on coffeed, and is considered by most to be the most famous purveyor of "molecular gastronomy." (click link for the wikipedia article) El Bulli is a moderate distance from Barcelona, Spain. I encourage the business-owners and money-movers out there, some competition or something HAS to have a grand prize of travel and dinner at El Bulli for two.

[Jim Schulman]

Thanks, Peter, your post clears up some things. The same tensions replay at various levels: for instance, the split between an ideal cappa profile and an adventurous espresso blend is mirrored by the one that develops when one tastes a coffee that's good or interesting, but very "atypical" for its region.

Nick, your experience with beer is illuminating. I think once a person gets into analytically tasting anything -- coffee, food, wine, etc -- they automatically bring the same attitude to anything else, like beer. Certain abstract relational concepts (the very ones the sensory evaluation text warns against asking about) like "complexity," and "balance" seem to apply to all tasting. Maybe this is why it's easier to judge quality in something completely new when one has experience in tasting something else.

Or perhaps coffee is so complex, that once one has done that, tasting everything else becomes a piece of cake? There is some support for this position in the chemistry -- 1200 odd aromatics in coffee versus around 700 in wine, the next highest item. This would make a great pitch for the foodie crowd; "Is your palate ready for the big leagues? Try coffee!"