Deep Thoughts on Coffee Review
Why is submitting a coffee to Coffee Review a good idea for small roasters?
Before I answer that question, I have decided to set this blog up in a question/answer format rather than in an essay format in order to better demonstrate the value of Coffee Review, how they operate, and how it could help your business. Additionally, I need to also say I have no affiliation with Coffee Review. But, like many coffee companies, I have submitted numerous coffees to them for review. With Bird Rock Coffee we sent in well over 50 coffees over the years and now with Chuck’s Roast, we have sent two since the company’s inception last Fall.
What is Coffee Review?
Coffee Review (CR) was founded by Ken Davids and Ron Walters in 1997. They had the idea to follow the wine industry’s lead and score coffees on a 100 point scale, not unlike Wine Spectator, in order to help consumers make educated buying decisions. They cup the coffees blindly and score the coffees based on Aroma, Acidity/Structure, Body, Flavor and Aftertaste. In addition to reviewing coffees that companies send in, they produce monthly educational articles that include reviews of coffees submitted for specific themes. For example, they may do a report one month on coffees from Kenya while another month they may delve into a particular coffee processing, like Honeyed Coffees.
Taken as a whole, the staff at Coffee Review review thousands of coffees each year. At the end of each year CR publishes their list of the Top 30 Coffees of the Year. All the coffee that make this list are exceptional and their ranking depends on score, uniqueness, and value.
Is it free to have your coffees reviewed?
No. Unless you are submitting coffee with the hope of being included in a monthly report, you must pay to have your coffee reviewed. At this time a review costs $375 per coffee.
If you are paying for a review, how can the consumer be sure it was not just a pay-to-play type of thing? Pay money and you get a good review.
A good and obvious question. Certainly there are pay-to-play ploys in the coffee industry; one annual coffee “competition” comes to mind.
At this point you need to look at the integrity of the operation. Ken Davids and Ron Walters are well-respected coffee people who have contributed much to the specialty coffee industry throughout their vaunted coffee careers. The reviews would not have been taken seriously for so long if the feeling was it was a pay-to-play operation nor would Ken or Ron risk their reputations on such a concept. Their longtime staff members, who do a lot of the reviewing, Kim Westerman and Jason Sarley, are two Q-grader coffee experts with stellar coffee industry reputations as well.
Coffee Review is a buisness. It is not run by a team of volunteers. I believe their only source of income is from website advertisements and commissioned reviews. So, yes, they charge for commissioned reviews but this certainly does not mean you are paying for a good score—more on that in a moment.
But all the reviews I see on Coffee Reviews are high scoring? There are a lot of 90 plus coffees on the CR website, which would be unheard if they were using the SCA coffee scoring protocol.
This is where things need to be clarified and analyzed a bit. Coffee Review scores are Customer-Facing scores. SCA scores (and Cup of Excellence scores), for example, are Professional Facing.
The SCA protocol allows coffee pros to talk about particular coffees in the language of coffee. For example, if a coffee broker is describing a coffee to a coffee roaster looking to buy green coffee, the broker might say the coffee “is an 84.” That score gives the buyer an idea of the quality of the coffee because the coffee roaster is used to using and has a deep understanding of how coffees are scored and cupped with the SCA methodology.
According to the SCA, “Specialty Coffee” — good, clean, sound coffee— would score 84 plus. A fantastic coffee on the SCA scale would score 87/88. This all makes perfect sense to the coffee professional due to an understanding of the process and the rigor with which the coffees are scored.
But to the consumer, an 80 point range coffee is… a “B” grade. Good. Not great. Since Coffee Review is modeled after the consumer-facing wine industry, the scores need to be meaningful for the consumers, not so much for coffee professionals.
CR would not be an effective tool or worthwhile for coffee roasters unless it made sense to people looking to buy good roasted coffee for home use so Coffee Reviews had to make a new scoring scale. Like the SCA, CR still analyses coffee based on acidity, body, complexity etc, but Coffee Review’s point system is skewed higher. While an 84 point score would not really register with the average consumer, a 90 point coffee does. The average consumer would equate a 90 point coffee with an “A” grade.
So one way to look at it is that the scores on Coffee Review are 4-5 points higher than that coffee would score using SCA protocols. A 90 point coffee on Coffee Review is a good, clean solid coffee — probably around 84/5 using the SCA scoring sheet. A 94 coffee on Coffee Review would probably be a 87/88 Coffee using SCA scoring while a 96/97 point coffee on Coffee Review will be pretty close to a 90-91 point coffee using the SCA scoring — a rare coffee indeed.
From Coffee Review’s site, here is how they define their score ranges:
97+ = Means: “We have not tasted a coffee of this style as splendid as this one for a long, long time”
95-96 = Perfect in structure, flawless, and shockingly distinctive and beautiful
93-94 = Exceptional originality, beauty, individuality and distinction, with no significant negative issues whatsoever
91-92 = An very good to outstanding coffee with excitement and distinction in aroma and flavor – or an exceptional coffee that still perhaps has some issue that some consumers may object to but others will love – a big, slightly imbalanced acidity, for example, or an overly lush fruit
89-90 = A very good coffee, drinkable, with considerable distinction and interest
87-88 = An interesting coffee but either 1) distinctive yet mildly flawed, or 2) solid but not exciting
85/86 = An acceptable, solid coffee, but nothing exceptional — the best high-end supermarket whole bean, for example
And, yes you don’t see many sub 90 point scores on coffee review because if you are paying them to review your coffee, you have the option of publishing the results or not. With a sub 90 score, most roasters would choose not to publish.
Ok so how does submitting coffee to Coffee Review help the coffee roaster sell coffee?
It only helps you if you get a good score. Obviously. Over the years, we found that scores of 94 and over do lead to more robust sales and that these 94 plus coffees drive more traffic to the coffee roaster’s website. (Coffee Review has a special category on their site for 94 scores and above, “Outstanding Coffees”. ) But if you own brick and mortar store, virtually any 90-plus Coffee Review score will move quickly if the score is displayed properly, either with a sticker on the bag or POS material that is easily understood by the consumer.
Let’s be honest, not every customer that walks into our shop is a hard-core coffee person with a deep understanding of coffee-processing, roasting, and varietals. The average consumer will walk into a coffee roasting company not knowing which coffee to buy. They often just, “want something good.” A well-educated barista is usually the best salesperson a coffee company can have, but if unavailable at the moment the customer is going to make that buying decision, the score on the bag tells the consumer, “Hey, this coffee is special. A third party identified it as good/outstanding/exceptional.” In our experience, 90% of the time the average consumer will choose the “scored” coffee over one that is not scored. So a coffee roaster can count on that coffee moving quickly—if priced fairly. And, generally the higher the score the quicker the coffee will move.
What is the downside of submitting a coffee?
The risk. Regardless of company size, $375 is a lot of money. It would be a bit of a bummer if you paid all that money and got what you perceive to be a poor or low score. Everyone who submits coffee to Coffee Review wants a 96 or 97 point score — a WOW score. Sadly, that is not going to happen most of the time. I have sent coffees to Coffee Review I was sure would be a 94 or 95 only to have the review come back at a 90 or 91. It stings a bit but, again, the score will still ultimately help sales if presented to the customer effectively.
Some coffee enthusiasts and professionals may also argue that Coffee Review does not have the impact that it used to have. All I can say is that one of the last coffees I submitted to Coffee Review as owner of Bird Rock Coffee Roasters was a Kenyan coffee back in 2016, the year before we sold the company. That coffee scored 97 and ultimately landed on the #1 spot on their Top 30 list. It had a huge impact on the company—even with all the other accolades BRCR achieved. Sales for the coffee were over-the-top and we got a lot of recognition for the first place finish on the Top 30 list.
Last November, 2024, we submitted the first Chuck’s Roast coffee to Coffee Review and it scored 96 and ended up finishing #4 on their top 30 list. We got a lot of recognition for doing so well and sales for that coffee were over-the-top. As far as I am concerned, the impact CR has is directly related to the quality of the coffee and the skill of the roaster. Thousands of people still visit the site regularly and still make buying decisions based on what is reviewed.
For me, using Coffee Review is sort of like spending money on advertising but has the potential for an immediate effect. I would pay the money for a commissioned review instead of spending that same amount of money, or more, on a passive advertisement that I may or may not ever see any real measurable benefit from.
What can a Coffee Roaster do to insure a high CR score?
There are no guarantees when it comes to getting a high score on Coffee Review. Nor is there a particular coffee “type” that will get you a 95 or 96. I have gotten great scores with naturals and washed, with Pacamaras and Geishas. I have also gotten so-so scores with naturals and washed and with Pacamaras and Geishas.
When choosing a coffee to send, I think the more interesting the better. If you have something truly unique on your offering sheet compared to your other coffees, it might be worth sending in. Coffee professionals like coffees that are different.
And, generally, coffees that really pop off the cupping table and are more dynamic, instead of subtle, have the potential to do a bit better.
Send your best and see what happens.
Any final thoughts?
Coffee roasters have many tools at their disposal that will help them to succeed. A coffee review by a third party is just one tool.
Certainly if you have enjoyed success and are a trusted name in coffee with a reputation for great sourcing and roasting, you may not need any extra help with a third party coffee review. But for the new or smaller coffee companies, having a coffee reviewed by a reputable third party can certainly help coffee lovers learn about your company and may help drive sales.
If you think that having a third party review your coffee might be a good idea for you, Coffee Review isn’t the only company that offers this service. Before sending your coffee to anyone for review, however, do your research.
How long has the company been around? How many reviews have they published? Who reads the reviews? How many people read the reviews? Do they detail their scoring protocol? Have you tasted coffee the company has reviewed and do you agree with their assessment?
Some people who review coffees on-line are merely hobbyists who just want free coffee (shocking, I know) and will promise you “exposure.” But beware because their review might not be worth the cost of your coffee or even the cost of shipping. Sometimes, you might be better off using a different tool.