If you cite a less than excellent, how can you still give out a 9?

1/19/2013

QUESTION:

I have for awhile now been amazed at some of the scoring thoughts versus final scores from some of our esteemed BBQ judges here. I have on occasion read some of your box critiques that were pretty harsh and justifiably true yet there was no hesitation handing out a score of 8 or 9. If you cite a less than excellent, how can you still give out a 9? If there are glaring problems that make you point them out how can it still be an 8?

There is mention made from time to time about the "7,8,9" judges who wouldn't give a 5 or 6 to save their souls.
My intention here is not to incite anyone to lower their scores just because. I'm merely suggesting boxes with problems are not "excellent".

I would like to propose a little exercise for all of us that when giving a score to these boxes that you also write out the KCBS descriptive word that is associated to the number score. If you judge a box and find fault with it and decide it's a 9 and can also give it the name "excellent" then do so, but if the term "very good" or "average" is more in line with what is in that box maybe you should adjust the number to match the description. The descriptive words and matching number scores are available on the same page we make our comments.

My only goal here is to cause some thought on the matter and make scoring more consistent among us judges.

Ed KCBS MCBJ
Montezuma IA
Competition Cook & MCBJ

Greg MCBJ/KCBS

1/19/2013 05:23:16 am

I agree with your thinking. Some of the critiques are way off in left field compared to the picture presented. Some judges need a refreher course.
The score given does depend on the imperfection. A slight misalignment of the meat on an otherwise excellant box would be a 9. Thesee presentation boxes are made by humans not machines, and most of them are our friends.

Karen

1/19/2013 05:24:58 am

I believe that a perfect box should be the only box given a "9" score. Any imperfections should lower the score. I too have been confused by other judges "9" scores....if the box has a flaw it should not be judged a "9".

Frank MCBJ

1/19/2013 07:23:58 am

In KCBS judging there is no "perfect" rating. The highest score is "Excellent" which does not imply "perfect".

Karen

2/13/2013 06:41:06 am

No need to be so literal. By "perfect" I meant that there are no visible flaws(appearance), good taste(obviously a subjective category) and no obvious errors in cooking (tenderness).

Bill F

1/19/2013 05:28:52 am

"Lack of perfection"? What the hell is that? I would never use that expression. A 9 is a 9, a box looks good in every respect. But the next 9 that comes along may look quite different and still rate a top score. Forget about "perfection". You can't define it and you'll never find it.

jaja01 KCBS MCBJ

1/19/2013 05:32:45 am

I feel if the submitted sample is not perfect, a judge CAN NOT give the box a perfect score in any of the three areas. I feel the BBQ societies lead the judges to giving higher scores when the reps remind the judges the cooks spend a lot of time and money to come to the contest. They have to remember the judges also spent time and money to judge, with no chance of any pay back. Judge the meat honestly.

Frank MCBJ

1/19/2013 07:26:31 am

In KCBS there is no "perfect" rating. The highest score is "Excellent" which does not imply "perfect".

Keith CBJ

1/21/2013 02:34:44 am

You may feel this way but unfortunately you're wrong. A score of 9 only indicates excellence -- not perfection. Is there any such thing as perfect barbecue?

jim v mcbj

1/19/2013 05:44:01 am

If I see a flaw i don't think it is fair to the other cooks to rate the box a 9. 9 or not 9 is an easy decision for me and I think for most judges but 6 & 7 are more subjective as far as appearance goes. Appearance is like beauty, could it be in the eyes of the beholder? Judging is not a science so lets not try to make it one. I try to shoot for consistency knowing that perfection is subjective.

Keith CBJ

1/21/2013 02:36:28 am

Excellence (i.e. a 9 rating) indicates just that -- not that it is flawless.

BBQ Critic

1/21/2013 07:36:39 am

Jim - I like your term "subjective." :-)

Sean KCBS CBJ link

1/19/2013 05:47:34 am

I have to disagree with your comment "I'm merely suggesting boxes with problems are not "excellent" Boxes with minor problems or flaws can still be an "excellent" box. A score of 9 is only considered excellent. There is no "perfect" designation, at least within the KCBS scoring system. To judge a box with the mindset that a 9 is considered a perfect box is doing the cook a disservice. Even the best of teams will have minor flaws in their boxes whether it be in the application of the sauce or a difference in the thickness of the slices.

TA KCBS MCBJ

1/19/2013 06:05:07 am

Stan... while agreed the best of teams will have minor flaws in their boxes... then they don't deserve a 9 for that box, of that meat, on that day. That team was not at their best and their box was not "excellent" if your noted comments were evident.

BBQ Critic

1/19/2013 07:20:31 am

Sean - So are you saying a box with minor flaws is just as good as a box without minor flaws (as far as you could see anyway)? Or do all boxes have minor flaws maybe? I am trying to understand why when you see "flaws" it is still EXCELLENT?

Frank MCBJ

1/19/2013 07:28:36 am

Spot on Sean.

Keith CBJ

1/21/2013 02:39:07 am

@ Critic -- IMO excellence (i.e. a 9 rating) indicates just that -- not that it is flawless. I believe that is the only point being made. I totally agree with Sean.

BBQ Critic

1/21/2013 07:39:39 am

Keith - Agreed. When we find minor flaws and are on the fense about a score, we are supposed to give the cook the "benefit of the doubt" just as we are taught in judge class (KCBS).

John

1/19/2013 05:50:15 am

A box shouldn't receive a 9 if an imperfection is noted. However, the weighted score between 7,8, & 9 is miniscule. Give us a break Ed!

Frank MCBJ

1/19/2013 07:30:40 am

Again, iIn KCBS judging there is no "perfect" rating. The highest score is "Excellent" which does not imply "perfect".

BBQ Critic

1/19/2013 07:49:31 am

Frank, agree with you on the "perfect" rating comment. However, let's get back to the flaws. If a box is turned in and you cannot find any flaws and score it a 9 for excellent and another box comes around the table and you find flaws and score it a 9 for excellent - why would you score BOTH boxes excellent after citing flaws one box?

This is the $2500 dollar Grand Champion question. ;-)

-- BBQ Critic

TA KCBS MCBJ

1/19/2013 06:01:29 am

Wow... someone with my thoughts exactly. However, I wouldn't use perfect or perfection when describing any competition box. Most of us can spot and know a 9 when you see it. But, you're very right... much more typical is the 8 and 7 scores, and being "lambasted" for being too harsh from other commenters. If I see and can verbalize "room for improvement" comments than it shouldn't be a 9.

Bill C. MCBJ/CTC KCBS

1/19/2013 06:03:59 am

Perfection? Haven't seen it yet and probably never will. I can tell you that if I lokk at a box and would return to buy it again, it's at least an 8. If I want the cook's recipe, then it's a 9.

Mike Cannon

1/19/2013 06:10:10 am

If a particular sanctioning organization states that a 9 is 'perfection' (if any do?), then no sample would ever receive a 9, as nothing is perfect.

In my many years of reping for MIM(not MBN), we told the judges that a 10 was not necessarily perfect, but that it should be at least Very Good to rate a 10 in a criteria. The only exception was Overall Impression, which is a comparative score, where 10 meant the best you had that day. Only one 10 could be given in Overall, whereas multiple samples could be given a 10 in the remaining criteria.

In short, whatever the sanctioning body of the event defines a 9 to be, is what the judges should do.

Brian P - CBJ, CTC, and Cook

1/19/2013 06:23:00 am

YYYou folks looking for "perfect" need to be retrained; period. Nowhere in any judge training, CD, or published information in KCBS does it say a 9 = perfect. It says in the training and right on the scorecard that a 9 = Excellent. And common sense and literal definition dictate that "Excellent" does NOT mean "Perfect". Feel free to disagree or be upset with my candor but either way you're still wrong and are treating cooks unfairly just like those folks that only give 789. Every judge should have the ability to give ANY score from 2-9 based on that the entry deserves. Anyone that can't do that has no business being at the table. Greg was spot on that an excellent looking box that simply has a slightly misaligned piece of chicken should still get a 9. Of course if five go one way and one is perpendicular then that's not slight and should score accordingly. We shouldn't be making up our own rules like refusing to give 9 except for perfection (which there is no such thing as a perfect box ever) or whatever other goofy thing folks simply make up. Those are the reasons you hear cooks complain about judges. If you think a box looks excellent in spite of tiny flaw or two then give it a 9; if things are horrendous then give that score be it 2, 3, 4, or whatever. The more judges we can get to abide by their actual training the better it will be across the board for all and folks won't have to ask questions like this (which btw I do think was a good question and calls out some obviously retraining needs).

BBQ Critic

1/19/2013 07:44:37 am

Brian - You are right, we should not use the term PERFECT. But it is human nature to confuse the words Excellent and Perfect. Dictionaries even use the word excellent to define perfect. Besides, when we are talking about the word Excellent - we all have our own ideas of what this word really means which is what we are trying to figure out here.

KCBS uses the word Excellent. AGREED. Give some slack. Someone inadvertently uses the term Perfect and suddenly they need "RETRAINED"?..."Treating cook unfairly"? ... REALLY? Wow.

Listen, we are not here on this thread to belittle judges. We want to get down to the whys of what we do. I agree, the word Perfect confused everything because that is not the word we use in KCBS.

I got it. Understood. Let's try to give opinions without insinuating judges should be fed to lions. ;-)

All is good...

-- Herb

BBQ Critic

1/19/2013 06:23:06 am

Editor's Note:

When Ed mentions the phrase "LACK OF PERFECTION," what he is really saying is LESS THAN EXCELLENT. So please try not to take the word perfection too literal. The point is simply this: If we comment on a box in terms of being "LESS THAN EXCELLENT," and yet score it a 9 which is EXCELLENT - why would we do this? That's the core of the question at hand.

-- BBQ Critic

Brian P

1/19/2013 06:31:23 am

Actually Ed's phrasing whether intentional or not brings to light a specific problem about judging for "perfection" that shouldn't exist but clearly does even though it was NEVER trained that way.

BBQ Critic

1/19/2013 07:15:47 am

The word Excellent and Perfect are simply easy to confuse. My fault I did not catch it before publishing. ;-)

Dan KCBS CTC,MCBJ

1/19/2013 06:45:12 am

I'm pretty stingy with my 9's at real live contests. This forum is quite different, no smell,usually poor lighting and less than excellent pics. For that reason I give 9's a little more liberally. Quit trying to take the fun out of it.

Ike, CBJ/KCBS

1/19/2013 12:13:52 pm

I agree with you Dan! I started posting on BBQ Critic as a way to keep up with how others are looking at things. The way I judge a contest has a whole different agenda. If leaving comments on this site is going to cause such an uproar then I won't be partaking anymore.

BBQ Critic

1/19/2013 07:42:16 pm

Ike - Anything you post online (whether here or anywhere else) is up for debate. However, one of the things I do to help protect judges from negative public commentary is close all judging when the scores are all in. That way no one can respond to your scoring and/or comments.

There were problems in the past when I first started the site which ended up getting in the way of helping the cook and becoming a big debate instead. The only time I allow posts all the time is in this section. Ask The BBQ judge encourages feedback back and forth when discussing issues. This has, indeed, become one of the more controversial questions asked so far. :-)

-- BBQ Critic

Russ H

1/21/2013 04:16:38 am

Ike,

I hope you don't take any of this as an uproar or personal attack against you. Personally, I am glad to have a place where CBJ's can compare their scoring of boxes and try to understand the differences among us. As a Table Captain I have seen the differences in scores, and most of the time can understand the small differences. But on occasion I've seen the one judge that stood apart from the rest - consistently. I had a table once where one judge would always score several points higher than all of the other judges, and then score several points lower than the other judges. Again, I understand that everything we do is subjective, but why is this one judge consistently the odd man out? This type of forum really gives us the opportunity to understand how other CBJ's are scoring, and more importantly, why they're scoring that way.

There will always be the handful of people on the internet that hide behind anonymity and cause problems, but please don't let them destroy something that can be used for the greater good of what we do - which is to judge BBQ to the best of our ability.

BBQ CRITIC

1/19/2013 06:54:13 am

Editor's Note:

I went ahead and edited the word perfection to excellent because that is what Ed meant and I want us to focus on the real intention of the question, not challenge the word perfection. I know I and some other judges have confused the two words in the past because they are almost one of the same. Please try to stay on track with what we are trying to learn here: If we score a box a 9 (Excellent) when we say it is not Excellent in terms of our comments. Why do we do this?

A WORD ABOUT PERFECT: Dictionaries use Excellent in terms of defining PERFECT:

EXAMPLE: "Perfect implies the soundness and the excellence of every part." http://www.merriam-webster.com/dictionary/perfect

EXAMPLE: Perfect - "Excellent in all respects. A perfect day." http://dictionary.reference.com/browse/perfect

EXAMPLE: Perfect - "Excellent or complete beyond practical or theoretical improvement." http://dictionary.reference.com/browse/perfect

Bottom line: KCBS uses the term Excellent and if our comments reveal a box is not excellent, why do we still SCORE IT excellent? A fair question indeed.

-- BBQ Critic

Russ H

1/19/2013 07:08:36 am

Perfect is a poor choice of words. Flawless may be a better used term. Some of the judges list a multitude of flaws/dislikes with a box, but go ahead and score a 9. I don't want to be overcritical in my judging, but a score of 9 should be reserved for a turn-in with no major flaws and the minor one(s) should be fairly insignificant.

A score of 9 means "excellent" and should be reserved for just that. It is a dishonor to other teams that have turned in 'excellent' boxes to give out 9's so freely. I understand that all of this is very subjective, that each judge will score differently, and that each judge has their own personal preferences that come into play. But when I see scores run from 6 to 9 I have to wonder if one of us truly misunderstood the meaning behind each of the scores as KCBS has designated them.

I am a big fan of the changes that KCBS has made. I think the mentoring program for new CBJ's is a great idea and should help to curtail many problems in the first few contests. I also like that the new scoring program will look at judges scores and watch for certain anomalies that can be addressed before it becomes a serious problem.

BBQ Critic

1/19/2013 07:14:18 am

Russ - I think KCBS is making awesome changes! Love it. it is nice to see an organization take judging seriously and try to improve it in every way. It's not perfect, though. ;-)

-- Herb

Dick-MCBJ/CTC

1/19/2013 08:12:44 am

This question sure opened a "can of worms" but, one which really needs to be opened. Properly trained judges should start the scoring at nine and then deduct points for any faults they notice. I know some misinformed judges start the scoring at 6 (avarage) and score up or down from there. A proper example.... A judge reviews a box and decides the meat is very slightly overcooked. They decide that since it is not overcooked enough to cause a one point deduction they would deduct one half a point. This box would be score a nine since we have always been taught to give the benefit to the cook and the 8 1/2 becomes a 9. Yes, this box is "not excellent" or "lacks prefection" but it is still properly and fairly scored at a nine. I have sat next to judges that could never give a box a 8 or 9 and also next to judges that gave boxes straight 9's on almost every entry in every catagory. Both of these judges should find a new hobby as this is not fair to cooking teams. Good judges owe it to the cooking team to take the training and allow the time to give the teams fair scores. They should not be afraid or intimadated to give any score from a two to a nine as long as it is a true, educated, fair, and just score. Most of my scores will be 7, 8, or 9's with an occasional 6 and very few scores below 6. I believe that most cooks do not spend the time and money to enter a KCBS event until they have a "above average" entry. I admit that when I began judging I was reluctent to give a nine as I considered this the perfect entry. I since have learned to realize that perfect usually does not exist. Most cooks could have made changes that leads to perfect and which one of us is so perfect that we even know perfect. These entrys might still deserve the excellent rating. Even a nine (rather than the eight) in one catagory doesn't necessary mean this entry will win as it is rare that an entry scores all nines. I would ask all judges to give their scores some serious consideration, discuss scores after the turn-in and review your score compared to the other judges scores, with an open mind. This is what makes BBQ Critic interesting. It gives a judge the chance to review the other judges scores and comments which in turn should teach us how in tune our scores are and the to rethink our scoring, if necessary. I believe that KCBS is on track to review judges scoring and hopefully require additional training or at least cause some serious discussion for judges that have the tendenacy to score high or low constantly. I for one hope this happens.

Tom - CBJ w' KCBS, MBN, NCPC & comp cook

1/19/2013 12:46:10 pm

KCBS CBJ training does NOT recommend that a judge start at 6 and adjust from there or start at 9 and deduct from there. The scores are listed on the scorecard and should be followed as indicated.
Excellent does not mean perfect, therefore, a box with minor flaws could still be excellent IMHO. Major flaws on the other hand cannot rate a 9.
Like Dick above, MOST of my scores are 7,8,9 with an occasional 6, but rarely less. I am NOT afraid of giving a 5 or below if a box truly rates such a score, but I believe that it is EXTREMELY rare that a professional competition team would submit such a box.

BBQ Critic

1/19/2013 07:17:43 pm

KCBS leaves it open as to what method we use to come to our scores as long as it reflects the descriptions outlined by KCBS (i.e., Excellent, Very Good, etc)

When boxes start coming my way, I try to view each box as a 9 until something tells me it is not a 9. Nothing wrong or right about that method - many judges do this to determine what is Excellent or Very Good or whatever.. There is no way I could personally start at 6, though. Makes no logical sense to me.

I think when "minor flaws" come into play, that is when judges tend to give cooks the "benefit of the doubt." Nothing wrong with that. I am right on with both of you in terns of mostly using 7,8,9. Usually the boxes are simply that good. Other times - well, not so good. ;-)

-- BBQ Critic

Tom - CBJ w/ KCBS, MBN, NCPC & comp cook

1/20/2013 11:57:14 am

Herb,
I took my CBJ from Mike Lake and sat in the class when my wife took here CBJ from Ron Harwell and Mickey Williams. None of these instructors said anything about starting at 9 and deducting from there. Every judge that I've spoken with on this topic have agreed that the KCBS listed the numeric score definition beside the score so that judges would have a descriptor for the score.

BBQ Critic

1/21/2013 07:46:19 am

Tom - I never said instructors taught that menthod of starting at 9. But in order to make sesne of it my mind (to come to the conclusion of what is Excellent, Very Good, etc), I do start at 9. So I agree - it is not taught, but many judges start out with a 9 on a box. If the box is EXCEENT, the score stays. There is no EXACT SCIENCE to the process, otherwise we take the word "subjectivity" out of the equation.

don cbj

1/19/2013 04:19:07 pm

I have always liked judgung these boxes. However its difficult forf anyone to give an accurate score.Photos are doctorable, Presentation is like a pass interferance call in football. Judgement. I see a box that the meat is mouthwatering but for some reason I cant tell exactly what garnish is in the box, Im leary and I have no one to answer to
And yes im pretty harsh, cant jut give away winners that dont earn it

BBQ Critic

1/19/2013 07:20:39 pm

yes - photos can be a two edge sword: 1) Hard to judge because of the quality and in person in simply more accurate. 2) We get to spend more time looking at a box which means we can find more flaws than if the box was passed by us at a table for a few seconds of viewing. :-)

-- BBQ Critic

Doug KCBS MCBJ & CTC

1/20/2013 12:52:02 am

We are taught not to compare one box to another and rightly so. But as we gain experience we should develop a sense as to what constitutes excellence and what does not.

BBQ Critic

1/20/2013 05:06:10 am

Doug - Time does play a big part in judging. Totally agree! Took me about 10 contests to really get a handle on it. And it gets easier and easier after that, too. Although I try to Table Captain more than I judge. I do not have to eat so much that way. :-)

Mike H. Master CBJ/ Backyard Bumbler

1/20/2013 07:52:47 am

Great comments. Clearly we've avid BBQ fanatics and thoughtful people.

I think the bottom line here is that good questions are being asked. Fiery discussion is GOOD. It gets things out in the open and we start thinking and understanding how & why scores are the way they are.

Bottom line to me is: If you don't like the system, make an honest attemt to fix it. There are several thing that KCBS could do to improve, and I talk with other judges and board members to try to make things better- like making suggestions on improved CBJ training and clarifications in the rules.

Frankly, This website is the result of discussion on how to improve the quality of BBQ judging and I, for one, think it's a good start. The BEST thing about this website is the improved communication between judges, pitmasters, board members, etc.

BBQ Critic

1/21/2013 09:12:36 am

Mike - Great post! I think one thing we are learning is how different people judge and determine a score for a competition box (whether it is Excellent, Very Good, etc.). KCBS does not tell us an exact method to do determine appearence scores, so we all discover our own personal ways of doing it over time.

All we can do is try to be as fair as possible when giving out scores and always remember to give the cook the "benefit of the doubt" when we are on the fense when giving out a score.

Connie

1/27/2013 04:55:24 am

There is no such thing as perfection in BBQ only excellent and if your sample is excellent and you'd like to have another...then score a 9...it is so individual... Even KCBS and FBA rules state if the sample is excellent...

Harv link

1/27/2013 12:29:39 pm

I also agree with your thought (option). As a KCBS Cert Judge trained in 2006, things (training) have changed. Take a refresher Judge's Class, at no charge, would help all teams have a level playing field..

Splat

2/26/2013 11:06:36 pm

Here's another way to view this debate.
No flaws = perfect
minor flaw(s) could equal Excellent.
Since there isn't a "perfect" score, perfect is a subset of Excellent.

Comments are closed.

If you cite a less than excellent, how can you still give out a 9?

BBQ FAQ for Cooks and Judges

FAQ on judging competition BBQ