Does discussing scores and cooking with a team benefit the judge?

4/21/2013

QUESTION

I’ve been judging for nearly 4 years now in both KCBS and FBA. As KCBS has less than 5 contests within reasonable driving distance of my home, I tend to judge more on the FBA circuit. One of the best parts of FBA to me, is that you tend to get about 60% of the same folks at each event. Friendships build quickly. At one of the FBA events about a year ago, one of the more senior judges (qualified by having judged over 100 FBA events and tons from KCBS and other associations), after we completed scoring each meat we wanted to see how close we were judging to each other. I was very pleased to see that I was consistently within a half point of him either up or down (FBA rates from 5 to 10 by .5 increments). Each time we’ve been at the same table since, we have continued this ritual. This other person was on the Judges Committee and we discussed opening this process up to more judges to more quickly educate the new judges. In early demonstrations of this process, the new judges did in fact become better at their scoring quicker. We are still contemplating making this more widespread within each contest with newer judges. Another point we discussed was cooking with a cook team a second time, once they’ve become Master Judges. Within FBA, to become a Master Judge, one must judge at 25 contests, of which 5 must be as a table captain, and the judge must cook with a cook team. A lot of the judges seem to get the part of cooking with a team done within the first 10 contests they attend. The appreciation for what the cook teams go through is an extremely important part of a judge’s education. Becoming aware of what a team goes through, the amount of time and money they spend on their entries can greatly improve the skill and “fairness” of their scores. Would you, as judges or cook teams see these efforts as good or bad for your BBQ Association?

Splat
Tampa, FL

Eric Devlin link

4/21/2013 02:19:56 pm

This is a great topic and I enjoyed reading the thoughts of Splat. I believe that self critiquing and looking for judging 'mentors' is a wonderful idea. I'm fully in support of continuing education and development of BBQ judges.

Although I'm receptive (and enthusiastically so) to new and innovative means to raise judging standards, I disagree with the importance placed on cooking with a team. The only reason that I can find for having a judge cook with a team is to reinforce the need to take their responsibility seriously. I don't think that there is a single thing that a team can learn about the judging process from cooking with a team.

We are not scoring a meat based on the time, effort and monetary expenditure that a team has invested. I couldn't care less if a team shows up an hour before turn-ins and cooks everything over a huge collection of matches or if a team shows up a week ahead of time and cooks over a pit they built out of cinderblocks. I'm judging the food that's presented to me, nothing more.

That's certainly nitpicking an important topic, so I apologize. I keep an email list of a couple of hundred judges in the NY area and let them know about upcoming contests on Long Island and in the metro-NY environs. Splats posting has spurred me to think a bit outside of the box about improving the training of judges. I think I'll start asking our most experienced judges to put their thoughts to (figurative) paper regarding the judging process and include them in the next mailing.

Eric Devlin

Keith KCBS CBJ/CTC

4/21/2013 03:00:53 pm

Eric, I appreciate your comments and you make some excellent points. I will allow however that it's my understanding that the cooking requirement is not meant to teach anything to cooks -- but rather to give judges first hand experience in what goes on out there prior to first turn-in. With that in mind I must disagree.

Having decided to actually enter and cook a competition rather than help an established team gave me and two other judges a deep appreciation and some invaluable experience. I highly recommend it and what's more recommend it be done ASAP in the judging career.

As for the cooks -- I recommend every cook judge on a regular basis to keep an idea of what the judges are looking for. But that should always certainly be optional as they are the ones who ultimately will win or lose each category based on the numbers on 6 cards.

Respectfully submitted...

Jim H - SBN/KCBS/SCBA

4/21/2013 02:37:02 pm

In my opinion, such a process can work very well if managed properly! Having judges get the experience of cooking with a team is invaluable to improving their judging experience but only if they actually learn something from the team. Having the criteria be only that they participated opens the door for a judge to spend the night with the team but sleep or just stay up "shooting the breeze" without actually learning anything. If judges have a list of measurable objectives they have to meet as a result of the cooking experience, it will more likely achieve some success! Regarding teaming a new judge with an experienced one, that two can be successful with proper oversight. Remember, habits are transferrable whether they are good or bad!

Eric Devlin link

4/21/2013 03:02:57 pm

'Having judges get the experience of cooking with a team is invaluable to improving their judging experience'

In what way, Jim? How will cooking with a team improve a judges ability to determine how food tastes, appears and feels?

Eric

Keith KCBS CBJ/CTC

4/22/2013 04:10:53 am

Because, put simply, it gives a well-rounded perspective on all of the elements of the actual competition. Speaking from personal experience this has made me a better, more deliberate judge, even if it has not impacted my ability to gauge appearance, taste and tenderness.

BBQ Critic

4/21/2013 04:17:59 pm

I think Eric makes a good point that when a judge cooks with a team, it is intended to help the judge think about the importance of judging, to help them understand how vital it is to focus while carrying out their duty in the judge tent. As Eric said, to “reinforce the need to take their responsibility seriously.”

A judge learns this when they see all the effort, time, money and skill it takes to create a turn-in box for competition. So it is all about focus, not lets give cooks a high score because all they spent so much money on a contest. Just simply understand this is something to be taken seriously. That's all.

My master cook taught me this and more. My buddy Mike Hall and I cooked with Tom and Char Manhart (Omaha, NE). We cooked with them in Council Bluffs, IA. We helped set up, trim meat, start the fire, tend the fire (all night long in cold, Iowa weather), prepare garnish boxes, load, unload the smoker, help assemble garnish boxes, and tear down. They let us do it all (within reason, of course). In the end you learn a lot about what a cook goes through in a competition. More importantly, you experience the comradery among teams. Just watch how everyone supports each other. It truly is a fun sport.

What taught me an important lesson was awards time. I thought we would kick ass. Nope! You see how the scores come back and that, in and of itself, is an eye opener for a new judge.

I've cooked with a few other teams since then and I one thing I would suggest for the master judge cook is to make the judge score the boxes as they go out the door to the turn-in table.Yep - I mean get out your KCBS score card and judge it! Judge the appearance of the actual box, and then use left over meats for scoring tenderness and taste. Then compare those scores to the actual judge scores. That is a HUGE learning experience for a judge and I recommend all teams require this of their judges who cook for the master judge qualification.

Let a judge compare their score to the actual contest scores, and you will instantly make a judge understand the important duty they have a ahead of them.

Herb K
Publisher,
BBQCRITIC.COM

jim v mcbj

4/22/2013 12:07:55 am

i am a strong believer in the practice of judges working with teams but this question has really made me think of why. Does it help me judge taste, tenderness and texture? now that i think about it, not really. what the the experience did do, however, was give me a rounded knowledge of the practice of preparing meat for competition. before the experience, i had no idea of all of the techniques, time, effort and money it takes to try to be successful. does that make me a better judge? i think so. because of the experience, i know more about what i am judging, i have expanded my friendships in the bbq community through the team i worked with and i think the teams feel a little better about the objectiveness of my judging knowing that i have at least given it a try. anyone can judge but not everyone can do! is it essential, no, but it is worthwhile and can't hurt!

Kent Harriss

4/22/2013 12:10:45 am

My wife and I judge both FBA and KCBS competitions and have judged around 75 contests. While I agree with most of what has been stated about the FBA requirement of cooking with a team to achieve your master judge status , I disagree that it should have much effect on the way that you score. I really like the idea of discussion of the entries immediately after judging at the table. I also like the idea of comparing the scores between others at your table. I would like to see an FBA policy of individually tracking your scores over the last 5 or 6 contests in relation to the others at your tables and if you were consistently lets say 1.5 points higher or lower than the table average that you would be mentored by another Master Judge, second time maybe the FBA rep and the third time you would not be allowed to further judge for the FBA. We have Master judges that are low low scorers. I Don't feel that cooking with a team a second time or continuing education will turn a bad judge into a good judge. Just my thoughts

Ike, CBJ/KCBS

4/22/2013 12:19:49 am

Splat brings up an interesting comment. I've been judging bbq for about 10 years and I enjoy it very much. I've judged more than the required times to become a Master Judge but I really have no interest in going that route. I know how much work it takes for the cooking teams because I have several friends that do cook and I keep that in mind when I judge a contest. The problem I have with a judge cooking with a team is that not every team prepares for the contest the same way, just as all judges do not judge the same way.
To me, it's more important for a judge to keep an open mind and just concentrate on what is being presented to him or her during the contest. If everyone cooked and/or judged the same there wouldn't be much competition left.

Greg MCBJ/KCBS

4/22/2013 01:20:07 am

In the previous comments I see very few constructive comments. It is good to have several suggestions and select the ones that may or will help, keeping in mind not to fix sumthin what ain't broke. some situations the exist in the East may not fit well in Western areas. That is why uniformity is difficult to achieve. The rules ion place now seem to be working well for most of us.

Ed KCBS MCBJ

4/22/2013 02:38:53 am

I personally have a real problem with comparing scores with my table, not because I'm uncertain about my scores but am afraid my scores might unduly influence another judge to match what I think of that box. BBQ scores should remain a completley indivdual effort and not trying to get eveyrbody to score in the same 1/2 point range.
It's bad enough now with the 7,8,9 mentality.
If your box is average in my opinion, you will get a 6 but I will also be able to vocalize why I thought it was average.
As far as having judges cook with teams I think this singlehandedly is the best way for a judge understand the other half of how BBQ contests work. I think they should be required to do it within their first 10 contests and again before becoming a Master Judge.
Of course I also think cooks should become a CBJ to see that half
of BBQ.

Tom - CBJ w/ KCBS, MBN, NCPC & comp cook

4/22/2013 07:42:43 am

Anything that helps a judge improve their skills at judging should be highly encouraged (IMHO). Cooking with a team (within their first 10 events or other), as well as shadowing the REPS at a contest, having a serious discussion with an Organizer relating to the work involved in putting on a contest, and even visiting a butcher shop to see where each cut of meat comes from originally, are ALL wonderful ways to improve a judge's overall knowledge of barbecue as a whole and I think that they are great ideas. That being said, I still agree with Eric that increasing a judge's knowledge of everything barbecue will NOT improve his/her ability to score a piece of meat on Appearance, Taste, and Tenderness. I cook in competition (non-KCBS) and know the expense in time, effort, and money that it takes to present a 9”x9” box of meat to be judged (AND judging at the grill), but that has NOTHING to do with the specific criteria a judge is to use when judging that box. Can a judge learn anything from a cooking team – YES, but will that information improve their ability to use the criteria required to judge – NO. Still I highly recommend a judge “work” with a team (“cook” with a team is probably NOT a good reference as to what the judge will be allowed to do!), to see the work that goes into presenting the box, just as I highly recommend a judge to shadow a REP, talk with an Organizer, visit a butcher shop, and increase their general wisdom of barbecue.

Concerning sharing “scores” with fellow judges at the table, I like the idea of discussing what you like and dislike about each entry after all score cards are in, but NOT discussing the actual scores given. Our TC this past weekend did mention that there were “mostly 9s and 8s” or something to that effect, but didn’t specifically state who gave what. While that may be pushing the envelope of acceptable, at least anonymity was maintained.

Mike S.; KCBS MCBJ &,BBQ-Brethren RJ

4/22/2013 08:19:45 am

I like to address the point if judges should at least cook with a team at least once. I feel whether they planning to be a master judge or not, a judge should cook with a team. The more cooks and judges interact may lessen any ill feelings between the two groups. A better understanding of what the other person does, and what they have to go through, is always a good thing. A cook should understand how much a lot of the judges have to spend to judge their food. Granted it may not be as much as the teams spend, but judges do not get a chance at any monetary reward. Often the teams are surprised as to how far judges will travel, and often stay in a motel for the weekend.
Having said that, I agree with Eric Devlin’s point, that when scoring, a judge should not take into consideration what the cook has to go through to prepare each entry box. That is part of the competition, and should put the competitors somewhere at the same starting point. Too often I have heard other judges say that they are scoring higher than they should have, because they take into account the overnight weather was windy, cold, rainy etc. My feeling is those factors should not be taken into account, part of competition is mastering your equipment and weather, and that type of scoring is not fair to the team or team that can do that. I will only judge on what the cook turns in, and what I am presented with.
I have been a KCBS master judge for about ten months, and have started my fifth barbeque season. I’ve judged 42 KCBS event, 4 BBQ-Brethren events, and about 6 NEBS events. I have mixed feelings about sitting with the same judges. For the past 4 – 6 contest I have been sitting with the same four judges, and I’m beginning to wonder how my scores are comparing with others. So I now plan to make sure I sit at another table.

Dan KCBS CTC,MCBJ

4/23/2013 03:16:41 am

No on comparing prior to turn-in. Yes to discussion afterward.
I think the comparison approach lessens the "blindness" factor and decreases individuality.

Deb KCBS MCBJ/CTC

4/23/2013 05:40:33 am

When one begins the process of becoming a Certified BBQ Judge, many judges have no clue as to the amount of work that goes into producing the fantastic product that we, as judges, are privileged to partake of. Thus, their scores are reflected according to their relative inexperience. And, as with anything, the more varied our experiences are, the better we become at something; in this case, judging BBQ'd meat. Seasoned judges need to mentor the newbies to assure a fair outcome, so having discussion following the judging is a reasonable thing to do.
While I personally do not have the desire to cook competitively, my experience of cooking with a BBQ team did give me a whole different perspective on the art.
Did it alter the way I judge? Maybe. If anything, the experience taught me to be thoughtful and very careful with the power my scores have in the realm of competition. If I am having a bad day, I need to correct my attitude and be sure to score fairly for the sake of the teams that work so hard to bring their entry to fruition. Having said that, judges are ultimately charged with judging the product as it is presented to them, not what it "might be".
Did the experience give me an insight into why there is animosity between the cooks and judges? Absolutely. When the team I cooked with was scored unfairly by a judge, it was very hard not to take it personally. Disappointment happens, but you learn from it and you move on. (Realistically, it is impossible to participate in such an endeavor without coming away with at least a nugget of knowledge you didn't previously possess.)
Most of the differences between cooks and judges seem to stem from a lack of understanding about what each aspect of the competition entails. The fact of the matter is, until you walk in another person's shoes, you won't "get" where they are coming from. We need to realize that without both cooks and judges participating, there is no competition at all. The bottom line is, we need each other.

Splat KCBS CBJ & CTC, FBA CMBJ

4/23/2013 06:54:14 am

I guess when I cooked with a team, my approach was a bit different. I knew I was there to acknowledge and learn what the team goes through to in presenting their entry. My goal was to learn as much about methods, smokers, wood, and the whys they did/do certain procedures. I was happy that I cooked with a team that had little or no problem with the barrage of questions I asked for the day and half I spent with them.
The type of things that were MOST beneficial to me were questions such as:
Why did you pull the skin off the thighs before you put them in the brine?
What kind of wood do you use for Chicken, Pork, etc.? What are the benefits of each.
Low and slow vs. Hot and Fast?
Why did you pick this smoker vs. that one? What are the pros/cons of each?
Why did you use that thing with hundreds of needles in it on the brisket?
How do you make burnt ends?
Some of the highlights from what I learned were such as:
Cherry wood will put a dark color on the meat. Some judges end up thinking this is over cooked, burnt or attribute it to “to much sauce”, so they don’t use it.
Onions in their BBQ sauce/glaze are taken to the stage of emulation as judges tend to think of those onions as “globs” on the meat and not take them for what they are.
About 2 months after I cooked with the team, I ordered my smoker. A baby Stumps and started making BBQ at home and testing all kinds of wood, rubs and sauces. I have not and have no desire whatsoever to cook in competition. I do this for my own/family benefit, and my wife and I both feel that we’ve become better judges in the overall because of this.

On showing scores to a new judge, I can only think of my first couple of contests I judged and had absolutely no real idea what “average” was. This past weekend I had a new judge seated next to me. After judging chicken, I allowed him to check mine to give them some idea of the scores I choose on the entries. I also expressed to him that mine were NOT the god numbers and that samples can be different from one piece to another within the same box. When we reviewed, his numbers they were 2 and 3 points off of mine lower. I expressed to him that unlike KCBS (this gent was judging for his very first time ever), FBA does not boot the lowest score, and that every judges input was counted. We did the same thing after ribs and his numbers were much more in line with the table.
Monday night, I was in a meeting that included the lead rep from that contest. Where she expressed that the new judge did very well after chicken. I didn’t ask her about the judge, it came up in the meeting as this person was a high up government official for the city the event was run in and was one of the folks that gave the “thanks for coming” speech at the beginning.

Comments are closed.

Does discussing scores and cooking with a team benefit the judge?

BBQ FAQ for Cooks and Judges

FAQ on judging competition BBQ