Are You Blind Ref?

Quite a bit of research has been done on the visual requirements of different sports, but very little on the requirements for sports officials.

There are, of course, several different kinds of officials. For simplicity, I’ll put them in four main groups.

Recorders include cricket scorers, and those who measure distances and times in athletics. Line judges would include line umpires in tennis, but would also include some who can occasionally intervene in play, such as touch judges in rugby. Referees and umpires are an integral part of the game, often making an enormous number of decisions, as in cricket, rugby and football. Finally, judges give a rating to a performance, but are external to it, for instance in gymnastics, diving and figure skating.

Some officials will cut across these boundaries, such as tennis umpires. There are also other officials who are further removed from the performance, such as tournament referees, 3rd and 4th officials.

I'm going to be concentrating on those officials who have to make decisions, but even recorders rely on vision. Cricket scorers in particular have quite high visual demands as they are a long way from the action. Recognising batsmen under helmets from outside the boundary can be difficult, and may require other features to be found, such as the markings on the bats. Scorers also need to be careful that they are ready to acknowledge any signal from the umpire, and aren’t looking down at the scorebook at that moment.

All officials, therefore, should have regular eye examinations to ensure that their judgments aren’t being distorted through any defects in their vision. This should usually include an assessment of their field of vision, as significant defects could result in some of the action being missed completely.

Assuming that the official has a good standard of vision, if necessary corrected with either spectacles or contact lenses as appropriate, we can look at how vision is an integral part of the decision making process for sports officials.

We've already seen how perception can affect line decisions in tennis (see “You cannot be serious” blog). This same effect could also cause assistant referees to flag a player offside who isn’t, and an umpire give a narrow run out decision in favour of the batsman.

Tennis line umpires are in a more or less fixed position, but the positioning of officials can be a crucial factor in their decisions. Oudejans et al (2000) proposed that many incorrect offside decisions in football were as a result of the referee’s assistant being behind, rather than level with, the last defender. The Dutch researchers correctly predicted that this would lead to more wrong calls of offside than missed calls of actual offside. But presumably, this would depend on whether the attacker was on the far side of the defender or near side, as viewed by the official. So it could be that they were actually confirming Whitney’s theory of perceptual delay (described in “You cannot be serious”). Nevertheless, it must be the case that poor positioning would cause errors due to simple parallax, even if they don’t necessarily favour the defender.

One of the things that can help with positioning is anticipation. For instance, if a rugby referee can anticipate a likely drop goal attempt, they’re more likely to be able to get in position to be able to tell whether or not the kick is successful.

Ste-Marie (1998) expert gymnastics judges showed better anticipation than novice judges. All judges were more accurate when they had seen the same performance earlier. If there was some minor difference in the two performances (e.g. bent knees versus straight knees) they were less accurate, and if the second performance had a completely different element to the first, this produced the least accurate judgements.

So whilst correct anticipation can lead to improved judging, incorrect anticipation can lead to inaccurate (though rapid) judgements. Judges therefore need to be able to recognise when a new element has occurred, and take the necessary time to re-evaluate it. They also need to keep an open mind at all times. Ste-Marie found that if judges were shown a move with an error, they were more likely to see non-existent errors when the move was repeated a second time.

A difference that is often seen between experts and novices in sport is how they make better use of visual information in decision making. Expert tennis players, for instance, will focus on certain parts of their opponent’s body when determining the direction of their shot. Is there any evidence for officials doing this?

Well, there is some. In gymnastics, for instance, Bard et al (1980) found that expert judges watched different areas of the body than did novice judges. Sed (2008) found a similar result in basketball. When asked about their thoughts when watching a DVD of a match, expert basketball officials focused on specific areas of the court, whereas novice officials tended to watch the ball. This enabled the experts to anticipate better the movements of the players.

Smith & Millslagle (2008) used head-mounted cameras to assess the gaze fixation of elite and novice umpires in softball. They found that when the pitcher released the ball, the elite umpires’ gaze location was 100% on the ball, whereas the novices’ gaze location was only 55% on the ball.

Considering officials that are an official part of the game, such as rugby and football referees, there are a large number of decisions that need to be made during a game. A study of football referees during the EURO 2000 championships found that they made, on average, 137 observable decisions. That didn’t include all the unobservable decisions (approximately 60), such as deciding not to blow the whistle. Most of these decisions have to be made within time constraints. Australian Football League referees were found to have less than a second in which to make their decisions (McLennan & Omodei, 1996).

Cricket umpires generally have more time to make their decisions, but one exception is in determining whether the bowler has no-balled or not. The umpire has to decide very quickly whether or not the bowler’s feet are legally placed before switching gaze to pick up the flight of the ball in anticipation of a possible LBW decision.

The relevant Law begins “In the delivery stride, the front must land……” When I was training to become a qualified umpire, we had to learn this off by heart, and if our answers didn’t have the words “in the delivery stride” we would lose marks. But when you’re umpiring you don’t have time to think about the words, you have to instantly recognise whether the feet are in an allowed position or not. The same goes for a rugby referee when judging whether a tackle is legal or not.

The training of umpires has now improved, and much time is spent demonstrating feet positions, for example. The idea is that these images are stored in the visual memory, providing an instant comparison to be used on the field of play.

If we are to improve at something, we need some kind of feedback to judge whether or not we are actually improving. One of the few studies on the use of feedback for sports officials was by Jendrusch et al (2002) for tennis line judges. Electronic devices were used to assess where the ball landed. Those line judges who received accurate feedback about their decisions showed marked improvements compared with those who didn’t. They showed no general improvement in perceptual skills, but simply learned what to look at when making decisions. A similar system could be used to improve the accuracy of cricket umpire’s LBW decisions using Hawk-Eye.

If we want to improve the decision making of referees and umpires, it would be helpful to know whether this can only be done on the field of play, or whether video training can also be used. One attempt to do this has been by Brand et al (2009) with the SET - Schiedsrichter-Entscheidngs-Training (Referees Decision-making Training). Videos from a variety of competitions are shown, including the German Bundesliga and the UEFA Champions League. Each video is stopped after a crucial incident, usually a contact between two players, and the participant clicks with a mouse to indicate their decision. This system has been cleverly designed to enable several parameters to be changed, such as how much time is allowed to make decisions, and whether feedback about the correctness of the decision is given immediately or delayed. Extra information can also be inserted, such as shirt colour and crowd noise, to explore the extent to which irrelevant information is used (See “Is the ref biased?”).

They've found that it’s better to give immediate, rather than delayed, feedback on the correctness of decisions. This resulted in the most effective increases in decision accuracy.

One limitation with the SET is that the video clips were taken from external pitch side cameras, so do not give the referee’s actual view. This was also a limitation of a Canadian study in which video clips were shown to rugby referees (MacMahon 1999). The experienced referees wanted to know more information than was provided, and often felt the videos gave them a poor view. One answer would be to fit a referee with a head-mounted camera. Another specific problem was that the referees wanted to see whether there was going to be advantage or not, despite the fact that they were told to ignore the question of advantage.

Another use of video has been in point sparring, which is a scoring system used in some martial arts. The determination of whether or not a point has been scored can be a subjective decision by the referee. Krueger (2008) found, unsurprisingly, that first-time referees were significantly less accurate in their scoring compared with more experienced ones. More surprising, perhaps, was the finding that accuracy was generally less when the standards were higher and the two fighters were more evenly matched. This meant that in the finals, when, at least in theory, the two best fighters met, the accuracy of decisions was lowest.

They also found that a referee with 20 years’ experience was generally no more accurate than a referee with only 2 years’ experience. This suggests that experience without feedback can lead to a plateau effect. Virtually all the referees showed systematic errors that could be easily corrected. One referee occasionally awarded a point to obviously the wrong person, apparently confusing which fighter was which. Another was generally accurate, but would make several errors in a short period, suggesting lapses in concentration.

Just as ProBatter has been used to give England’s batsmen a realistic idea of what it’s like facing Test bowlers, virtual technology should in time be able to give officials experience in an environment where mistakes aren’t too costly. Important factors such as crowd noise could easily be incorporated.

In the meantime, I’m always surprised how little officials are used in players’ practice sessions. Not only would it give novices a chance to develop their experience, but established officials could use it to “get their eye in” before the start of the season.

This would also benefit the players. How many bowlers overstep in the nets? Would there be so many offences at the break down if there were referees at practice games?

A novel way of giving trainee officials relevant experience and feedback has been tried in the Australian Football League. First year umpires are given green shirts, and are monitored by an experienced referee during a game. They may be asked to concentrate on just one call, and may stay on the field for a limited amount of time, giving them a chance to reflect. In rugby union, for instance, a similar system might have trainees concentrating on the straightness of the throw or the spacing between players at the line out.

I’m sure a lot could be learned from time on the pitch with an experienced referee, learning how to look in the right places for likely offences.

David Donner