A summer project sponsored by Element14 and Texas Instruments to create an electronic referee to assist in close calls for ball sports
BLOG 2: TI Component Overviews and initial findings The Electronic Referee: eZ430-Chronos, Tiva C Series TM4C1294 LaunchPad, CC110L booster Pack.. Getting to know them
BLOG 3: Sensor components and interfacing initial view The Electronic Referee: Electronic Sensing 101 and isolation.. .did I mention Lasers
BLOG 4: Sensors and Sunshine The Electronic Referee: Sunshine, Lasers and LEDs, How do the sensors fare in the great outdoors
BLOG 5: Radios and more Radios The Electronic Referee: Wireless 915Mhz, 5 Nodes, 5 Different Launchpads
BLOG 6: Update on progress now I have more parts in hand The Electronic Referee - Update + Interface Board almost done (1st one anyway)
BLOG 7: Launch Pad Sandwich's Anyone ? The Electronic Referee - Launch Pad Sandwich's Anyone
BLOG 8, 9, 10: Interfacing designs and options used for the 24V sensors Driving BIG loads with your micro controller - no isolation Drive BIG things with added safety. OPTO Isolators and Sense BIG things with your Launchpad or Arduino, OPTO Isolated inputs.
Blog 11: POC testing and Summer wrap up (The TI Bloging will continue but the summer project has to end somewhere ) The Electronic Referee: POC testing and wrap up for the summer project but not the Bloging
This is now concluded , the POC was a success in that I have a working prototype with line detection, radio transmission of the result to an LCD display, the ability to reset the status remotely etc, but it is with personal reservations due to library compatibility issues and board interoperability and none TI parts not really living up to their expectations (LED Line Sensors), these will be resolved over time but did cause me to not demonstrate everything I wanted to which I am a little sad about as I was really keen to make this as successful as I could and I did not reach my goal.
But there are so many things now waiting to be shown based on my experience, I will continue with a series of TI Launchpad and Booster posts for an extensive range of boards including the brand new TI Stepper Motor Controller Booster Pack so stay tuned and keep the questions coming
BLOG 1: The Project
The seasonal context of this project is summer. There are a number of major sporting events happening throughout the world at this time; one of the frustrations common to spectator sports is the perceived myopia of referees and umpires. From tennis courts to baseball diamonds, a common refrain to officials is
“Are you blind?!?”
An electronic sensor placed on a goal or foul line which could detect when it has been crossed by a ball or a player would be of great value; by utilizing digital accuracy, the reliance on the perspicacity of the referee is diminished. If accurate information could be communicated to the official via a piece of wearable technology, e.g., a watch, quicker as well as more accurate judgements could be made.
So what to choose, there is only enough time to support one sport, if time ends up permitting then a second may be chosen
This year we have at least two major world stage sporting events.
“The World Cup”, Soccer (Football for you Brits) tournament currently underway in Brazil,
“Wimbledon”, the world Tennis tournament in the UK starting today (June 23rd).
There are other sports that require keen eyed umpires and referees too, Baseball for instance and also Basketball.
All have different rules and ways of determining if a ball is in play, out of play and if a player has initiated a foul play resulting in a penalty or other repercussion
All are plagued with the officials being barraged with calls from fans and players for making a bad call as to the state of play (Fans always have a better view and knowledge of the rules in their opinion ),
So how can we help the referees and other officials? An impartial and unbiased referee that cannot be argued with or cares about the verbal abuse, that can make fast and precise calls for those difficult challenging near the line issues and deliver the call right to the wrist of the umpires.
So what are the ground rules for the project:
- Mid June through to end of Summer (August 29),
- Weekly updates in the form of blog entries,
- Uses tech from TI and Newark online store (I’m like a kid in a candy store right now, Oh yea)
- Has a chance of completion in the time scales given without it being a full time job for the next three months?
- Does not break the bank. I have not been given an expense limit but I will be self-imposing limits to be reasonable given the scope and audience of this project and will find the best economical choices for the build
- Must include the TI Chronos watch in the solution Chronos: Wireless development tool in a watch| TI.com
- Use the Tiva C Series TM4C1294 Connected Launchpad Tiva C Series TM4C1294 Connected LaunchPad - EK-TM4C1294XL - TI Tool Folder
- Blog into this space http://www.element14.com/community/community/knode/dev_platforms_kits/element14_dev_kits/ti_launchpad
- And my Favourite and added by me… Must have Lasers somewhere if possible, everyone loves lasers.
So what are we dealing with as far as play is concerned and what potential issues will we need to contend with?
- How to track the ball? Potentially tracking the ball in three dimensional space
- Detect line hit vs miss
- Distinguish between ball and players
- Speed of play? How fast is the ball going thereby setting the needed performance of any sensors?
- Not presenting a hazard for players (Except for the lasers )
Fields and Pitches
Rules about onside, off side, penalty, foul etc. rely on not just the position of the ball but also the players in relationship to each other and the actual pitch (Home team vs away team), all of the ball related infractions can happen in the air as well as on the ground
Tracking twenty two players and a ball in three dimensional space, being able to identify each player and his location to within inches along with the ball will require high speed high definition cameras and very high performance processing power to predict the trajectory of the ball and evaluate if an infraction has occurred even if the ball cannot be seen by the system at the time of the infraction. Ground based line sensors can help but will only cover part of the problem.. Even limiting the sensing to the goal line will not address the goalie being in a beam or on the line and the ball can still pass at any height and angle to cross the goal line. This in my opinion cannot be achieved in the time allotted and within a reasonable cost
There have been many systems tried for Soccer/Football, with Video triangulation being the most successful so far, see here for more details on the systems http://en.wikipedia.org/wiki/Goal-line_technology
and a quick video from TI's web site
In baseball, a foul ball is a batted ball that Settles on foul territory between home and first base or between home and third base, or bounds past first or third base on or over foul territory, or First falls on foul territory beyond first or third base, or while on or over foul territory, touches the person of an umpire or player, or any object foreign to the natural ground.
A foul fly shall be judged according to the relative position of the ball and the foul line, including the foul pole, and not as to whether the fielder is on foul or fair territory at the time he touches the ball.
Foul territory or foul ground is defined as the area of the field outside of the foul lines. The foul lines and foul poles are not part of foul territory.
In general, when a batted ball is ruled a foul ball, the ball is dead, all runners must return to their time-of-pitch base without liability to be put out, and the batter returns to home plate to continue his turn at bat. A strike is issued for the batter if he had fewer than two strikes. If the batter already has two strikes against him when he hits a foul ball, a strike is not issued unless the ball was bunted to become a foul ball, in which case a third strike is issued and a strikeout recorded for the batter and pitcher. A strike is, however, recorded for the pitcher for every foul ball the batter hits, regardless of the count.
Again this is an instance where significant play is in the air thereby requiring video or expensive and complex tracking systems that can predict or track the ball along a trajectory to determine if it is in play or crossing the line into foul land. Any ground based sensors would again have to deal with batter, catcher or fielder interference negating the chances of beams on the lines at ground level for those close calls.
Most basketball play is in the air or the ball is in the hands of the player, many of the foul plays are associated with releasing the ball to late when against the clock or stepping to many times before passing the ball and of course, it would be almost impossible to track the ball with any other system than video and when evaluating the existing systems, it seems “Instant Replay” is used most often to determine if infractions have occurred. The ball is not subject to the same forces as in soccer, baseball or tennis.
Here are some ways a player can lose a point in tennis:
An attempt at a serve may result in one of the following outcomes:
- a good serve
- a let
- a fault
- (very rarely) immediate award of the point to the server.
A good serve occurs when a legally delivered ball lands in the cross-court service box without touching anything in flight. The point then continues with normal play.
A let occurs when a legally delivered ball lands in the cross-court service box having touched the net cord (but not the net post or any other object). Play stops immediately, and the server is allowed another attempt.
There are multiple reasons why a service attempt may result in a fault.
- A foot fault takes place when the server assumes an illegal position while serving. The server's feet may touch only the ground behind the baseline, between the extensions of the centre line and the sideline. The server's feet must not touch the baseline or the extension of the centre line or sideline at any time before the ball is struck. The server is also required to stay roughly on the same position to prevent the opponent from being misled as to where the serve will originate. Running or walking while serving is not allowed, however jumping is permitted.
- Illegal release of the ball. The server must release the ball from one hand (exceptions are made for one-handed players, who may use the racket to release the ball).
- A miss. If the server swings but misses the ball, it is a fault. However, if a server releases the ball but does not attempt to hit it, there is no fault and the server may repeat the service attempt.
- If the ball, before bouncing, hits any object other than the net cord, the opponent's racket or body, it is a fault. So, for example, if the ball hits the net post and then bounces into the correct court, the service is still a fault.
- If the ball fails to clear the net, or bounces anywhere other than the cross-court service box, it is a fault.
Here are the laws of tennis http://www.itftennis.com/media/136148/136148.pdf
For the ball related faults, it is easy for an umpire to call the ones that are significantly out but detecting a ball traveling at up to 140MPH to within an inch or less of the line is beyond most people’s capability even when trained.
Tennis is one of the few ball sports that can benefit from ground based (Or close to the ground) sensor grids that will detect those close calls in real time and alert an umpire to the in or out play status. A simple sensor on the net will detect even a graze if required and because the rules are absolute when it comes to on or off the lines, is can be assumed that during a serve, the sensor can be trusted as a player should not be activating it.
So given the available sports applicable to the theme, the time to implement and complexities of the solution, the logical choice has to be Tennis, so with this in mind, lets look deeper into what we need to understand to build the solution
Yes there are many commercial systems to help umpires with their calls where a ball is marginal or even impossible to determine by the human eye but these have been traditionally expensive or complex and using custom electronics.
What we want to do is leverage the available set of TI Launch pads and couple them with commodity components Newark in order to get the job done
So, what about tennis. Ever heard someone say this with passion:-
You cannot be serious!
Household words these days but originated from the mouth of a very famous tennis player,
John McEnroe. He is world famous for his multiple grand slams but also for his abuse of umpires and staff on the courts of Europe and the USA, this is a great reason to create Electronic Umpires.
Technical fact to be considered in the system
Some facts to determine the requirements of equipment to sense the ball being in or out
Top speed of a tennis serve – 140Mph, or 62.6M/s (205.38 feet/Sec), this is the best tennis players in the world
Equates to 1M in 15mS (1ft in 4.8mS) or 1cm in 15uS (1” in 40uS)
Size of a tennis ball - 6.7 cm (2.63”), the ball will travel its length in 100uS
So this will require a sensor with a response time of 100uS or maybe a little slower as it will be impacting the ground and have slowed down sue to air friction etc., The best I can find on the Newark store is quoted as less than 1mS, allowing for the fact that I’m not going to get a Pro serving for me and factoring in it will have slowed down and be compressing on the ground then re-bounding this should suffice without having to make custom sensors. We will evaluate the capability as we go and adjust if necessary.
Based on standard court dimensions, we will require about 20M sensor range for the cross court service line as the above ground sensors must not trip players or umpires so requires a range of about 60 feet to get to the edge of the whole area.
Additionally we will need one sensor right on the outer edge of the line to detect an IN ball and an additional four sensors about 5cm apart covering about 20cm of ground toward the base line. This should be enough to allow even a poor sited umpire to call an OUT if beyond this range
The first sensor I found on Newark web site was an Omron industrial 20M sensor (part no 86W6683) using an LED emitter, it did not specify the beam width at 20 meters but once I received one and tested it the beam was not narrow enough and even after a few feet was diverged enough to energize several sensors if they were a few centimeters apart at a 20M range
So on further investigation I found is a Banner engineering 20M laser Receiver and transmitter (Part 21H6089 and 21H6111), these use a laser diode so the beam was more than adequate even after 20M but the response time is only quoted as better than 1mS.
So this is how the court can be monitored:
The service line lasers are 5 deep with one laser right on the edge of the line, the base line are a single laser in order to simply detect a foot over the line, the base line show an optional microphone that may be used to detect a ball strike indicating the start of the serve, this would be used to disable the baseline detectors if they had not been tripped before the impact was heard. the service line sensors can be disabled at the next impact being detected or a reasonable flight time has elapsed.
Detecting the side lines along the length of the court has issues due to the length of the court, in addition none of the available sensors have a range of 40M (Length of court plus gap). Placing sensors at the net could work to mitigate the range and this will be included after the initial POC has been completed to prove the technology and approach as the lasers are currently the most expensive single component
Placing some kind of sensor on the Net line will allow the additional fault detection of a net strike, I have not yet figured out the best way to detect this. Some kind of vibration sensor should work but must be immune to wind and other sources of vibration. 2 additional laser sensors will work (One for each side) but these would not account for sag along the length of the net. 2 sensors per side aiming slightly down from each end could improve on this. For the sake of a prototype we could assume the net is straight and go with one sensor per side.
For the prototype, only one side will be created and monitored and one net hit sensor, this includes a sensor to monitor the “IN” side and 4 sensors to monitor the “OUT” side of the service line.
I want to make this more of a community project so if you have ideas, suggestions and constructive feedback regarding improvements on sensors, coding and general solution I am open to all and will welcome the help.
I have some latitude as to the launch pads and accessories to use for this and currently have several components for the system already as follows, I still need a good way to detect the ball strike and some suggestions on how best to approach that. nyway this is my starter kit to begin to build the proof of concept
Banner Engineering 20M laser Receiver
Banner Engineering Laser Emitter
BOOSTXL-IOBKOUT - GPIO Breakout BoosterPack