Few college students were alive the last time Jim Kelly threw an NFL pass or Barry Sanders ran for a touchdown. But they’ve probably never watched a football game on television that didn’t have a yellow line superimposed on the screen to show how far away the offense is from picking up a first down.

The technology is called different names by the various networks. By any name, however, it’s one of the great innovations in the marriage between TV and sports.

ESPN was the first to display the first-down line

Watching football on TV without a yellow line 10 yards up the field from the ball on first down is like driving on a two-lane highway without the yellow stripe between lanes; it’s possible but just doesn’t feel right.

The “1st and Ten” system from Sportvision Inc. made its debut on ESPN on Sept. 27, 1998, as the Cincinnati Bengals took on the Baltimore Ravens. ESPN had exclusive access to the technology that first season, paying $25,000 a game, according to Vox.com.

The yellow first-down line quickly became a fixture shortly afterward on every network showing NFL or college games as other tech firms developed versions. Variations of the technology are used in other sports, too. In baseball, for instance, the strike zone is superimposed over home plate and ads that are invisible to fans attending the game are superimposed on the base of the backstop.

Simple in theory, complicated in reality

Drawing a superimposed straight line on the playing field isn’t a simple procedure for a variety of reasons beginning with the fact that fields aren’t flat. However subtle, all fields have a crown that helps with drainage. In addition, no two fields have the exact same colors under the same lighting conditions. Since the networks have to “teach” the software what colors it can and can’t replace on a TV screen, getting it right can be time-consuming.

To understand why the latter is important, think of the weather forecast on the nightly local news. The meteorologist standing in front of the map displaying temperatures across the region or a satellite image of an approaching storm is actually standing in front of a blue or green screen. A computer generates the images on TV monitors that the meteorologist is watching while he or she points. The same image is incorporated into the digital signal sent to home televisions.

The whole system breaks down if the on-air talent is wearing clothing that is the same color as the screen behind them. The first-down line can also be fooled by uniform colors unless the TV technical crew carefully tweaks the settings.

How do networks show the first-down line on TV?

The single biggest hurdle that engineers faced in designing the system that displays the first-down line on TV was getting accurate live-time data from cameras on their pan, zoom, and tilt (PZT) angles, according to GlobalSpec.com. A close second was getting the computers to recognize which shades of green (grass), brown (dirt), and white (yard lines) are OK to cover with the yellow first-down line. Team and league logos on the field also have to be considered.

The groundbreaking “1st and Ten” system required six cameras feeding images and PZT data into a computer. That computer’s calculations were fed to another computer that held a carefully measured and constructed 3D model of the field.

The line was generated on the overlay from the 3D model, which the computers kept in sync with the live shot. The “flattened” image was then sent out as the TV signal for viewers to see at home.

In the early days, the amount of hardware required filled a tractor-trailer. As computing power has continued to grow exponentially, the amount of equipment required has dropped drastically.