FCC Invents Negative Distance

The trials of correcting accuracy miscalculations within commission rules
Publish date:
Social count:
The trials of correcting accuracy miscalculations within commission rules

SAN FRANCISCO—Remember the wiring error in the FCC seal when the commission rolled out its new web site, as shown in Fig. 1? A bit embarrassing for the agency, and probably drove the small number of engineers at the FCC nuts every time they saw it. But it was a quirk, right? Unfortunately, there is a sequel: The FCC has inadvertently created a rule that includes negative distance. Think I am kidding? Read on.

Image placeholder title

Fig. 1: The “oopsie” FCC seal (left) versus the correct FCC seal (right).

In the June 14, 1983, Broadcasting and Cable Bureau (BC, now the FCC Media Bureau) Docket 80-90 Report and Order (R&O), the commission converted the tables in the Part 73 rules giving the length of a degree of latitude and longitude to the trigonometric series that generated those tables; their source was Table 6, Length of a Degree of Latitude and Longitude, from The American Practical Navigator by Nathaniel Bowditch (1773–1838), still a great read to this day.


The approach is to determine the mid-latitude (ML) in degrees between the two points of interest, calculate the length of a degree of latitude and longitude for that ML, and multiply by the change in latitude and longitude to form the two sides of a right triangle. The Pythagorean Theorem is then used to determine the length of the resulting hypotenuse. The formulas are:

Dlat = 111.13209 – 0.56605cos(2ML) + 0.0012cos(4ML) Dlong = 111.41513cos(ML) – 0.09455cos(3ML) + 0.00012cos(5ML) where Dlat and Dlong are the length in kilometers.

Because these trigonometric series are for an ellipsoid model of the Earth, this method is more accurate than the spherical earth model for distances of up to about 400 km. Section 73.208 of the FCC rules specifies this method for calculation of station separation distances, and other Part 73 and Part 74 rules are also linked to it. As a historical side note, the BC Docket 80-90 R&O truncated the trigonometric series coefficients to just three digits to the right of the decimal point and used only the first two terms in each series.

This not surprisingly caused accuracy problems. So, in Mass Media (MM, now MB) Docket 86-144, the commission asked for input regarding accuracy issues, and as a result of H&E comments pointing out that this would result in needing two sets of formulas—one for FCC distance calculations, and another set of full-precision formulas that would match USGS topographic maps—the Sept. 25, 1987, MB Docket 86-144 Second R&O adopted the H&E suggestion of using the full-precision coefficients and all of American Practical Navigator published terms.


The distance calculating method in Section 73.208 worked so well that in the Oct. 21, 1998, WT Docket 98-20 R&O, the commission created a supposedly matching rule section in Part 1 of the FCC rules: namely, Section 1.958, Distance Computation.

Unfortunately, a typographical error was made, and the adopted formula for the length of a degree of longitude had as its first term 111.41513cos(5ML) instead of 111.41513cos(ML). Since this was the first term of the series, it was a huge error. Indeed, as shown by Fig. 2, a graph of the length of a degree of longitude as a function of the mid-latitude, it causes that calculated length to go negative for mid-latitudes from about 18° North to about 49° North. Since contiguous U.S. latitudes run from about 24° North to about 48° North, this is a problem.

Image placeholder title

Fig. 2: The consequence of having 5ML in the first Dlong cosine term instead of 1ML.

The error is mitigated by the fact that when you square a negative delta longitude distance you get a positive number to then be added to the square of the delta latitude distance, but you will still end up with an incorrect calculation of the distance unless the two points are exactly N-S of each other, or at the two "sweet spot" mid latitudes.

For example, the correct distance between KGO-TV in San Francisco and KCRA-TV in Sacramento is 101.65 km, but using the incorrect Section 1.958 formula gives 119.68 km. What makes this error so dangerous is that you still get a reasonable looking result; but one that could cause you to think that a minimum separation requirement was met, when in reality it wasn't, and the reverse, if you're calculating the distances between points with mid latitudes of less than about 30° or greater than about 45°.

So you would think that the commission should be able to correct this obvious error administratively when it was pointed out, and give public notice that it has been done, right?

Wrong. Because the WT Docket 98-20 rulemaking has been completed, the legal side of the FCC said that it is too late to fix the formula error. Accordingly, on March 21, 2011, H&E filed a Petition for Rulemaking, formally asking the commission to fix the error, plus some other tweaks. But, several months later, no action has been taken; not even a Rule Making (RM) number has been assigned.


Turns out the FCC Secretary's office managed to lose the H&E petition not once but twice! They have it now, although it's not clear if it's our first, March 22 filing, or the second, April 6 filing. Nevertheless the Section 1.958 longitude distance formula, booby-trapped 13 years ago, remains uncorrected. Sigh.

Perhaps it's time for more engineers at the FCC. And in this writer's opinion it would be wonderful if the attempt 20 years ago for a statutory requirement that at least one FCC commissioner be an engineer, H.R. 3501, the Federal Communications Engineering Sciences Qualification Act of 1991, could be re-introduced as the big brother of Maine Republican Senator Snowe's S.2881 (now S.661), which of course is the FCC Commissioners' Technical Resource Enhancement Act. S.661 is certainly a step in the right direction, but why stop there? For a copy of the think-bigger H.R. 3501, which had garnered sixteen co-sponsors when it died in Committee at the end of the 102nd Congress, see eibass.org/images/filings/senator snowe dec23.pdf

The author is Co-Chair, Engineers for the Integrity of Broadcast Auxiliary Services Spectrum (EIBASS) and a senior engineer with Hammett & Edison, Inc., Consulting Engineers, San Francisco.

This article originally appeared in TV Technology.