SPOF is a four-letter acronym in common use today.
“SPOF” refers to a single point of failure and can be applied to almost system. Some examples where you can have sneaky SPOFs hide are: audio chain paths, transmitters, towers, backup power systems, cooling systems, pressurization, internet circuits and well, almost anything.
In spite of your best efforts and well-crafted systems, some strange problems can arise, particularly when your systems are under stress. When I say “under stress,” I mean to indicate those times when your facility is operating at less than nominal conditions — as would be the case during a hurricane, wildfires, extended power outages and the like.
At one facility in a major market, my team uncovered a serious SPOF. Now, understand this was a well-equipped facility with redundant audio chains, redundant PPM encoders, redundant STL systems and studios. But with all of that, this facility was hanging by a very thin thread. It turns out that two of the AM stations shared a distribution amplifier — one of those two sets of eight outputs style units — and both air chains of both stations were fed by this one DA. This meant that, if the unit’s power supply fuse opened, both stations would be instantly off-air, with no way to bypass this failure (without a long drive to the studio facility).
This oversight was by no means limited to this facility.
Across the country on the other coast, I found a similar situation, in which the station had two exciters, two transmitters, two antennas, two audio processors and emergency power and yet, was hanging by an equally thin thread. The engineers had installed an AES DA for both audio chains powered by a single wall wart (not even secured to the power strip — it could easily have fallen out). So with all this equipment, that wall wart’s potential failure spelled big trouble.
These two examples likely seem obvious to the reader and may leave you scratching your head, wondering what the engineers were thinking. (Answer: They weren’t.)
But let’s dig deeper and discuss more hidden, insidious SPOFs, which may not manifest themselves until times of stress emerge.
MORE EXAMPLES
Recently, a widespread and extended power outage occurred in a large market in which a circuit breaker tripped — which happened to feed the generator block heater. (Recall a previous Tech Tip column which discussed generator systems monitoring, including block heater, battery float voltage, etc.)
That in and of itself wasn’t the real issue. The major concern is that the generator day tank fuel pump was on the selfsame circuit breaker as the block heater, so when the block heater breaker tripped, the circuit to the day tank fuel pump went down with it. I believe you can figure what happened next.
In another market where I was the CE, Hurricane Irene caused widespread power outages that lasted for days. Our studio generator and those at our transmitter sites worked well (happily), and all seemed fine until — we suddenly had a great deal of water falling in our break room — not from the hurricane rain but from one of the studio AC units in the ceiling. The drain pan pump was on the non-generator supported electrical circuit and had not been operational since we switched to generator power.
This is the sort of thing the engineer may not be aware of (unless the engineer installed the unit with his/her own two hands). But it’s certainly something you may wish to check during your next AC inspection.
SPOFs can be even more subtle than those I discussed above. One last example I’ll share is also derived from the Hurricane Irene experience — which I promise you’ll agree is a “gotcha” or even just plain bad luck.
At one of my transmitter sites, my generator is an LP unit. I won’t get into fuel system advantages/disadvantages here, but I prefer diesel, if only based upon what happened at this site during the storm. So power is out and has been for several days, and the LP generator was running smoothly, but the fuel level was beginning to be a concern. This fact, coupled with the utility estimate of “several days” before restoration of utility power was quite worrisome. I couldn’t reach the LP vendor because, well, they were also without power and were not operational. I reached another LP vendor only to learn that by law, they were not permitted to fill another vendor’s tank!
Fortunately, I have a lower-power backup site for this station and the LP vendor showed up as if by magic when the tank gauge indicated 15 percent fuel remaining. Take this under advisement, if you have an LP-fueled generator.
Our job is more complex and intricate than most people (including ourselves) will ever understand, and there’s a great deal to consider. But remember: The time to fix your AC is when the snow is falling — or some variation of that saying.
Dennis Sloatman is the VP of engineering for SummitMedia Corp.
