A current Radio World ebook explores trends in control and monitoring for radio broadcast infrastructure.
Jeff Welton, CBRE, is Nautel’s regional sales manager for the eastern United States. He is a sought-after public speaker and award-winning engineer.
Radio World: Jeff what’s your vote for the most important trend in how broadcasters control and monitor their transmission facilities?
Jeff Welton: Almost certainly it’s the implementation of network operations center-type systems so that a few people can monitor many stations.
SNMP has vastly simplified this, although there are other technologies that could probably do the same job, but the concept of having a small staff continually monitoring dozens, or even hundreds, of stations, with the ability to instantly see if anything is out of the norm and to drill down for details — and sometimes even be able to resolve or patch around the issue, rather than needing to have somebody go there immediately — is a significant change.
It’s critical, given the aging out of the broadcast engineering profession. Fewer engineers are needing to be able to keep an eye on more facilities, so anything that makes that process easier is going to be valuable.
RW: To what extent have broadcasters created centralized infrastructures for monitoring and control?
Welton: Especially in large groups it’s very common, but even smaller groups are using home-brewed centralized monitoring, or one of the premade systems, or one of the contracted monitoring services.
I just came back from a trip to upstate New York where a small group was monitoring 70-odd transmitters by overlaying their coverage map and using the Zabbix SNMP manager to create a custom interface that allowed them to click on any of the signals on the map, see all of the operating parameters, get near-real-time notification of any out-of-tolerance parameters, and send resets or make the determination as to whether a site visit is immediately warranted.
There are options out there and they can be as simple or as complex as the group wants to make them, ranging from free to one-time purchase, to monthly subscriptions.
RW: What kind of customization is available in remote control and monitoring?
Welton: It’s limited only by the budget and imagination of the person configuring it.
Back in the ’90s, PLCs (programmable logic controls) were considered the gold standard for top-end customized remote control, and you could tap off a circuit to use a comparator to trigger an alarm for functions beyond what the equipment natively provided.
Today, with SNMP monitoring, hundreds or even thousands of data points can be managed, such as fan speed, device currents, heatsink temperatures — the ability to customize way beyond conventional forward/reflected power and plate voltage/current is practically unlimited.
If a signal isn’t monitored by the device, it’s possible to create custom interfaces to make it happen. You could always do this, but there were limits, whether time, cost, physical space, etc. Now those limits have been moved back exponentially, with the ability to create custom interfaces so that you can drill down for additional detail, have color codes to indicate severity (off-air vs. reduced power, for example), or even have it notify support staff.
RW: What role does AI play in all this?
Welton: I’m not really able to answer that. However, as to what role it CAN play: Just in the time I’ve been writing this response, the AI app installed on my cell phone was beside me compiling a list of suggestions for creating an interface to monitor forward and reflected power for 40 transmitters simultaneously, displaying all on a 40-inch monitor. It reached out to the web for the MIBs for the devices I referenced, gave ideas for configuring the display and setting up the monitoring system, for use with the Zabbix SNMP manager. It created several pages of tips and suggestions, broken down by topic and function, then ended with a list of ideas for additional improvements, shown in the accompanying image.
AI in general provides a powerful tool for getting things done. It definitely is NOT a replacement for a skilled task person, but absolutely it can scour for information, provide insights, perhaps give ideas the originator hadn’t considered and assist with the creation of a better product.
As with all things AI, though, care must be taken with respect to information security. An open AI engine like ChatGPT is definitely not the place to put any information that could help a threat actor compromise your network or equipment.
RW: Can you recommend best practices for setting notifications and alarms?
Welton: That’s like asking me, “What kind of vehicle should I buy?” This will be determined by the scope of the system, the infrastructure to be monitored, whether it’s one site or multiple, who’s doing the monitoring (an experienced engineer or somebody looking for a red light on the screen) and a plethora of other things.
As a minimum, I like to be notified for audio loss and reduced-power or off-air situations. I set alarms for all of those.
If the site has a generator, it would be a really good idea to have a notification configured for if the generator is running — and monitor whether AC power is on or off, because if the generator is running while AC is on and it’s NOT a scheduled test time, something may be awry! Tower lights are another good thing.
The caveat: It is easy, especially with SNMP, to fall into the trap of monitoring an overwhelming number of things. You probably should have a red light on the screen for a PA failure in a large solid-state transmitter, but you might not want that red light to translate to a 3 a.m. text or alarm, when it’s a problem that could easily be handled during business hours.
RW: How can off-site monitoring and management services cope with clients whose facilities have a lot of customized components?
Welton: That depends on the service. Some can provide interfaces to connect to custom components. If the device in question has SNMP functionality, it’s as simple as loading the MIB for it. If it doesn’t, perhaps an external RC with SNMP output and programmable I/O will be required. If there is a situation with a lot of custom devices, especially in these days of SDR and Raspberry Pi devices that can be configured for almost any conceivable function, it may require significant back and forth with the monitoring service to determine what needs to be monitored and how to achieve that.
Absolutely make the request — some will be more comfortable with standard setups and minimal wizardry, others may be more flexible. Ask before assuming, no matter what.
RW: How can a user protect their systems and related infrastructure from cyberattacks?
Welton: This comes back to the AI reference. First and foremost, but surprisingly not as common as it should be, firewall the system. There are still a shocking number of broadcast devices connected wide open and unprotected on the internet. If it’s visible, even if it’s not using default usernames and passwords, it’s a potential target for a brute force or DDoS attack.
I’m a fan of VPNs, although it’s critical not to poke holes in it — don’t leave any ports open that aren’t required for functionality. Use IP whitelisting, configure for zero trust (i.e., nobody gets access if they don’t need it). Run periodic scans, whether with something like a paid Shodan membership, or just monitoring the network traffic with WireShark over the course of a few days, to verify that nothing is reaching out needlessly — and that nothing is allowed in that hasn’t been approved.
In some cases, it might be beneficial to contract an outside IT security firm to run penetration testing. Costs on this can vary so get numbers ahead of time, but an independent expert can frequently point out holes that you might have missed.
RW: Please send a photo of yourself, if possible in a radio work environment.
Welton: I’m going to use the one shown here, but with a caveat.
There’s a lot of fuss about the amount of information people enter to get an accurate AI caricature, and they’re correct. My prompt for this image was “Create a caricature of me using only the information that you already have.” I’ve used ChatGPT for a few SDR projects and, of course, after three decades of presentations and articles, I have profile pictures scattered all over the internet.
People do need to understand, though, that any information they put into any open AI engine, such as ChatGPT, is available on the public internet forever. So a lot of discretion is required, as with the question about cyberattacks. Because it’s not just you who can access this information.
Read more in the ebook “Trends in Remote Control & Facility Management.”
