Transmitting audio over the Internet is a reality for almost every radio station these days, and Rhode Island Public Radio is no exception.
In fact, we use it even for mission-critical purposes like getting our main programming audio from our studios to our transmitters in Seekonk — WELH(FM) — and South County — WRNI(FM). But while audio-over-the Internet is so ubiquitous that everyone takes it for granted, it’s important to remember that, strictly speaking, it’s a terrible idea and the Internet was never designed for it.
That’s where nifty devices like the new APT SureStreamer come in.
By “never designed,” I mean that the core design characteristic of the Internet is resiliency. If a nuclear war happens and half the network that made up ARPANET (the military precursor to the Internet) was destroyed, the packet-based nature of the data flowing across it would keep things working. And use of TCP meant the far end would recognize any missing packets and ask for them to be re-sent. This is great if you’re downloading a file or sending an email, but it can be problematic for streaming media, like audio or video.
STREAMING SOLUTIONS
There are several solutions to this thorny problem. First is that instead of TCP/IP, we use User Datagram Protocol. UDP doesn’t have a bi-directional nature. Lost a packet? Too bad! That packet is dead to us! Leave him behind! Without that bi-directional nature, you have far, far less overhead and processing time needed. UDP works fine even under far poorer network conditions.
You can also use some clever tricks like Forward Error Correctionand Reliable UDP. Both add a little more delay (but not so much that it’s a problem), and they make the overall system much better at successfully getting packets from Point A to Point B.
You could also throw money at the problem by investing in more reliable infrastructure, like fiber-optic Internet delivery; and once you’ve got that in place, you can make use of Quality of Service technologies, like MPLS,’which direct the network to give your packets a certain priority from end to end, thus better ensuring they arrive on-time and in-order. These technologies are quite expensive, though, often costing $1,000 to $4,000/month. Although, for that much money, you tend to get much better customer service during a problem!
Even with a pricey fiber line, things can still go wrong — very wrong sometimes. A tree can fall on a telephone pole and snap all the wires. A construction worker can dig right through the wrong conduit and destroy your fiber line. Even the best service contract in the world isn’t going to change the fact that it’ll be hours, if not days, before that cable can be repaired.
Enter the APT SureStreamer. Instead of relying on any one Internet connection, and trying to make that connection as reliable as it can be, the SureStreamer aggregates up to four separate Internet connections, all simultaneously. Put a SureStreamer at either end, plug your codec (in our case, Comrex BRIC Links) into the local LAN, and the local LAN into the SureStreamer. Then connect as many Internet Service Provider connections as you want into the remaining three Ethernet jacks. You’ll want at least two, although unless the third follows a physically redundant path (e.g. the same backhoe can’t fade all your ISP links at once) then two is all you really need.
Lost a bunch of packets on the fiber line? No big deal — exact copies of those packets made it just fine over the cable-modem. The audio didn’t even hiccup!
In late July 2015, RIPR acquired the first two APT SureStreamers in the entire USA for demo units/beta testing. It took a lot of fiddling but we eventually got them set up properly, thanks to APT’s help. Once the SureStreamers are connected to each other, you just configure your audio codec to send a Realtime Transport Protocol stream to the local SureStream and voila! It comes out the other end for your codec to accept.
On the studio end, we have a Cox fiber-optic Internet connection, and a Cox Business cable modem. On the transmitter end, there’s another Cox Business cable modem and a Verizon DSL line. This is important because cable modems don’t work when power is out in the area; even if you’ve got a generator on-site (as we do) the cable modems need power on the poles for the equipment to function. DSL does not; as long as there’s power at the local central office — the DSL will function. Similarly, even though fiber optics are usually quite reliable, they do need to be taken down sometimes for maintenance. (See Fig. 1, our network diagram.)
The important thing to remember is this: If you lose connectivity on either IP link, the SureStreamers don’t care. They just chug right along, and the audio comes across the BRIC Links with nary a hiccup. APT’s own tests show zero or near-zero actual packet lost over months and even years of testing, because even though each connection will lose some packets here and there, they almost never lose the same packets at the same time.I’ve verified this myself by simply unplugging the network from one of the ISPs while watching the network statistics page on BRIC Link. You can see a screen capture of it, above in Fig. 2.
The row of blue columns in the middle shows the last 60 seconds of network performance in terms of target delay, jitter, and actual delay. Below that is a row that’s currently black. Black is good — if there were frame loss (dropped packets) you’d see red columns, and in fact if you look closely right above the “T-30 Seconds” in the middle, you’ll see a tiny smidge of red, indicating a slight frame loss, not enough to cause an audio outage or distortion. It’s hard to see, but flush with the bottom is a tan/brown line indicating the frame loss at the far end of the connection is also at 0 percent.
So to summarize, RIPR now has a system that can deliver better reliability than an expensive fiber optic line at the Narragansett tower, and is using a mere cable modem — and DSL line. Instead of spending $1,500/month we’re spending more like $200/month for both connections.
Technically, we are still vulnerable, since the DSL and cable modem both come via telephone poles along Point Judith Road and Westmoreland Road to the tower. A falling tree could wreck our whole day, although that’s always been a concern with WRNI(FM) … even when we had a Verizon PAS circuit for our STL (Verizon discontinued the service in March 2015).
At the moment the software does not support any 3G/4G cellphone Internet modems for use via the USB ports, but it is planned for a future firmware release. I’m keeping my fingers crossed for some time in 2016.
Read is the director of IT and engineering for Rhode Island Public Radio.
