How do engineers in a big group communicate? Are owners reversing the trend of losing technical people to I-T? Who controls the LAN at a radio facility?
(click thumbnail)Top engineers discuss management issues. From left: David Baden, Frank McCoy, Margaret Bryant, Al Kenyon and Tony Masiello.
A highlight of the NAB2001 convention this year was the “Radio Engineering Roundtable,” which brought together five prominent radio engineering managers who discussed these questions.
The conversation, moderated by Radio World Editor Paul McLane, focused on challenges facing top-level engineers, including how to manage in the group environment; how to communicate with engineers at hundreds of stations; what role engineers should play in planning Internet strategy; and how broadcast and I-T departments should interact.
The conversation also touched on facets of the huge new digital facility operated by XM Satellite Radio.
Here is the transcript of the session, which took place in April 2001 at the NAB convention in Las Vegas.
The participants are David Baden, chief technology officer of Radio Free Asia; Margaret Bryant, director of engineering and technical operations for ABC Radio Networks; Al Kenyon, vice president of projects and technology for Clear Channel Radio; Tony Masiello, vice president of operations at XM Satellite Radio; and Frank McCoy, vice president of engineering for American Media Services.
McLane: In 2001, how does one manage a radio group’s engineering functions? Let’s start with Clear Channel.
Kenyon: We have Jeff Littlejohn, who is the vice president of engineering services, who operates out of our radio corporate division offices in Covington, Ky. – which is just across the river from Cincinnati, but we’ve been open all the time, instead of closing at night like Cincinnati has recently. (Laughter).
(click thumbnail)Al Kenyon
Under Jeff there are 15 regional engineers who handle geocentric areas of the country. Tom Cox has the largest station load, up around 120 radio stations. The lowest station count for a regional engineer is somewhere in the forties.
We’re somewhere between 1,200 and 1,300 domestic radio stations, which if you check the back of Broadcasting magazine is roughly 10 percent of the radio stations in the country.
The regional engineers each work with local management, the local general manager, the local chief engineer, the local director of engineering, on purchases, on maintaining the equipment. This infrastructure is also put to use a lot during our capital process.
Like almost every organization, you take capital requests (that) start out at the station level. In Clear Channel, they go from the station level to the regional engineer, who goes through and vets it. He takes out silly things, like the request for two pop-up tents for remotes because they only last six months and they want to depreciate that over a five-year period.
He then discusses it with local engineers and they put together a package. It’s submitted to Steve Davis for review. He accretes it all and we figure out where we are and try to combine it with the reality of where we have to go. It goes back and forth between people so that everyone is involved in the ultimate selection process.
Some of it is pushed all the way back down to the local engineer. “You’ve got a choice: you’ve got this much money that you can spend. What’s the most important project for your station?”
That’s the only way we’ve been able to come up with to manage that large number of stations.
McCoy: The worst you ever got was a request for two pop-ups? (Laughter) I’ve had a request for a margarita machine!
Maybe it was in the capital budget as a target. “If you’re gonna cut something, this will go.”
Kenyon: Did they try to capitalize the first load of tequila?
Baden: We’re fortunate because we’re a fairly new company, only five years old. Before I came to Radio Free Asia, I worked at Radio Free Europe for 15 years – I’m not a very clever person at changing careers.
At Radio Free Europe, almost two technical departments developed. As computers became more prevalent, there was a whole I-S department built up, and they found themselves with the technical group that was doing the transmitters and the broadcasts, and the technical group that was handling all the computers – two separate networks, two separate computer requests and capital budget logs.
But we’ve been fortunate at Radio Free Asia that we control the I-S function as well as the broadcast function.
McCoy: One of the challenges is not having the folks at the local market level think, “Oh, gee whiz, it’s those ‘no’ people again.” It’s hard not to be a “no” person, but obviously there’s a finite pie that can be spent on various different things.
I believe that every market-level operation is filled with people who are genuinely committed to their work and really want to do a great job and look at themselves and compare themselves to the competitor across the street. They see that those folks have a shiny new vehicle and they make those assessments. Those are reduced to a capital budget and sent in.
Sensitivity becomes pretty important. It’s easy in a large environment with many, many stations to sometimes lose sight of the fact that every single item on this list, some of which you’re going to take out, has a constituency at the local level. Somebody is going to be disappointed they didn’t get this tool, and they feel that it would give them the opportunity to win, where maybe they’re not now.
Kenyon: That’s why we developed the process, sort of a washing process, where the requests come up and the budget restraints go back down and you try to make the choices at the grass-roots level. You get a buy-in so that the people in the individual markets feel empowered to make decisions and have some understanding of what constraints the entire company is under.
McCoy: When you shorten the list then, you give the local folks some opportunity to have some input on taking things out, choose this or that?
Kenyon: Yes. It takes longer but it creates buy-in and creates a sense of teamwork. We’re working together to achieve a goal instead of having things come down from on high.
At a company I worked for a number of years ago, you’d send a capital request in and it’d be approved, and it would go over to somebody in the TV division. You’d get a DA that was designed for television use that didn’t have any application in radio. It just showed up at your doorstep.
McCoy: Then there becomes the perception that whoever is up at the top doing the purchasing, clearly they’re getting taken out to dinner by somebody.
McLane: Maggie, how do people communicate at ABC Radio Networks? Is e-mail a much-used tool for engineering communication? Is there a regular engineering discussion?
Bryant: Dallas is the headquarters for ABC Radio Networks. The news stuff comes out of New York; Paul Harvey of course is in Chicago. We have a facility in L.A.
It’s very interesting to see how things have changed. I think a part of it is the style of the personalities involved. It used to be that almost everything was e-mail, and there’s still a lot, but more and more of it is on the telephone.
It has to do with the new vice president of engineering, Christine Ianuzzi. It is very much for the better. There are more conference calls, which one may think is a waste of time, but we have so many diverse locations that are working on so many different projects.
We are more in tune to what all the various areas are doing now, more than ever. Conference calling or just being on the phone and talking with people, there is more one-on-one rather than e-mails. I like the change.
McLane: Tony, you’ve been on both ends of the terrestrial/satellite, traditional/new-media divides, what’s your take on this?
Masiello: Prior to joining XM, I spent 11 years in charge of the technical aspects of the CBS radio division. In the late ’80s, early ’90s, we employed the same regional technical management.
At the time, we only had 26 O&Os, not the thousands that big groups have now, so it was somewhat more manageable.
But there is a totally different model at work at XM for technical management. Since all of our stations are located at one very large facility, it takes on a different style of technical management.
We employ something that’s used a lot in I-T-type, Internet-type, large telcom environments. We operate on the principle of SLAs, or service-level agreements.
My department is responsible for the operation and maintenance of the facility, and we have different tiers. Tier 1 is the operators in the various control centers. We have a broadcast operations center, a network operations center and I-T operations center. I am responsible for the I-T as well. Those are the front-line operators in the seats, working shifts around the clock.
If something breaks, there’s Tier 2, the maintenance folks who come out and fix it. Then there’s Tier 3, the engineering people who have to solve a problem. It’s beyond the maintenance people, it’s a system design problem requiring extra expertise.
McCoy: Where does the CEO’s laptop figure in to the service level agreement?
Masiello: Usually near the top. (Laughter)
McLane: I’ve heard this from many engineers. Particularly in smaller operations, it seems the engineer is still the jack-of-all-trades, who is expected to know how to fix the toilet and the laptop.
Masiello: There’s no doubt about it. We have a large facility, but it’s the same structure. The guy that’s watching six or seven stations is all of those levels, Tiers 1, 2 and 3.
But we have the luxury of being able to start from scratch, use a different methodology and having such a large facility. Everything is in one place, so we can have that kind of structure, but you’re right. A guy out there who’s doing his job is all three of those levels.
Baden: But is that a bad thing, to be in charge of anything?
I’ve been in organizations where they’ll start up a whole I-S department. They’ll pay these guys who (are) nothing but Microsoft-certified, and they make twice the money as the guy who built the console. It’s because they’re in the new field and thanks to this new I-T thing, they have more earning power. That creates a lot of problems, to have a whole separate support division.
Masiello: We took great pains to make sure with the different service levels that the personnel, who may have different titles, are all within the same salary ranges. They may call them different things. “Principle Software Engineer I” is akin to someone on a manager level. Broadcast ops or maintenance may have the same pay scales for that very reason.
Within an organization, those providing the services on the same level are at the same pay scale.
Baden: We use a faceless automated help desk system where we don’t even take calls from users; they have to send an e-mail.
McCoy: So Tony, someone who writes code gets paid the same money as the guy who works on a bench?
Masiello: If the guy is what we call an analyst, not writing the code but fixing it, yes.
McCoy: So software maintenance and hardware maintenance get paid the same. Wow.
Baden: How do you justify that? Aren’t you either overpaying one engineer or underpaying another?
Masiello: I don’t think either one applies. Nowadays, when you want to get a broadcast maintenance tech, you don’t want someone who knows how to fix an Ampex 350, because we don’t have any.
Actually I have one that I took from CBS that they gave me as a souvenir.
McCoy: What if you do know how, but you can do other stuff, too?
Masiello: Well, that’s it. If you know how to open up a PC and change hard drives and do troubleshooting and can take an AES analyzer and all of that …
I think the levels of the two between the software person and the broadcast maintenance person are converging because more knowledge is necessary that’s closer to the software than to the broadcast side. The software guy cannot fix the cart machine if it breaks; and even though we have an all-digital facility, there are some cart machines for the odd cart that shows up.
McCoy: You still have two disparate groups, don’t you? Basically software/information technologies and then the folks who are responsible for audio delivery? Even if it is AES?
Masiello: No, because at our facility when you start talking about digital in general, there’s a convergence. There are hundreds of Compaq workstations that are the actual playback devices.
Bryant: But are they also the ones who are taking care of the laptops that are not used for on-air?
Masiello: They certainly could, but they don’t. There are people in the I-T department that take care of corporate I-T.
Bryant: That’s kind of the way that we have. We have an I-T department that takes care of the desktops and all that. And the engineering department is responsible for anything on-air.
Masiello: Anything that’s broadcast related. Absolutely.
McCoy: Could this eventually result in a turf dispute over the LAN?
Masiello: Yes it does, but we set those priorities out early.
Baden: That’s why it’s better to seize control over the whole thing. You really don’t want to be fighting over your LAN.
Bryant: There’s another solution. All of our LANs from engineering are separate intentionally from everything that’s I-T.
Masiello: Totally separate LANs on the broadcast floor. Broadcast is on the second floor of our building and we have 14 LANs dedicated to doing specific things that relate to air, totally apart from corporate I-T.
McCoy: That just eventually moves the dispute to the router. (Chuckle)
Masiello: Yeah, but the corporate I-T is so minimal. What are we talking? An occasional attachment?
It’s getting into the argument with the corporate people as we try to decide, because we’re on a limited budget, who gets the new generation of PCs. We’ll tell them flat out: only power users.
“Using an Excel spread sheet and Word is very impressive to you, but you’re not a power user.”
McLane: You have how many workstations in your facility?
Baden: About 350.
Masiello: Our TOC or rack room has close to 300 racks and I think 20 are dedicated to corporate I-T. The rest are broadcast-related. We monitor the activity closely and we let them in at our will and pleasure. We don’t let them go in and mess things up.
McCoy: But they do have keys to the building?
Masiello: Yes, we let them in the building occasionally.
Baden: The problem is if you don’t control all the technology, you’ll have some executive traveling with a brand-new Pentium VIII laptop with a billion Gigahertz, and you’re being told that your broadcaster has to use that old 386 because you can’t afford any better.
McCoy: Or a variation is, the CEO comes to rely on and become more connected personally with the I-T people than the broadcast engineers. Then you’re fighting a politically uphill battle there.
McLane: Now chief engineers have to be I-T-trained; but once we get them that training and we get them their degrees, they’re marketable and they can double their money by working for some non-radio entity. Another problem is getting entry-level people interested in developing these special radio skills.
So how do we train people, keep them and compensate them well?
Kenyon: You definitely do not want to keep training from your people. Research has shown that people will develop more job loyalty and more job satisfaction when they have more training opportunities.
It’s a better reward than dollars. It increases their flexibility. One of the reasons most of us got into this industry was that we got to play with electronics, with everything from DC to daylight. You get to dabble in civil engineering, mechanical engineering, with towers, HVAC systems.
We’re the jack-of-all-trades at many of our properties. Most of us started out at that level, where we did everything from marching around with a plumber’s assistant to working with computer systems.
If you can offer training, it shows that you’re interested in your people and interested in making them more valuable. Yes, you may lose some of them to the I-T industry, but broadcast engineering offers things that I-T doesn’t.
And frankly, we’ve gone after some of the brighter I-T people who want to do more than play with ones and zeros and do board-level replacements on things.
We’ve brought them in to broadcast engineering because they get to play with RF systems and big pieces of copper that they don’t really relate to. People who find that part of the I-T discipline confining are the broadcast engineers of the future.
McLane: Do your organizations have a policy of helping to pay for employee tuition?
Masiello: Yes. We send our technicians for training on specific devices and systems. If that requires knowledge of NT or Windows 2000 or Cisco routers and switchers, they are sent for that training.
We had an interesting case. A couple of very good technicians who were hired by the I-T staff actually decided they’re going to quit because they couldn’t take the “come fix my PC,” “my mouse is dead,” or this or that. “How come you’re not here in 30 seconds?”
They came over to the broadcast engineering side and found it much more orderly. They got to expand their horizons into doing things that they consider more meaningful. They’re not helping someone get their Windows application running so they can play solitaire. They’re helping to get programming on the air.
Kenyon: I have to point out that, in Tony’s organization, it’s more orderly because they don’t have to put up with AM DAs quite as much as the rest of us!
In our organization, we have an I-T department. It’s often said in technical circles that “things” happen and you can’t spell that particular thing without “i-t.”
We also communicate by e-mail. We have an e-mail list server. For some strange reason, just about all the engineers subscribe to the I-T listserver and all the I-T guys subscribe to the broadcast engineer listserver.
McLane: The lines are blurring.
Kenyon: Definitely. Frankly in smaller markets, the engineer is the I-T person. A number of our regional I-T people are former broadcast engineers because they understand the aspect of the business that it has to be “up” 24 hours a day.
And that’s one of the things that makes broadcast engineers very desirable to the I-T industry. We’ve been working in an industry where we understand things have to be up 24 hours a day and address needs with some urgency. A lot of I-T people have come out of business where the attitude is, “Oh, it’s after 5:30, we can shut down the server.”
We had a gentleman who was very well-intentioned who tried to save some money on our Denver build-up by putting a very large intelligent routing-switch system that would support sub-routers in software in the system.
I said, “That’s all well and good, but we’re running Klotz consoles, and the consoles and the Vadis frames need to connect through this router. When do you expect to maintain that?”
He said, “Well, I’ll just come in here and shut it down some evening.”
I said, “You realize that when you shut it down, every audio source that’s open stays open; every audio source that’s off stays off. We have no level control. Just how fast do you think you can do maintenance?”
The guy turned a little pale and decided his plan wasn’t going to work out all that well.
Masiello: Just to reinforce what Al said. There is, in a big organization, and even in a small one, this rivalry between the factions that have computer knowledge, I-T and broadcast-related.
But the real big difference is that most I-T departments are in support of the corporation and the business process. (But) I-T as it’s used in broadcast is the business. We are carrying the signals. They are generating the audio and even the video. So there’s a big difference. Everything broadcast does is “mission-critical.” Most corporate is not.
If the file server for e-mail is down, you’re still in business. If the file server that’s putting all the audio out is down, you’re out of business.
And that is something that most I-T managers don’t understand, that you can’t just turn things off willy-nilly. You have to have response levels that are two hours or less to replace dead components.
That means that when you design a facility that uses a lot of computer infrastructure, it must be redundant. Because no matter what any salesman tells you, it’s not reliable. It will break. So you protect yourself by having multiple versions of same and a system for switching it in.
McCoy: Back to what Al said, there are individuals who are recruiting future or present I-T personnel from the broadcast industry … there’s probably some truth to that, but it might be because we’re willing to wear a pager and have it next to our bedside all night long as opposed to the computer geek they got now. Doesn’t make me feel good! It’s a little unsettling.
Is anybody here in the audience not a broadcast engineer but exclusively in an I-T function for the company they work for? Can I see hands? One person …
Baden: Yeah, but he works for me and he also maintains the Linux MPEG3 archive. His responsibility goes from the corporate H-R database to the corporate finance database, but also all the servers that keep us on the air, the archiving system, the streaming for the Internet.
McCoy: So I’m going to take that to mean either one or no hands in this room. We’re all people who are accustomed to wearing a pager and having whatever activity we’re involved with interrupted by pages from the radio station for things that are sometimes meaningful or sometimes not. We just take it to be a part of our life.
Baden: What we’re saying is we don’t have hobbies.
McCoy: Does anybody here see a dichotomy in terms of the way the I-T personnel are treated and the expectations associated with them as against the broadcast engineers in this particular respect? I’m not looking to create a schism here …
Kenyon: In our company there was, but it’s merging more and more as we go forward.
Our I-T department was originally based on an I-T structure that developed out of the outdoor industry. And I made the argument similar to what you presented, in one of the I-T meetings when these guys were up.
I said that you have to understand that when a server goes down in a broadcast station these days, the audio stops. When a server goes down in an outdoor plant, the billboard faces don’t go blank. They will in the future, but in the moment, they’re not quite there yet.
McCoy: So the service response time for changing the lights on the outdoor sign is longer, and should be longer, than the service response time for getting the radio station back on the air.
Kenyon: The changing face cycle for outdoor – they use very large printers to print these wraps. … The face stays up for whatever period it’s ordered. If they don’t replace it, it’s not like there’s dead air. It’s like yesterday’s programming is still playing.
McCoy: So somebody gets two extra days for free.
Question from the audience: It’s all right you all to understand the difference between a broadcast system and an I-T system and what you need for broadcast.
How do you express this to upper management when they’re drawing up the budget, making priorities, when everybody’s saying “World Wide Web is so important,” but it might have an audience of 10 people while your broadcast system has an audience of 10 million?
Kenyon: I think our company just recently came to and reinforced that conclusion when all streaming activity was ordered stopped by Mark Mays.
McCoy: But that wasn’t for reasons of strategy, it was for reasons of intellectual property rights.
Kenyon: That’s the official posture.
McLane: Is it advisable for broadcast engineers to be involved with management as part of the decision-making process and the content and structure process of what’s on your organization’s Web? Should that be part of your management domain as an engineer?
Baden: Streaming is the model of how all broadcast is going to be in the future. I mean, if you’re not getting it now you’re not going to get it when everything goes digital and you’re serving IP down to a transmitter with text file.
Masiello: To get back to the question the gentleman raised about how do you get management’s attention buying digital equipment for broadcast when the focus appears to be something that’s exciting for the Web.
A strategy we’ve used is to present the purchase and the equipment in the light that it serves both masters; that you’ve purchased this equipment to upgrade the broadcast side and as a feature, this play-out system knows how to do Web authoring, or it can put what’s on the air into templates for Webs, etc.
Leverage the two. Most management sees the two differently, but actually it’s the same because the content’s coming off the same engine, the same creation, what’s going to air, modified sometimes for Internet. One way to leverage that is to show that the same can be used for both sides.
McCoy: As an all-digital company, are you going to be doing any streaming (at XM)?
Masiello: There’s some plan for it, but we have different restrictions than terrestrial broadcasters. We are not as badly controlled, but we’re kind of between terrestrial and the Internet Web sites by the Digital Millennium Copyright Act.
That is a word I wish I’d never heard. It is a law, and it now forces you to change technical implementation and change processes in order to accomplish something – having to do in our case even with the way you can broadcast songs.
There’s two parts of that. Part A applies to satellite, having to do with how many songs in a row from the same artist you can play. How many songs off the same album within a certain time period you can play.
Bryant: If you have somebody appearing live, and the same time you would play cuts from their latest CD, how much of them singing live and cuts from the CD can you have there?
Masiello: Right, it gets really cumbersome. And the reporting requirements the present terrestrial broadcasters are not faced with and which we and the Internet folks are faced with … anyone who streams … that is for the RIAA. They’ve sent a regulation with 12 fields of information that must be completed for every song and you have to report that by the 20th of the month following its use, etc.
Those restrictions have a lot to do with why you want to stream and do stream, and in our case how you actually structure your system.
McCoy: If you were to call the interview you just described playing a few cuts and maybe the artist sings live, a “commercial”? Does that get you off the hook?
Masiello: No, if the guy signs a waiver and his recording company allows him to, then it’s not considered a recording and you can do that. A lot of this is not tested and there are attorneys who go and try to negotiate these as much as possible.
Baden: All the sites say they’re making up the rules as they go along.
Bryant: That’s the reason for what you had said earlier, about engineering getting more involved in the streaming part of it. It’s not just taking your signal and putting it on the Web, and it’s not just worrying about the equipment to do the streaming.
Because of the Digital Millennium Copyright Act, there is so much more involved. You start having to report on the Internet. You would have to say song title and artist and all sorts of information that would be displayed on the Web at the same time that the music is being played.
There are so many different requirements that, from a technical standpoint, as Tony said, to get it so that it’s seamless to the operators in the studio, there’s an awful lot of technology that has to go into so it becomes more automatic.
Otherwise the operators in the studios will be going absolutely nuts trying to service both the broadcast, which is your bread and butter, and your streaming function.
McLane: We see suppliers serving that market with ad insertion software and content substitution.
Bryant: It’s not just content substitution; it’s your regular product, it’s the people talking, it’s the music and all of that affected by that.
Baden: We’re even getting away from being called “radio broadcasters.” We’re more content providers. We (at Radio Free Asia) obviously don’t have the music problems as we do news and information.
McCoy: Well, the Chinese don’t care about copyrights anyway …
Baden: We’ve had that argument with people. Our broadcast chain going to air, going to Internet is the same. Also the archival process.
We’re trying to get everything in the formats that we’re predicting will be around for a while, like MPEG3 and XML for script files; using XML databases and SQL databases to search the content. The digital shortwave, which will be coming up next, basically has the same parameters as streaming to the Internet. You can take a text line into it, you can get some still pictures with it. So it’s all kind one and the same.
McLane: I’m assuming most of you are not tape-based anymore. Where are we going with broadcast and storage standards?
Masiello: We have one of the larger storage systems around. All of our music is stored digitally with 22 Terabytes of storage dedicated to the audio. We have 3.5 Terabytes just for the indexing.
All of the audio, all of our music, the titles, is stored as MPEG Layer II, 384 kilobits, so it’s the most benign amount of compression we can apply to it. If we didn’t do that, we’d have 88 or 100 Terabytes of storage, which begins to get crazy.
In addition to moving the audio around back and forth from the storage device to the playout devices on LANs, we employ a lot of fiber interconnect to do that. But at some point things do get congested.
Everything is stored compressed, and we have both “online” and “near-online.” The online is hard drives; near-online is tapes, digital linear tapes.
You can archive things for broadcast, not legal archiving but high quality on the tapes. That access is usually four to 12 seconds, vs. the online, which is instantaneous.
Everything in the facility is stored compressed at 384. If there is a standard, Layer II is the one we use for storage. Most of the vendors can deal with it, are familiar with it, have products that can do that.
McLane: Al, thoughts on standards?
Kenyon: Clear Channel owns Prophet Systems Innovations, and that’s our designated digital audio system. In fact we’re putting those in at the rate of about six a month. That’s six facilities a month, not six stations a month.
What the next generation is, I’m not too sure. We’re pretty busy implementing the “NexGen” of the system.
McCoy: Tony, I’m just doing the math in my head, and 27 Terabytes and 384 kilobits, that’s close to a million songs.
Masiello: One-point-two million songs. (Applause.) You know that commercial, “Every song ever recorded, I’d like to access”? Well just about. If it’s commercially available, it’s in the system.
McLane: Eight hundred versions of “Louie Louie.” (Laughter)
Masiello: It includes various versions, including “best of” and re-releases. Because – and this is important, it’s part of the Digital Millennium Copyright Act – I can’t make a copy at my facility! We bought all of those CDs and have the audio ripped.
I made my first digital copy according to the law. Now, when I use it throughout the facility, I cannot make a copy.
Bryant: Our set-up (at ABC) is a little bit different, because our end user is not the listener, our end user is a radio station.
To the extent that we can, most of our stuff is uncompressed. It’s linear because after it gets to the radio station, we don’t have any control over it. We try to keep it as uncompressed as we possibly can so there isn’t a problem with it later down the line with whatever anybody else might do with it.
McLane: Tell us briefly about the most recent facility build or rebuild you’ve worked in and any insights you’ve learned that you can share with us.
Frank, you did a Tech Tour two years ago across the country to learn about the GulfStar stations to learn what was happening at its facilities.
McCoy: That was kind of a goofy thing. It was almost three years ago that we realized we needed a firm implementation of the possibility of having disk jockeys in one place do radio programs in another on a custom basis.
So we built a facility in Austin, Texas, to do that, the Star System, based on the Prophet technology model, their product and their software … and a wide-area network that we bought connectivity from.
I think Sprint was the principle provider, but it was frame relay connectivity that was fairly expensive by today’s standards. Certainly you would order up DSL or whatever today, but we had dedicated nailed-up circuits in places like Lufkin, Texas. It wasn’t the easiest thing to get connected or keep connected.
Although only it’s only three or four years old, that facility is so far behind technically what Tony Masiello is doing. It’s amazing how little, if you held them side by side, they would have in common.
My only comment is that two years time, three years time is an eternity in this line of work.
Bryant: The Dallas facility was built in ’95, a long time ago, and what we’ve been doing is adding on to it. And if I had to pick the one thing that’s changed the most, is that most of the copper is CAT-5 these days.
Kenyon: Our most advanced facility was the Denver build, which Radio World did an article on, and that was based on Klotz consoles, the Vadis system, and most of the audio chases around on glass.
About six months ago, we took a direct lightning strike on the building. All the racks and the technical centers are in the basement level, and we have studios on the third and fourth floor. There’s a penthouse with STL equipment and RPU receive equipment.
The only thing that happened is that it pulled the fuses on the output of the UPS. I can’t say enough good things about an APC UPS. It tried to counter the induced power in the power lines and took out its output fuses, which is what it’s supposed to do. We threw the switches for the emergency bypass, brought the servers back up and everything else worked.
We lost three fuses, with a direct lightning strike to a building that’s essentially five stories tall and has racks at the bottom and STL equipment at the top. It’s seven stations and it’s strung together like Tony says with most of the audio running around on glass.
We’re moving stations into a complex that will eventually house 13 stations in San Diego. You get to 13 because a lot of the call letters start with “X.”
McLane: Can you approximate how many facilities projects you have going on in Clear Channel now?
Kenyon: We’ve got about eight active builds going on at the moment, but the total list of projects either closing this year or in start-up phase this year is about 36. Those are projects that are large enough to require the attention of an architect and contracting.
One of the biggest things we’re running into now is finding an architectural firm that has a strong enough MEP – Mechanical/Electrical/Plumbing – side, within the organization.
I can look at the prints and make sure the acoustics work out fairly well, and that’s within the architectural discipline, but as far as getting the HVAC right in various situations, we’ve found it to be fairly problematic.
In fact we located one station’s new facilities next to a building that’s an icehouse. The strange thing there is, everything is fine, they got it all put in, but the toilets won’t flush when the ice factory is running because there’s not enough water pressure. This goes back to the wandering around with a plunger days!
It’s important to have an architect that not only understands studios and inter-relationships in these larger complexes, but also has an MEP division to make sure all this other stuff works.
In my first build, I made the mistake of figuring any idiot could design office space, so I just concentrated on the studios. And we ended up with office space that was horrible.
(At this point in the session, Masiello shows a short video about the new facility at XM Satellite Radio in Washington, D.C.)
Masiello: What made our facility come together, and it is the largest in the United States, is planning. You can’t design something like that without it.
We spent over half the time it took to put it together doing the planning. We used a software package called VidCAD to design the facility, lay out all the cabling, lay out all the racks, and since it interfaces with AutoCAD, which is what most architects use, the physical space was related to the technical space.
The ability to populate equipment racks on the screen in two different locations, push a button and it figures out all the cable lengths and all of that, was most useful in bringing the facility together.
McCoy: The narration (on the XM video) said 22 Terabytes and you said 27.
Masiello: They didn’t count the index.
Baden: But I guarantee you’ll be here in a year saying 22 Terabytes isn’t enough.
Masiello: It’s already not enough, and (we’re) out of space. That’s why we have the off-line storage, because program people want to save things.
The critical part was the planning and a decision early on to do the wiring in a very specific way, in that all the real-time or live audio would travel on fiber optics and any of the file transfers, compressed audio or audio shuttled between devices would be done on CAT-5 cable.
The only traditional audio wiring that you’ll find is from the microphone to the input of the console, where it’s converted to digital.
McLane: Was that a contentious decision, the CAT-5 decision?
Masiello: No it was not. Part of the reason it was easier was my boss, Senior VP of Operations Jack Wormington. (Ed. Note: Wormingon has since left XM.)
His background prior to joining XM was at Hughes, and before that in the military. He is a retired Air Force general who ran Cape Canaveral for years.
They have a lot of experience with big systems. He told me that if he saw anyone running around with a punch-down tool, I would be fired. So the decision was pretty easy not to go that way.
McLane: You can have a plumber’s assistant?
Masiello: Yes, but no punch-down tool.
McLane: Is CAT-5 the future of the industry?
Baden: We’ve been using CAT-5 (at Radio Free Asia) since the beginning for everything. When we first did it, it was like comparing the specs and asking, “Why couldn’t it work?” We couldn’t find much information five years ago. And we basically went out and got a 1,000-foot spool and ran digital audio through it back and forth, bidirectional and one way. It sounded good, so we went with it.
Masiello: One of the decisions we made early was to go with the Radio Systems Studio Hub system, which is CAT-5-based.
It’s a shielded system, all the connections are shielded and we used shielded CAT-5 cable. We even run the occasional analog audio over it. All of our I-T plant (is) wired, even the corporate stuff, using the same platform that was designed for broadcast, and it works extremely well. We have a unified CAT-5 approach in the building and a unified fiber optic approach.
McLane: Any other questions?
Audience member: The (XM) facility looks very nice. In the old days broadcast facilities would last for years and years. With the new I-T paradigm, there rapidly has to be change.
How soon do you think your facility will be obsolete and is your management prepared to refurbish it at that point?
Masiello: The basic infrastructure won’t get obsolete unless there is a complete change in the methodologies. What really changes is speed of processes, amount of storage, etc., and that you follow along.
All the drives in our workstations in the TOC for playout are removable. We can expand to larger sizes as the vendor does. They’re all Compaq computers. The big storage system is an IBM storage area network. There are over 600 36-gig drives. They’ll be coming out shortly with 72-gig and I’m sure it will go from there.
So the expansion within the existing model is there; it’s scalable.
You need to have something on the horizon. We decided we’re going to see light storage and something that’s a hologram on a little cube. We’re talking about something that could be way out, and that could be 10 years. If that changes, the whole paradigm is shifted. Then I think we have a problem.
We took pains to make it very broadcast-traditional. The Klotz work surfaces look like ordinary consoles, and they didn’t have to. All they are are just control surfaces. There’s an RJ-45 plugged in. There’s no audio. It goes into a computer, which talks to the frame.
So if the presentation needs to change, change the work surface. What’s behind it is still going to be there. We’re still doing AES audio.
McCoy: What did it all cost?
Masiello: The second floor, the broadcast floor, studios and equipment is about $40 million.
McCoy: So basically, about $40 a cut? (Laughter)
Masiello: It’s interesting to note that to launch and build two satellites is half a billion dollars. So for once, I was not the major expense in the tent. I was just picking up the dust of the money that it takes to build and launch satellites.
McLane: XM represents a totally different model of radio broadcasting in the United States. Do you panelists feel like our traditional model of radio broadcasting can co-exist? Are you worried that this is the Death Star?
Kenyon: Well, we (Clear Channel) are investors in XM…
McCoy: They’ve hedged their bets.
McLane: Do you feel that there will be a substantial difference in the competitive environment within five years because in our local markets, suddenly station managers are competing with a subscription service? Digital to the car?
Baden: Do you think they’ll be here in five years?
McCoy: That’s the key question as to whether you can get them into automobiles. If you can get them into automobiles, then you probably lose Tom Joyner.
People in the network delivery business, they’re on your target list, aren’t they, Tony?
Masiello: If there’s a model that’s close, it’s probably close to network delivery. But we go directly to the end user.
It’s like when cable came out and spelled the reported demise of terrestrial television broadcasting. And then direct-to-satellite came out and was prophesied to spell the demise of cable systems. That didn’t happen.
It just adds to the available inventory of things people can watch and listen to. When it comes down to it, it matters not. It comes down to the programming that you’re carrying.
Most listeners, after they get over the “gee-whiz” factor, will go where the programming is. And people are still listening to AM radio stations carrying Paul Harvey and knocking the hell out of anything in the market, just because.
So the fact that it’s AM or FM or whatever, it’s where the programming is … It’s where the listener’s going to go. We just offer more potential venues to attract their attention.
Baden: Where I think the model falls down is that it’s something everyone got for free, and now you’re making them pay for it.
Masiello: People did that with cable, too. It was all for free and …
Baden: But now you have HBO and the girlie channels.
Masiello: You got HBO and commercial-free channels and most people watch their terrestrial television stations over a cable. So the model is somewhat similar, but it’s been proven.
I’m a tech-y guy. The marketing folks all swear that this is where it’s going and they have this background and material and market surveys and focus groups to back them up. Otherwise people like General Motors and Clear Channel and Honda wouldn’t invest money in it.
McCoy: You get a separate bill for your cable television service. If it got rolled into the automobile monthly charge, it might be pretty painless. I think they’ve at least got that much of a leg up.
From the audience: Changing the subject, since several of you represent the major program supplier networks to a lot of stations … small-market radio here…
Back in the early 1980s the networks got together and worked a consortium, and DATS was born. At that time the four major networks agreed. And then SEDAT came along and everybody went with that.
The new post-SEDAT technology that’s coming out now … it doesn’t seem like you guys are all talking.
From our standpoint, even though, for example, several of the networks are going with Starguides, we can’t use them between the different networks. What I’m ending up with now is instead of one or two satellite receivers to satisfy our needs in a small-market array of cluster of three stations, now I’ve got five satellite receivers, and I think I’m going to need two more.
That’s a nice scheme for Starguide because they’re getting a lot of money. In smaller markets, we don’t always get free receivers from the network. Why is the platform not common, and why are we not able to have some of the standardization we enjoyed under SEDAT where we could pluck around the different channels as we signed agreements with the networks and used that common infrastructure to receive you?
Bryant: There’s a lot more in common than you realize, but I think in some cases it’s a business decision; the receivers can be authorized for a variety of different services.
From the audience: But you can’t get them unless you change carriers. So we can’t keep, for example, Westwood One on one side and ABC on another side in the same receiver.
Kenyon: In the old days with the DATS receiver, you remember having to open that little window and put a new crystal in it for a different transponder? It’s the same thing. You’re changing transponders. If it’s on the same transponder you can pick it up.
The difference is, the DATs and SEDATs receivers were not addressable. There was absolutely no control for program providers as to who was downloading the program. Either using it or monitoring it. With the Starguide receivers, they’re all addressable. And should a program provider desire, they can set it up so only those authorized receivers can receive the program.
Back in my days at WLW, there were several stations that were (Cincinnati) Reds affiliates that were not authorized. Our only remedy was to involve lawyers. But there was a loss and there was nothing we could do about it other than litigate. There was no way to close the door. Now there is.
Bryant: And if you’ve got multiple receivers, especially if they’re all one network or another, some people are ending up with multiple receivers when actually they can accomplish the same thing with extra cards in individual ones. I think a lot of people are not aware of this. You can populate them with more cards. As long as they’re on the same transponder, it takes up a lot less rack space.
McCoy: That SEDAT receiver has got to be 20 years old? Maybe you’ve run the wheels off of that.
McLane: The first low-power FM CPs were granted this spring. Is low-power FM going to come about as the former FCC chairman Bill Kennard envisioned it?
McCoy: It’s an interesting area because in some cases, there are shared time arrangements involved. That was the mechanism they were going to use to determine in cases where there were mutually exclusive applications. They were going to grant everybody that they deemed qualified, and then individuals would arrive, or perhaps the commission would broker who’s going to be on the air when. And it would be two hours on, four hours off, whatever it is.
The difficulty is, this is a secondary service in the same sense as translators and boosters. It’s entirely possible that there may exist an opportunity at some future time for a real, live full-service FM in that location, maybe a Class A or whatever it is.
So eventually when somebody proposes a way to put a full-service FM in that location, what sort of pushing and shoving match are we going to get into? At least in theory, the rules say that the secondary service – if you have a translator in a market and someone proposes to put a full-service FM in a location that would make the translator no longer permitted, it’s all on the basis of spacing. Then the translator has to go away. That’s the way the rules read.
The LPFM should be subject to the same set of rules. It’s just that now you have an interesting circumstance where there might be a number of licensees, most of whom are operating what might be deemed to be charitable organizations, since these are all noncommercial.
I would look for some real interesting squabbles over this in the upcoming years.
McLane: Is IBOC DAB going to happen in the next year to 18 months?
Kenyon: Our current timetable is more like 24 months or so. However I defer all that stuff to management.
Masiello: Our service launch is scheduled for late summer and we’ll have receivers you can get at Best Buy, Circuit City, and start buying probably the August timeframe. The 2002 model year Cadillacs will have as options XM radio.
McCoy: Does the home box require an outside antenna?
Masiello: It depends. If you are within the coverage area of a terrestrial repeater, you do not. It’ll pick it up indoors. Otherwise yes. Vendors are working on how do you get a satellite-only signal delivered into the home.
But in major-market metropolitan areas, you can pick it up indoors.
Bryant: I wouldn’t think that would be a problem, look how many DBS dishes you see everywhere.
Kenyon: How many of you have a receiver or set-top box that’s capable of receiving a DTV signal? Hands?
OK, the five or six of us can meet later on. But how long has digital television been out? It’s gonna take a while for satellite radio to ramp up.
By the way, I’m impressed that there were that many people here with DTV receivers.
McLane: Other topics?
Kenyon: Digital audio consoles. I’m firmly in the camp that Tony jumped into, that a digital audio console needs to be more than just a specific little box you stick in a room.
All you’re doing there is putting in a digital system that’s a drop-in for an analog console that can, in many cases, meet the same audio specs. The power of the digital console is interfacing with a routing system and being able to control all these audio sources back and forth through the use of a wholly integrated system.
There’s an interesting product being introduced by Telos. It’s a Super Surface. They’re making a control surface which they hope will drive a Klotz Vadis frame or will output control information that could interface with the switcher of your choice, should the manufacturers want to go that way.
I recommended that they contact someone like AudioPoint, that’s a DSP-based routing switch. I was looking at consoles and I wanted to find somebody that would interface with a DSP router. The only one that was available was Klotz and it’s working very well for us in Denver. It’s a very good proof of concept.
Masiello: We made a conscious decision to go with Klotz. … The reason for our decision was that having tried to do this before at CBS and CBS Radio Network, we had a digital router, then an intercom that you needed, an IFB system, to generate mix-minuses. Nothing talked to anything.
And you had these little digital boxes, and we ran out and bought little outside relay boxes. I can tell you that at CBS we had more Henry relay boxes that were controlling digitally outputted things.
The Klotz platform provided for us a way to keep it digital, but all of the control, everything was centralized.
Its work surface is a console. A panel that’s set up to be an intercom, is an intercom. How many mix-minuses do you need out of this studio? One, two, 20? It makes no never-mind. Change the configuration.
All of it is inter-connected by fiber. It sends all the AES block around as well, so everything is locked in time. It’s also a routing switcher. The facility is about 2,500 by 2,500 square, but it’s a virtual router. Wherever there’s a frame, it’s part of the routing system. It can route RS-232 as well for some of my program-associated data.
The other benefit is from a maintenance standpoint. I could have a technician in the broadcast operations center looking on a computer that looks at all of Klotz frames. If there’s a failure, it will point me to Rack 6, Frame 12, Card 2.
But more importantly, if the disk jockey in the studio hoses the setup big-time and gets all confused, I can reset the console. Or say I’m expecting a feed from our New York studio, a two-way interview, and the jock doesn’t know how to figure it out. While he’s still on the air talking, I just go a computer with a few clicks, and I can actually, right under his fingers, if you will, change the routing and put that feed from New York, set up the IFB, do all of that seamlessly.
So it’s not just the audio, but the other control.
And there are situations that come up at a station that you need to attack and it’s good to have a platform where you don’t need to run around with punch down tools, relay boxes and the like to solve a problem.
But (this is) what vendors should be looking at, and the user should be insisting upon. Don’t just sell me a digital console. It needs to talk to things. It needs to be able to address other problems within the facility. You really do need a digital engine as a backbone.
Baden: And please, when vendors put these things together, let’s use some standards.
McCoy: Would you then support a blank slate, unified control surface device, just a big flat-screen, touch-screen gizmo that …
Masiello: Well that’s what you decide. Some people hate touch screens, so don’t make it touch screen. We helped Klotz with their console design DCII because one of their standard designs was too confusing to the jocks. So we worked with them and now they have their DCII platform.