How to Select the Right Network Telemetry System
Picking telemetry technology to monitor your network is very important - and most companies don't seem to recognize it. It's all too easy to ignore remote monitoring when you're busy rolling out new network sites and services.
Telemetry equipment is quite a bit like insurance. Most of the time, you think it's too expensive. When you have a problem and it's too late to react, you'd pay almost anything for it.
But you have to make the right choice. Not all telemetry gear is created equal.
Make no mistake. Your career will be strongly linked to the success or failure of a telemetry system you specify. This is a major purchase for any organization. You need to choose wisely.
In decades past, it was hard to know which to choose. Salespeople trying to sell you something had much more information than you did. Now, you can easily find reference information online like this guide:
Choosing a System: Where to start?
As with any project, you have to start educating yourself before you can select specific gear. Ironically, one decent way to do that is to sift through the product listings of specific gear. You will learn eventually, but that isn't the best approach. Different manufacturers use different trade names for the same things, which will only confuse you if you're just getting started.
It's almost always better to start with a general checklist that's manufacturer-independent. You have to establish your buying criteria before you look at specific hardware and software. Here's a checklist that will get you started:
Make sure your monitoring system supports these essential functions:
- Alarm collection and device management: Don't settle for a limited remote network telemetry system. Get multiprotocol support for every monitoring device in your network, plus discrete alarms, analog alarms, ping alarms, and redundant path reporting. Closed, proprietary systems severely limit your future growth. Once you start buying from one manufacturer, you're stuck with that system virtually forever. To switch, you have to scrap your system and start over. A system that supports multiple, open standards has a future - even if the manufacturer goes out of business or you simply decide to choose a different one. Don't ever fall into the trap of a proprietary system. What does it say about a company if they have to lock you into their system?
- Alarm sorting and analysis: Make sense of alarm cascades with automatic intelligent alarm sorting, filtering, processing, and trend analysis. If you have a medium or large network, alarm overload can easily happen. You need a network telemetry system that shows you what you need and hides what you don't. One great way to do this is by sorting alarms into logical groups (severity, site, etc.) and analyzing them for overarching patterns. Your attention is limited. Focus on what matters.
- Alarm presentation and notification: Send detailed alarm descriptions and correction instructions to NOC and field techs via SMS notifications and web interfaces. Your techs spend most of their time out in the field. Don't make them come back to the central office to get new alarm dispatches. That wastes time, fuel, and budget dollars. For the dispatches you do send, make them clear. You want your techs heading out with the right plan and the right parts.
Select one of the following options to start evaluating network telemetry systems.
Essentials Checklist: Network Telemetry System
Here's a handy checklist of essential network telemetry system features that you should look for in your next system. Print this checklist out and use it to rate the systems you're evaluating. If a system can't meet these basic requirements, cross it off your list.
Critical alarm collection and device management functions
You can't effectively manage your network if your telemetry system can't talk to all of your equipment. Some companies are running two or more systems side-by-side to cover all of their gear. That is painful to watch. You should never deliberately run concurrent systems. Find something that is universally compatible. This includes both collection of alarms and issuance of control commands. Here are the keys to broad compatibility:
- Live analog monitoring: You can't adequately monitor battery levels, temperature, and humidity with one-threshold contact closures. Look for support for analog inputs, including live monitoring of actual analog values. Analog inputs should support both 0-5v and 4-20mA sensor standards. In voltage mode, your analog inputs should have a wide range (ex. -90 to +90v DC) to accommodate battery voltages and power input voltages.
- Back-up reporting: Don't rely on your primary network to bring back alarms. If anything goes wrong with your transport, you'll lose your telemetry data just when you need it the most. Look for a system that supports alternate path reporting. Traditionally, this was serial or dial-up. Today, you're more likely to have cellular, T1, or fiber available. No matter what transport tech you have available, plan ahead when ordering your RTU. Find one that supports the right transport & protocols.
- Control relays: Many common site problems, from power outages to high temperature alarms, can be solved by quickly turning on a generator or an air conditioner. Remote operation of site devices is the best way to eliminate unnecessary site visits and it's a lot faster than going in the truck. Remember that control relays come in several flavors. 1-amp relays are good for controlling things that accept discrete contact closure inputs. High amperage (ex. 10 amps) is needed to handle direct toggling of power to enable power-cycling of frozen equipment. Anything 10 amps or higher is PDU-grade power switching, even if it's included as part of an RTU.
- SNMP support and ping alarms and more: If you're responsible for both telecom and IP equipment, consolidate all your monitoring on one system. It's not enough to handle only SNMP or only manage "IT" gear. You need to monitor everything in your network. Choose a telemetry system that is capable of accepting data in all the necessary formats to accomplish that goal. Remember that these 2 categories are very broad. You also need to keep various protocols in mind, as "telecom" and "IT" equipment encompass a wide range of protocols. It's not as though you only have to support 2 standards with your telemetry system. There are many. You need to support all that you use (or will use) in your network.
- Multiprotocol support for your existing devices: Make sure your next master system collects alarms from all your existing devices, including your older legacy gear. You can get rid of all your specialized consoles and monitor your network from one screen. If you have proprietary systems, they're probably very old. Replacing master stations that could die any day is a big priority. Once you've done that, you can replace your proprietary RTUs over time as needed. This spreads the cost of replacement out over several budgeting cycles. It's easier to get approval for expenses that are both expected and gradual.
Alarm sorting and analysis functions
Your alarms must be organized to be effective. Beyond that, your telemetry system must translate individual clues into clear information you can use to take action. Here are specific functions that allow your system to handle alarms intelligently:
- Root cause analysis: Finding the underlying cause between alarm cascades can take hours of patient detective work. Look for a system that can automatically correlate repeated combinations of alarms. For example, you don't need to see hundreds of "equipment offline" alarms at dozens of sites simultaneously. That's overload that hurts your situational awareness and, therefore, your ability to respond quickly. A smart telemetry system will see the few alarms that matter, move them to another screen, and put a single large "regional power failure" alarm on your main screen. That's a usable summary of what's happening instead of information overload. That's real intelligence instead of just data.
- Nuisance alarm filtering: Even the best NOC staff stops taking alarms seriously if they're bombarded with status alerts, oscillating conditions, and unimportant alarms. Look for a system that filters these out. Status alerts are easily to hide if you filter by severity. Even simple color coding is a big help. You can only fight problems you can see, and one of the leading causes of blindness in telemetry systems is TOO MUCH data. Choose network telemetry software that shows you only what's important. The rest can simply be dumped to a log file for review upon request.
- Alarm sorting: A large, complex network can create a cascade of alarms. Some are unimportant, but others are critical. You need to see the critical ones without being encumbered by anything unimportant. Look for a system that can automatically sort and prioritize this flood of information for you.. Sorting is really just a variation on filtering. Read down the list in descending order. Once you reach the layer of "unimportant stuff" that requires no response, you're done. That's the power of alarm sorting.
- History and trend analysis: Identify problem areas and eliminate recurring problems with a system that keeps a complete alarm history that's exportable for trend analysis.
- Custom combination alarms: A low battery isn't a serious problem, and neither is a failed generator, but they're pretty serious when the happen at the same time. Look for a system that can watch many different alarm inputs and spot critical alarm combinations. When a pre-programmed combination is detected, your telemetry system will generate a "soft alarm". This alarm point is not tied directly to a hardware input, but is purely a software construction. It is the result of your system detecting a group of alarm conditions that need your immediate attention.
Key alarm presentation and notification functions
These functions revolve around showing you alarms when you're at your monitoring console and sending you notifications (email, SMS, etc.) when you're not. It's all about clarity and detail. You need to quickly determine whether an alarm requires action. Once you do, you need to know where the problem is, which technician to send, and what parts or info are required.
- Alarm correction instructions: Detailed instructions included in alarm notifications ensure that system operators, without extra training, will know precisely what to do and who to call if an alarm happens. You can't let your network depend on people who don't know how to respond. Historically, that probem was best addressed by paying expensive travel costs so your people could attend expensive factory training. Now, you can simply choose a smarter telemetry system. The interface will be intuitive. The alarm database will include your custom response instructions. This allows your technicians to be functional from their first days on the job.
- Web interface: Everybody knows how to use a Web browser. A Web interface makes sure all your field techs can access your alarm system, from any computer from any location. Also, modern systems will now present a "responsive" web interface that looks great on smartphones. You'll get the same data, but it will be scaled for smaller handheld screens. Fonts will be bigger. Buttons will be easier to press with your finger. Web interfaces are a great way to improve alarm visibility and reduce training requirements. Smartphone web interfaces deliver those same advantages in a portable package your technicians can take into the field.
- Detailed alarm notifications: Summary "major/minor" alarms don't give you enough information to make dispatch decisions. Look for a network telemetry monitoring system that includes detailed diagnostic information in each alarm. Imagine you're a technician. What spare parts do you take for a "major alarm"? Compare that to "Card 2 on 10/100 Network Switch FAIL". Good telemetry makes maintenance and repairs a no-brainer.
- SMS and e-mail notifications: SMS and e-mail notifications let your field techs respond to alarms while they're still in the field, speeding repairs and reducing windshield time. Look for a system with SMS support, which can send detailed alarm notifications to cell phones. In the wireless age, there's no sense in requiring your technicians to drive back to the CO every time they finish with one alarm. There might be another important alarm just down the road. Don't burn time and fuel if you don't have to.
What happens if I don't follow this checklist exactly?
One or two minor deviations from this checklist probably won't cause a major debacle, but you certainly won't do yourself any favors. This list was designed to protect you no matter which telemetry manufacturer you choose. By following this list of functions closely, you're dramatically reducing the chance that something will go wrong.
And let's not kid ourselves here. Network telemetry solutions are a major infrastructure purchase. Your career will rise and fall with the success of the system you choose. Imagine the horror of having to scrap your first system and start over. That's an unforgivable waste of resources.
Where can I go to find equipment that does everything on that list?
You're not alone if you feel that way. Network telemetry is surprisingly complex. It has to monitor every major part of your complex network, though, so it could hardly be any other way.
This complexity is a big hint that you should choose a service-oriented manufacturer. You have a substantial project ahead of you, and buying commodities from a traditional vendor will make it very difficult. You need to work with an expert who cares about your success.
One great way to cut through the noise and find a good vendor is to choose one who is as vertically integrated as possible. Do they manufacture the equipment they sell? Do they provide their own tech support? These are strong clues that you're hiring a partner, not just buying from a vendor.
Your typical know-nothing vendor will simply be buying equipment from a distributor and trying to move it out the door as quickly as possible. You will have to be 100% of the brains for the project. The problem isn't that you don't have the brains for the job. You just don't have the time. You have a lot of other work to do. That's why you need an experienced vendor who will be your partner.
What if I need something special that wasn't mentioned on the checklist above?
Again, working with an expert who will listen to you is the key here. You can ask technical questions if you work with a company like that. They'll consider it with their wealth of experience, and give you a useful answer.
If you decide that you do, in fact, need that function, your vendor should be able to develop it for you. You might have to satisfy a reasonable minimum order (maybe 10-12 units or so) or pay a small development fee, but you shouldn't have to order thousands of units. You may have to submit a PO to get the process started, but be sure you're protected with a money-back guarantee.
"Every network telemetry system I make is backed by my no-risk, money-back guarantee. If you buy my equipment and you're not satisfied for any reason during the first 30 days, simply return it for a full refund. If my product doesn't solve your problem, I don't want you to have it."
- Bob Berry, CEO, DPS Telecom This is just a checklist of basic monitoring functions. If you want to find a network telemetry system that meets and exceeds your requirements, select one of the options below...
