4304

How to Troubleshoot Your SCADA System

Morgana Siggins
Morgana Siggins
Monitoring Specialist

Many industries rely on SCADA systems to monitor and manage nearly every aspect of their remote operations. That's because, over the past 40 years, SCADA functionality has evolved from being a simple on/off process system to an intricate one with modern devices.

The main objective of these supervisory systems is to collect data about remote equipment or operation and send it back to a central master station. This master station will display the information in an organized and user-friendly way.

SCADA systems can be small and simple (imagine a single sensor connected to a simple workstation) or very large and complex, such as the ones that monitor and control the activities going on at nuclear power plants, refinery, or at large water systems.

However, like any other type of system, the uptime for a SCADA system won't be at 100% all the time. We that, because, as a trusted SCADA equipment manufacturer, we not only get calls from future clients but also from current clients that need tech support. They already have a SCADA system so their challenge is to keep it online.

SCADA system
A Supervisory Control and Data Acquisition (SCADA) system refers to a centralized monitor and process control system of a complete network.

When you have a problem, a good practice is to verify the easy stuff first. This is because that's normally where most of the issues come from, so your techs will be able to solve the problems quickly.

Also, most of the time, if you know where to look at, you'll be able to pinpoint where an issue is coming from. Even if you are not able to fix the issue by yourself, being aware of what's going on and explain the situation to whoever will be taking care of the problem will make the process of solving the issue way faster.

Knowing how to troubleshoot your SCADA system is a very important asset. So, let's take a look at some ways you can successfully locate and solve your SCADA system problems.

Start by Checking on Small Things First

Your SCADA system collects and sends out a lot of information in real-time and your mission-critical operations depend on this visibility. It's understandable if you feel overwhelmed when you have a problem or if things don't seem to be working as expected.

When you have a SCADA issue, troubleshooting is critical. However, just like for your personal PC (or any other system that consists of many different hardware and software parts) finding the problem can be complicated and take a long time.

First of all, as I've said before, it's better if you start checking on simple things first. If your error happened due to a small problem that only requires a small fix (which is true most of the times), you'll save a lot of time.

If your equipment is deployed at very remote locations, perform verify the equipment that is close to you first, then go to your remote sites. Before performing a more in-depth examination, try assessing the following key points.

Take a look at your power supply

Are all your SCADA devices receiving power? Examine your power supplies. Batteries, for example, often have lights on them that will tell you if the power is connected.

Also, check on your power supply connections. Are your batteries properly attached to all components?

Look for broken or damaged equipment

Most SCADA devices are left at remote sites where they are more susceptible to the elements. So, if an issue happens, it's important to check and make sure that all the components of your equipment are not broken and they are all properly connected to where they should be.

Check out all your cables and ensure that they are not water damaged

It's always important to make sure that all your cables and connections are sealed with waterproof tape to avoid water damage.

Are your connections properly grounded?

If your equipment is not grounded correctly, it can cause power surges and even equipment failures.

Do you use antennas? If so, ensure correct placement

Your signal can be affected if the antenna is moved even to the slightest. So, make sure you have it at the best place possible for the best functionality.

If you use a computer workstation, take a look at the keyboard keys

If you forget your CAPS lock or NUM lock keys on it will cause misspelling issues that will lead to errors.

Examine your surge protectors

Also, check on your surge protectors and make sure that they are attached correctly and are not malfunctioning.

A good practice is to have spare parts or spare devices handy. This way, your technician can simply swap those out for troubleshooting purposes. The spare components usually include Ethernet routers, radio modems, and any other parts that are pre-configured for use in your system.

If You Didn't Find The Problem, Then Take a Closer Look

If the previous inspections didn't shed a light on your issue, then it's time for an in-depth approach. The following are some of the things you can do.

Make sure that all the communication processes are happening

The main purpose and core of all SCADA systems is communication. Network communications, as well as telemetry communication processes, are the connection responsible for transferring information from sensors to deployed RTUs at your remote sites to your HMI.

The data transfer happens through whatever method you have at your sites, such as Ethernet, Frame Relay/T1, or even POTS. So, locate where your communication ports are and confirm that they are talking through the wire.

Usually, the connector itself will have an indicator with a blinking light that will help you to confirm that. If the light is off, basically no signal is getting through the wire. Also, if you can, check on the signal of your telemetry communication. Is the radio getting interferences? Is your cellular network down?

And, don't forget to take the time to make sure that your signal is being sent to the correct places. For example, are there lights on the device's interface? Is your configuration, settings, and firmware all up-to-date? Do all your wires seem to be connected properly (look for frayed strands, not too many wires on the contact, etc)? Are all the components on the equipment's interface connected to something in the first place?

Another thing is don't forget to take a look at the correct connections using your device's user manual.

Verify your HMI

The SCADA HMI can be anything from the display on a device's interface to a heavy-duty dedicated hardware and software combo to which the data is being sent to.

A well-designed HMI will monitor all system communications, give you a user-friendly graphical interface showing all the details about your network, and provide alarm notifications to you or to the appropriate people. If your HMI is not receiving data, your system will not be functional.

SCADA HMI
The HMI allows you to understand your SCADA data, and it gives you an extensive reports related to network alarms.

Often times simple settings change in your HMI can cause problems in the whole SCADA system. These problems will usually involve a failure in collecting data and not sending commands properly. So, it's important to check if all your settings and configurations are set as they should be. To avoid overlook important aspects of your HMI, have your most experienced techs review it and confirm with your SCADA vendor's tech support to make sure everything is working correctly.

If you rebooted, debugged, and commissioned your SCADA devices, then it's safe to say that your issue is not software-related. You'll probably be looking at a hardware failure of some sort. These can be power supply failures, I/O issues, transient surges, and etc. Make sure to check on your basics first and then work backward to the origin of a given signal, input or output.

Complete field verification

An advantage of having monitoring equipment that gives you visibility over remote sites is that you'll know what is happening without having to be physically there. However, if you are not receiving information, then this probably means that you have a problem with your monitoring equipment.

So, if all the previous verifications showed that everything is in order then your next step is to make a trip to your remote site.

Once you get there, you'll want to check on a few things. First, if you have a radio system, visually inspect your antennas for lightning strikes. If there are no apparent issues, check your RTU units to make sure that they have power and appear to be working as expected.

If you use cellular networks, make sure that they are online. If you don't find any issues there, then your next step is to take a look at your instrumentation.

The easiest way to verify if the readings from your instrumentation are correct is to manipulate the expected value to a known quantity (usually zero) and check on the SCADA readings. So, for example, if you have a flow sensor, the easiest way to verify that the its readings are correct is to turn off the gear that this sensor is connected to and confirm that the readings read zero flow. If it doesn't, or if get a reading of zero when the gear is still running, then you probably got a malfunctioning sensor.

Performing a complete field verification will probably be hard work depending on your applications, so it's best to save this step until all other possible problems have been discarded.

Maintenance and Tuneup Services Are Essential

Maintaining and tuning up your SCADA system is critical.

Keep in mind that your SCADA is just like your car and any other mechanical system. So, it needs regular maintenance to ensure the best possible performance and a better lifespan.

There are some maintenance best practices that are important to be pointed out. Some of them are:

It's also very important to be able to count on your SCADA vendor to perform tuneup services. They should come to you, assess your network, and develop a list of maintenance projects that will immediately improve your performance. They should also teach you and your techs the best ways to use your SCADA system for your needs. At the end of their visit, you should have a system that is running optimally and that you know how to keep it that way.

The Bottom Line

SCADA systems help keep visibility over remote networks, lower maintenance costs, and optimize processes. The devices monitored by these systems will be different from application to application. For example, the water/wastewater industry will need SCADA for monitoring and controlling their plants and assets, while manufacturing industries need SCADA for monitoring their processes.

This means that troubleshooting SCADA systems are not cookie-cutter processes. Each company will have to assess their own unique network.

The lists that we provided here are a good place to start if you are not very familiar with all the details about your SCADA system. But, they don't include everything you can do to troubleshoot your system, by any means.

As I've said previously, even if you can't solve the problem by yourself, you can still know what's going on and explain it all to the people who will take care of the issues. This greatly reduces the time needed to fix any potential problem.

Regardless if you are able to troubleshoot your SCADA system by yourself or not, it's vital that you can count on your vendor to provide support post-sale. Make sure you can get a quality, US-based tech support with the engineers that actually helped design your system. Factory training is also important to teach you how to properly operate your system and avoid unexpected problems.

SCADA engineering
Being able to rely on the engineers that actually designed your SCADA system is critical.

As a trusted remote monitoring and control solutions provider, we stand behind our products. We offer free training and tech support to make sure you can always keep your system running efficiently. If you would like to know more about our custom SCADA devices and the post-sale services we offer, just send us a message.

Learn More

DPS is here to help.

1-800-693-0351

Have a specific question? Ask our team of expert engineers and get a specific answer!

Ask an Expert DPS Telecom Get a Fast Answer!

Click here for more information.

Get the Alarm Fundamentals White Paper

Download our free Monitoring Fundamentals Tutorial.

An introduction to Monitoring Fundamentals strictly from the perspective of telecom network alarm management.

Download Alarm Fundamentals White Paper Download White Paper

Click here for more information.

Learn the Easy Way

Sign up for the next DPS Factory Training!

DPS Factory Training

Whether you're new to our equipment or you've used it for years, DPS factory training is the best way to get more from your monitoring.

Reserve Your Seat Today

Click here for more information.

Having trouble finding the perfect solution?

Get Help

No other network on the planet is exactly like yours. We manufacture hundreds of product variations per year that are customized to our clients' exact spec, all while providing training, tech support, and upgrade availability.

Send us a quick online message about what you're trying to accomplish. We'll give you a call and work with you to design a perfect-fit solution for your network.

Hours: Monday - Friday
7:00 a.m. - 6:00 p.m. PST
Tech Support: (559) 454-1600 / support@dpstele.com
Sales: Domestic: (800) 693-0351
International: 1+ (559) 454-1600