It's long been an issue that OT networks are almost impossible to secure and more so to gain visibility and metrics from.
Any cybersecurity initiative is doomed to fail without these obstacles being overcome, given that effective cybersecurity requires implementation of controls and policies aimed at risk reduction.
This has been stopping enterprise and property managers who manage retail, commercial and industrial assets from moving ahead with any risk reduction initiatives in their organization’s environment. Therefore these networks remain without inventory and without any documented understanding of the communications in and out bound, nor any policy or vendor management.
With the risk of Cyber Attacks constantly increasing, these companies face increasing challenges in defending their OT environments and networks against cyber attacks.
The truth though, is that these issues are not insurmountable and there is precedent where these have been overcome. In fact, in recent projects we have had great success at documenting topologies of these networks, uplifting the architecture and securing with a method that provides un-precedented detailed visibility. Further, with the aid of Site Audit against NIST compliance, and intelligent remediation, strong vendor management capabilities have also been added.
What are the challenges to managing OT networks and gaining visibility ?
1) Non-Standard Technology and Network Design
Although the IT industry has had the last three decades to evolve and develop standards in Cabling, Technology, Topology and Design, the OT industry has remain largely the same. In many cases, the OT Networks are made up of Domestic Grade equipment that is a a mixture of old legacy and new equipment. Worst still, most of this equipment is distributed across the asset rather than being hosted in a consolidated comms room. This poses both a management problem as the inventory is unknown, as well as a Security or Cyber problem over time where the location of the unknown equipment is also unknown. This non-standard nature also adds another risk when an IT or Cyber Engineer attempts to use standard tools for Vulnerability Scanning or other types of Discovery or Visibility. Most IT Scanners (in particular QUALYS) will likely drop OT devices such as DVR, BMS and HVAC Controllers. This is also true for some SCADA equipment. The main reason is that these devices are legacy, EOL or un-patched and they respond poorly to malformed packets for example. An outage of a piece of control equipment which is performing a critical function may cause an outage to entire site.
2) Disparate Vendors
The lack of management standards in OT Environments, means that there is a reliance on the vendor to provide the skillset needed to manage the particular network and provide issue resolution as and when needed. For example, CCTV Vendors would install and manage their equipment independent of BMS/HVAC vendors. As there is no standard for network design or management, each vendor would manage different parts of the network and therefore there is no integration. It's also quite common that where a company owns many properties, a different vendor will provide their service at each property thereby adding further complication and Cyber Risk.
3) Non-Compliant Protocols
Leading from the non-standard issue of item #1, most OT equipment uses proprietary protocols created and managed by the manufacturing vendor. These protocols are certainly not standard and are incompatible with monitoring or IT Tools that use traditional open protocols. Therefore, NOC platforms that monitor and manage IT networks will not be able to do the same with most OT networks. Having said that, some Cyber platforms which are designed with OT in mind, (eg Nozomi's Guardian) have the ability to import or edit proprietary protocols.
4) Remote Access and Cyber Protection
At best a firewall may be installed at the highest spot of a network hierarchy, however this is rare and even then misconfigured with ANY/ANY rules and open ports rendering it useless. OT assets that require Remote Access for Off-Site management, are usually connected directly to the internet via a shared domestic grade modem onsite, or a 4G dongle connected directly to the device. The remote engineers typically load insecure Remote Access software on the target OT device and the associated agent on their laptop. This exposes the OT device and likely the entire OT Network to the internet increasing Cyber Risk exponentially.
What Solutions are available to OT networks to be secured, managed and visible to monitoring platforms ?
1) Site Audit and Network Discovery
The first step towards network discovery is to understand the topology of the network. This can be achieved in two ways;
- Physical Onsite Network Audit: This involves a tech visiting the site visually inspecting the equipment. Cable tracing and identifying equipment such as Routers, Switches and Servers will help to build a picture of connectivity. The assistance of the Facility Manager or similar Operational personnel will be required as they have local knowledge form which the tech can make educated guesses of where the equipment is likely to be.
- Onsite Network Scan: This will assist the discovery of devices and also the connectivity of the devices in the context of a network topology, by executing a scan from a switch or router to determine which devices are currently live and communicating. Typical scanners that are safe for OT Equipment include NMAP running on a laptop connected to the target switch, with a light scan mode set of `Quick Scan Plus'. There other scanners which are more typically installed onto a device or a mini-pc which is then connected to the network. Either scan will produce a basic inventory that includes IP Address, Operating System and Manufacturer as defined by the MAC Address. This will give a clue of how many and what types of devices are present on the network at the time of the scan. This is of course not conclusive as there will be devices on different networks, or device may be offline at the time of the scan. It will be a good start though and in some cases, will provide more information
2) install an ICN or Dedicated OT Cyber Appliance.
An Integrated Communications Network is fast becoming a standard in large enterprise service providers who are tasked with managing OT Networks. This is in essence a Best Practices Design borrowed from the IT world that comprises of a Core and Access Layer. This can be scaled to accommodate the size of most sites, however may still be outside the budget allocation of most companies. In that case, a dedicated device may be installed that can act as a gateway providing security for OT Devices and Remote Access at the same time. Although there an increasing number of these coming up, one that we're very familiar with having been involved in product development as well as implementation is the Tempered Airwall. This device is a dedicated OT security appliance that uses HIP (Host Identity Protocol) to `cloak' the downstream devices. Anyone who requires remote access to the device can do so by installing a very lightweight client onto their phone, PC or laptop. This then creates an encrypted tunnel from the source laptop or device, to the destination device directly. All other devices remain cloaked until an administrator allocates access to the remote tech.
3) Network Visibility
Once an ICN or an appliance (in this case the Tempered Airwall) is installed on the OT Network, visibility can be derived in real-time that indicates the topology of the network, and the relationship of each of the devices to each other and to the remote personnel. Historical data that is captured and logged can be accessed via reports or manually exported and imported to other software platforms.
4) Cyber Policy and Compliance
Although technology standards and network visibility will go a long way towards securing the network and uplifting it to a level that makes management possible, from a Cyber perspective, vulnerabilities will still exist if the vendors are not managed and made to uplift their own Cyber policies. To this end, a site assessment focused on NIST Compliance would provide a score and a list of remediation items that will be required to bring that site to an appropriate level. This audit will uncover issues with server operating system vulnerabilities, as well as insecure implementations of the hardware however it's main benefit will be to understand the Policies that need to be implemented or uplifted to allow better vendor management and to ensure their acceptance of their responsibilities and compliance to Policy.
In summary, the steps above will help all organizations with a large, unmanaged OT environment to gain visibility into their assets and implement intelligent processes for maintaining Cyber, Vendor and Process Control.
This can be summarised into three categories;
- Device Discovery and Visibility: This includes such items as device Model and Type, Firmware Revision and Status, Network Interfaces and their Connectivity, Active Applications, Currently Configured User etc. These audit items are very important to identify vulnerabilities and then assess the remediation requirements to lower the risk profile of that device.
- Network Discovery, Management and Visibility: Understanding the topology of the network, the baseline bandwidth and the interaction between the devices is crucial to determine the baseline operation of that OT network. Once this baseline is set, monitoring systems will be able to identify anomalies and will be able to alert either an automated process or human intervention to take action as needed. Beyond potential cyber breaches, this also assists day to day management of the network and allows effective Change Management and Configuration Management.
- Policy and Vendor Management and Visibility: The ability to know at any point in time, who accessed a part of the network and what actions they took whilst connected, is important in Access Control. This includes physical site access as well as Remote Access by staff as well as vendors. Policies that cover Cyber situations, Security incidents and Vulnerability Management will be an imp0ortant overlay to the technology implemented at each asset.