Executive Summary
As companies shift towards greater automation, it is crucial to examine the potential dangers of excessive automation, especially in regards to automating response actions. Without proper oversight, this can result in significant disruptions to daily operations. This article examines a scenario where an automated response action caused unintended disruptions to business operations.
Investigation
The Alarm
One evening, an alarm was triggered that indicated a software package attempting to execute on a server was auto-mitigated by SentinelOne. The software package was behaving in a way that was taken as attempting to evade detection, and so the auto-mitigation steps included killing and quarantining the process. On this server, a “Protect” policy was applied, meaning automated response actions were taken without any human evaluation.
The intrusion level of those automated response actions can be customized, but they all perform an automated action without a person looking at the situation first. The image below is an alarm for malware which ended up being process automation software, but nonetheless was auto-mitigated by SentinelOne as shown in the log excerpt below.
The Business Impact
The next morning, the customer expressed concern about the result of the automated response action. The software package was a critical part of their business infrastructure and should never have been stopped from executing. The customer questioned why the agent suddenly believed the software package was malicious. It was not possible to answer this definitively, as the decision-making behind identifying and rating a process as “Malicious” versus “Suspicious” or benign is a proprietary logic.
What could be said is that any EDR solution worth its price will continually update indicator of compromise (IOC) signatures. Any worthwhile EDR solution will also include not only static detection but also behavior-based dynamic detection. Therefore, any software package run on a server is subject to updates for security, efficiency, or product feature upgrades.
Lessons Learned
Just as we learn that security is a balancing act between confidentiality, integrity and availability, there is a balance to be struck between the use of immediate automated response actions and the slower reasoning of human evaluation prior to response actions. Automation, machine learning, artificial intelligence, and the like have their place but the human component will always be necessary.
The SOC and customers must work together to define the critical assets and business processes that should never be touched by automated intrusion. They must also find the space in the environment where those swift and ruthless automated response actions are an advantage. It is a very human decision to conclude how much risk can be tolerated in each implementation.
Key Points
- Automation, machine learning, and artificial intelligence have their place, but the human component is essential.
- Critical assets and business processes should never be touched by automated intrusion.
- Automated response actions should be used only in situations where their swift and ruthless nature is an advantage.
- It is a human decision to decide how much risk is acceptable in each implementation.