We are back! We didn’t go anywhere we have just been busy like everyone else. Today, we carry on my favorite miniseries of the best way to not get eaten is to not look like food; proverbially of course. Part 2 of this is regarding critical asset protections. This includes, but again not limited to, domain controllers and critical asset backups, business continuity planning, IT/OT segmentation considerations, and virtualization infrastructure protections, and finally as always, my favorite part, security controls and detections that can be implemented to help monitor and respond to threats these controls may face.
Domain Controller and Other Critical Asset backups
Yes, were talking about backups! Do them often and test them. Backups, especially for domain controllers and other critical assets, should be verified that they are available and protected against unauthorized access or modification. These backups should be protected and stored within secured enclaves that include both network and identity control segmentations.
If for example your organization’s active directory were to become compromised or otherwise unavailable due to a ransomware or otherwise destructive attack; your only choice to get these services back online may be to restore from a backup. For domain controllers, the following are some recommended best practices that your organization could develop.
- Ensure that Windows Server backup is an installed feature on the domain controller and are known to be in a good state, as well as backups conducted on the domain controller and all SYSVOL shares. i.e. C:\Windows\SYSVOL – A simple command such as, wbadmin start systemstatebackup -backuptarget:<targetDrive> Will initiate a system state backup.
- Conversely, to back up SYSVOL the following command could also be run. (Keep in mind that proper security and audit permissions must be set on the account performing the backup) robocopy c:\windows\sysvol c:\sysvol-backup /copyall /mir /b /r:0 /xd.
- It would also be prudent to identify domain controllers that hold FSMO (flexible single master operations) roles as these domain controllers will need to be prioritized if a full recovery of the domain is to occur.
Finally, a few other points listed below for consideration regarding domain controller backups.
- Again, ensure that online backups are kept and secured separate from offline backups
- Backups should be encrypted at rest and in transit over the wire or when duplicated for offsite storage.
- DSRM (Directory Services Restore Model) passwords are documented and set. These will be required for both authoritative and non-authoritative restorations of domain controllers.
- Security alerting should be configured for all backup operations. Any alerts relating especially to the integrity and availability of backups should be configured to be triggered and monitored.
- Authoritative and non-authoritative domain controller restore processes should be documented and tested on a regular basis. This same level of backup planning and testing should also be applied to any critical assets.
Detection Opportunities for Enterprise Backups
As stated in part 1 of this miniseries RSM Defense, we map all of our detections to MITRE’s ATT&CK framework. If you are not familiar with the framework, please take a moment here to go back to part 1 and see the MITRE ATT&CK getting started guide linked at the bottom of the post to learn more. Nonetheless, these detections below are not meant to be all encompassing detections, but a mere starting point so that once an organization has these tools and solutions in place there are many opportunities where professional security teams can begin to start to make detections around anomalous or malicious activity and behavior using telemetry from those solutions.
| Use Case | MITRE ID | Description |
| Volume Shadow Deletion | T1490 – Inhibit System Recovery | Alerting for instances where a threat actor will delete volume shadow copies to inhibit system recovery. This can be accomplished using legitimate tools such as command line, PowerShell, and other utilities. |
| Unauthorized Access Attempt | T1078 – Valid Accounts | Alerting for unauthorized users attempting to access the media and applications that are used to manage data backups. |
| Suspicious Usage of DSRM Password |
|
Event ID 4794 – An attempt was made to set the Directory Services Restore Mode administrator password
Monitoring the following registry key on domain controllers: HKLM\System\CurrentControlSet\Control\Lsa\DSRMAdminLogonBehavior 0: (default): The DSRM Administrator account can only be used if the domain controller is restarted in Directory Services Restore Mode. 1: The DSRM Administrator account can be used for a console-based log on if the local Active Directory Domain Services service is stopped. 2: The DSRM Administrator account can be used for console or network access without needing to reboot a domain controller |
Business Continuity Planning (BCP)
BCP must be discussed at least briefly here, as it requires in depth planning and integration into an organization’s plan for any critical asset recovery to effectively take place. Critical asset recovery plans should include the following core components below at a minimum.
- What are the mission critical business applications and their operations?
- A well-defined guide of what the organizations, “crown jewel” applications and data applications are and especially how that data and those applications align with backup, failover, and recovery methodologies.
- A defined process that sets forth what asset and recovery sequences must take place
- Trained personnel to support recovery tasks.
- Processes to validate recovery.
- Clear demarcations of managing responsibility of application backup and validation of said data once its restored.
- Readily available and digestible polices and their frequencies how they are initiated, there verification and how they are tested.
- Establish service level agreements with vendors that are in direct support of any of the in-scope applications and processes.
Finally, I have observed BCP plans that have started out great but are often never updated or maintained to reflect the current and ever-changing environment or to reflect personnel or vendor changes etc. over time. I have observed coordinating recovery exercises or table tops as well and continuous training exercises help keep organizations fit and these plans up to date in regard to BCP in the event of a disaster.
IT/OT Segmentation (Independent and without issue)
I’ll preference this section with the statement that a direct compromise of an enterprise identity or asset should never give a threat actor the ability to directly pivot to an asset that has the ability to control or disrupt any OT process. Organizations should ensure that there is BOTH physical and logical segmentation between any and all enterprise domains, identities, networks and assets and those that are responsible for the direct operation and support of any OT process or control. Once the above is solidified it can reassure an organization that a threat actor will be forbidden from pivoting from any corporate domain to any OT domain by using any of the organizations accounts or access pathways.
OT environments where applicable should support applications and controls that may have a dual use in the corporate environment such as Anti-Virus, Backup solutions, end point detection tools, jump boxes etc. This also extends into other IT/OT segmentation recommendations such as inputting explicit firewall rules to restrict incoming corporate traffic and outgoing traffic from the OT environment. In general, all firewalls in these environments should operate under the “deny all” principle and only authorized traffic is permitted. There should also be attack surface reduction exercises and reviews done periodically to ensure that only necessary ports, services, and protocols are needed for operation are accessible from within the OT environment and only within the OT environment. Finally, any remote access that is leveraged in the corporate environment should not directly translate into the OT environment. In summary, these OT environments should be designed in such a way that if something disastrous were to happen in the corporate environment it would ensure the OT function would be able to operate independently and without issue.
Detections for IT and OT Segmented Environments
| Use Case | MITRE ID | Description |
| Network Service Scanning | T1046 – Network | Searching for instances where a threat actor is performing internal network discovery to identify open ports and services between segmented environments. |
| Unauthorized Authentication Attempts Between Segmented Environments | T1078 – Valid | Searching for failed logins for accounts limited to one environment attempting to login within another environment. This can detect threat actors attempting to reuse credentials for lateral movement between networks. |
Virtualized Infrastructure Protections
Virtualized infrastructure isn’t immune from threat actor targeting (i.e., vCenter, Hyper-V). As part of threat actors’ goals and objectives once they gain access to your environment, threat actors attempt to either laterally move inside of virtualized environments or escalate privileges to gain access to virtualized environments. Some easy best practices to secure and reduce the attack surface of these environments are, restrict management interfaces through VLAN segmentation to only allow connections from dedicated subnets. There are several other features and considerations that I won’t go into depth on here, but controls like VMKernal interfaces not being bound to the shared VMware interfaces that are shared with virtual machines on the host. Also, implementation of lockdown mode where only console access can be access from vCenter servers themselves is another protection that can be implemented.
One of the major things we notice and pay close attention to here in RSM Defense is the inspection and traffic, particularly around port 22 (SSH) that is heavily used for administration and maintenance of these virtualized environments. Just as your administrators use SSH for access to these systems threat actors leverage the same protocols in their access. As above, outside of enclaving network interfaces SSH access should also be disabled and then reenabled for specific administrative use cases. Further, implementation of network-based (access control lists) ACLs, on firewalls, to further limit and restrict access should also considered to limit where SSH activity can originate from on the network to these environments.
Finally, additional authentication and identity segmentation mitigations should be considered for virtualized environments. All accounts that access these environments should be dedicated and unique accounts. A best practice for this would be to use PAM solution such as, but not limited to Beyond Trust, CyberArk, Thycotic etc. We could go even further by even limiting access to these environments from specific workstations where we can enforce policy’s that wont store or cache passwords that are used to access this critical infrastructure. I will provide some links at the bottom of this post that will help you direct what ports and protocols should be targeted to isolate these vCenter environments as well as a link to best practices in securing Hyper-V environments as well.
Detections for monitoring and securing virtualized environments
| Use Case | MITRE ID | Description |
| Non-Authorized access attempts to a virtualized environment or system | T1078 – Valid | SIEM Alerts can be generated for any account that is used that doesn’t meet the access requirements. I.e., if you have a group of users and someone outside that group attempts to access this infrastructure. This will require you to maintain an updated list of authorized users and accounts for these systems so an alert can be generated. |
| Unauthorized SSH
Connection Attempt |
T1021.004 – Remote Services: SSH |
SIEM alerts can be triggered for instances where an SSH connection is attempted when SSH has not been enabled for an approved purpose or is not expected from a specific origination network or user or workstation. |
Wrap Up
As I stated in my previous post on this topic, even with a half decent business continuity plan and hardening your critical domain assets, IT/OT systems, and virtualized environments to at least pass the proverbial, “sniff test” from a would-be threat actor may just take your organization off the target list in favor of easier prey where the posture of their systems is more favorable to trivial exploitation and attack. As always, to not get eaten in the ravenous food chain of cyberattacks, it’s important to not make yourself look like food.
Links
Admin ports associated with VMware VCenter (for isolation)
Best practices for securing Hyper-V
Whoarewe?
RSM Defense and our Unit26 security team brings decades of global cyber defense operations experience to your doorsteps. We entered this arena with an innovative cloud-native security solution that aims to stop cyber threats in whatever realm or vertical your business operates, including multi-cloud, third-party hosted, or remote deployments. If you have an existing security stack that is growing, RSM Defense and Unit26 can help manage, triage and respond to your cyber threats within that environment.
If your organization is looking for help with responding to the growing number of cyber threats, let’s get in touch and talk through how we can introduce you to the RSM Defense approach to obtaining a more secured cyber presence.
-Todd
