The Best Way to Manage SSH Keys

The Secure Shell (SSH) protocol is used extensively by system administrators, providing a secure way to access remote critical systems, devices, and data over an unsecured network. However, their widespread use and privileged access beg the question, “what is the best way to manage SSH keys?”

With the proliferation of emerging technologies like IoT and the cloud, the importance of SSH keys has risen dramatically. These keys are now deployed across organizations, not just to enable secure remote access but, also to secure machine-to-machine sessions in highly automated environments.

Want to skip the blog and hear it right from the experts? Just click “Register” below to join our webinar on the risks of SSH key sprawl and best practices to take back control.

However, poorly managed SSH keys quickly expand an organization’s attack surface and create opportunities for compromise by malicious actors. As cybercriminals increasingly seek out SSH keys to weaponize the trusted access they provide, the risk of attack is high.

Even a single key is enough for attackers to gain undetected privileged access to critical systems and data. Once compromised, they can then move laterally within the organization, find additional keys, circumvent security controls to inject fraudulent data, subvert encryption software, install persistent malware, or actually destroy systems.

Recent large-scale coordinated malware and botnet campaigns (e.g. TrickBot, FritxFrog, Lemon_Duck, etc.) are warning signs of the real threat that misused SSH keys pose in the security landscape.

There was a time when poorly managed SSH keys could slide under the radar. However, today regulators are far more concerned with privileged access and key management than ever before. Any missteps can lead to access that violates compliance mandates such as PCI DSS, HIPAA, GDPR, CCPA, or NIST 800-53. These regulations all require controlling who can access what systems and data.

It is therefore imperative to maintain strong standards and best practices for securing the SSH keys. NIST has published the NISTIR 7966 report, which offers guidance for organizations and businesses on proper security controls for SSH deployments. The NIST recommendations emphasize SSH key inventory, vulnerability identification, risk remediation, continuous monitoring, and of course, key rotation.

Before we dig into the best way to manage SSH keys, it is important to highlight two issues.

First, these practices will be ineffective if they are not supported by clear policies that solidify operational and security requirements. That means addressing existing SSH deployments and ensuring that new SSH implementations are properly deployed and tracked.

Second, securing SSH keys means automating all related processes. With the vast numbers of SSH keys possessed by organizations, manual processes is infeasible. All processes related to the management of SSH keys can be optimized by automating the SSH key lifecycle. Most importantly, automation can significantly improve security, efficiency, and availability.

Now, let’s examine the best ways to manage SSH keys.

Many organizations have thousands of unknown and untracked SSH keys granting access to mission-critical systems. Existing legacy and orphaned keys pose a substantial security risk, which is difficult to mitigate if these keys are not discovered and inventoried.

Discovering your SSH keys means creating an inventory of the location of every SSH key pair and their trust relationships. The inventory must then be evaluated against defined corporate security policies. In addition, the inventoried SSH keys must be mapped against their owners.

Mapping all trust relationships that result from deployed identity keys and requiring review, and approval from their owners, is a crucial part of the inventory process. If not properly understood, these chains of trust can be used by criminal actors to infiltrate and pivot undetected between systems.

Any identified keys that were not approved could be an indication of a backdoor left behind by an attacker and should be removed immediately.

The next step to establishing a secure SSH implementation is to identify any weaknesses and vulnerabilities in the existing scheme. Therefore, organizations should perform a thorough audit of their existing SSH key management process and procedures.

SSH implementation vulnerabilities could be anything from configuration weaknesses (use of weak encryption algorithms) to keys with unnecessary root access. Such vulnerabilities can be easily exploited to gain unauthorized access to corporate systems.

In addition, the audit should identify any improperly configured access controls. These controls are usually configured through an Identity and Access Management (IAM) or a Privilege Access Management (PAM) platform and a powerful asset to any security program.

However, improperly configured access controls can enable SSH access accounts with elevated privileges or unauthorized access to other accounts.

Finally, the audit should discover any unused keys that sit dormant in the organization’s structure waiting to be exploited. Creating a “backdoor” key and adding it to the authorized keys file could allow an attacker to bypass any privileged access management system.

Remediation is important to close gaps and secure your organization against any attackers seeking to exploit your vulnerable SSH keys to wreak havoc.

In addition, remediation is required for regulatory compliance. All security and privacy regulations require transparency and accountability when securing sensitive systems and data. Identifying authorized users and their access rights, applying the principle of least privilege, and managing authorized keys are predominant requirements.

Your remediation plan can be quite time consuming and a sizable effort, but it is critical for security and compliance. The project might include the following actions, depending on the findings of the key inventory and vulnerability identification:

• Remove any orphaned keys as they may be “backdoor keys”
• Remove any duplicate identity keys to reduce the attack surface
• Ensure all interactive identity keys are protected with a passphrase
• Replace all keys not complying with industry standards for key length and encryption algorithm
• Assign owners (applications, business processes, individuals) for all keys granting access to servers
• Monitor log data to determine which keys are not being used and remove any unused keys that are not approved for retaining

The purpose of continuous monitoring is to ensure that all SSH key life cycle management established policies are followed and enforced. Monitoring is also important for detecting unauthorized access performed as part of a criminal activity using backdoor keys.

Here are some examples of what to look for:

• Detect legitimate SSH keys not being used and propose their removal to remediate unnecessary access grants
• Detect and monitor connections from remote locations and ensure that they are properly approved
• Detect connections using an authorized key from not approved hosts, potentially involving compromised credentials
• Identify keys that are used from outside the managed environment as “external keys” requiring manual rotation

Any deviations from the approved SSH configuration should be identified in a dashboard or report to initiate remediation actions.

Finally, the best way to manage SSH keys must involve key rotation to minimize the risks of stolen or compromised credentials. The lifetime of SSH keys should be limited and the keys should be rotated on a defined cycle.

The key cycle should be clearly defined and in accordance with the corporate security and audit policies. In addition, key owners should disallow using the same passphrases across multiple accounts.

During rotation, new keys are generated and switched with the old ones, while the unused keys are removed. This constant change of keys decreases any chances of compromised keys being used to get access to the remote servers. Key rotation is also required for administrative issues when key owners rotate to other functions or depart from the organization.

Key rotation should be performed prior to the end of a pre-configured “crypto period.” It is important to ensure that any new or rotated keys are automatically deployed to servers based on access policies controlled by the security team. A self-service console can simplify the process for system admins while automating the backend deployment, allowing them to move quickly while staying within security requirements.

The key to an effective SSH key lifecycle management strategy is to automate and streamline existing processes. Automation not only secures your SSH implementation, minimizing the risks of poor management, but it also removes from the equation error prone manual procedures. In large organizations possessing thousands of SSH keys, manual management is just not feasible.

Ready to implement a new SSH key management strategy and prevent the risk of SSH key sprawl in our organization? Register for our upcoming webinar – Ghost Hunt: How to Find and Manage Your SSH Keys.