SSH key management best practices and implementation tips

Why is SSH key management important?

Proper SSH key management can mean the difference between safely using one of the best network security tools and opening an organization's entire network to exploitation. While individual users are able to create SSH keys using the ssh-keygen utility, this ad hoc approach is counter to modern cybersecurity DevOps principles that emphasize oversight and control.

SSH makes it possible to bypass password protection on critical servers when a user logs in from a host that has authenticated itself by sharing a public key and a signature using the associated private key. SSH keys can quickly multiply -- without controls -- and they don't expire. In addition, there's no default central authority that can keep track of who is signing in to sensitive systems. This means organizations often find themselves with a large number of SSH keys -- in some cases, as many as millions of SSH key pairs -- to manage.

SSH keys present a problem for network and security professionals who manage access to their organizations. SSH keys give users access to network resources in the same way user IDs and passwords are used.

This means SSH key management is as critical to enterprise security as any other systems for protecting user credentials. Other important reasons for getting a grasp on SSH key management in large organizations include the following:

• Widespread use of SSH, especially by network and security professionals who use it as an ad hoc tool for connecting to remote resources, means many undocumented SSH keys may be distributed on many different network hosts. Other security risks related to unmanaged SSH keys include stolen keys used by hackers and the use of SSH keys with improper permissions levels.
• Regulatory compliance rules may require SSH key management. SSH keys are access credentials, which means the same type of compliance required for any other access credentials apply to SSH. These include regulations and standards related to HIPAA, PCI DSS and Sarbanes-Oxley.
• Too many unexpired SSH keys present attackers with opportunities to exploit enterprise resources. SSH Communications Security reported research results that showed a high degree of key sprawl with as many as 90% of SSH keys in its customers' systems that were not being used.
• Granting permissions improperly is also an issue. SSH Communications Security found 10% of SSH keys were granted root access on its systems, which itself raises the stakes for organizations using SSH authentication. A compromise of a single key -- if that key is permissioned improperly -- could enable attackers to bypass firewalls, install malware or otherwise subvert more than one remote server.

Research shows SSH keys created for use in large organizations can quickly rise to such high numbers, with as many as one in 10 of those keys representing authenticated identities with root access. While SSH may be considered a method for strongly authenticating remote users, when SSH keys get out of hand, it may also be considered a source of serious vulnerabilities.

What is SSH key management?

SSH key management is the process by which an organization can use automation to identify all cryptographic keys associated with the SSH application, determine levels of permissions assigned to those keys and administer the use of those keys for SSH access.

SSH key management systems address the following issues:

• provisioning of public keys
• inventorying SSH public keys
• monitoring use of SSH public keys
• terminating SSH public keys that should be revoked

SSH key management systems must be implemented along with, and in line with, a complete set of key management policies and procedures that can help reduce the risks associated with using SSH, especially on critical systems that hold sensitive information or that perform mission-critical functions.

SSH key management systems also may be used to administer the creation of new SSH keys.

Some of the functions SSH key management systems can help implement are the following:

• key management policies to maintain control over access to restricted resources;
• SSH key rotation, a strategy for removing or replacing an SSH key that is no longer required -- key rotation calls for removing or replacing unneeded keys from the authorized_keys file on an SSH server; and
• SSH key lifecycle to put key management policies in place to control processes for the following:
  • requesting new SSH keys;
  • approving requests;
  • provisioning requests;
  • logging use of approved keys;
  • regular reviews of key-based access;
  • reauthorization of SSH keys in use; and
  • termination of SSH keys when their use is no longer valid.

When properly implemented, an SSH key management system gives organizations a way to standardize control over the creation and use of SSH keys. This also enables the organization to maintain control over who may access internal resources and what permission levels they have.

What is at stake with SSH key management?

Although the SSH protocol itself is considered secure, many vulnerabilities are still related to using SSH for privileged access to remote systems. Executives should consider addressing these vulnerabilities as part of the process of planning an SSH key access management system.

Some types of vulnerabilities relating to SSH key management are the following:

• Vulnerabilities in the SSH implementations being used. This includes issues related to the software implementing the SSH protocol, as well as issues with system configurations and protocol weaknesses. For example, legacy servers may still support connections using the original version of SSH, SSH1, also called SSH v1 or SSH-1. Because SSH1 has known security vulnerabilities, most experts recommend using version SSH2
• Access control issues. These refer to cases where too-generous user privileges are configured in accounts used by individuals or by processes to access and control resources they should not be permitted to access or control. In other words, one should configure user accounts to limit their permission levels to run commands as the root user. This type of configuration issue must be addressed at multiple levels to take advantage of the organization's existing OS directory systems, as well as other mechanisms, such as Kerberos, for authentication of user identities.
• Compromised keys. Keys that have been stolen, leaked or otherwise revealed are a big risk -- likewise for old keys that have not been terminated, which could easily be in the next hack or leak. The vulnerability is compounded when the compromised key is a backdoor key, a key designated to bypass the organization's PAM system. While such keys may make some processes easier for authorized users -- such as logging in to any organizational network resource without passwords -- when compromised, these keys give attackers full access to the organization's network.

SSH has a well-deserved reputation as the gold standard for securing network connections. But, while the SSH protocol itself is recognized as secure, the way the protocol is implemented and used may still leave it vulnerable to various attacks. SSH key management is one way to prevent many such attacks.

Tips for implementing an SSH key management system

Identifying the requirements is the first step to implementing an SSH key management, according to the NIST report, "Security of Interactive and Automated Access Management Using Secure Shell (SSH)." Some of the recommended best practices for setting up and using SSH key management in that document are the following:

• Implement SSH securely. This means being judicious in where SSH server processes are enabled, as well as keeping all SSH software updated and patched, configuring systems securely and limiting permissions using the principle of least privilege.
• Harden SSH software. This includes disabling SSH1 access, disabling other authentication methods, limiting groups and accounts authorized to use SSH authentication, and disabling SSH port forwarding.
• Maintain secure policies for SSH identities and authorized keys. This includes setting standards for creating keys. For example, some of the policies that should be addressed are the following:
  • minimum key lengths and permitted algorithms;
  • key lifetimes and key expiration processes;
  • identity key access control;
  • using passphrases for interactive users;
  • setting IAM policies related to copying identity keys -- they should not be allowed, but if allowed, guidelines should limit risk; and
  • setting limits on who is allowed to install keys.
• Set up a continuous monitoring and audit process. The only way to ensure policies are implemented correctly and system access is secure is to monitor and audit the systems that are implementing those policies.
• Identify all SSH servers, SSH keys and SSH trust relationships. Enumerating all the SSH entities -- SSH servers, SSH keys and the trust relationships those keys represent -- is just the first step. Once identified, these entities need to be reviewed to make sure they comply with organizational requirements and policies. If they don't, they should be replaced as required.

Balancing the potentially high number of unsanctioned keys with the ongoing need to use SSH for legitimate purposes, it makes sense to automate as many processes as possible. Organizations may already have some of the tools necessary. For example, identifying and enumerating SSH entities in a network can often be done using existing vulnerability scanning systems to identify SSH servers.

Other processes that should be automated -- because they are done so often -- include the provisioning and installation of new SSH keys independently of system administrators and the continuous monitoring of SSH exchanges to make sure all systems are working properly.