OpenSSH stores user-specific keys and configuration inside the .ssh directory in the user’s home folder.

Common locations include:

• Linux and macOS: ~/.ssh
• Windows: %USERPROFILE%\.ssh

Multiple private keys can be stored in this directory and assigned to specific servers through the SSH client configuration file.

In this blog post, we will add multiple SSH keys to a local user profile and configure SSH to select the correct key for each connection.

• Review Existing SSH Keys
• Generate an Additional SSH Key
• Set the File Permissions
• Add the Key to SSH Agent
• Configure the SSH Client
• Connect Using the Host Alias
• Configure SSH Agent on Windows
• Install the Public Key Remotely
• Troubleshooting
• Conclusion
• Additional Resources

Review Existing SSH Keys

List the existing files in your SSH directory:

ls -la ~/.ssh

Common key files include:

• id_ed25519
• id_ed25519.pub
• id_rsa
• id_rsa.pub
• config
• known_hosts

Files ending in .pub are public keys. Files without .pub are private keys and should never be shared.

Generate an Additional SSH Key

Generate a separate Ed25519 key with a descriptive filename:

ssh-keygen -t ed25519 -C “work-server” -f ~/.ssh/id_ed25519_work

Enter a strong passphrase when prompted.

This creates:

• ~/.ssh/id_ed25519_work
• ~/.ssh/id_ed25519_work.pub

Use descriptive names such as:

• id_ed25519_work
• id_ed25519_personal
• id_ed25519_github
• id_ed25519_servername

Set the File Permissions

On Linux or macOS, configure restrictive permissions:

chmod 700 ~/.ssh

chmod 600 ~/.ssh/id_ed25519_work

chmod 644 ~/.ssh/id_ed25519_work.pub

Private keys should only be readable by the local user who owns them.

Add the Key to SSH Agent

Start SSH agent when necessary:

eval “$(ssh-agent -s)”

Add the new private key:

ssh-add ~/.ssh/id_ed25519_work

Display the currently loaded keys:

ssh-add -l

Remove a specific key with:

ssh-add -d ~/.ssh/id_ed25519_work

Configure the SSH Client

Create or edit the local SSH configuration file:

vi ~/.ssh/config

Add a separate host entry for each key:

Host work-server
    HostName server.example.com
    User admin
    IdentityFile ~/.ssh/id_ed25519_work
    IdentitiesOnly yes

Host personal-server
    HostName personal.example.com
    User jamison
    IdentityFile ~/.ssh/id_ed25519_personal
    IdentitiesOnly yes

Set the configuration file permissions:

chmod 600 ~/.ssh/config

The important settings are:

• Host creates a local connection alias.
• HostName specifies the actual server address.
• User specifies the remote username.
• IdentityFile specifies the private key.
• IdentitiesOnly yes prevents SSH from trying unrelated keys loaded in SSH agent.

Connect Using the Host Alias

After creating the configuration, connect using the alias:

ssh work-server

OpenSSH automatically uses the hostname, username, and private key defined for that alias.

You can still specify a private key manually:

ssh -i ~/.ssh/id_ed25519_work admin@server.example.com

The SSH configuration file is more convenient for frequently accessed systems.

Configure SSH Agent on Windows

Open PowerShell as Administrator and enable the OpenSSH Authentication Agent:

Set-Service -Name ssh-agent -StartupType Automatic

Start-Service ssh-agent

Add the private key:

ssh-add $HOME.ssh\id_ed25519_work

List loaded keys:

ssh-add -l

The Windows SSH configuration file is stored at:

%USERPROFILE%\.ssh\config

It uses the same Host, IdentityFile, and IdentitiesOnly settings as Linux and macOS.

Install the Public Key Remotely

The corresponding public key must be added to the remote server or service.

For a Linux server with ssh-copy-id available, run:

ssh-copy-id -i ~/.ssh/id_ed25519_work.pub admin@server.example.com

The public key is normally added to the remote user’s:

~/.ssh/authorized_keys

Services such as GitHub, GitLab, Bitbucket, and Gitea provide an account setting where the contents of the .pub file can be added.

Never upload or copy the private key.

Troubleshooting

Display detailed connection information with:

ssh -v work-server

Confirm that the expected key appears after:

Offering public key

If SSH tries too many keys or displays Too many authentication failures, confirm that the host entry contains:

IdentitiesOnly yes

Also verify:

• The correct private key is referenced.
• The public key exists on the remote account.
• The local key permissions are restrictive.
• The remote username is correct.
• SSH agent contains the expected key.
• The Host alias matches the connection command.

Conclusion

OpenSSH can manage multiple private keys from the same local user profile. Each key should use a descriptive filename and be assigned to the appropriate server through ~/.ssh/config.

Using IdentityFile with IdentitiesOnly yes ensures that SSH presents the correct key instead of attempting every identity loaded in SSH agent. This creates a cleaner and more reliable setup for accessing multiple servers, Git accounts, and organizations.

Additional Resources

• OpenSSH SSH Client Configuration
• OpenSSH ssh-add Documentation
• Microsoft OpenSSH Key Management
• GitHub Managing Multiple Accounts