How to Use OpenSSH on Linux: Beginner’s Guide to Secure Remote Access

By Bo MorganLINUX ADMIN

How to Use OpenSSH on Linux: Beginner’s Guide to Secure Remote Access

Managing Linux servers remotely is one of the first skills every sysadmin or Linux enthusiast should master. The tool that makes this possible? OpenSSH. In this guide, I’ll walk you through step by step:
  • ✅ What OpenSSH is and why it matters
  • ✅ How to connect to a Linux server with SSH
  • ✅ Creating and using SSH keys for secure, password-free logins
  • ✅ Simplifying connections with an SSH config file
  • ✅ Hardening your server for better security
  • ✅ Troubleshooting common SSH issues
Whether you’re brand new to Linux or just brushing up your skills, this tutorial has you covered.

What Is OpenSSH?

OpenSSH (short for Open Secure Shell) is the standard tool for secure remote administration in the Linux world. Originally developed by the OpenBSD project, it quickly became the go-to implementation because it’s open source, fast, and extremely reliable. Today, it’s included by default in nearly every Linux distribution, ships with macOS, and even comes pre-installed on Windows 10/11. With OpenSSH, you can:
  • 🔑 Open a secure shell session on a remote server
  • 🖥️ Run commands as if you were sitting at the keyboard
  • 📂 Transfer files securely (using scp or sftp)
  • 🌐 Forward ports or tunnel traffic for advanced use cases
Simply put, OpenSSH is how Linux administrators (and developers) work from anywhere in the world without sacrificing security.

How to Connect with SSH

On Linux and macOS, the SSH client is almost always installed by default. On Windows 10/11, it’s also built in — no need for PuTTY anymore. To check if it’s installed: 
which ssh
If you see /usr/bin/ssh, you’re good to go. A basic connection looks like this: 
ssh username@server-ip
The first time you connect, SSH will show you the server’s fingerprint and ask if you trust it. If it’s your server, type yes. That fingerprint is then saved in ~/.ssh/known_hosts so you won’t be asked again.

Using an SSH Config File

Remembering IP addresses, usernames, and custom ports can get annoying fast. Instead, you can use an SSH config file to simplify connections. Edit ~/.ssh/config: 
Host myserver
    HostName 203.0.113.25
    User bo
    Port 2222
Now you can connect with a simple: 
ssh myserver
If you manage multiple servers, this trick will save you tons of time.

SSH Keys: The Secure Way to Log In

Passwords work, but they’re risky. The better way is to use SSH keys:
  • 🔒 More secure than passwords
  • ⚡ Faster and easier to use
  • 🌍 The industry standard for Linux authentication
Generate a new key (ED25519 is the modern, recommended option): 
ssh-keygen -t ed25519 -C "bo@laptop"
This creates two files in your ~/.ssh directory:
  • id_ed25519 → your private key (keep this secret)
  • id_ed25519.pub → your public key (share this with servers)
Copy the public key to your server with: 
ssh-copy-id -i ~/.ssh/id_ed25519.pub bo@server-ip
From now on, you’ll log in without typing a password.

Using a Passphrase and ssh-agent

When you generate an SSH key, you have the option to protect it with a passphrase. A passphrase works like a password for your private key — if someone steals the key file, they still can’t use it unless they also know the passphrase. This adds an extra layer of security. The downside? You’ll be asked for the passphrase every time you use the key. That’s where ssh-agent comes in. ssh-agent is a background process that holds your private keys in memory. Once you enter your passphrase and add the key with ssh-add, the agent will handle authentication for you. On most Linux desktop environments, ssh-agent runs by default, so you often don’t need to start it manually unless you’re on a server or a headless system. Example: 
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
Now you can connect to servers without retyping your passphrase each time, while still keeping the added security of a protected key. 💡 Pro tip: Create a shell alias for starting ssh-agent and adding your keys, so you can run it with a single command instead of typing the full sequence every time.

Hardening the SSH Server

Once you’re comfortable connecting with SSH keys, take a moment to harden your server. Small changes go a long way. Configuration lives in: 
/etc/ssh/sshd_config
Two critical security wins: 
PermitRootLogin no
PasswordAuthentication no
  • Disable root login – attackers love brute-forcing the root account. Use a normal user instead.
  • Disable password authentication – once SSH keys work, turn off passwords completely.
Want to reduce random bot noise? Change the default port: 
Port 2222
Restart the SSH service: 
sudo systemctl restart ssh
These simple steps make your server far harder to compromise.

Troubleshooting SSH

If something goes wrong, here are the most common fixes:
  1. Networking Make sure port 22 (or your custom port) is open. Test with: 
    ssh -v username@server-ip
  2. Permissions
    • ~/.ssh should be 700
    • Private keys should be 600
  3. Logs You can watch the auth.log file in real time while a user attempts to log in. This shows whether authentication succeeds or fails and is one of the quickest ways to diagnose SSH issues. 
    tail -f /var/log/auth.log

Final Thoughts

OpenSSH is the backbone of Linux administration. With just a few steps — using SSH keys, creating an SSH config file, disabling root login, and turning off password authentication — you’ll have both security and speed on your side.