How to Secure SSH Access on Linux Servers: Key Authentication & Fail2Ban Setup (2026 Guide)

hostmaster

6 hours ago

SSH (Secure Shell) remains the primary gateway to every Linux server on the internet. According to recent cybersecurity reports, brute-force SSH attacks have increased by over 200% since 2022, with automated bots scanning the entire IPv4 address space in under an hour. If your Linux server is exposed to the internet with default SSH configuration, you are essentially leaving your front door unlocked.

Proper SSH hardening is the single most important step in securing any Linux server.

Why Default SSH Configuration Is Dangerous

The default OpenSSH configuration on most Linux distributions prioritizes compatibility over security. Password authentication is enabled by default. Root login is often permitted. These defaults make your server vulnerable to:

Brute-force attacks — automated scripts trying thousands of password combinations per minute
Credential stuffing — attackers using leaked credentials from other services
Dictionary attacks — bots cycling through common passwords and username combinations
Zero-day exploits — vulnerabilities targeting the SSH daemon itself

Let’s walk through a complete SSH hardening process that will dramatically reduce your attack surface.

Step 1: SSH Key Pair Authentication

The single most impactful change you can make is switching from password-based to key-based authentication. SSH keys use asymmetric cryptography — a public key (safe to share) and a private key (never share this) — making them virtually immune to brute-force attacks.

Generating Your SSH Key Pair

On your local machine (your laptop or desktop, not the server), generate a new Ed25519 key pair. Ed25519 is faster and more secure than the older RSA 2048/4096:

ssh-keygen -t ed25519 -a 100 -f ~/.ssh/id_ed25519 -C "your-email@example.com"

If you need compatibility with older systems that don’t support Ed25519, use RSA with a 4096-bit key instead:

ssh-keygen -t rsa -b 4096 -a 100 -f ~/.ssh/id_rsa -C "your-email@example.com"

You will be prompted to set a passphrase. Always use a passphrase — it encrypts your private key on disk so even if someone steals the file, they cannot use it without the passphrase.

Copying Your Public Key to the Server

Use the ssh-copy-id utility to transfer your public key:

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@your-server-ip

If ssh-copy-id is not available, manually append the public key:

cat ~/.ssh/id_ed25519.pub | ssh username@your-server-ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Step 2: Hardening the SSH Daemon Configuration

Now edit the SSH daemon configuration file on the server:

sudo nano /etc/ssh/sshd_config

Apply the following changes for a production-grade configuration:

# Disable password authentication (critical)
PasswordAuthentication no

# Disable root login via SSH
PermitRootLogin no

# Use only key-based authentication
PubkeyAuthentication yes

# Restrict which users can SSH
AllowUsers your-username

# Change the default port (optional but recommended)
Port 2222

# Limit login attempts
MaxAuthTries 3

# Disable empty passwords
PermitEmptyPasswords no

# Use protocol 2 only
Protocol 2

# Set idle timeout (seconds)
ClientAliveInterval 300
ClientAliveCountMax 2

After making changes, always test the configuration before restarting:

sudo sshd -t

If the command produces no output, the configuration is valid. Now restart SSH:

sudo systemctl restart sshd

Critical safety tip: Before closing your current session, open a second terminal and test that you can log in with your new key. If something is wrong, you still have your original session to fix it. Never kill yourself out of a server!

Step 3: Setting Up Fail2Ban for SSH

Fail2Ban is an intrusion prevention framework that scans log files for repeated authentication failures and temporarily bans offending IP addresses using firewall rules. It’s lightweight, battle-tested, and runs on virtually every Linux distribution.

Installing Fail2Ban

# Debian / Ubuntu
sudo apt update && sudo apt install fail2ban -y

# RHEL / CentOS / AlmaLinux
sudo dnf install epel-release -y
sudo dnf install fail2ban -y

# Fedora
sudo dnf install fail2ban -y

Configuring Fail2Ban for SSH

Never edit the default jail.conf file directly — it will be overwritten by updates. Instead, create a local override:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

Find or add the [sshd] jail section and configure it:

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

# For aggressive mode (ban after first failed attempt if it matches known patterns)
# This catches port scanners and script kiddies faster
mode = aggressive

Key parameters explained:

maxretry = 3 — ban after 3 failed attempts
bantime = 3600 — ban duration in seconds (1 hour; set to -1 for permanent ban)
findtime = 600 — the window of time (10 minutes) in which maxretry attempts must occur
mode = aggressive — catches more attack patterns including port scans

If you changed the SSH port to something other than 22, update the port directive:

port = 2222

Starting and Enabling Fail2Ban

sudo systemctl enable --now fail2ban
sudo systemctl status fail2ban

Monitoring Fail2Ban Activity

Check the current status of the SSH jail:

sudo fail2ban-client status sshd

This shows the number of currently banned IPs and total failures. View the ban log:

sudo tail -f /var/log/fail2ban.log

To manually unban an IP address (if you locked yourself out):

sudo fail2ban-client set sshd unbanip 192.168.1.100

Step 4: Additional Security Enhancements

4.1 Use a Firewall

Combine Fail2Ban with a good firewall. UFW makes this simple on Ubuntu:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2222/tcp comment 'SSH on custom port'
sudo ufw enable

4.2 Disable SSH Protocol 1

Protocol 1 is ancient and insecure. Ensure it’s disabled:

# In /etc/ssh/sshd_config
Protocol 2

4.3 Use Strong Ciphers Only

Restrict SSH to modern, secure encryption algorithms:

# In /etc/ssh/sshd_config
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group16-sha512
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com

4.4 Rate Limiting with iptables

Add an additional layer of rate limiting directly in iptables:

sudo iptables -A INPUT -p tcp --dport 2222 -m state --state NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 2222 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP

This drops packets from any IP that establishes more than 3 new connections to port 2222 within 60 seconds.

Automated SSH Hardening Script

For server administrators managing multiple machines, here is a reusable hardening script you can deploy via Ansible or SSH:

#!/bin/bash
# ssh-hardening.sh - Apply SSH security hardening

SSHD_CONFIG="/etc/ssh/sshd_config"

# Backup
cp $SSHD_CONFIG ${SSHD_CONFIG}.bak.$(date +%Y%m%d)

# Apply settings
sed -i 's/^#*PasswordAuthentication.*/PasswordAuthentication no/' $SSHD_CONFIG
sed -i 's/^#*PermitRootLogin.*/PermitRootLogin no/' $SSHD_CONFIG
sed -i 's/^#*PubkeyAuthentication.*/PubkeyAuthentication yes/' $SSHD_CONFIG
sed -i 's/^#*MaxAuthTries.*/MaxAuthTries 3/' $SSHD_CONFIG
sed -i 's/^#*Protocol.*/Protocol 2/' $SSHD_CONFIG

# Validate
sshd -t && systemctl restart sshd || echo "ERROR: Invalid config - check ${SSHD_CONFIG}.bak.$(date +%Y%m%d)"

Verifying Your Hardening

After applying all changes, verify your server’s SSH security from an external machine:

# Test that password auth is disabled
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no user@your-server
# Should return: Permission denied (publickey)

# Test key authentication works
ssh user@your-server -p 2222 -i ~/.ssh/id_ed25519
# Should connect successfully

# Test that root login is disabled
ssh root@your-server -p 2222
# Should return: Permission denied (publickey)

Conclusion

Securing SSH access on your Linux server is not a one-time task but an ongoing practice. The combination of SSH key authentication and Fail2Ban provides a formidable defense against the vast majority of automated attacks targeting Linux servers on the internet today.

Start with key-based authentication and Fail2Ban today. Even these two steps alone will reduce your server’s attack surface by over 99% compared to a default installation. As your infrastructure grows, consider adding centralized authentication (LDAP or FreeIPA), SSH certificate authority, and multi-factor authentication for an even stronger security posture.

Have questions or run into issues? Drop a comment below — the Inertz community is here to help each other build more secure infrastructure.