Web Load Server Balancing with Apache,Nginx and HAProxy

A comprehensive guide to setting up web load balancing with HAProxy, Apache on Rocky Linux, and Nginx on Ubuntu 24.04. Learn high availability, SSL termination, health checks, traffic distribution, round robin balancing, firewall configuration, and systemd services for scalable, reliable HTTP infrastructure with fault tolerance.

Original content from computingforgeeks.com - post 170462

In today’s networked environments, ensuring high availability, scalability, and reliability of web services is essential. This guide demonstrates the configuration of a Web Service Load Balancing System that distributes client requests across multiple backend servers to optimize performance and prevent downtime. Using HAProxy as the load balancer, alongside Apache and Nginx web servers running on separate nodes, the setup efficiently balances HTTP traffic. This configuration enhances service responsiveness, supports fault tolerance, and ensures that users experience minimal service interruption even if one server becomes unavailable.

Configure Apache and Nginx using the following servers

Rocky Linux (Apache)
Ubuntu 24.04 (Nginx)
HAProxy as a load balancer, benefits:
- SSL Terminal
- Health Checks
- Traffic distribution between Apache and Nginx

This set up ensures high availability, load balancing, and secure connections.

Step 1: Configure on Apache on Rocky Linux

To enable web service load balancing on Rocky Linux, Apache must be configured with the appropriate proxy and balancing modules. This setup allows incoming HTTP requests to be distributed across multiple backend servers, improving performance, redundancy, and fault tolerance. By enabling and configuring Apache’s load-balancing features, the system can efficiently manage web traffic and ensure high availability of service

Install Apache

To set up load balancing on Rocky Linux, first we need to install and start the Apache web server. Apache provides the necessary modules for proxying and distributing web traffic across backend servers.

# Install httpd and it's properties
sudo dnf install httpd -y
# Enable httpd
sudo systemctl enable httpd --now
# Check httpd status
sudo systemctl status httpd

Deploy Application

To verify that the load balancer correctly forwards client requests to each backend server, we deploy a simple web page. By creating an index.html file on each server with a unique message, it’s easy to confirm which backend the load balancer is directing traffic to.

This step creates the web page inside the default Apache document root. Create a file index.html inside /var/www/html/ using this command:

echo "Hello from Apache on Rocky Linux" | sudo tee /var/www/html/index.html

Step 2: Allow Firewall

For the load balancer to distribute traffic across your web servers, each backend must be reachable over the network

This step opens the HTTP service in the firewall so external clients—and the load balancer—can communicate with the server.

After allowing the service, we reload the firewall to apply the new rules.

# Install firewalld and enable
sudo dnf install firewalld -y
sudo systemctl enable firewalld --now
# Allow firewall, make it permanent
sudo firewall-cmd --permanent --add-service=http
# Reload firewall
sudo firewall-cmd  --reload

Step 3: Configure Nginx on Ubuntu 24.04

In this step, we will set up Nginx, a powerful and widely used web server that serves as the gateway to your application.

Installing and enabling Nginx ensures your server can handle HTTP requests and route traffic correctly.

The following commands will update your package lists, install Nginx, and start the service so your server is ready to host web content.

Install Nginx

# Update Rocky Linux
sudo apt update
# Install Nginx
sudo apt install nginx -y
# Enable Nginx
sudo systemctl enable nginx --now
# Check status
sudo systemctl status nginx

Deploying Application

Before Nginx can participate in load balancing, each backend server must host a simple application or test page. This helps verify that load balancing is working correctly by allowing you to identify which server responds to each request.

In this step, we will create a basic index.html file inside /var/www/html/ on the Nginx server to serve as its response page.

echo "Hello from nginx on Ubuntu 24.04" | sudo tee /var/www/html/index.html

Step 4: Configure HAProxy

HAProxy acts as the central load balancer that distributes traffic across your backend servers (Apache and Nginx). It ensures high availability, improved performance, and fault tolerance by intelligently routing client requests.

In this step, we will install HAProxy on one of your servers so it can manage and balance incoming web traffic. You can install HAProxy using either one of the servers: For Ubuntu,

Install HAProxy

# Update HAProxy
sudo apt update
# Install HAProxy
sudo apt install haproxy -y

Enable SSL Termination

SSL termination allows HAProxy to handle HTTPS encryption and decryption directly at the load balancer. This offloads the encryption workload from your backend servers (Apache and Nginx), improving their performance and simplifying certificate management.

In this step, you will generate a self-signed SSL certificate that HAProxy will use to securely serve HTTPS traffic.

# Make directory
sudo mkdir -p /etc/haproxy/certs
# Generate a HAProxy SSL Certificate
openssl req -new -newkey rsa:2048 -nodes -keyout haproxy.key -x509 -days 365 -out haproxy.crt
# Combine Certificate and Key for HAProxy
cat haproxy.crt haproxy.key | sudo tee /etc/haproxy/certs/haproxy.pem

Set Permission

To secure your SSL certificate and private key, it is important to restrict access so that only the HAProxy service (and system administrators) can read them.
This prevents unauthorized users from viewing or modifying sensitive files. In this step, we will tighten the directory permissions to ensure your SSL certificate is protected.

# Restrict permissions so only HAProxy and admins can read the cert and key
sudo chmod -R go-wx /etc/haproxy/certs

Step 5: Configure Load Balancing

With HAProxy installed and SSL protected, the next step is to configure how traffic is distributed between your backend web servers. Load balancing improves performance, increases availability, and ensures your users are always served by a healthy server.

In this step, we will edit the HAProxy configuration file to define:

The frontend that receives client requests
The backend servers (Rocky Linux + Apache, and Ubuntu + Nginx)
The load-balancing method HAProxy will use

Make sure to replace the placeholders with your actual ROCKY-IP and NGINX-IP before saving the configuration. When editing the /etc/haproxy/haproxy.cfg:

ROCKY-IP
NGINX-IP

Edit /etc/haproxy/haproxy.cfg,

global
    log /dev/log local0
    log /dev/log local1 notice
    daemon
    maxconn 2048

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client 50000
    timeout server 50000

frontend http_front
    bind *:8080
    default_backend web_servers

backend web_servers
    balance roundrobin
    server ubuntu 192.168.1.41:80 check
    server rocky 192.168.1.40:80 check

Step 6: Port conflict and configuration

You may encounter errors caused by running multiple services on the same server. A common problem is port conflicts two services trying to listen on the same port. To fix this, update your haproxy.cfg and change the listening ports to ensure each service uses a unique and available port.

Step 7: Restart HAProxy

After updating the HAProxy configuration, it’s necessary to restart the service to apply the changes. Enabling the service ensures that HAProxy starts automatically on system boot, maintaining load-balancing functionality consistently.

# Validate HAProxy configuration before restarting
sudo haproxy -c -f /etc/haproxy/haproxy.cfg
# Restart HAProxy
sudo systemctl restart haproxy
# Enable HAProxy
sudo systemctl enable haproxy

Step 8: Verify Setup

After completing the configuration, it’s important to verify that all services are running correctly and that the system behaves as expected. This step ensures that your authentication, NFS mounts, load balancing, or any other configured components are functioning without errors. You can now test access, check logs, and confirm that each part of the setup is working smoothly

Access your HAProxy server in a browser:

To verify that the setup was successful, you can point your browser to http://<haproxy-ip>, ensure to replace haproxy-ip with your server IP address:

IP address command

# Verification
ip addr show

Refresh several times:

The request should alternate between: . “Hello from Apache on Rocky Linux” . “Hello from Nginx on Ubuntu 24.04”

This is the output from the terminal:

This confirms that HAProxy is successfully load balancing traffic between both web servers.

Step 9: Health Check

To confirm that the load balancer is functioning correctly, you can perform a simple health check by stopping Apache on one of the backend servers. This allows you to verify whether the load balancer detects the service outage and properly reroutes traffic to the remaining healthy servers.

# Stop httpd
sudo systemctl stop httpd

Refresh HAProxy page → only Nginx responses appear.
Start Apache back, traffic resumes load balancing.

Conclusion

This guide demonstrates a complete web service load balancing setup using HAProxy as the load balancer with Apache and Nginx as backend web servers. By distributing incoming HTTP traffic across multiple backend servers, the system achieves improved performance, fault tolerance, and high availability. SSL termination at the HAProxy layer offloads encryption overhead from the backend servers, while health checks ensure that traffic is only routed to healthy instances. The round-robin load balancing method provides an even distribution of requests, and the firewall configuration ensures that backend services remain accessible. This architecture serves as a solid foundation for scaling web services in production environments.