Configure Nginx Upstream Load Balancing

Nginx upstream load balancing lets one public virtual host send traffic to two or more backend servers. It is useful when an application runs on several ports, containers, virtual machines, or private hosts, and you want Nginx to spread requests, avoid failed backends temporarily, and keep one stable hostname for users.

For reverse-proxied HTTP applications, open source Nginx upstream blocks can handle round-robin balancing, least_conn, ip_hash, consistent hashing, passive failure handling, backup servers, upstream keepalive connections, verification, rollback, and common failure patterns. If your basic reverse proxy is not working yet, configure the Nginx reverse proxy first, then add upstream load balancing after one backend returns the correct response.

Understand Nginx Upstream Load Balancing

An Nginx load-balancing setup usually has two parts: an upstream block in the http context and a proxy_pass directive that points to that upstream group. Nginx receives the client request, chooses a backend from the upstream group, proxies the request, then returns the backend response to the client.

The main directives and parameters fit these contexts:

Directive or Parameter	Context	Purpose
upstream	http	Defines a named group of backend servers.
server	upstream	Adds a backend address, port, Unix socket, or server parameter to an upstream group.
least_conn	upstream	Sends a new request to the backend with the fewest active connections, while still considering weights.
ip_hash	upstream	Keeps requests from the same client IP on the same backend when possible.
hash	upstream	Chooses a backend from a key such as $request_uri, optionally with consistent hashing.
keepalive	upstream	Keeps idle upstream connections open for reuse by worker processes.
max_fails	server parameter	Controls how many failed attempts can mark a backend temporarily unavailable.
fail_timeout	server parameter	Sets the window for counting failures and the temporary unavailable period.
backup	server parameter	Marks a backend that receives traffic only when primary backends are unavailable.
proxy_next_upstream	http, server, location	Controls which upstream failures allow Nginx to try another backend.

Open source Nginx uses passive failure handling. It reacts to connection errors, timeouts, and selected upstream responses while handling real requests. Active health probes with the health_check directive belong to NGINX Plus, so do not copy health_check into a normal open source Nginx configuration and expect it to pass nginx -t.

Change load-balancing rules during a maintenance window or with a tested rollback path. A wrong upstream block can move every request for a hostname from a working backend to a broken one.

Configure Basic Nginx Upstream Load Balancing

Check Nginx Upstream Requirements

Confirm that Nginx is installed and available:

nginx -v

Expected output shows the installed branch:

nginx version: nginx/1.x.x

Each backend should respond directly from the Nginx host before you place it in an upstream group. Test the first backend with the curl command:

curl -fsS http://127.0.0.1:3000/health

A small health endpoint might return output like this:

ok

Repeat the same direct check for every backend address you plan to place in the upstream group:

curl -fsS http://127.0.0.1:3001/health
curl -fsS http://127.0.0.1:3002/health

If the direct backend request fails, fix the application listener, container port mapping, firewall rule, or service state first. Nginx load balancing cannot repair a backend that is unreachable from the proxy host.

Choose an Nginx Upstream File Location

The upstream block belongs in the http context, not inside a server or location block. On many packaged installs, files under /etc/nginx/conf.d/*.conf are included from the main http block, which makes that directory a practical place for shared upstream definitions. Debian and Ubuntu site files under sites-available are also normally loaded inside the http context, but keep shared upstream groups separate from individual virtual hosts when you reuse them.

Back up the main Nginx configuration before editing a working server:

sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup

Also back up the active virtual host file after you identify it. On a conf.d layout, that might be a file such as /etc/nginx/conf.d/example.com.conf. On Debian and Ubuntu, it might be the matching file under /etc/nginx/sites-available/.

Create an Nginx Upstream Group

Create a shared upstream file:

sudo nano /etc/nginx/conf.d/app-upstream.conf

Add the upstream group. Replace the ports and addresses with your application backends:

upstream app_backend {
    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 max_fails=3 fail_timeout=30s;
}

With no load-balancing method specified, Nginx uses weighted round-robin. Because the three backends have equal weight in this example, Nginx rotates requests across them as connections are available.

Point an Nginx Server Block at the Upstream Group

Edit the virtual host that should use the upstream group. On Debian and Ubuntu, that might be a file under /etc/nginx/sites-available/. On Fedora, RHEL, Rocky Linux, nginx.org packages, and many container layouts, it is often a file under /etc/nginx/conf.d/.

server {
    listen 80;
    listen [::]:80;
    server_name example.com www.example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The proxy_pass http://app_backend; line points to the upstream name, not to a literal domain. Do not add a trailing path after the upstream name unless you intentionally want Nginx URI replacement behavior. If path handling is the problem you are debugging, read the Nginx proxy_pass trailing slash explanation before changing the upstream group.

Test and Reload Nginx Upstream Changes

Test the Nginx configuration before reloading:

sudo nginx -t

Expected output confirms that Nginx can parse the upstream group and virtual host:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Reload Nginx after the syntax test passes:

sudo systemctl reload nginx

The reload command normally returns no output on success. If the reload fails, run sudo nginx -t again and fix the exact file and directive named in the error.

Choose an Nginx Upstream Load Balancing Method

The right method depends on the application. Stateless applications usually work well with the default round-robin method. Long-running requests often benefit from least_conn. Applications with local session state may need ip_hash temporarily, although shared session storage is usually the better long-term fix.

Use Default Nginx Round-Robin Load Balancing

Round-robin is the default method, so no extra directive is required. Use weight when one backend should receive more requests than another:

upstream app_backend {
    server 127.0.0.1:3000 weight=3 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 weight=2 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 weight=1 max_fails=3 fail_timeout=30s;
}

With these weights, the first backend receives roughly three times as many selected requests as the third backend when all servers are healthy and equally available. Weights are selection hints, not strict per-minute quotas.

Use Nginx least_conn for Uneven Request Durations

The least_conn method sends new requests to the backend with the fewest active connections. It helps when some requests stay open longer than others, such as reporting pages, uploads, streaming responses, or APIs with mixed response times.

upstream app_backend {
    least_conn;

    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 max_fails=3 fail_timeout=30s;
}

least_conn still considers weights when you set them. A high-capacity backend with weight=3 can still receive more traffic than a smaller backend with weight=1.

Use Nginx ip_hash for Basic Session Stickiness

The ip_hash method keeps requests from the same client IP on the same backend when that backend is available. This can reduce session problems when an application stores login state in local memory or local files, but it is not a full substitute for shared sessions.

upstream app_backend {
    ip_hash;

    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

Use ip_hash as a compatibility workaround, not as a permanent session design. Users behind the same corporate NAT, VPN exit, or mobile carrier can share one public IP address and crowd onto the same backend.

Use Nginx Consistent Hashing for Cache or Shard Affinity

The hash method chooses a backend from a key. The consistent parameter reduces remapping when backends are added or removed, which can help cache-like workloads or services that benefit from stable key-to-backend routing.

upstream app_backend {
    hash $request_uri consistent;

    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;
}

This example hashes the full request URI, so repeated requests for the same path and query string tend to reach the same backend. Choose the key deliberately. Hashing on $request_uri is useful for some cache-affinity cases, but it is a poor session key for normal web applications.

Tune Nginx Upstream Failure Handling

Load balancing is not only about distributing healthy traffic. A practical upstream group also needs safe behavior when a backend stops accepting connections, times out, or returns gateway-class errors.

Use max_fails and fail_timeout in Nginx Upstreams

The max_fails and fail_timeout parameters tell Nginx when to treat a backend as temporarily unavailable. In this example, three failed attempts within 30 seconds can cause Nginx to avoid that backend for the same 30-second window:

upstream app_backend {
    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 max_fails=3 fail_timeout=30s;
}

These settings work with Nginx passive failure handling. They do not send background probes to the backend. If an upstream group has only one server, Nginx has nowhere else to send traffic, so failure parameters are much less useful in practice.

Add a Backup Server to an Nginx Upstream

A backup server receives traffic only when primary servers are unavailable. It is useful for a degraded fallback instance, a static maintenance backend, or a smaller standby server that should not handle normal traffic.

upstream app_backend {
    server 127.0.0.1:3000 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:3002 backup;
}

Keep backup-server designs simple. Nginx does not allow backup with ip_hash, hash, or random load-balancing methods, so test the exact combination with nginx -t before relying on it.

Control Nginx Retries with proxy_next_upstream

proxy_next_upstream controls when Nginx may try another upstream server after a failure. Keep retry behavior conservative for normal web applications so a backend failure does not turn one slow request into several expensive attempts.

location / {
    proxy_pass http://app_backend;
    proxy_http_version 1.1;

    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 3;
    proxy_next_upstream_timeout 30s;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

This example lets Nginx retry another backend for connection errors, timeouts, and selected gateway or service errors. Avoid broad retry rules for requests that can create, charge, upload, or mutate data unless the application is designed for safe retries.

Reuse Nginx Upstream Connections with keepalive

Upstream keepalive lets Nginx reuse idle connections to backend servers instead of opening a new TCP connection for every proxied request. It is useful for high-request reverse proxies, API gateways, and applications where connection setup overhead is noticeable.

upstream app_backend {
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
    server 127.0.0.1:3002;

    keepalive 32;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The keepalive 32 value is the maximum number of idle upstream connections kept per worker process for this upstream group. It is not a global connection limit and it is not the number of active client connections Nginx can serve. Keep the value modest until you have backend-side connection limits and monitoring in place.

Verify Nginx Upstream Load Balancing

After reloading Nginx, send several requests through the proxy. Use the public hostname when DNS is ready, or use a local request with the expected host header when testing on the Nginx server:

for i in 1 2 3 4; do
    curl -fsS -H "Host: example.com" http://127.0.0.1/
done

If your test backends return their own names, repeated requests may show traffic moving across the group:

backend-a
backend-b
backend-c
backend-a

Do not expect that exact order on every system. Active connections, weights, failed backends, client IP hashing, consistent hashing, and keepalive behavior can all change which backend handles a specific request.

Logs are often the best verification tool after a real deployment. If you need to see which upstream handled each request, add upstream variables to a temporary or site-specific access log format while you test the change.

log_format upstream_debug '$remote_addr $host $request '
                          'status=$status upstream=$upstream_addr '
                          'upstream_status=$upstream_status '
                          'upstream_response_time=$upstream_response_time';

access_log /var/log/nginx/app-upstream.log upstream_debug;

After enabling a custom log format, test and reload Nginx, make a few requests, then inspect the site-specific log with the tail command:

sudo nginx -t && sudo systemctl reload nginx
sudo tail -n 10 /var/log/nginx/app-upstream.log

Relevant log lines include the selected upstream address and response status:

127.0.0.1 example.com GET / HTTP/1.1 status=200 upstream=127.0.0.1:3000 upstream_status=200 upstream_response_time=0.012
127.0.0.1 example.com GET / HTTP/1.1 status=200 upstream=127.0.0.1:3001 upstream_status=200 upstream_response_time=0.010

Use Nginx Upstream Load Balancing Safely

Load balancing changes how users move through an application. Check these items before sending production traffic through a multi-backend upstream group.

• Use shared session storage when users must stay logged in across backends. Redis, a database, or another shared session store is better than relying on client IP stickiness.
• Keep backend versions compatible during rolling deploys. If backend A and backend B expect different cookie, API, or database behavior, load balancing can expose users to inconsistent responses.
• Make backend health endpoints cheap and honest. A backend that returns 200 while its database connection is broken is not actually healthy.
• Scope firewall rules so backend ports accept traffic only from the Nginx proxy or private network. Do not expose internal app ports publicly unless that is the intended architecture.
• Keep WebSocket locations separate when they need longer timeouts or upgrade headers. A normal HTTP proxy location is not enough for persistent upgraded connections.
• Add caching only after routing works. Configure proxy cache after you have verified that upstream selection and headers are correct.

Roll Back Nginx Upstream Load Balancing

The safest rollback is to point the virtual host back to the single backend that worked before the upstream group was added. Replace the upstream proxy_pass target with the direct backend address:

location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_http_version 1.1;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

If you created a separate upstream-only file and no other virtual host uses it, remove that file after the virtual host no longer references the upstream name:

sudo rm -f /etc/nginx/conf.d/app-upstream.conf
sudo nginx -t && sudo systemctl reload nginx

If Nginx reports host not found in upstream or no resolver defined after rollback, search for leftover references to the old upstream name with the grep command:

sudo nginx -T 2>/dev/null | grep -n "app_backend"

Remove or update every stale reference, then run sudo nginx -t again before reloading.

Troubleshoot Common Nginx Upstream Load Balancing Issues

Most upstream load-balancing problems come from one of four layers: the Nginx syntax, the backend listener, the retry/failure policy, or the application state behind the backends.

Nginx Reports upstream Directive Is Not Allowed Here

This error means the upstream block is in the wrong context, usually inside a server or location block.

nginx: [emerg] "upstream" directive is not allowed here

Move the upstream block to a file that is included from the main http block, then test again:

sudo nginx -t

Nginx Sends All Traffic to One Upstream Backend

First confirm that you did not configure ip_hash, hash, or uneven weights. Then check whether your test keeps one long-lived connection open, because a browser or HTTP client can reuse connections in ways that make backend selection look less balanced than it is.

for i in 1 2 3 4 5 6; do
    curl -fsS -H "Host: example.com" http://127.0.0.1/
done

If all requests still hit one backend, inspect the active configuration for load-balancing directives:

sudo nginx -T 2>/dev/null | grep -E "upstream|least_conn|ip_hash|hash|weight|proxy_pass"

Nginx Shows 502 Bad Gateway with an Upstream Group

A 502 Bad Gateway usually means Nginx tried to reach a backend and could not complete the upstream exchange. Check the error log for the specific upstream address:

sudo tail -n 50 /var/log/nginx/error.log

Relevant error patterns include connection refusals, bad upstream hostnames, and prematurely closed responses:

connect() failed (111: Connection refused) while connecting to upstream
upstream prematurely closed connection while reading response header from upstream

Test each backend directly from the Nginx host. If direct access fails, repair the backend before changing Nginx:

curl -fsS http://127.0.0.1:3000/health
curl -fsS http://127.0.0.1:3001/health

Use the Nginx 502 Bad Gateway troubleshooting guide when the error points to PHP-FPM sockets, dead services, container port mapping, or upstream application crashes.

Nginx Shows 504 Gateway Timeout with an Upstream Group

A 504 Gateway Timeout means Nginx connected to the upstream path but did not receive a complete response in time. Check whether one backend is slow while others are healthy:

curl -fsS -o /dev/null -w 'backend 3000: %{time_total}s\n' http://127.0.0.1:3000/health
curl -fsS -o /dev/null -w 'backend 3001: %{time_total}s\n' http://127.0.0.1:3001/health

If the backend is legitimately slow, tune the application first. Then adjust Nginx timeouts only for the location that needs more time, and retest the slow backend directly before changing the whole upstream group.

Nginx Backup Server Never Receives Traffic

A backup backend is not part of normal load balancing. It receives traffic only when primary servers are unavailable according to Nginx passive failure handling. If you want the server to receive a smaller amount of normal traffic, use a lower weight instead of backup.

upstream app_backend {
    server 127.0.0.1:3000 weight=5;
    server 127.0.0.1:3001 weight=1;
}

Nginx ip_hash Still Logs Users Out

ip_hash does not fix every session problem. The client IP can change, several users can share the same public IP, and an unavailable backend can move the user to another server. The durable fix is shared application state, such as shared sessions, shared uploads, and compatible backend versions.

Nginx Upstream Keepalive Causes Backend Errors

If errors appear only after enabling upstream keepalive, confirm that the backend supports persistent HTTP connections and that the backend connection limit is high enough. Temporarily remove keepalive and proxy_set_header Connection "";, reload Nginx, and compare behavior.

sudo nginx -t && sudo systemctl reload nginx

If the errors disappear, reintroduce keepalive with a lower value and watch backend connection usage before raising it again.

Conclusion

Nginx can now route one hostname through a verified upstream group, with direct backend checks, nginx -t, log review, and rollback protecting the change. Keep round-robin as the default until the application needs least_conn, ip_hash, hashing, backup servers, or keepalive, then retest backend selection before sending production traffic through the new method.