← All posts Deployments

Zero-downtime deployment checklist for ordinary Linux servers

The Tryssh team ·

Zero-downtime deployment checklist for ordinary Linux servers is the kind of production question that attracts confident shortcuts. The safer approach is more useful: define the user-visible failure, collect evidence from the narrowest relevant layers, and change one thing only after the observations support it.

A deployment pipeline is a production system with credentials, state, concurrency, and failure modes. Fast delivery comes from small reversible changes and trustworthy evidence.

What this question is really asking

The search intent behind this problem is coordinating capacity, health, compatibility, drain, and rollback. That wording matters because it sets the boundary of the investigation. A vague request to “fix production” invites unrelated changes; a precise question produces a testable hypothesis and a clear finish line.

Before connecting to a host, record the affected service, environment, first known failure time, last successful check, and most recent deploy or configuration change. Preserve timestamps in UTC. If several users are investigating, choose one incident lead and one shared log of actions.

Start with evidence, not remediation

Run a compact read-only sequence:

systemctl status api
ss -lntp
curl -sS http://127.0.0.1:8080/health
journalctl -u api --since '15 minutes ago'

Each command should answer one question. Do not collect output simply because it looks technical. Mark what confirms normal behavior, what contradicts the working theory, and what is still unknown. If a command is unavailable, record that fact rather than silently replacing it with a riskier action.

The central interpretation for this case is: Zero downtime requires overlap between compatible old and new versions, meaningful readiness, traffic draining, and enough capacity. A process restart alone cannot guarantee it.

Build the failure chain

Describe the system as a path from the user to the dependency that completes the request. Then place each observation on that path. A strong explanation accounts for the symptom, the timing, and why healthy-looking components did not prevent the failure.

Use this sequence:

  1. Reproduce the exact external symptom.
  2. Bound which hosts, tenants, regions, or requests are affected.
  3. Compare desired configuration with effective runtime state.
  4. Read the smallest log window around the first failure.
  5. Check saturation, errors, traffic, and recent changes.
  6. State one hypothesis and the observation that could disprove it.
  7. Choose the smallest reversible test.

If the evidence does not converge, widen one boundary at a time. Jumping from application logs to a fleet-wide restart skips the layers most likely to explain the problem.

Changes that look fast but create debt

Restart everything. This can restore service while deleting the process state, queue position, connection history, or logs needed to explain the outage.

Increase every limit. More memory, connections, retries, or replicas can move pressure downstream and make the next failure larger.

Change multiple layers together. When DNS, firewall, credentials, and application configuration change in one attempt, success has no attributable cause and rollback becomes ambiguous.

Trust a green dashboard. Health checks observe a chosen path. They can be green while the real user path is failing, slow, or serving stale data.

Make the repair safely

Write the proposed command, expected effect, verification, and rollback before executing it. Validate syntax before reload. Keep a recovery session open during access or firewall work. For data changes, confirm backup age and restoration procedure—not merely that a backup job reported success.

After the change, test the original symptom from outside the host. Then check the adjacent failure modes: latency, errors, resource pressure, retry volume, and data correctness. “The process started” is not the same as “the service recovered.”

Turn this article into a runbook

Add the commands that were genuinely useful, expected healthy output, escalation owner, rollback location, and links to dashboards or logs. Remove speculative steps. A short runbook that the on-call engineer rehearses is more valuable than a comprehensive document nobody trusts.

For the underlying model, consult GitHub Actions documentation. Product behavior and defaults change, so first-party documentation should win over copied snippets.

Related Tryssh guides

Continue with incident deploy broke production, docker compose production, and best ssh client for mac. These connect the immediate symptom to access safety, incident response, and durable operating practice.

Investigate it with Tryssh

Tryssh is a native macOS SSH workspace built for this evidence-first loop. Its copilot can run safe read-only checks, retain host-specific context, and explain combined output. The visible terminal remains separate, SSH secrets stay in the macOS Keychain, and commands that change state wait for explicit approval.

That boundary is especially useful under pressure: automation gathers facts quickly, while the operator remains responsible for blast radius. Download Tryssh for macOS and rehearse the runbook on a non-production host before the next incident.