This guide is for operators maintaining an existing installation, including the managed service. Tenant application workflows live in Build applications. Initial installation belongs in Self-hosting.
Select the installation before running commands
All commands run from the repository root. Targets without a profile prefix operate on the local single-node demo; they do not select the hosted service.
| Task | Local single node | Local six-node fleet | Reference public Latitude host |
|---|---|---|---|
| Converge platform services | make site |
make multi-site |
make latitude-site |
| Inspect service health | make status |
make multi-status |
make latitude-status |
| Open an operator shell | make ssh |
make multi-ssh |
make latitude-ssh |
| Verify existing curated apps | make examples-test |
make PROFILE=fleet examples-test |
make latitude-check |
Convergence applies Ansible configuration and may restart changed services.
Application deployment is a separate Wrangler operation. Use
make latitude-deploy only when you intend to redeploy the hosted curated
examples. To update just the documentation, use
Publish the documentation site.
The Latitude targets address the checked-in hosted inventory. Operators of another installation should use their own inventory, state, and credentials; see Public hosting.
Inspect and support
The operator console provides platform activity, organization support, per-host capacity and service readiness, and lifecycle events. The control plane reports ready only when its required identity, quota, Access, data-service, proxy, and runtime dependencies are ready.
Use Invite and manage customers for customer access. Owners manage their own organizations and choose their usage tier. Operators can suspend an organization, reactivate it, set a quota override, or explicitly delete it. Suspension retains data; deletion purges it.
Use Platform observability to investigate collection health, service errors, and capacity pressure. A failing customer Worker alone does not indicate a platform outage.
Upgrade the runtime
The runtime version and Linux archive checksums are pinned together in
infra/ansible/inventory/group_vars/all.yml. Validate a change through the
repository checks and relevant integration checks before applying the selected
installation’s convergence command.
Runtime upgrades restart services and may interrupt requests. Schedule a maintenance window. Keep application compatibility dates unchanged unless adopting new API behavior deliberately. Worker-code rollback is separate from a runtime upgrade and does not restore database schema or application data.
The development guide lists validation tools. Runtime isolation explains the database and Images execution boundaries that must survive upgrades.
Protect persistent state
State belongs to each capability’s service. On a compact installation these directories share one host; on a fleet they reside on the selected service hosts. Additional hosts provide capacity, not replicas.
Protect the identity and quota catalogs, control-plane versions and secrets, object blobs and metadata, database files, queues, key-value state, vectors, Models encryption keys, native Durable Object state, and certificate state. Their locations and placement are described by the service roles and architecture.
Native Durable Objects and Cron state live under
/var/lib/ermahgerd-durable on their assigned shard host. A restart retains
state. Adding a shard does not migrate or replicate existing objects. There is
no built-in backup, restore, replication, or automatic failover workflow;
plan and verify consistent recovery for the whole installation.
For the reference public profile, back up .local/latitude/secrets.yml and
its Vault password independently. Restoring fresh credentials over an existing
installation is not a safe rotation procedure. See
Public hosting.
Local make down retains VM disks. Local make destroy deliberately removes
the demo’s data. Recreating a demo is not a recovery strategy for hosted data.
Maintain security and certificates
Keep the administrator portal and backing APIs on private listeners. Tenant and administrator sessions have separate origins and audiences. Public ingress validates the hostname and replaces trusted routing headers.
Workers share runtime processes. Tenant authorization, bindings, hostname routing, and workerd/V8 isolates enforce the platform boundary; it is not a VM boundary between individual Workers.
Certificates and domains covers issuance, renewal, and publication. Local HTTPS documents the independent development CA. Public installations require their own DNS, real certificates, secure portal cookies, and fresh credentials.
Telemetry retention
Prometheus retains up to seven days or 8 GB; Loki and Tempo retain seven days. Journald is bounded to 512 MiB persistent and 128 MiB runtime storage. Retention settings do not replace disk-capacity monitoring. See Platform observability for signal interpretation and operator-only verification workflows.