Deployment Center/Operations

Day-two operations

Update and operate X2 safely

Storage-node updates are consensus workflows. KMS maintenance is an independent operator procedure. Use immutable manifests, preserve quorum, and verify convergence after every change.

Command platform

Linux

Updates

Use the correct workflow for each component

Rolling X2 Node update

  1. Publish a signed immutable release.
  2. Open Updates in the X2 Admin UI.
  3. Review the automatically discovered stable release.
  4. Select Start Rolling Update.
  5. Monitor drain, replacement, restart, catch-up, and write restoration.

X2 checks https://x2.edgedrive.com/releases/latest.json and pins the matching immutable version manifest before creating the consensus workflow. Private mirrors can override the catalog URL and signing trust through node configuration.

X2 KMS maintenance

Update one member at a time. Preserve quorum, encrypted storage, TLS identities, client-key grants, and unseal custody. Every restarted member returns sealed and must be unsealed locally.

XC update

Verify and replace the client binary, then run xc --version. User aliases remain outside cluster consensus.

Release evidence

Record the immutable manifest URL, parent revision, UI revision, signatures, node order, and post-update convergence result.

Health and logs

Start with readiness, then inspect the node

Linux

systemctl status x2-node
tail -n 100 /var/log/x2/x2-node.log
curl --fail <X2_PUBLIC_URL>/health/ready

Windows

Get-Service X2Node
Get-Content 'C:\ProgramData\X2\Logs\x2-node-service.log' -Tail 100
Invoke-WebRequest <X2_PUBLIC_URL>/health/ready

macOS

sudo launchctl print system/com.edgedrive.x2-node
tail -n 100 /Library/Logs/X2/x2-node.stderr.log
curl --fail <X2_PUBLIC_URL>/health/ready

Reference

Network, paths, and security boundaries

Default ports

8443/TCP
Public HTTPS, console, S3, API, MCP
9443/TCP
Inter-node mTLS mesh
18200/TCP
X2 KMS HTTPS API
18201/TCP
X2 KMS Raft traffic

Persistent paths

Linux
/usr/lib/x2, .x2 beneath the first metadata path, and logs in /var/log/x2
Windows
C:\Program Files\X2, .x2 beneath the first metadata path, and logs in C:\ProgramData\X2\Logs
macOS
/usr/local/lib/x2, .x2 beneath the first metadata path, and logs in /Library/Logs/X2

Security boundaries

  • KMS root token and shares remain with KMS operators.
  • Enrollment tokens are single-use and short-lived.
  • Private keys remain node-local.
  • Normal startup never erases storage.

Production runbooks

Respond without changing the storage contract

Use health and placement evidence from every affected node. Do not reuse a removed node identity, rewrite metadata by hand, or treat a load-balancer check as proof that storage is healthy.

01

Replace a failed disk

  1. Confirm the failed disk identity and affected placement in the Console or node diagnostics.
  2. Stop directing new work to the failed device using the supported maintenance workflow; do not delete its metadata files manually.
  3. Replace or repair the device, register the intended storage path, and wait for scanner and healing work to finish.
  4. Verify object reads and placement health from another node before closing the incident.
02

Replace a node

  1. Capture the failed node ID, cluster membership, addresses, storage paths, and KMS generation.
  2. Remove or replace membership through the cluster administration workflow only after confirming quorum and surviving-node health.
  3. Install the replacement as a new node, enroll it with a fresh join link, and configure its KMS connection against the compatible logical keyspace.
  4. Wait for KMS compatibility attestation, catch-up, and healing, then test S3 reads through a different load-balanced route.
03

Expand capacity

  1. Confirm the supported target layout and free-space state across all nodes.
  2. Add storage through the X2 configuration surface; do not mount a new path over an existing data root.
  3. Monitor scanner, placement, and healing activity until the new capacity is recognized and stable.
  4. Recheck failure-domain balance before increasing application load.
04

Recover from a failed update

  1. Stop the rollout and preserve the failed node’s logs and release identity.
  2. Do not continue to more nodes or replace on-disk state with files from another version.
  3. Use only an explicitly supported rollback path; otherwise restore the prior binary with its unchanged configuration and data, then verify readiness and object access.
  4. Resume only after every upgraded and non-upgraded node is on a release combination approved by the release record.
05

Handle a KMS outage

  1. Identify whether the provider is unreachable, sealed, unauthorized, or presenting invalid TLS.
  2. Restore an endpoint that exposes the compatible logical keys; do not initialize an unrelated keyspace for the affected X2 node.
  3. Verify the node-specific provider identity and grants, then read an existing encrypted object through more than one X2 node.
  4. Keep older provider generations until all historic ciphertext is proven readable.
06

Recovery evidence

  1. Record request IDs, node IDs, timestamps, release versions, provider generation, and the exact failed operation.
  2. Correlate public-gateway and node/storage logs rather than attributing a timeout to disk from a single message.
  3. Validate readiness, representative writes, reads, deletes, and protected-object behavior from the published endpoint.
  4. Document any backend product gap separately; this website does not change storage or security semantics.

Troubleshooting

Common deployment questions

Can several nodes run on one development machine?

Yes. Give each process unique public and internal ports plus distinct metadata and disk paths. X2 derives each node's state beneath its metadata root.

Can the bootstrap password change on restart?

No. Bootstrap credentials are consumed only when the administrator is first created. Change existing credentials through IAM.

What happens when KMI is disabled?

X2 remains functional, but credential envelopes and object data are stored without provider-backed encryption.

Can a legacy multi-service cluster upgrade in place?

No. The unified architecture requires a destructive clean-cluster migration with no state import or mixed-version runtime.

Where is node.yaml documented?

x2-node configure generates <METADATA_PATH>/.x2/config/node.yaml from CLI or environment values. Operators normally do not maintain a large YAML manually.

Where do I add another node?

Follow the dedicated cluster expansion guide; do not bootstrap another independent cluster.