Persistent volumes in Chef 360 Platform
Chef 360 Platform runs on Kubernetes and uses persistent volumes to store data that must survive pod restarts, upgrades, and node failures. This article explains what each service stores, what a persistent volume protects, and which data requires customer-managed disaster recovery backups.
Overview
Chef 360 Platform uses persistent storage to keep service data available across restarts and upgrades. A persistent volume protects data when Kubernetes replaces or reschedules a pod. A persistent volume isn’t a backup: disaster recovery requires separate backups of critical data.
The following services use persistent storage:
- PostgreSQL database
- Object storage, such as MinIO or an external S3-compatible service
- RabbitMQ messaging
- Valkey caching
- OpenSearch search and indexing
Storage and recovery by service
PostgreSQL database
PostgreSQL stores Chef 360 Platform’s core server data, including users, organizations, roles, policies, and transactional data. PostgreSQL stores customer transactional data in multiple databases. Each database represents a specific domain area, such as DSM, Courier state, or node management. All PostgreSQL databases must use persistent storage and must be included in backups.
Object storage
Object storage stores Courier job artifacts, audit logs, usage reports, Habitat packages hosted by the embedded Chef Habitat Builder, and other structured but variable action results, such as Compliance reports and Infra Client runs. It also stores data in Parquet format for reporting and analytics. This data is critical and must be included in backups. You can externalize object storage by using MinIO or another external S3-compatible service.
RabbitMQ messaging
RabbitMQ queues messages between Chef 360 Platform services. This section describes the current internal RabbitMQ behavior in supported Chef 360 Platform releases and may change if the platform’s messaging implementation changes in a future release. Kubernetes pods are temporary, so data stored only in a container’s local filesystem is deleted when Kubernetes replaces the pod. A persistent volume claim (PVC) connects RabbitMQ to persistent storage. RabbitMQ also requires persistent storage for stable node identities and internal state. Without this state, a RabbitMQ cluster can lose its topology or queues and may experience split-brain scenarios.
RabbitMQ uses two types of queues:
- Courier job signals use ephemeral queues. These signals contain lightweight JSON payloads that tell Courier when a job is ready and prompt it to retrieve the job data from PostgreSQL.
- Interservice events use durable queues. These events communicate application state changes, such as a new tenant or a node check-in.
Persistent storage preserves durable queues, messages, message metadata, and RabbitMQ’s internal state across pod restarts. When RabbitMQ experiences memory pressure, it can page data to PVC, so reliable persistent storage also provides space for this operation. RabbitMQ isn’t externalized and isn’t included in disaster recovery backups. After a restore, queued events are considered stale and aren’t restored. RabbitMQ event payloads contain only identifiers and data signals, not customer data. Customer data remains in PostgreSQL and object storage.
Valkey caching
Valkey caches frequently accessed data to improve performance. Although Valkey keeps data primarily in memory, it writes snapshots (RDB) and append-only files (AOF) to persistent storage. A persistent volume claim (PVC) lets Valkey reconnect to the existing storage volume if Kubernetes moves the pod to another node. This persistence reduces cache loss during pod restarts, maintenance, or other failures and helps the platform avoid rebuilding the cache from scratch. Valkey data isn’t required for disaster recovery and doesn’t need to be backed up or restored because the platform can rebuild the cache. The PVC supports Valkey performance, scalability, and availability; it doesn’t make Valkey data part of disaster recovery.
OpenSearch search and indexing
OpenSearch stores the search indices used for search and reporting. A persistent volume keeps the indices available when a pod restarts, so the platform doesn’t need to reindex them after a routine pod failure or restart. OpenSearch backups are optional for disaster recovery. If you restore OpenSearch data from a snapshot or persistent volume backup, reindex to refresh search and reporting data after recovery. If you don’t restore OpenSearch data, Chef 360 Platform can rebuild the indices as nodes check back in, but search and reporting data may remain incomplete until reindexing or repopulation finishes.