> WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable.
This is true. For anyone getting alarmed that this is due to a bug in PostgreSQL, it's not - it's PostgreSQL protecting the customer from attempting to write data that it cannot durable commit - "I am going to go unavailable because I don't have enough space to save more data".
There are multiple ways to handle this, the easiest, most hands on way is to keep a monitor and alert that watches the WAL size like a hawk and then alerts OPS the moment it breaches a threshold.
Barman Cloud was a convenient choice for CloudNativePG (CNPG) because it was developed by the same team that created Barman originally (I am part of both teams). When we started CNPG, we never anticipated it would become so popular, which has obviously resulted in some technical debt. The issue you mentioned concerns our decision not to integrate pgBackRest into CNPG's core, as we aimed to develop a pluggable interface (CNPG-I).
As a community, we have decided to support volume snapshot backups and offer the Barman Cloud plugin to ensure we provide the same level of service. Our aim is to encourage other organisations or developers to create plugins for their preferred backup solutions.
Currently, as maintainers of CNPG, we must concentrate on the core capabilities and allow the ecosystem to grow with both community and, potentially, commercial solutions based on CNPG-I.
Great to see you posting on HN! I still remember making the decision to replace Crunchy Data PGO. CNPG was newer and smaller at the time, so maybe riskier, but seeing the way your team had responded to issues, had a real engineering mindset and clear knowledge of how PostgreSQL should be operated made it an easy choice!
Thank you for your efforts, I hope the balance of commercial and community works out well for you, the product is great.
Completely understand your motivations considering the lineage, and I appreciate that you guys put in the effort for the backup plug-in system. I think you guys made the right choices for that project. I appreciate it much more than the Crunchy k8s project which didn't have open images.
I was definitely hoping that a pgbackrest plug-in would mature for cnpg, but I didn't realize there were troubles on the horizon for that project.
The way I read that issue and the linked discussion was, that pgBackRest handles a lot of details itself that's otherwise handled by Kubernetes. Hence, a lot of functionality in pgBackRest is not only redundant but incompatible with how Kubernetes CSI could be used to provide incremental and differential backups. Hence, Barman and `barman-cloud` plugins are a better, natural fit for a Kubernetes environment than pgBackRest.
right and here's why CloudNativePG chose Barman over pgBackRest: https://github.com/cloudnative-pg/cloudnative-pg/issues/3077
> WAL limits need to be set carefully or you just end up filling WAL volumes and the database becoming unavailable.
This is true. For anyone getting alarmed that this is due to a bug in PostgreSQL, it's not - it's PostgreSQL protecting the customer from attempting to write data that it cannot durable commit - "I am going to go unavailable because I don't have enough space to save more data".
There are multiple ways to handle this, the easiest, most hands on way is to keep a monitor and alert that watches the WAL size like a hawk and then alerts OPS the moment it breaches a threshold.