258 questions
3
votes
1
answer
98
views
Advance `confirmed_flush_lsn` in PostgreSQL 18 without a heartbeat table when `pgoutput` skips empty transactions
After upgrading to PostgreSQL 18, pgoutput no longer sends BEGIN/COMMIT for transactions that touch only tables outside the publication. This is by design (see related question). Our replication slot'...
- 67
2
votes
1
answer
147
views
`pgoutput` does not send BEGIN/COMMIT for empty transactions in PostgreSQL 18 — replication slot LSN does not advance
Problem
After upgrading from PostgreSQL 17 to PostgreSQL 18, our logical replication consumer stops advancing confirmed_flush_lsn on the replication slot when workload consists mainly of empty ...
- 67
Advice
0
votes
2
replies
75
views
What are problems with my reasoning that a WAL (write-ahead log) is redundant for my TSDB (time-series DB)?
I am building a time series database in C++. Changes to record sets are restricted to appends. This is not a production system, rather for learning.
The motivation and purpose for a WAL (write-ahead ...
- 1
0
votes
1
answer
842
views
How to properly verify `pg_isready`
Using PG 17.
In case running PG instance under recovery flow (with recover.signal in PGDATA) pg_isready is actually signals the instance is not ready yet but not until the actual database is ready to ...
- 525
1
vote
0
answers
43
views
What is PG archive_timeout alignment
Actually for the archive_timeout PG config option what is the alignment point.
Is it counting
from the last archive time (or the server start)
or just something like now() % archive_timeout
Thanks
- 525
-1
votes
1
answer
64
views
Match Postgres WAL Segments to According Base Backup
As both the archive_command and restore_command defined with related config options looks to be injected only with %f and %p parameter how actually to identify on top of what base backup the current ...
- 525
0
votes
1
answer
82
views
What means that checkpointing leads to the wal file be synced to the data file? [closed]
In my course "Foundations of PostgreSQL v13" the instructor tells:
"Whenever the wal file has been checkpointed, which means: has been synced to the data file ..."
I read the ...
- 2,353
0
votes
0
answers
99
views
Lag increasing for Slot created by debezium kafka connect
I'm using confluent kafka for streaming of data and using postgres plugin for source and jdbc for sink connectors
While creating a connector with the following configuration:
{"name": "...
- 1,214
0
votes
0
answers
74
views
PostgreSQL SSL Replication: Why is WAL Sender Encrypted but Not WAL Receiver?
I am setting up SSL encryption for PostgreSQL replication and have the following configuration:
PostgreSQL Version: 14.3 OpenSSL Version: 3.1.0
Configuration Details:
I have enabled encrypted ...
- 41
0
votes
2
answers
455
views
Postgres CDC failure and WAL filling up disk space?
Suppose a CDC data loading tool (Fivetran, Airbyte, Debezium, etc) gets into a failure state and is not successfully running its incremental syncs of a Postgres data source. Also suppose that ...
0
votes
1
answer
320
views
Postgres logical replication, need to get only the changed column values, not all column values, on update event row
I am writing a client implementation of the Postgres logical replication protocol, but I encountered an unexpected behavior, in the DML UPDATE event
I always get all the values of the table columns, ...
- 101
0
votes
1
answer
873
views
How to reset Debezium offset to start streaming from different DB server after migration
We are using Debezium connectors to stream PostgreSQL data into kafka. We will have a major version upgrade on PostgreSQL(13 to 16) by side by side approach using logical replication to new server. ...
- 143
0
votes
0
answers
170
views
Setting from UNLOGGED to LOGGED takes a long time
I am trying to load data with pictures stored in a BLOB column. Each picture is around 1MB in size. I have about 200K rows.
First try, I copy the table straight away. After about an hour, it stops and ...
- 265
0
votes
0
answers
499
views
PostgreSQL Logical Replication Slot size is getting bigger without new records beeing generated
I have a postgresql source database where I have enabled logical replication to replicate data in another postgresql destination database. Publication contains only one table which is arround 3MBs of ...
- 1,517
0
votes
0
answers
811
views
How to get postgres to clean up pg_wal at patroni standby instance?
We have a lot of postgresql clusters set up with patroni.
Most of the time and most of all clusters are behaving as expected.
But at one of the standby nodes WAL files are not removed and so pg_wal ...
- 11