Postgresql.csv log consumes all disk space
See original GitHub issueBug report
Describe the bug
When running supabase, the docker overlay 2 runs full. I could track it down to the postgresql.csv logs under var/lib/docker/overlay2/**/diff/var/log/postgresql
. It seems that logs don’t have an upper limit.
Expected behavior
I would expect, that the logs won’t grow indefinitely. At least, there most be some settings to specify the upper size.
Screenshots
My LXC container had 15GB diskspace and the csv file contained 6.4 GB.
System information
- OS: Ubuntu 20.10 Server in an LXC Container
Additional context
Issue Analytics
- State:
- Created 2 years ago
- Comments:14 (2 by maintainers)
Top Results From Across the Web
Dealing with disk space full in postgresql
"truncate does not log the full data, only the fact that a truncate happened. In order to be able to roll it back,...
Read more >My PostgreSQL Database is Out of Disk Space | Severalnines
It could be a problem for your disk space because it will store the WAL files until they have been received by all...
Read more >Postgres Logs keep eating up all disk space | by sandeep arneja
Now Transaction logs should not grow and hold on to space forever. RDS by default is setup to archive them as they grow...
Read more >Resolve DiskFull errors on Amazon RDS for PostgreSQL
I have a small Amazon Relational Database Service (Amazon RDS) for PostgreSQL database. The instance's free storage space is decreasing, ...
Read more >Re: Plz help: PostgreSQL takes too much disk-space
the COPY command is converted from a text file we receive every evening. ... Our database currently consumes 20GB (!) of disk space!...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hey @Manuelbaun - this one is probably not a bug. It’s up to you to manage the configuration of your Postgres instance to suit your particular use-case.
You could check out the
log_rotation_size
config option (https://postgresqlco.nf/doc/en/param/log_rotation_size/) and possibly dig into a logrotate solution like https://vector.dev (which is very good!)Seems like https://github.com/supabase/supabase/commit/4a70284e47f9ef00d1ff7f65096e4319e25333fd fixes this issue in a better way? It’s excluding just the realtime polling queries, which seem to be the cause of fast growing log files in the first place?
Can be closed?