question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Postgresql.csv log consumes all disk space

See original GitHub issue

Bug report

Describe the bug

When running supabase, the docker overlay 2 runs full. I could track it down to the postgresql.csv logs under var/lib/docker/overlay2/**/diff/var/log/postgresql. It seems that logs don’t have an upper limit.

Expected behavior

I would expect, that the logs won’t grow indefinitely. At least, there most be some settings to specify the upper size.

Screenshots

My LXC container had 15GB diskspace and the csv file contained 6.4 GB.

System information

  • OS: Ubuntu 20.10 Server in an LXC Container

Additional context

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:14 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
kiwicopplecommented, Jan 17, 2022

Hey @Manuelbaun - this one is probably not a bug. It’s up to you to manage the configuration of your Postgres instance to suit your particular use-case.

You could check out the log_rotation_size config option (https://postgresqlco.nf/doc/en/param/log_rotation_size/) and possibly dig into a logrotate solution like https://vector.dev (which is very good!)

0reactions
ymilhahncommented, Oct 29, 2022

Seems like https://github.com/supabase/supabase/commit/4a70284e47f9ef00d1ff7f65096e4319e25333fd fixes this issue in a better way? It’s excluding just the realtime polling queries, which seem to be the cause of fast growing log files in the first place?

Can be closed?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Dealing with disk space full in postgresql
"truncate does not log the full data, only the fact that a truncate happened. In order to be able to roll it back,...
Read more >
My PostgreSQL Database is Out of Disk Space | Severalnines
It could be a problem for your disk space because it will store the WAL files until they have been received by all...
Read more >
Postgres Logs keep eating up all disk space | by sandeep arneja
Now Transaction logs should not grow and hold on to space forever. RDS by default is setup to archive them as they grow...
Read more >
Resolve DiskFull errors on Amazon RDS for PostgreSQL
I have a small Amazon Relational Database Service (Amazon RDS) for PostgreSQL database. The instance's free storage space is decreasing, ...
Read more >
Re: Plz help: PostgreSQL takes too much disk-space
the COPY command is converted from a text file we receive every evening. ... Our database currently consumes 20GB (!) of disk space!...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found