From 4299f2c21ff4e13a911f8b41d99d1fcef7823ec6 Mon Sep 17 00:00:00 2001 From: David Luevano Alvarado Date: Thu, 15 Jun 2023 04:23:12 -0600 Subject: add arch logs entry --- live/blog/a/arch_logs_flooding_disk.html | 182 +++++++++++++++++++++++++++++++ live/blog/a/manga_server_with_komga.html | 6 + live/blog/index.html | 1 + live/blog/rss.xml | 44 ++++++++ live/blog/sitemap.xml | 6 + live/blog/tag/@code.html | 1 + live/blog/tag/@english.html | 1 + live/blog/tag/@server.html | 1 + live/blog/tag/@short.html | 1 + live/blog/tag/@tools.html | 1 + live/blog/tag/@tutorial.html | 1 + 11 files changed, 245 insertions(+) create mode 100644 live/blog/a/arch_logs_flooding_disk.html (limited to 'live/blog') diff --git a/live/blog/a/arch_logs_flooding_disk.html b/live/blog/a/arch_logs_flooding_disk.html new file mode 100644 index 0000000..4b8bd19 --- /dev/null +++ b/live/blog/a/arch_logs_flooding_disk.html @@ -0,0 +1,182 @@ + + + + + + +Configure system logs on Arch to avoid filled up disk -- Luévano's Blog + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + + +
+ +
+
+ +
+

Configure system logs on Arch to avoid filled up disk

+ +

It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.

+

Turns out that out of the box, Arch has a default config for systemd‘s journald that keeps a persistent journal log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal journal directory, nginx directory and my now unused prosody instance.

+

For prosody it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space. +For journal I did a combination of configuring SystemMaxUse and creating a Namespace for all “email” related services as mentioned in the Arch wiki: systemd/Journal; basically just configuring /etc/systemd/journald.conf (and /etc/systemd/journald@email.con with the comment change) with:

+
[Journal]
+Storage=persistent
+SystemMaxUse=100MB # 50MB for the "email" Namespace
+
+

And then for each service that I want to use this “email” Namespace I add:

+
[Service]
+LogNamespace=email
+
+

Which can be changed manually or by executing systemctl edit service_name.service and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running systemctl daemon-reload and systemctl restart service_name.service (probably also restart systemd-journald).

+

I also disabled the logging for ufw by running ufw logging off as it logs everything to the kernel “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal [UFW BLOCK] log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.

+

Finally to clean up the nginx logs, you need to install logrotate (pacman -S logrotate) as that is what is used to clean up the nginx log directory. nginx already “installs” a config file for logrotate which is located at /etc/logrotate.d/, I just added a few lines:

+
/var/log/nginx/*log {
+    rotate 7
+    weekly
+    dateext
+    dateformat -%Y-%m-%d
+    missingok
+    notifempty
+    create 640 http log
+    sharedscripts
+    compress
+    postrotate
+        test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
+    endscript
+}
+
+

Once you’re ok with your config, it’s just a matter of running logrotate -v -f /etc/logrotate.d/nginx which forces the run of the rule for nginx. After this, logrotate will be run daily if you enable the logrotate timer: systemctl enable logrotate.timer.

+ + + + +
+ +
+ + + + \ No newline at end of file diff --git a/live/blog/a/manga_server_with_komga.html b/live/blog/a/manga_server_with_komga.html index 70095c3..21c5305 100644 --- a/live/blog/a/manga_server_with_komga.html +++ b/live/blog/a/manga_server_with_komga.html @@ -468,6 +468,12 @@ default:other::r-x