From 4299f2c21ff4e13a911f8b41d99d1fcef7823ec6 Mon Sep 17 00:00:00 2001
From: David Luevano Alvarado <david@luevano.xyz>
Date: Thu, 15 Jun 2023 04:23:12 -0600
Subject: add arch logs entry
---
live/blog/a/arch_logs_flooding_disk.html | 182 +++++++++++++++++++++++++++++++
live/blog/a/manga_server_with_komga.html | 6 +
live/blog/index.html | 1 +
live/blog/rss.xml | 44 ++++++++
live/blog/sitemap.xml | 6 +
live/blog/tag/@code.html | 1 +
live/blog/tag/@english.html | 1 +
live/blog/tag/@server.html | 1 +
live/blog/tag/@short.html | 1 +
live/blog/tag/@tools.html | 1 +
live/blog/tag/@tutorial.html | 1 +
11 files changed, 245 insertions(+)
create mode 100644 live/blog/a/arch_logs_flooding_disk.html
(limited to 'live/blog')
diff --git a/live/blog/a/arch_logs_flooding_disk.html b/live/blog/a/arch_logs_flooding_disk.html
new file mode 100644
index 0000000..4b8bd19
--- /dev/null
+++ b/live/blog/a/arch_logs_flooding_disk.html
@@ -0,0 +1,182 @@
+<!DOCTYPE html>
+<html class="theme-dark" lang="en
+"
+ prefix="og: https://ogp.me/ns#">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="icon" href="https://static.luevano.xyz/images/icons/favicon.ico">
+<title>Configure system logs on Arch to avoid filled up disk -- Luévano's Blog</title>
+ <meta name="description" content="How to configure the system logs, mostly journald, from filling up the disk, on Arch."/>
+<link rel="alternate" type="application/rss+xml" href="https://blog.luevano.xyz/rss.xml" title="Luévano's Blog RSS">
+ <!-- general style -->
+ <link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/style.css">
+ <link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/fork-awesome/css/fork-awesome.min.css">
+ <link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/font-awesome/css/all.min.css">
+ <!-- theme related -->
+ <script type="text/javascript" src="https://static.luevano.xyz/scripts/theme.js"></script>
+ <link id="theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/theme.css">
+ <!-- misc functions-->
+ <script type="text/javascript" src="https://static.luevano.xyz/scripts/return_top.js"></script>
+ <!-- extra -->
+ <!-- highlight support for code blocks -->
+<script type="text/javascript" src="https://static.luevano.xyz/hl/highlight.min.js"></script>
+<script type="text/javascript">
+ hljs.initHighlightingOnLoad();
+</script>
+<link id="code-theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/hl/styles/nord.min.css">
+
+
+
+
+
+
+
+ <!-- og meta -->
+ <meta property="og:title" content="Configure system logs on Arch to avoid filled up disk -- Luévano's Blog"/>
+ <meta property="og:type" content="article"/>
+ <meta property="og:url" content="https://blog.luevano.xyz/a/arch_logs_flooding_disk.md"/>
+ <meta property="og:image" content="https://static.luevano.xyz/images/b/default.png"/>
+ <meta property="og:description" content="How to configure the system logs, mostly journald, from filling up the disk, on Arch."/>
+ <meta property="og:locale" content="en"/>
+ <meta property="og:site_name" content="Luévano's Blog"/>
+ </head>
+
+ <body>
+ <header>
+<nav>
+ <ul>
+ <li>
+ <a href="https://luevano.xyz/"><i class="fas fa-home" alt="Home"></i><span>Home</span></a>
+ </li>
+
+ <li>
+ <a href="https://blog.luevano.xyz/"><i class="fas fa-book-open" alt="Blog"></i><span>Blog</span></a>
+ </li>
+
+ <li>
+ <a href="https://art.luevano.xyz/"><i class="fas fa-paint-brush" alt="Art"></i><span>Art</span></a>
+ </li>
+
+ <li><i class="fab fa-git" alt="Git"></i><span>Git</span>
+ <ul>
+ <li><a href="https://git.luevano.xyz/" target="_blank"><i class="fab fa-git-alt" alt="Git-alt"></i></a></li>
+
+ <li><a href="https://github.com/luevano" target="_blank"><i class="fab fa-github" alt="Github"></i></a></li>
+
+ <li><a href="https://gitlab.com/dluevano" target="_blank"><i class="fab fa-gitlab" alt="Gitlab"></i></a></li>
+ </ul>
+ </li>
+
+ <li><i class="fas fa-box-open" alt="Stuff"></i><span>Stuff</span>
+ <ul>
+ <li><a href="https://gb.luevano.xyz/"><i class="fas fa-gamepad" alt="Gameboy"></i><span>Gameboy</span></a></li>
+ </ul>
+ </li>
+ </ul>
+</nav>
+
+<button class="theme-switcher" onclick="toggleTheme()"><i class="fas fa-moon"></i><i class="fas fa-sun"></i></button>
+
+ </header>
+
+ <main>
+ <div class="return-top">
+ <button class="return-top" onclick="returnTop()" id="returnTopButton">
+ <i class="fas fa-arrow-up" alt="Return to top"></i>
+ </button>
+ </div>
+ <h1>Configure system logs on Arch to avoid filled up disk</h1>
+
+ <p>It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.</p>
+<p>Turns out that out of the box, Arch has a default config for <code>systemd</code>‘s <code>journald</code> that keeps a persistent <code>journal</code> log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal <code>journal</code> directory, <code>nginx</code> directory and my now unused <code>prosody</code> instance.</p>
+<p>For <code>prosody</code> it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space.
+For <code>journal</code> I did a combination of configuring <code>SystemMaxUse</code> and creating a <em>Namespace</em> for all “email” related services as mentioned in the <a href="https://wiki.archlinux.org/title/Systemd/Journal#Per_unit_size_limit_by_a_journal_namespace">Arch wiki: systemd/Journal</a>; basically just configuring <code>/etc/systemd/journald.conf</code> (and <code>/etc/systemd/journald@email.con</code> with the comment change) with:</p>
+<pre><code class="language-conf">[Journal]
+Storage=persistent
+SystemMaxUse=100MB # 50MB for the "email" Namespace
+</code></pre>
+<p>And then for each service that I want to use this “email” <em>Namespace</em> I add:</p>
+<pre><code class="language-conf">[Service]
+LogNamespace=email
+</code></pre>
+<p>Which can be changed manually or by executing <code>systemctl edit service_name.service</code> and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running <code>systemctl daemon-reload</code> and <code>systemctl restart service_name.service</code> (probably also restart <code>systemd-journald</code>).</p>
+<p>I also disabled the logging for <code>ufw</code> by running <code>ufw logging off</code> as it logs everything to the <code>kernel</code> “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal <code>[UFW BLOCK]</code> log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.</p>
+<p>Finally to clean up the <code>nginx</code> logs, you need to install <code>logrotate</code> (<code>pacman -S logrotate</code>) as that is what is used to clean up the <code>nginx</code> log directory. <code>nginx</code> already “installs” a config file for <code>logrotate</code> which is located at <code>/etc/logrotate.d/</code>, I just added a few lines:</p>
+<pre><code>/var/log/nginx/*log {
+ rotate 7
+ weekly
+ dateext
+ dateformat -%Y-%m-%d
+ missingok
+ notifempty
+ create 640 http log
+ sharedscripts
+ compress
+ postrotate
+ test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
+ endscript
+}
+</code></pre>
+<p>Once you’re ok with your config, it’s just a matter of running <code>logrotate -v -f /etc/logrotate.d/nginx</code> which forces the run of the rule for <code>nginx</code>. After this, <code>logrotate</code> will be run daily if you <code>enable</code> the <code>logrotate</code> timer: <code>systemctl enable logrotate.timer</code>.</p>
+
+ <div class="page-nav">
+
+ <span class="index">
+ <a href="https://blog.luevano.xyz" alt="Index">
+ <i class="fas fa-home" alt="Home"></i>
+ <span>Index</span>
+ </a>
+ </span>
+
+ <span class="previous">
+ <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html" alt="Previous">
+ <i class="fas fa-arrow-right" alt="Arrow right"></i>
+ <span>Previous</span>
+ </a>
+ </span>
+</div>
+
+
+ <hr>
+ <div class="article-info">
+ <p>By David Luévano</p>
+ <p>Created: Thu, Jun 15, 2023 @ 10:22 UTC</p>
+ <div class="article-tags">
+ <p>Tags:
+<a href="https://blog.luevano.xyz/tag/@code.html">code</a>, <a href="https://blog.luevano.xyz/tag/@english.html">english</a>, <a href="https://blog.luevano.xyz/tag/@server.html">server</a>, <a href="https://blog.luevano.xyz/tag/@short.html">short</a>, <a href="https://blog.luevano.xyz/tag/@tools.html">tools</a>, <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> </p>
+</div>
+
+ </div>
+ </main>
+
+ <footer>
+<span>
+ <i class="fas fa-address-card" alt="Contact"></i>
+ <a href="https://blog.luevano.xyz/contact.html">Contact</a>
+</span>
+
+<span>
+ <i class="fas fa-donate" alt="Donate"></i>
+ <a href="https://blog.luevano.xyz/donate.html">Donate</a>
+</span>
+
+<span>
+ <i class="fas fa-rss" alt="RSS"></i>
+ <a target="_blank" href="https://blog.luevano.xyz/rss.xml">RSS</a>
+</span>
+
+<br>
+<span class="created-with">
+ <i class="fas fa-hammer" alt="Hammer"></i>
+ Created with <a href="https://github.com/luevano/pyssg">pyssg</a>
+</span>
+
+<br>
+<span class="copyright">
+ Copyright <i class="far fa-copyright" alt="Copyright"></i> 2023 David Luévano Alvarado
+</span>
+
+ </footer>
+ </body>
+</html>
\ No newline at end of file
diff --git a/live/blog/a/manga_server_with_komga.html b/live/blog/a/manga_server_with_komga.html
index 70095c3..21c5305 100644
--- a/live/blog/a/manga_server_with_komga.html
+++ b/live/blog/a/manga_server_with_komga.html
@@ -468,6 +468,12 @@ default:other::r-x
</ul>
<div class="page-nav">
+ <span class="next">
+ <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html" alt="Next">
+ <i class="fas fa-arrow-left" alt="Arrow left"></i>
+ <span>Next</span>
+ </a>
+ </span>
<span class="index">
<a href="https://blog.luevano.xyz" alt="Index">
diff --git a/live/blog/index.html b/live/blog/index.html
index efd84fe..21f71a1 100644
--- a/live/blog/index.html
+++ b/live/blog/index.html
@@ -88,6 +88,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html">Updated the how-to entries titles</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">I had to learn Go and Lua the hard way</a></li>
diff --git a/live/blog/rss.xml b/live/blog/rss.xml
index 6a21a21..9c7de3e 100644
--- a/live/blog/rss.xml
+++ b/live/blog/rss.xml
@@ -22,6 +22,50 @@
<title>Luévano's Blog</title>
<link>https://blog.luevano.xyz</link>
</image>
+ <item>
+ <title>Configure system logs on Arch to avoid filled up disk</title>
+ <link>https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</link>
+ <guid isPermaLink="true">https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</guid>
+ <pubDate>Thu, 15 Jun 2023 10:22:20 GMT</pubDate>
+ <category>Code</category>
+ <category>English</category>
+ <category>Server</category>
+ <category>Short</category>
+ <category>Tools</category>
+ <category>Tutorial</category>
+ <description>How to configure the system logs, mostly journald, from filling up the disk, on Arch.</description>
+ <content:encoded><![CDATA[<p>It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.</p>
+<p>Turns out that out of the box, Arch has a default config for <code>systemd</code>‘s <code>journald</code> that keeps a persistent <code>journal</code> log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal <code>journal</code> directory, <code>nginx</code> directory and my now unused <code>prosody</code> instance.</p>
+<p>For <code>prosody</code> it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space.
+For <code>journal</code> I did a combination of configuring <code>SystemMaxUse</code> and creating a <em>Namespace</em> for all “email” related services as mentioned in the <a href="https://wiki.archlinux.org/title/Systemd/Journal#Per_unit_size_limit_by_a_journal_namespace">Arch wiki: systemd/Journal</a>; basically just configuring <code>/etc/systemd/journald.conf</code> (and <code>/etc/systemd/journald@email.con</code> with the comment change) with:</p>
+<pre><code class="language-conf">[Journal]
+Storage=persistent
+SystemMaxUse=100MB # 50MB for the "email" Namespace
+</code></pre>
+<p>And then for each service that I want to use this “email” <em>Namespace</em> I add:</p>
+<pre><code class="language-conf">[Service]
+LogNamespace=email
+</code></pre>
+<p>Which can be changed manually or by executing <code>systemctl edit service_name.service</code> and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running <code>systemctl daemon-reload</code> and <code>systemctl restart service_name.service</code> (probably also restart <code>systemd-journald</code>).</p>
+<p>I also disabled the logging for <code>ufw</code> by running <code>ufw logging off</code> as it logs everything to the <code>kernel</code> “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal <code>[UFW BLOCK]</code> log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.</p>
+<p>Finally to clean up the <code>nginx</code> logs, you need to install <code>logrotate</code> (<code>pacman -S logrotate</code>) as that is what is used to clean up the <code>nginx</code> log directory. <code>nginx</code> already “installs” a config file for <code>logrotate</code> which is located at <code>/etc/logrotate.d/</code>, I just added a few lines:</p>
+<pre><code>/var/log/nginx/*log {
+ rotate 7
+ weekly
+ dateext
+ dateformat -%Y-%m-%d
+ missingok
+ notifempty
+ create 640 http log
+ sharedscripts
+ compress
+ postrotate
+ test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
+ endscript
+}
+</code></pre>
+<p>Once you’re ok with your config, it’s just a matter of running <code>logrotate -v -f /etc/logrotate.d/nginx</code> which forces the run of the rule for <code>nginx</code>. After this, <code>logrotate</code> will be run daily if you <code>enable</code> the <code>logrotate</code> timer: <code>systemctl enable logrotate.timer</code>.</p>]]></content:encoded>
+ </item>
<item>
<title>Set up a manga server with Komga and mangal</title>
<link>https://blog.luevano.xyz/a/manga_server_with_komga.html</link>
diff --git a/live/blog/sitemap.xml b/live/blog/sitemap.xml
index 976623d..0efd230 100644
--- a/live/blog/sitemap.xml
+++ b/live/blog/sitemap.xml
@@ -45,6 +45,12 @@
<priority>1.0</priority>
</url>
+ <url>
+ <loc>https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</loc>
+ <lastmod>2023-06-15</lastmod>
+ <changefreq>weekly</changefreq>
+ <priority>1.0</priority>
+ </url>
<url>
<loc>https://blog.luevano.xyz/a/manga_server_with_komga.html</loc>
<lastmod>2023-06-13</lastmod>
diff --git a/live/blog/tag/@code.html b/live/blog/tag/@code.html
index 56f95fd..bcb9349 100644
--- a/live/blog/tag/@code.html
+++ b/live/blog/tag/@code.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<h3>2021</h3>
<li><span class="page-list-item">Aug 01</span> - <a href="https://blog.luevano.xyz/a/vpn_server_with_openvpn.html">Set up a VPN server with OpenVPN</a></li>
diff --git a/live/blog/tag/@english.html b/live/blog/tag/@english.html
index 3b7f7c4..e0dad45 100644
--- a/live/blog/tag/@english.html
+++ b/live/blog/tag/@english.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html">Updated the how-to entries titles</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">I had to learn Go and Lua the hard way</a></li>
diff --git a/live/blog/tag/@server.html b/live/blog/tag/@server.html
index c5f41c4..d1292e8 100644
--- a/live/blog/tag/@server.html
+++ b/live/blog/tag/@server.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<h3>2021</h3>
<li><span class="page-list-item">Aug 01</span> - <a href="https://blog.luevano.xyz/a/vpn_server_with_openvpn.html">Set up a VPN server with OpenVPN</a></li>
diff --git a/live/blog/tag/@short.html b/live/blog/tag/@short.html
index de3d874..a9b5c7d 100644
--- a/live/blog/tag/@short.html
+++ b/live/blog/tag/@short.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html">Updated the how-to entries titles</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">I had to learn Go and Lua the hard way</a></li>
<li><span class="page-list-item">May 09</span> - <a href="https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html">Al fin tengo fibra ópticona</a></li>
diff --git a/live/blog/tag/@tools.html b/live/blog/tag/@tools.html
index 26e5b05..8f9f380 100644
--- a/live/blog/tag/@tools.html
+++ b/live/blog/tag/@tools.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<li><span class="page-list-item">Jun 03</span> - <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">I had to learn Go and Lua the hard way</a></li>
<li><span class="page-list-item">May 06</span> - <a href="https://blog.luevano.xyz/a/updated_pyssg_pymdvar_and_website.html">Updated pyssg to include pymdvar and the website</a></li>
diff --git a/live/blog/tag/@tutorial.html b/live/blog/tag/@tutorial.html
index 7d8d4c7..fa88ac2 100644
--- a/live/blog/tag/@tutorial.html
+++ b/live/blog/tag/@tutorial.html
@@ -78,6 +78,7 @@
<h2>Articles</h2>
<ul class="page-list">
<h3>2023</h3>
+ <li><span class="page-list-item">Jun 15</span> - <a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html">Configure system logs on Arch to avoid filled up disk</a></li>
<li><span class="page-list-item">Jun 10</span> - <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html">Set up a manga server with Komga and mangal</a></li>
<h3>2021</h3>
<li><span class="page-list-item">Aug 01</span> - <a href="https://blog.luevano.xyz/a/vpn_server_with_openvpn.html">Set up a VPN server with OpenVPN</a></li>
--
cgit v1.2.3-70-g09d2