<!DOCTYPE html>
<html class="theme-dark" lang="en
"
prefix="og: https://ogp.me/ns#">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" href="https://static.luevano.xyz/images/icons/favicon.ico">
<title>Configure system logs on Arch to avoid filled up disk -- Luévano's Blog</title>
<meta name="description" content="How to configure the system logs, mostly journald, from filling up the disk, on Arch."/>
<link rel="alternate" type="application/rss+xml" href="https://blog.luevano.xyz/rss.xml" title="Luévano's Blog RSS">
<!-- general style -->
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/style.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/fork-awesome/css/fork-awesome.min.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/font-awesome/css/all.min.css">
<!-- theme related -->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/theme.js"></script>
<link id="theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/theme.css">
<!-- misc functions-->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/return_top.js"></script>
<!-- extra -->
<!-- highlight support for code blocks -->
<script type="text/javascript" src="https://static.luevano.xyz/hl/highlight.min.js"></script>
<script type="text/javascript">
hljs.initHighlightingOnLoad();
</script>
<link id="code-theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/hl/styles/nord.min.css">
<!-- og meta -->
<meta property="og:title" content="Configure system logs on Arch to avoid filled up disk -- Luévano's Blog"/>
<meta property="og:type" content="article"/>
<meta property="og:url" content="https://blog.luevano.xyz/a/arch_logs_flooding_disk.md"/>
<meta property="og:image" content="https://static.luevano.xyz/images/b/default.png"/>
<meta property="og:description" content="How to configure the system logs, mostly journald, from filling up the disk, on Arch."/>
<meta property="og:locale" content="en"/>
<meta property="og:site_name" content="Luévano's Blog"/>
</head>
<body>
<header>
<nav>
<ul>
<li>
<a href="https://luevano.xyz/"><i class="fas fa-home" alt="Home"></i><span>Home</span></a>
</li>
<li>
<a href="https://blog.luevano.xyz/"><i class="fas fa-book-open" alt="Blog"></i><span>Blog</span></a>
</li>
<li>
<a href="https://art.luevano.xyz/"><i class="fas fa-paint-brush" alt="Art"></i><span>Art</span></a>
</li>
<li><i class="fab fa-git" alt="Git"></i><span>Git</span>
<ul>
<li><a href="https://git.luevano.xyz/" target="_blank"><i class="fab fa-git-alt" alt="Git-alt"></i></a></li>
<li><a href="https://github.com/luevano" target="_blank"><i class="fab fa-github" alt="Github"></i></a></li>
<li><a href="https://gitlab.com/dluevano" target="_blank"><i class="fab fa-gitlab" alt="Gitlab"></i></a></li>
</ul>
</li>
<li><i class="fas fa-box-open" alt="Stuff"></i><span>Stuff</span>
<ul>
<li><a href="https://gb.luevano.xyz/"><i class="fas fa-gamepad" alt="Gameboy"></i><span>Gameboy</span></a></li>
</ul>
</li>
</ul>
</nav>
<button class="theme-switcher" onclick="toggleTheme()"><i class="fas fa-moon"></i><i class="fas fa-sun"></i></button>
</header>
<main>
<div class="return-top">
<button class="return-top" onclick="returnTop()" id="returnTopButton">
<i class="fas fa-arrow-up" alt="Return to top"></i>
</button>
</div>
<h1>Configure system logs on Arch to avoid filled up disk</h1>
<p>It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.</p>
<p>Turns out that out of the box, Arch has a default config for <code>systemd</code>‘s <code>journald</code> that keeps a persistent <code>journal</code> log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal <code>journal</code> directory, <code>nginx</code> directory and my now unused <code>prosody</code> instance.</p>
<p>For <code>prosody</code> it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space.
For <code>journal</code> I did a combination of configuring <code>SystemMaxUse</code> and creating a <em>Namespace</em> for all “email” related services as mentioned in the <a href="https://wiki.archlinux.org/title/Systemd/Journal#Per_unit_size_limit_by_a_journal_namespace">Arch wiki: systemd/Journal</a>; basically just configuring <code>/etc/systemd/journald.conf</code> (and <code>/etc/systemd/journald@email.conf</code> with the comment change) with:</p>
<pre><code class="language-ini">[Journal]
Storage=persistent
SystemMaxUse=100MB # 50MB for the "email" Namespace
</code></pre>
<p>And then for each service that I want to use this “email” <em>Namespace</em> I add:</p>
<pre><code class="language-ini">[Service]
LogNamespace=email
</code></pre>
<p>Which can be changed manually or by executing <code>systemctl edit service_name.service</code> and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running <code>systemctl daemon-reload</code> and <code>systemctl restart service_name.service</code> (probably also restart <code>systemd-journald</code>).</p>
<p>I also disabled the logging for <code>ufw</code> by running <code>ufw logging off</code> as it logs everything to the <code>kernel</code> “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal <code>[UFW BLOCK]</code> log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.</p>
<p>Finally to clean up the <code>nginx</code> logs, you need to install <code>logrotate</code> (<code>pacman -S logrotate</code>) as that is what is used to clean up the <code>nginx</code> log directory. <code>nginx</code> already “installs” a config file for <code>logrotate</code> which is located at <code>/etc/logrotate.d/</code>, I just added a few lines:</p>
<pre><code>/var/log/nginx/*log {
rotate 7
weekly
dateext
dateformat -%Y-%m-%d
missingok
notifempty
create 640 http log
sharedscripts
compress
postrotate
test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
endscript
}
</code></pre>
<p>Once you’re ok with your config, it’s just a matter of running <code>logrotate -v -f /etc/logrotate.d/nginx</code> which forces the run of the rule for <code>nginx</code>. After this, <code>logrotate</code> will be run daily if you <code>enable</code> the <code>logrotate</code> timer: <code>systemctl enable logrotate.timer</code>.</p>
<div class="page-nav">
<span class="next">
<a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html" alt="Next">
<i class="fas fa-arrow-left" alt="Arrow left"></i>
<span>Next</span>
</a>
</span>
<span class="index">
<a href="https://blog.luevano.xyz" alt="Index">
<i class="fas fa-home" alt="Home"></i>
<span>Index</span>
</a>
</span>
<span class="previous">
<a href="https://blog.luevano.xyz/a/manga_server_with_komga.html" alt="Previous">
<i class="fas fa-arrow-right" alt="Arrow right"></i>
<span>Previous</span>
</a>
</span>
</div>
<hr>
<div class="article-info">
<p>By David Luévano</p>
<p>Created: Thu, Jun 15, 2023 @ 10:22 UTC</p>
<p>Modified: Thu, Jun 15, 2023 @ 10:24 UTC</p>
<div class="article-tags">
<p>Tags:
<a href="https://blog.luevano.xyz/tag/@code.html">code</a>, <a href="https://blog.luevano.xyz/tag/@english.html">english</a>, <a href="https://blog.luevano.xyz/tag/@server.html">server</a>, <a href="https://blog.luevano.xyz/tag/@short.html">short</a>, <a href="https://blog.luevano.xyz/tag/@tools.html">tools</a>, <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> </p>
</div>
</div>
</main>
<footer>
<span>
<i class="fas fa-address-card" alt="Contact"></i>
<a href="https://blog.luevano.xyz/contact.html">Contact</a>
</span>
<span>
<i class="fas fa-donate" alt="Donate"></i>
<a href="https://blog.luevano.xyz/donate.html">Donate</a>
</span>
<span>
<i class="fas fa-rss" alt="RSS"></i>
<a target="_blank" href="https://blog.luevano.xyz/rss.xml">RSS</a>
</span>
<br>
<span class="created-with">
<i class="fas fa-hammer" alt="Hammer"></i>
Created with <a href="https://github.com/luevano/pyssg">pyssg</a>
</span>
<br>
<span class="copyright">
Copyright <i class="far fa-copyright" alt="Copyright"></i> 2023 David Luévano Alvarado
</span>
</footer>
</body>
</html>