1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
|
<!DOCTYPE html>
<html class="theme-dark" lang="en
"
prefix="og: https://ogp.me/ns#">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" href="https://static.luevano.xyz/images/icons/favicon.ico">
<title>Set up a manga server with Komga and mangal -- Luévano's Blog</title>
<meta name="description" content="How to set up a manga server with Komga as media server and mangal for downloading manga, on Arch. Tachiyomi integration is available thanks to Komga."/>
<link rel="alternate" type="application/rss+xml" href="https://blog.luevano.xyz/rss.xml" title="Luévano's Blog RSS">
<!-- general style -->
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/style.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/fork-awesome/css/fork-awesome.min.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/font-awesome/css/all.min.css">
<!-- theme related -->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/theme.js"></script>
<link id="theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/theme.css">
<!-- misc functions-->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/return_top.js"></script>
<!-- extra -->
<!-- highlight support for code blocks -->
<script type="text/javascript" src="https://static.luevano.xyz/hl/highlight.min.js"></script>
<script type="text/javascript">
hljs.initHighlightingOnLoad();
</script>
<link id="code-theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/hl/styles/nord.min.css">
<!-- og meta -->
<meta property="og:title" content="Set up a manga server with Komga and mangal -- Luévano's Blog"/>
<meta property="og:type" content="article"/>
<meta property="og:url" content="https://blog.luevano.xyz/a/manga_server_with_komga.md"/>
<meta property="og:image" content="https://static.luevano.xyz/images/b/default.png"/>
<meta property="og:description" content="How to set up a manga server with Komga as media server and mangal for downloading manga, on Arch. Tachiyomi integration is available thanks to Komga."/>
<meta property="og:locale" content="en"/>
<meta property="og:site_name" content="Luévano's Blog"/>
</head>
<body>
<header>
<nav>
<ul>
<li>
<a href="https://luevano.xyz/"><i class="fas fa-home" alt="Home"></i><span>Home</span></a>
</li>
<li>
<a href="https://blog.luevano.xyz/"><i class="fas fa-book-open" alt="Blog"></i><span>Blog</span></a>
</li>
<li>
<a href="https://art.luevano.xyz/"><i class="fas fa-paint-brush" alt="Art"></i><span>Art</span></a>
</li>
<li><i class="fab fa-git" alt="Git"></i><span>Git</span>
<ul>
<li><a href="https://git.luevano.xyz/" target="_blank"><i class="fab fa-git-alt" alt="Git-alt"></i></a></li>
<li><a href="https://github.com/luevano" target="_blank"><i class="fab fa-github" alt="Github"></i></a></li>
<li><a href="https://gitlab.com/dluevano" target="_blank"><i class="fab fa-gitlab" alt="Gitlab"></i></a></li>
</ul>
</li>
<li><i class="fas fa-box-open" alt="Stuff"></i><span>Stuff</span>
<ul>
<li><a href="https://gb.luevano.xyz/"><i class="fas fa-gamepad" alt="Gameboy"></i><span>Gameboy</span></a></li>
</ul>
</li>
</ul>
</nav>
<button class="theme-switcher" onclick="toggleTheme()"><i class="fas fa-moon"></i><i class="fas fa-sun"></i></button>
</header>
<main>
<div class="return-top">
<button class="return-top" onclick="returnTop()" id="returnTopButton">
<i class="fas fa-arrow-up" alt="Return to top"></i>
</button>
</div>
<h1>Set up a manga server with Komga and mangal</h1>
<p>I’ve been wanting to set up a manga media server to hoard some mangas/comics and access them via Tachiyomi, but I didn’t have enough space in my vultr VPS. Now that I have symmetric fiber optic at home and my spare PC to use as a server I decided to go ahead and create one. As always, <mark>i use arch btw</mark> so these instructions are specifically for it, I’m not sure how easier/harder it is for other distros, I’m just too comfortable with arch honestly.</p>
<p>I’m going to run it as an exposed service using a subdomain of my own, so the steps are taking that into account, if you want to run it locally (or on a LAN/VPN) then it is going to be easier/with less steps (you’re on your own). Also, as you might notice I don’t like to use D*ck*r images or anything (ew).</p>
<p>As always, all commands are run as root unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#yay">yay</a><ul>
<li><a href="#install">Install</a></li>
<li><a href="#usage">Usage</a></li>
</ul>
</li>
<li><a href="#mangal">mangal</a><ul>
<li><a href="#install-from-source">Install from source</a></li>
<li><a href="#configuration">Configuration</a></li>
<li><a href="#usage_1">Usage</a><ul>
<li><a href="#headless-browser">Headless browser</a></li>
<li><a href="#tui">TUI</a></li>
<li><a href="#inline">Inline</a></li>
<li><a href="#automation">Automation</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#komga">Komga</a><ul>
<li><a href="#reverse-proxy">Reverse proxy</a></li>
<li><a href="#ssl-certificate">SSL certificate</a></li>
<li><a href="#starting-using-komga">Starting using Komga</a></li>
<li><a href="#library-creation">Library creation</a><ul>
<li><a href="#set-default-directory-permissions">Set default directory permissions</a></li>
<li><a href="#populate-manga-library">Populate manga library</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#alternative-downloaders">Alternative downloaders</a></li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries, if you want it as a subdomain:</p>
<ul>
<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>komga</code> (or whatever you want).</li>
<li>An SSL certificate, if you’re following the other entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> entry), add a <code>komga.conf</code> and run <code>certbot --nginx</code> (or similar) to extend/create the certificate. More details below: <a href="#reverse-proxy">Reverse proxy</a> and <a href="#ssl-certificate">SSL certificate</a>.</li>
</ul>
<h2 id="yay">yay<a class="headerlink" href="#yay" title="Permanent link">¶</a></h2>
<p>This is the first time I mention the <strong>AUR</strong> (and <code>yay</code>) in my entries, so I might as well just write a bit about it.</p>
<p>The <a href="https://aur.archlinux.org/">AUR</a> is the <strong>A</strong>rch Linux <strong>U</strong>ser <strong>R</strong>epository and it’s basically like an extension of the official one which is supported by the community, the only thing is that it requires a different package manager. The one I use (and I think everyone does, too) is <code>yay</code>, which as far as I know is like a wrapper of <code>pacman</code>.</p>
<h3 id="install">Install<a class="headerlink" href="#install" title="Permanent link">¶</a></h3>
<p>To install and use <code>yay</code> we need a normal account with sudo access, <mark>all the commands related to <code>yay</code> are run as normal user and then it asks for sudo password</mark>. <a href="https://github.com/Jguer/yay#installation">Installation</a> its straight forward: clone <code>yay</code> repo and install. Only dependencies are <code>git</code> and <code>base-devel</code>:</p>
<p>Install dependencies:</p>
<pre><code class="language-sh">sudo pacman -S git base-devel
</code></pre>
<p>Clone <code>yay</code> and install it (I also like to delete the cloned git repo):</p>
<pre><code class="language-sh">git clone git@github.com:Jguer/yay.git
cd yay
makepkg -si
cd ..
sudo rm -r yay
</code></pre>
<h3 id="usage">Usage<a class="headerlink" href="#usage" title="Permanent link">¶</a></h3>
<p><code>yay</code> is used basically the same as <code>pacman</code> with the difference that it is run as normal user (then later requiring sudo password) and that it asks extra input when installing something, such as if we want to build the package from source or if we want to show package diffs.</p>
<p>To install a package (for example Komga in this blog entry), run:</p>
<pre><code class="language-sh">yay -S komga
</code></pre>
<h2 id="mangal">mangal<a class="headerlink" href="#mangal" title="Permanent link">¶</a></h2>
<p><a href="https://github.com/metafates/mangal">mangal</a> is a CLI/TUI manga downloader with anilist integration and custom Lua scrapers.</p>
<p>You could install it from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S mangal-bin
</code></pre>
<p>But I’ll use my <a href="https://github.com/luevano/mangal">fork</a> as it contains some fixes and extra stuff.</p>
<h3 id="install-from-source">Install from source<a class="headerlink" href="#install-from-source" title="Permanent link">¶</a></h3>
<p>As I mentioned in my past <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a> I had to <a href="https://github.com/luevano/mangal">fork</a> <code>mangal</code> and related repositories to fix/change a few things. Currently the major fix I did in <code>mangal</code> is for the built in <a href="https://mangadex.org/">MangaDex</a> scraper which had really annoying bug in the chunking of the manga chapter listing.</p>
<p>So instad of installing with <code>yay</code> we’ll build it from source. We need to have <code>go</code> installed:</p>
<pre><code class="language-sh">pacman -S go
</code></pre>
<p>Then clone my fork of <code>mangal</code> and <code>install</code> it:</p>
<pre><code class="language-sh">git clone https://github.com/luevano/mangal.git # not sure if you can use SSH to clone
cd mangal
make install # or just `make build` and then move the binary to somewhere in your $PATH
</code></pre>
<p>This will use <code>go install</code> so it will install to a path specified by the <code>go</code> environment variables, for more run <code>go help install</code>. It was installed to <code>$HOME/.local/bin/go/mangal</code> for me because my env vars, then just make sure this is included in your <code>PATH</code>.</p>
<p>Check it was correctly installed by running <code>mangal version</code>, which should print something like:</p>
<pre><code>▇▇▇ mangal
Version ...
Git Commit ...
Build Date ...
Built By ...
Platform ...
</code></pre>
<h3 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h3>
<p>I’m going to do everything with a normal user (<code>manga-dl</code>) which I created just to download manga. So all of the commands will be run without sudo/root privileges.</p>
<p>Change some of the configuration options:</p>
<pre><code class="language-sh">mangal config set -k downloader.path -v "/mnt/d/mangal" # downloads to current dir by default
mangal config set -k formats.use -v "cbz" # downloads as pdf by default
mangal config set -k installer.user -v "luevano" # points to my scrapers repository which contains a few extra scrapers and fixes, defaults to metafates' one; this is important if you're using my fork, don't use otherwise as it uses extra stuff I added
mangal config set -k logs.write -v true # I like to get logs for what happens
</code></pre>
<p><strong>Note</strong>: For testing purposes (if you want to explore <code>mangal</code>) set <code>downloader.path</code> once you’re ready to start to populate the Komga library directory (at <a href="#populate-manga-library">Komga: populate manga library</a>).</p>
<p>For more configs and to read what they’re for:</p>
<pre><code class="language-sh">mangal config info
</code></pre>
<p>Also install the custom Lua scrapers by running:</p>
<pre><code class="language-sh">mangal sources install
</code></pre>
<p>And install whatever you want, it picks up the sources/scrapers from the configured repository (<code>installer.<key></code> config), if you followed, it will show my scrapers.</p>
<h3 id="usage_1">Usage<a class="headerlink" href="#usage_1" title="Permanent link">¶</a></h3>
<p>Two main ways of using <code>mangal</code>: </p>
<ul>
<li><strong>TUI</strong>: for initial browsing/downloading and testing things out. If the manga finished publishing, this should be enough.</li>
<li><strong>inline</strong>: for automation on manga that is still publishing and I need to check/download every once in a while.</li>
</ul>
<h4 id="headless-browser">Headless browser<a class="headerlink" href="#headless-browser" title="Permanent link">¶</a></h4>
<p>Before continuing, I gotta say I went through some bullshit while trying to use the custom Lua scrapers that use the <em>headless</em> browser (actually just a wrapper of <a href="https://github.com/go-rod/rod">go-rod/rod</a>, and honestly it is not really a “headless” browser, <code>mangal</code> “documentation” is just wrong). For more on my rant check out my last <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a>.</p>
<p>There is no concrete documentation on the “headless” browser, only that it is automatically set up and ready to use… but it doesn’t install any library/dependency needed. I discovered the following libraries that were missing on my Arch minimal install:</p>
<ul>
<li>library -> arch package containing it</li>
<li>libnss3.so -> nss</li>
<li>libatk-1.0.so.0 -> at-spi2-core</li>
<li>libcups.so.2 -> libcups</li>
<li>libdrm.so.2 -> libdrm</li>
<li>libXcomposite.so.1 -> libxcomposite</li>
<li>libXdamage.so.1 -> libxdamage</li>
<li>libXrandr.so.2 -> libxrandr</li>
<li>libgbm.so.1 -> mesa</li>
<li>libxkbcommon.so.0 -> libxkbcommon</li>
<li>libpango-1.0.so.0 -> pango</li>
<li>libasound.so.2 -> alsa-lib</li>
</ul>
<p>To install them::</p>
<pre><code class="language-sh">pacman -S nss at-spi2-core libcups libdrm libxcomposite libxdamage libxrandr mesa libxkbcommon pango alsa-lib
</code></pre>
<p>I can’t guarantee that those are all the packages needed, those are the ones I happen to discover (had to <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the lua libs and add some logging because the error message was too fucking generic).</p>
<p>These dependencies are probably met by installing either <code>chromedriver</code> or <code>google-chrome</code> from the AUR (for what I could see on the package dependencies).</p>
<h4 id="tui">TUI<a class="headerlink" href="#tui" title="Permanent link">¶</a></h4>
<p>Use the TUI by running</p>
<pre><code class="language-sh">mangal
</code></pre>
<p>Download manga using the TUI by selecting the source/scrapper, search the manga/comic you want and then you can select each chapter to download (use <code>tab</code> to select all). This is what I use when downloading manga that already finished publishing, or when I’m just searching and testing out how it downloads the manga (directory name, and manga information).</p>
<p>Note that some scrapters will contain duplicated chapters, as they have multiple uploaded chapters from the community, usually for different <em>scanlation groups</em>. This happens a lot with <a href="https://mangadex.org/">MangaDex</a>.</p>
<h4 id="inline">Inline<a class="headerlink" href="#inline" title="Permanent link">¶</a></h4>
<p>The inline mode is a single terminal command meant to be used to automate stuff or for more advanced options. You can peek a bit into the “<a href="https://github.com/metafates/mangal/wiki/Inline-mode#command-examples">documentation</a>” which honestly it’s ass because it doesn’t explain much. The minimal command for inline according to the <code>mangal help</code> is:</p>
<pre><code class="language-sh">mangal inline --manga <option> --query <manga-title>
</code></pre>
<p>But this will not produce anything because it also needs <code>--source</code> (or set the default using the config key <code>downloader.default_sources</code>) and either <code>--json</code> which basically just does the search and returns the result in <code>json</code> format or <code>--download</code> to actually download whatever is found; I recommend to do <code>--json</code> first to check that the correct manga will be downloaded then do <code>--download</code>.</p>
<p>Something not mentioned anywhere is the <code>--manga</code> flag options (found it at the source code), it has 3 available options:</p>
<ul>
<li><code>first</code>: first manga entry found for the search.</li>
<li><code>last</code>: last manga entry found for the search.</li>
<li><code>exact</code>: exact manga title match. This is the one I use.</li>
</ul>
<p>Similar to <code>--chapters</code>, there are a few options not explained (that I found at the source code, too). I usually just use <code>all</code> but other options:</p>
<ul>
<li><code>all</code>: all chapters found in the chapter list.</li>
<li><code>first</code>: first chapter found in the chapter list.</li>
<li><code>last</code>: last chapter found in the chapter list</li>
<li><code>[from]-[to]</code>: selector for the chapters found in the chapter list, index starts at 0.<ul>
<li>If the selectors (<code>from</code> or <code>to</code>) exceed the amount of chapters in the chapterlist it just adjusts to he maximum available.</li>
<li>I had to fix this at the source code because if you wanted <code>to</code> to be the last chapter, it did <code>to + 1</code> and it failed due to index out of range.</li>
</ul>
</li>
<li><code>@[sub]@</code>: not sure how this works exactly, my understanding is that it’s for “named” chapters.</li>
</ul>
<p>That said, I’ll do an example by using <a href="https://mangapill.com">Mangapill</a> as source, and will search for <a href="https://mangapill.com/manga/2285/kimetsu-no-yaiba">Demon Slayer: Kimetsu no Yaiba</a>:</p>
<ol>
<li>Search first and make sure my command will pull the manga I want:</li>
</ol>
<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --json | jq # I use jq to pretty format the output
</code></pre>
<ol start="2">
<li>I make sure the json output contains the correct manga information: name, url, etc..</li>
</ol>
<ul>
<li>You can also include the flag <code>--include-anilist-manga</code> to include anilist information (if any) so you can check that the correct anilist id is attached. If the correct one is not attached (and it exists) then you can bind the <code>--query</code> (search term) to a specific anilist id by running:</li>
</ul>
<pre><code class="language-sh">mangal inline anilist set --name "Kimetsu no Yaiba" --id 101922
</code></pre>
<ol start="3">
<li>If I’m okay with the outputs, then I change <code>--json</code> for <code>--download</code> to actually download:</li>
</ol>
<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --download
</code></pre>
<ol start="4">
<li>Check if the manga is downloaded correctly. I do this by going to my download directory and checking the directory name (I’m picky with this stuff), that all chapters where downloaded, that it includes a correct <code>series.json</code> file and it contains a <code>cover.<img-ext></code>; this usually means it correctly pulled information from anilist and that it will contain metadata Komga will be able to use.</li>
</ol>
<h4 id="automation">Automation<a class="headerlink" href="#automation" title="Permanent link">¶</a></h4>
<p>The straight forward approach for automation is just to bundle a bunch of <code>mangal inline</code> commands in a shell script and schedule it’s execution either via <a href="https://wiki.archlinux.org/title/cron">cron</a> or <a href="https://wiki.archlinux.org/title/systemd/Timers">systemd/Timers</a>. But, as always, I overcomplicated/overengineered my approach, which is the following:</p>
<ol>
<li>Group manga names per source.</li>
<li>Configure anything that should always be set before executing <code>mangal</code>, this includes anilist bindings.</li>
<li>Have a way to track the changes/updates on each run.</li>
<li>Use that tracker to know where to start downloading chapters from.<ul>
<li>This is optional, as you can just do <code>--chapters "all"</code> and it will work but I do it mostly to keep the logs/output cleaner/shorter.</li>
</ul>
</li>
<li>Download/update each manga using <code>mangal inline</code>.</li>
<li>Wrap everything in a <code>systemd</code> <code>service</code> and <code>timer</code>.</li>
</ol>
<p>Manga list example:</p>
<pre><code class="language-sh">mangapill="Berserk|Chainsaw Man|Dandadan|Jujutsu Kaisen|etc..."
</code></pre>
<p>Function that handles the download per manga in the list:</p>
<pre><code class="language-sh">mangal_src_dl () {
source_name=$1
manga_list=$(echo "$2" | tr '|' '\n')
while IFS= read -r line; do
# By default download all chapters
chapters="all"
last_chapter_n=$(grep -e "$line" "$TRACKER_FILE" | cut -d'|' -f2 | grep -v -e '^$' | tail -n 1)
if [ -n "${last_chapter_n}" ]; then
chapters="$last_chapter_n-9999"
echo "Downloading [${last_chapter_n}-] chapters for $line from $source_name..."
else
echo "Downloading all chapters for $line from $source_name..."
fi
dl_output=$(mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "$chapters" -d)
if [ $? -ne 0 ]; then
echo "Failed to download chapters for $line."
continue
fi
line_count=$(echo "$dl_output" | grep -v -e '^$' | wc -l)
if [ $line_count -gt 0 ]; then
echo "Downloaded $line_count chapters for $line:"
echo "$dl_output"
new_last_chapter_n=$(echo "$dl_output" | tail -n 1 | cut -d'[' -f2 | cut -d']' -f1)
# manga_name|last_chapter_number|downloaded_chapters_on_this_update|manga_source
echo "$line|$new_last_chapter_n|$line_count|$source_name" >> $TRACKER_FILE
else
echo "No new chapters for $line."
fi
done <<< "$manga_list"
}
</code></pre>
<p>Where <code>$TRACKER_FILE</code> is just a variable holding a path to some file where you can store the tracking and <code>$DOWNLOAD_FORMAT</code> the format for the mangas, for me it’s <code>cbz</code>. Then the usage would be something like <code>mangal_src_dl "Mangapill" "$mangapill"</code>, meaning that it is a function call per source.</p>
<p>A simpler function without “tracking” would be:</p>
<pre><code class="language-sh">mangal_src_dl () {
source_name=$1
manga_list=$(echo "$2" | tr '|' '\n')
while IFS= read -r line; do
echo "Downloading all chapters for $line from $source_name..."
mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "all" -d
if [ $? -ne 0 ]; then
echo "Failed to download chapters for $line."
continue
fi
echo "Finished downloading chapters for $line."
done <<< "$manga_list"
}
</code></pre>
<p>The tracker file would have a format like follows:</p>
<pre><code># Updated: 06/10/23 10:53:15 AM CST
Berserk|0392|392|Mangapill
Dandadan|0110|110|Mangapill
...
</code></pre>
<p>And note that if you already had manga downloaded and you run the script for the first time, then it will show as if it downloaded everything from the first chapter, but that’s just how <code>mangal</code> works, it will actually just discover downloaded chapters and only download anything missing.</p>
<p>Any configuration the downloader/updater might need needs to be done before the <code>mangal_src_dl</code> calls. I like to configure mangal for download path, format, etc.. I found that it is needed to clear the <code>mangal</code> and <code>rod</code> browser cache (headless browser used in some custom sources) from personal experience and from others: <a href="https://github.com/metafates/mangal/issues/170">mangal#170</a> and <a href="https://github.com/oae/kaizoku/issues/89">kaizoku#89</a>.</p>
<p>Also you should set any anilist binding necessary for the downloading (as the cache was cleared). An example of an anilist binding I had to do is for Mushoku Tensei, as it has both a <a href="https://anilist.co/manga/85470/Mushoku-Tensei-Jobless-Reincarnation/">light novel</a> and <a href="https://anilist.co/manga/85564/Mushoku-Tensei-Jobless-Reincarnation/">manga</a> version, which for me it’s the following binding:</p>
<pre><code class="language-sh">mangal inline anilist set --name "Mushoku Tensei - Isekai Ittara Honki Dasu" --id 85564
</code></pre>
<p>Finally is just a matter of using your prefered way of scheduling, I’ll use <code>systemd/Timers</code> but anything is fine. You could make the downloader script more sophisticated and only running every week on which each manga gets (usually) released but that’s too much work; I’ll just run it once daily probably.</p>
<p>A feature I want to add and probably will is sending notifications (probably through email) on a summary for manga downloaded or failed to download so I’m on top of the updates. For now this is good enough and it’s been working so far.</p>
<h2 id="komga">Komga<a class="headerlink" href="#komga" title="Permanent link">¶</a></h2>
<p><a href="https://komga.org/">Komga</a> is a comics/mangas media server.</p>
<p>Install from the AUR:</p>
<pre><code class="language-sh">yay -S komga
</code></pre>
<p>This <code>komga</code> package creates a <code>komga</code> (service) user and group which is tied to the also included <code>komga.service</code>.</p>
<p>Configure it by editing <code>/etc/komga.conf</code>:</p>
<pre><code class="language-sh">SERVER_PORT=8989
SERVER_SERVLET_CONTEXT_PATH=/ # this depends a lot of how it's going to be served (domain, subdomain, ip, etc)
KOMGA_LIBRARIES_SCAN_CRON="0 0 * * * ?"
KOMGA_LIBRARIES_SCAN_STARTUP=false
KOMGA_LIBRARIES_SCAN_DIRECTORY_EXCLUSIONS='#recycle,@eaDir,@Recycle'
KOMGA_FILESYSTEM_SCANNER_FORCE_DIRECTORY_MODIFIED_TIME=false
KOMGA_REMEMBERME_KEY=USE-WHATEVER-YOU-WANT-HERE
KOMGA_REMEMBERME_VALIDITY=2419200
KOMGA_DATABASE_BACKUP_ENABLED=true
KOMGA_DATABASE_BACKUP_STARTUP=true
KOMGA_DATABASE_BACKUP_SCHEDULE="0 0 */8 * * ?"
</code></pre>
<p>My changes (shown above):</p>
<ul>
<li>Port on <code>8989</code> because <code>8080</code> its too generic.</li>
<li><code>cron</code> schedules.<ul>
<li>It’s not actually <code>cron</code> but rather a <code>cron</code>-like syntax used by <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronSequenceGenerator.html">Spring</a> as stated in the <a href="https://komga.org/installation/configuration.html#optional-configuration">Komga config</a>.</li>
</ul>
</li>
<li>Added the remember me key.</li>
<li>For more check out <a href="https://komga.org/installation/configuration.html">Komga: Configuration options</a>.</li>
</ul>
<p>If you’re going to run it locally (or LAN/VPN) you can start the <code>komga.service</code> and access it via IP at <code>http://<your-server-ip>:<port>(/base_url)</code> as stated at <a href="https://komga.org/installation/webui.html">Komga: Accessing the web interface</a>, then you can continue with the <a href="#mangal">mangal</a> section, else continue with the next steps for the reverse proxy and certificate.</p>
<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3>
<p>Create the reverse proxy configuration (this is for <code>nginx</code>). In my case I’ll use a subdomain, so this is a new config called <code>komga.conf</code> at the usual <code>sites-available/enabled</code> path:</p>
<pre><code class="language-nginx">server {
listen 80;
server_name komga.yourdomain.com; # change accordingly to your wanted subdomain and domain name
location / {
proxy_pass http://localhost:8989; # change 8989 to the port you want to use
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
}
</code></pre>
<p>If it’s going to be used as a subdir on another domain then just change the <code>location</code> with <code>/subdir</code> instead of <code>/</code>; be careful with the <code>proxy_pass</code> directive, it has to match what you configured at <code>/etc/komga.conf</code> for the <code>SERVER_SERVLET_CONTEXT_PATH</code> regardless of the <code>/subdir</code> you selected at <code>location</code>.</p>
<h3 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h3>
<p>If using a subdir then the same certificate for the subdomain/domain should work fine and no extra stuff is needed, else if following along me then we can create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>That will automatically detect the new subdomain config and create/extend your existing certificate(s). In my case I manage each certificate’s subdomain:</p>
<pre><code class="language-sh">certbot --nginx -d domainname.com -d subdomain.domainname.com -d komga.domainname.com
</code></pre>
<h3 id="starting-using-komga">Starting using Komga<a class="headerlink" href="#starting-using-komga" title="Permanent link">¶</a></h3>
<p>We can now <code>start</code>/<code>enable</code> the <code>komga.service</code>:</p>
<pre><code class="language-sh">systemctl enable komga.service
systemctl start komga.service
</code></pre>
<p>And access the web interface at <code>https://komga.domainname.com</code> which should show the login page for Komga. The first time it will ask to create an account as shown in <a href="https://komga.org/installation/webui.html#create-user-account">Komga: Create user account</a>, this will be an admin account. Fill in the email and password (can be changed later). The email doesn’t have to be an actual email, for now it’s just for management purposes.</p>
<p>Next thing would be to add any extra account (for read-only/download manga permissions), add/import libraries, etc.. For now I’ll leave it here until we start downloading manga on the next steps.</p>
<h3 id="library-creation">Library creation<a class="headerlink" href="#library-creation" title="Permanent link">¶</a></h3>
<p>Creating a library is as simple as creating a directory somewhere and point to it in Komga. The following examples are for my use case, change accordingly. I’ll be using <code>/mnt/d/mangal</code> for my library (as stated in the <a href="#configuration">mangal: configuration</a> section):</p>
<pre><code class="language-sh">mkdir /mnt/d/mangal
</code></pre>
<p>Where I chose the name <code>mangal</code> as its the name of the downloader/scrapper, it could be anything, this is just how I like to organize stuff.</p>
<p>For the most part, the permissions don’t matter much (as long as it’s readable by the <code>komga</code> user) unless you want to delete some manga, then <code>komga</code> user also needs write permissions.</p>
<p>Then just create the library in Komga web interface (the <code>+</code> sign next to <em>Libraries</em>), choose a name <em>“Mangal”</em> and point to the root folder <code>/mnt/d/mangal</code>, then just click <em>Next</em>, <em>Next</em> and <em>Add</em> for the defaults (that’s how I’ve been using it so far). This is well explained at <a href="https://komga.org/guides/libraries.html">Komga: Libraries</a>.</p>
<p>The real important part (for me) is the permissions of the <code>/mnt/d/mangal</code> directory, as I want to have write access for <code>komga</code> so I can manage from the web interface itself. It looks like it’s just a matter of giving ownership to the <code>komga</code> user either for owner or for group (or to all for that matter), but since I’m going to use a separate user to download manga then I need to choose carefully.</p>
<h4 id="set-default-directory-permissions">Set default directory permissions<a class="headerlink" href="#set-default-directory-permissions" title="Permanent link">¶</a></h4>
<p>The desired behaviour is: set <code>komga</code> as group ownership, set write access to group and whenever a new directory/file is created, inherit these permission settings. I found out via <a href="https://unix.stackexchange.com/a/1315">this</a> stack exchange answer how to do it. So, for me:</p>
<pre><code class="language-sh">chown manga-dl:komga /mnt/d/mangal # required for group ownership for komga
chmod g+s /mnt/d/mangal # required for group permission inheritance
setfacl -d -m g::rwx /mnt/d/mangal # default permissions for group
setfacl -d -m o::rx /mnt/d/mangal # default permissions for other (as normal, I think this command can be excluded)
</code></pre>
<p>Where <code>manga-dl</code> is the user I created to download manga with. Optionally add <code>-R</code> flag to those 4 commands in case it already has subdirectories/files (this might mess file permissions, but it’s not an issue as far as I konw).</p>
<p>Checking that the permissions are set correctly (<code>getfacl /mnt/d/mangal</code>):</p>
<pre><code>getfacl: Removing leading '/' from absolute path names
# file: mnt/d/mangal
# owner: manga-dl
# group: komga
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x
</code></pre>
<p>You can then check by creating a new subdirectory (in <code>/mnt/d/mangal</code>) and it should have the same group permissions.</p>
<h4 id="populate-manga-library">Populate manga library<a class="headerlink" href="#populate-manga-library" title="Permanent link">¶</a></h4>
<p>You can now start downloading manga using <code>mangal</code> either manually or by running the <code>cron</code>/<code>systemd/Timers</code> and it will be detected by Komga automatically when it scans the library (once every hour according to my config). You can manually scan the library, though, by clicking on the 3 dots to the right of the library name (in Komga) and click on “Scan library files”.</p>
<p>Then you can check that the metadata is correct (once the manga is fully indexed and metadata finished building), such as title, summary, chapter count, language, tags, genre, etc., which honestly it never works fine as <code>mangal</code> creates the <code>series.json</code> with the <code>comicId</code> field with an upper case <code>I</code> and Komga expects it to be a lower case <code>i</code> (<code>comicid</code>) so it falls back to using the info from the first chapter. I’ll probably will fix this on <code>mangal</code> side, and see how it goes.</p>
<p>So, what I do is manually edit the metadata for the manga, by changing whatever it’s wrong or add what’s missing (I like adding anilist and MyAnimeList links) and then leave it as is. This is up to you.</p>
<h2 id="alternative-downloaders">Alternative downloaders<a class="headerlink" href="#alternative-downloaders" title="Permanent link">¶</a></h2>
<p>Just for the record, here is a list of downloaders/scrapers I considered before starting to use <code>mangal</code>:</p>
<ul>
<li><a href="https://github.com/oae/kaizoku">kaizoku</a>: NodeJS web server that uses <code>mangal</code> for its “backend” and honestly since I liked <code>mangal</code> so much I should use it, the only reason I don’t is because I’m a bitch and I don’t want to use a D*ck*r image and NodeJS (ew) (in general is pretty bloated in my opinion). If I get tired of my solution with pure <code>mangal</code> I might as well just migrate to it as It’s a more automatic solution.</li>
<li><a href="https://github.com/manga-py/manga-py">manga-py</a>: Python CLI application that’s a really good option as far as I’ve explored it, I’m just not using it yet as <code>mangal</code> has been really smooth and has everything I need, but will definitely explore it in the future if I need to. The cool thing out of the box is the amount of sources it can scrape from (somethign lacking from <code>mangal</code>).</li>
<li><a href="https://github.com/mylar3/mylar3">mylar3</a>: Python web server that should be the easier way to download manga with once correctly set up, but I guess I’m too dumb and don’t know how to configure it. Looks like you need to have access to specific private torrent trackers or whatever the other ways to download are, I just couldn’t figure out how to set it up and for public torrent stuff everything will be all over the place, so this was no option for me at the end.</li>
</ul>
<p>Others:</p>
<ul>
<li><a href="https://hakuneko.download/">HakuNeku</a>: It looks pretty easy to use and future rich, only thing is that it’s not designed for headless servers, just a normal app. So this is also not an option for me. You could use it on your computer and <code>rsync</code> to your server or use some other means to upload to your server (a <em>nono</em> for me).</li>
<li><a href="https://github.com/riderkick/FMD">FMD</a>: No fucking idea on how to use it and it’s not been updated since 2019, just listing it here as an option if it interests you.</li>
</ul>
<div class="page-nav">
<span class="next">
<a href="https://blog.luevano.xyz/a/arch_logs_flooding_disk.html" alt="Next">
<i class="fas fa-arrow-left" alt="Arrow left"></i>
<span>Next</span>
</a>
</span>
<span class="index">
<a href="https://blog.luevano.xyz" alt="Index">
<i class="fas fa-home" alt="Home"></i>
<span>Index</span>
</a>
</span>
<span class="previous">
<a href="https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html" alt="Previous">
<i class="fas fa-arrow-right" alt="Arrow right"></i>
<span>Previous</span>
</a>
</span>
</div>
<hr>
<div class="article-info">
<p>By David Luévano</p>
<p>Created: Sat, Jun 10, 2023 @ 19:36 UTC</p>
<p>Modified: Tue, Jun 13, 2023 @ 10:48 UTC</p>
<div class="article-tags">
<p>Tags:
<a href="https://blog.luevano.xyz/tag/@code.html">code</a>, <a href="https://blog.luevano.xyz/tag/@english.html">english</a>, <a href="https://blog.luevano.xyz/tag/@server.html">server</a>, <a href="https://blog.luevano.xyz/tag/@tools.html">tools</a>, <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> </p>
</div>
</div>
</main>
<footer>
<span>
<i class="fas fa-address-card" alt="Contact"></i>
<a href="https://blog.luevano.xyz/contact.html">Contact</a>
</span>
<span>
<i class="fas fa-donate" alt="Donate"></i>
<a href="https://blog.luevano.xyz/donate.html">Donate</a>
</span>
<span>
<i class="fas fa-rss" alt="RSS"></i>
<a target="_blank" href="https://blog.luevano.xyz/rss.xml">RSS</a>
</span>
<br>
<span class="created-with">
<i class="fas fa-hammer" alt="Hammer"></i>
Created with <a href="https://github.com/luevano/pyssg">pyssg</a>
</span>
<br>
<span class="copyright">
Copyright <i class="far fa-copyright" alt="Copyright"></i> 2023 David Luévano Alvarado
</span>
</footer>
</body>
</html>
|
