1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
<!DOCTYPE html>
<html class="theme-dark" lang="en
"
prefix="og: https://ogp.me/ns#">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" href="https://static.luevano.xyz/images/icons/favicon.ico">
<title>I had to learn Go and Lua the hard way -- Luévano's Blog</title>
<meta name="description" content="Thanks to the issues of a program (mangal) I'm starting to use for my manga media server, I had to learn Go and Lua the hard way so that I can fix it and use it."/>
<link rel="alternate" type="application/rss+xml" href="https://blog.luevano.xyz/rss.xml" title="Luévano's Blog RSS">
<!-- general style -->
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/style.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/fork-awesome/css/fork-awesome.min.css">
<link rel="stylesheet" type="text/css" href="https://static.luevano.xyz/font-awesome/css/all.min.css">
<!-- theme related -->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/theme.js"></script>
<link id="theme-css" rel="stylesheet" type="text/css" href="https://static.luevano.xyz/css/theme.css">
<!-- misc functions-->
<script type="text/javascript" src="https://static.luevano.xyz/scripts/return_top.js"></script>
<!-- extra -->
<!-- og meta -->
<meta property="og:title" content="I had to learn Go and Lua the hard way -- Luévano's Blog"/>
<meta property="og:type" content="article"/>
<meta property="og:url" content="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.md"/>
<meta property="og:image" content="https://static.luevano.xyz/images/b/default.png"/>
<meta property="og:description" content="Thanks to the issues of a program (mangal) I'm starting to use for my manga media server, I had to learn Go and Lua the hard way so that I can fix it and use it."/>
<meta property="og:locale" content="en"/>
<meta property="og:site_name" content="Luévano's Blog"/>
</head>
<body>
<header>
<nav>
<ul>
<li>
<a href="https://luevano.xyz/"><i class="fas fa-home" alt="Home"></i><span>Home</span></a>
</li>
<li>
<a href="https://blog.luevano.xyz/"><i class="fas fa-book-open" alt="Blog"></i><span>Blog</span></a>
</li>
<li>
<a href="https://art.luevano.xyz/"><i class="fas fa-paint-brush" alt="Art"></i><span>Art</span></a>
</li>
<li><i class="fab fa-git" alt="Git"></i><span>Git</span>
<ul>
<li><a href="https://git.luevano.xyz/" target="_blank"><i class="fab fa-git-alt" alt="Git-alt"></i></a></li>
<li><a href="https://github.com/luevano" target="_blank"><i class="fab fa-github" alt="Github"></i></a></li>
<li><a href="https://gitlab.com/dluevano" target="_blank"><i class="fab fa-gitlab" alt="Gitlab"></i></a></li>
</ul>
</li>
<li><i class="fas fa-box-open" alt="Stuff"></i><span>Stuff</span>
<ul>
<li><a href="https://gb.luevano.xyz/"><i class="fas fa-gamepad" alt="Gameboy"></i><span>Gameboy</span></a></li>
</ul>
</li>
</ul>
</nav>
<button class="theme-switcher" onclick="toggleTheme()"><i class="fas fa-moon"></i><i class="fas fa-sun"></i></button>
</header>
<main>
<div class="return-top">
<button class="return-top" onclick="returnTop()" id="returnTopButton">
<i class="fas fa-arrow-up" alt="Return to top"></i>
</button>
</div>
<h1>I had to learn Go and Lua the hard way</h1>
<p><mark>TL;DR</mark>: I learned Go and Lua the hard way by forking (for fixing):</p>
<ul>
<li><a href="https://github.com/luevano/mangal">mangal</a>: main manga scraper written in Go.</li>
<li><a href="https://github.com/luevano/mangal-lua-libs">mangal-lua-libs</a>: <a href="https://github.com/yuin/gopher-lua">gopher-lua</a> libraries for mangal.</li>
<li><a href="https://github.com/luevano/mangal-scrapers">mangal-scrapers</a>: custom mangal scrapers written in Lua.</li>
</ul>
<p>In the last couple of days I’ve been setting up a <a href="https://komga.org/">Komga</a> server for manga downloaded using <a href="https://github.com/metafates/mangal">metafates/mangal</a> (upcoming set up entry about it) and everything was fine so far until I tried to download One Piece from <a href="https://mangadex.org/">MangaDex</a> of which <code>mangal</code> has a built-in scraper. Long story short the issue was that MangaDex’s API only allows requesting manga chapters on chunks of 500 and the way that was being handled was completely wrong, specifics can be found on my <a href="https://github.com/luevano/mangal/commit/395bce96e439ee828d0180328a5cf9204bfd818a">commit</a> (and the subsequent minor fix <a href="https://github.com/luevano/mangal/commit/6bf709fe9b333ec9d4375ed80f9b055d07a40c1c">commit</a>).</p>
<p>I tried to do a PR, but the project hasn’t been active since Feb 2023 (same reason I didn’t even try to do PRs on the other repos) so I closed it and will start working on my own <a href="https://github.com/luevano/mangal">fork</a>, probaly just merging everything <a href="https://github.com/Belphemur">Belphemur</a>‘s <a href="https://github.com/Belphemur/mangal">fork</a> has to offer, as he’s been working on <code>mangal</code> actively. I could probably just fork from him and/or just submit PR requests to him, but I think I saw some changes I didn’t really like, will have to look more into it.</p>
<p>Also, while trying to use some of the custom scrapers I ran into issues with the headless chrome explorer implementation where it didn’t close on each manga chapter download, causig my CPU and Mem usage to get maxed out and losing control of the system, so I had to also <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the <a href="https://github.com/metafates/mangal-lua-libs">metafates/mangal-lua-libs</a> and “fixed” (I say fixed because that wasn’t the issue at the end, it was how the custom scrapers where using it, shitty documentation) the issue by adding the <code>browser.Close()</code> function to the <code>headless</code> Lua API (<a href="https://github.com/luevano/mangal-lua-libs/commit/97fba97ab23efe88278dfacbeed2dd83c5472de0">commit</a>) and merged some commits from the original <a href="https://github.com/vadv/gopher-lua-libs">vadv/gopher-lua-libs</a> just to include any features added to the Lua libs needed.</p>
<p>Finally I <a href="https://github.com/luevano/mangal-scrapers">forked</a> the <a href="https://github.com/metafates/mangal-scrapers">metafates/mangal-scrapers</a> (which I actually forked <a href="https://github.com/NotPhantomX">NotPhantomX</a>‘s <a href="https://github.com/NotPhantomX/mangal-scrapers">fork</a> as they had included more scrapers from some pull requests) to be able to have updated custom Lua scrapers (in which I also fixed the <code>headless</code> bullshit) and use them on my <code>mangal</code>.</p>
<p>So, I went into the rabbit hole of manga scrapping because I wanted to set up my Komga server, and more importantly I had to quickly learn Go and Lua (Lua was easier) and I have to say that Go is super convoluted on the module management, all research I did lead me to totally different responses, but it is just because of different Go versions and the year of the responses.</p>
<div class="page-nav">
<span class="next">
<a href="https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html" alt="Next">
<i class="fas fa-arrow-left" alt="Arrow left"></i>
<span>Next</span>
</a>
</span>
<span class="index">
<a href="https://blog.luevano.xyz" alt="Index">
<i class="fas fa-home" alt="Home"></i>
<span>Index</span>
</a>
</span>
<span class="previous">
<a href="https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html" alt="Previous">
<i class="fas fa-arrow-right" alt="Arrow right"></i>
<span>Previous</span>
</a>
</span>
</div>
<hr>
<div class="article-info">
<p>By David Luévano</p>
<p>Created: Sat, Jun 03, 2023 @ 03:32 UTC</p>
<div class="article-tags">
<p>Tags:
<a href="https://blog.luevano.xyz/tag/@english.html">english</a>, <a href="https://blog.luevano.xyz/tag/@rant.html">rant</a>, <a href="https://blog.luevano.xyz/tag/@short.html">short</a>, <a href="https://blog.luevano.xyz/tag/@tools.html">tools</a> </p>
</div>
</div>
</main>
<footer>
<span>
<i class="fas fa-address-card" alt="Contact"></i>
<a href="https://blog.luevano.xyz/contact.html">Contact</a>
</span>
<span>
<i class="fas fa-donate" alt="Donate"></i>
<a href="https://blog.luevano.xyz/donate.html">Donate</a>
</span>
<span>
<i class="fas fa-rss" alt="RSS"></i>
<a target="_blank" href="https://blog.luevano.xyz/rss.xml">RSS</a>
</span>
<br>
<span class="created-with">
<i class="fas fa-hammer" alt="Hammer"></i>
Created with <a href="https://github.com/luevano/pyssg">pyssg</a>
</span>
<br>
<span class="copyright">
Copyright <i class="far fa-copyright" alt="Copyright"></i> 2023 David Luévano Alvarado
</span>
</footer>
</body>
</html>
|
