I had to learn Go and Lua the hard way

TL;DR: I learned Go and Lua the hard way by forking (for fixing):

• mangal: main manga scraper written in Go.
• mangal-lua-libs: gopher-lua libraries for mangal.
• mangal-scrapers: custom mangal scrapers written in Lua.

In the last couple of days I’ve been setting up a Komga server for manga downloaded using metafates/mangal (upcoming set up entry about it) and everything was fine so far until I tried to download One Piece from MangaDex of which mangal has a built-in scraper. Long story short the issue was that MangaDex’s API only allows requesting manga chapters on chunks of 500 and the way that was being handled was completely wrong, specifics can be found on my commit (and the subsequent minor fix commit).

I tried to do a PR, but the project hasn’t been active since Feb 2023 (same reason I didn’t even try to do PRs on the other repos) so I closed it and will start working on my own fork, probaly just merging everything Belphemur‘s fork has to offer, as he’s been working on mangal actively. I could probably just fork from him and/or just submit PR requests to him, but I think I saw some changes I didn’t really like, will have to look more into it.

Also, while trying to use some of the custom scrapers I ran into issues with the headless chrome explorer implementation where it didn’t close on each manga chapter download, causig my CPU and Mem usage to get maxed out and losing control of the system, so I had to also fork the metafates/mangal-lua-libs and “fixed” (I say fixed because that wasn’t the issue at the end, it was how the custom scrapers where using it, shitty documentation) the issue by adding the browser.Close() function to the headless Lua API (commit) and merged some commits from the original vadv/gopher-lua-libs just to include any features added to the Lua libs needed.

Finally I forked the metafates/mangal-scrapers (which I actually forked NotPhantomX‘s fork as they had included more scrapers from some pull requests) to be able to have updated custom Lua scrapers (in which I also fixed the headless bullshit) and use them on my mangal.

So, I went into the rabbit hole of manga scrapping because I wanted to set up my Komga server, and more importantly I had to quickly learn Go and Lua (Lua was easier) and I have to say that Go is super convoluted on the module management, all research I did lead me to totally different responses, but it is just because of different Go versions and the year of the responses.