diff options
Diffstat (limited to 'live/blog/rss.xml')
| -rw-r--r-- | live/blog/rss.xml | 5070 |
1 files changed, 0 insertions, 5070 deletions
diff --git a/live/blog/rss.xml b/live/blog/rss.xml deleted file mode 100644 index 775006f..0000000 --- a/live/blog/rss.xml +++ /dev/null @@ -1,5070 +0,0 @@ -<?xml version="1.0" encoding="UTF-8" ?> -<rss version="2.0" - xmlns:atom="http://www.w3.org/2005/Atom" - xmlns:content="http://purl.org/rss/1.0/modules/content/"> - <channel> - <title>Luévano's Blog</title> - <link>https://blog.luevano.xyz</link> - <atom:link href="https://blog.luevano.xyz/rss.xml" rel="self" type="application/rss+xml"/> - <description>My personal blog where I post about my thoughts, some how-to's, or general ranting.</description> - <language>en-us</language> - <category></category> - <copyright>Copyright 2023 David Luévano Alvarado</copyright> - <managingEditor>david@luevano.xyz (David Luévano Alvarado)</managingEditor> - <webMaster>david@luevano.xyz (David Luévano Alvarado)</webMaster> - <pubDate></pubDate> - <lastBuildDate></lastBuildDate> - <generator>pyssg v0.9.0</generator> - <docs>https://validator.w3.org/feed/docs/rss2.html</docs> - <ttl>30</ttl> - <image> - <url>https://static.luevano.xyz/images/b/default.png</url> - <title>Luévano's Blog</title> - <link>https://blog.luevano.xyz</link> - </image> - <item> - <title>Final improvements to the FlappyBird clone and Android support devlog 3</title> - <link>https://blog.luevano.xyz/g/flappybird_godot_devlog_3.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/flappybird_godot_devlog_3.html</guid> - <pubDate>Fri, 01 Mar 2024 10:00:57 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Gdscript</category> - <category>Godot</category> - <description>Notes on the final improvements to my FlappyBird clone made in Godot 4.x. Also details on the support for Android.</description> - <content:encoded><![CDATA[<p>Decided to conclude my FlappyBird journey with one last set of improvements, following up on devlogs <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html">1</a> and <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_2.html">2</a>. Focusing on <strong>refactoring</strong>, <strong>better UI</strong>, <strong>sprite selection</strong> and Android support.</p> -<p>I missed some features that I really wanted to get in but I’m already tired of working on this toy project and already eager to move to another one. Most of the features I wanted to add are just <em>QoL</em> UI enhancements and extra buttons basically.</p> -<p>The source code can be found at <a href="https://github.com/luevano/flappybirdgodot">luevano/flappybirdgodot</a>. Playable at <a href="https://lorentzeus.itch.io/flappybirdgodot">itch.io</a>:</p> -<p style="text-align:center"><iframe src="https://itch.io/embed/1551015?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/flappybirdgodot">FlappyBirdGodot by Lorentzeus</a></iframe></p> - -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#refactoring">Refactoring</a></li> -<li><a href="#improvements">Improvements</a><ul> -<li><a href="#background-parallax">Background parallax</a></li> -<li><a href="#sprite-switcher">Sprite switcher</a></li> -<li><a href="#save-data">Save data</a></li> -</ul> -</li> -<li><a href="#android">Android</a></li> -<li><a href="#misc">Misc</a></li> -</ul> -</div> -<h2 id="refactoring">Refactoring<a class="headerlink" href="#refactoring" title="Permanent link">¶</a></h2> -<p>The first part for my refactor was to move everything out of the <code>src/</code> directory into the root directory of the git repository, organizing it a tiny bit better, personal preference from what I’ve learned so far. I also decided to place all the raw aseprite assets next to the imported one, this way its easier to make modifications and then save directly in the same directory. Also, a list of other refactoring done:</p> -<ul> -<li>The way I handled the gameplay means that I needed to make the camera, background and the (ceiling and tiles) “detectors” move along with the player, while restricting their movement in the <code>x</code> axis, really hacky. Instead, I did what I should’ve done from the beginning… just let the tiles move backwards and keep everything static with the player only moving up an down (as how I stated at the beginning of <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html">FlappyBirdgodot devlog 1</a> but didn’t actually follow).</li> -<li>Moved the <code>set_process</code> methodology to their own scripts, instead of handling everything in <code>main.gd</code> while also taking advantage of how signals work now. Instead of doing:</li> -</ul> -<pre><code class="language-gdscript">func _ready(): - Event.game_pause.connect(_on_game_pause) - -func _on_game_pause(pause: bool): - set_process(pause) -</code></pre> -<p>Just connecting to <code>set_process</code> is enough:</p> -<pre><code class="language-gdscript">func _ready(): - Event.game_pause.connect(set_process) - # and when the signal doesn't send anything: - Event.game_start.connect(set_process.bind(true)) - Event.game_over.connect(set_process.bind(false)) -</code></pre> -<h2 id="improvements">Improvements<a class="headerlink" href="#improvements" title="Permanent link">¶</a></h2> -<h3 id="background-parallax">Background parallax<a class="headerlink" href="#background-parallax" title="Permanent link">¶</a></h3> -<p>First thing was to add a moving background functionality, by adding 2 of the same <code>Sprite2D</code>‘s one after another and everytime the first sprite moves out of the screen, position it right after the second sprite. Some sample code to accomplish this:</p> -<pre><code class="language-gdscript">func _ready(): - # Sprite2D and CompressedTexture2D nodes - background_orig.texture = background_texture - texture_size = background_orig.texture.get_size() - - backgrounds.append(background_orig.duplicate()) - backgrounds.append(background_orig.duplicate()) - backgrounds[1].position = background_orig.position + Vector2(texture_size.x, 0.0) - - add_child(backgrounds[0]) - add_child(backgrounds[1]) - background_orig.visible = false - -# ifirst (index first) it's a boolean value starting with false and -# its a hacky way of tracking the first sprites -# (the one closest to the left of the screen) in the array -func _process(delta: float): - for background in backgrounds: - background.move_local_x(- SPEED * delta) - - # moves the sprite that just exited the screen to the right of the upcoming sprite - if backgrounds[int(ifirst)].position.x <= - background_orig.position.x: - backgrounds[int(ifirst)].position.x = backgrounds[int(!ifirst)].position.x + texture_size.x - ifirst = !ifirst -</code></pre> -<p>Then I added background parallax by separating the background sprites in two: background and “foreground” (the buildings in the original sprites). And to move them separately just applied the same logic described above with 2 different speeds.</p> -<h3 id="sprite-switcher">Sprite switcher<a class="headerlink" href="#sprite-switcher" title="Permanent link">¶</a></h3> -<p>Also added a way to select between the bird sprites and the backgrounds, currently pretty primitive but functional. Accomplished this by holding textures in an exported array, then added a bit of logic to cycle between them (example for the background):</p> -<pre><code class="language-gdscript">func _get_new_sprite_index(index: int) -> int: - return clampi(index, 0, background_textures.size() - 1) - - -func _set_sprites_index(index: int) -> int: - var new_index: int = _get_new_sprite_index(index) - if new_index == itexture: - return new_index - for bg in backgrounds: - bg.texture = background_textures[new_index] - for fg in foregrounds: - fg.texture = foreground_textures[new_index] - itexture = new_index - return new_index -</code></pre> -<p>Then, in custom signals I just call <code>_set_sprites_index</code> with a <code>texture_index +/- 1</code>.</p> -<h3 id="save-data">Save data<a class="headerlink" href="#save-data" title="Permanent link">¶</a></h3> -<p>Moved from manual <code>ConfigFile</code> (which is an <code>.ini</code> file basically) to <code>Resource</code> which is easier to work with and faster to implement.</p> -<p>Accomplished by defining a new <code>data_resource.gd</code>:</p> -<pre><code class="language-gdscript">class_name DataResource -extends Resource - -@export var high_score: int -@export var volume: float -@export var mute: bool -@export var bird: int -@export var background: int - -func _init(): - high_score = 0 - volume = 0.5 - mute = false - bird = 0 - background = 0 -</code></pre> -<p>Where the <code>@export</code>s are not needed unless you need to manage the <code>.tres</code> resource files for testing in-editor.</p> -<p>Then, the <code>data.gd</code> script needs to be changed accordingly, most notably:</p> -<ul> -<li>The file extension is <code>.tres</code> instead of <code>.cfg</code>.</li> -<li>No need for “config sections”.</li> -<li>Saving changes to:</li> -</ul> -<pre><code class="language-gdscript">func save(): - var err: int = ResourceSaver.save(_data, DATA_PATH) - if err != OK: - print("[ERROR] Couldn't save data.") -</code></pre> -<ul> -<li>The loader gets simplified (mostly because the default values are set in the <code>_init()</code> function of the <code>data_resource.gd</code>) to:</li> -</ul> -<pre><code class="language-gdscript">func _load_data(): - if ResourceLoader.exists(DATA_PATH): - _data = load(DATA_PATH) - else: - _data = DataResource.new() - save() -</code></pre> -<ul> -<li>The attributes/config/saved data can be retrieved directly by the <code>data_resource.gd</code> variable name, for example: instead of <code>_data.get_value(SCORE_SECTION, "high_score")</code> it’s now simply <code>_data.high_score</code>. And similar for setting the values.</li> -</ul> -<p>Compared to the <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html#saved-data">3.x version</a> it is a lot more simple. Though I still have setters and getters for each attribute/config (I’ll se how to change this in the future).</p> -<h2 id="android">Android<a class="headerlink" href="#android" title="Permanent link">¶</a></h2> -<p>I did add android support but it’s been so long since I did it that I actually don’t remember (this entry has been sitting in a draft for months). In general I followed the official guide for <a href="https://docs.godotengine.org/en/stable/tutorials/export/exporting_for_android.html">Exporting for Android</a>, setting up Android studio and remotely debugging with my personal phone; it does take a while to setup but after that it’s as simple as doing <em>“one click deploys”</em>.</p> -<p>Most notably, I had to enable touch screen support and make the buttons clickable either by an actual mouse click or touch input. Some of the <em>Project Settings</em> that I remember that needs changes are:</p> -<ul> -<li><em>display/window/handheld/orientation</em> set to <code>Portrait</code>.</li> -<li><em>input_devices/pointing/emulate_touch_from_mouse</em> and <em>input_devices/pointing/emulate_mouse_from_touch</em> both set to <code>on</code>.</li> -</ul> -<h2 id="misc">Misc<a class="headerlink" href="#misc" title="Permanent link">¶</a></h2> -<p>Found a bug on the <code>ScoreDetector</code> where it would collide with the <code>Ceiling</code>. While this is really not a problem outside of me doing tests I fixed it by <a href="https://blog.luevano.xyz/g/godot_layers_and_masks_notes.html">applying the correct layer/mask</a>.</p>]]></content:encoded> - </item> - <item> - <title>Godot layers and masks notes</title> - <link>https://blog.luevano.xyz/g/godot_layers_and_masks_notes.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/godot_layers_and_masks_notes.html</guid> - <pubDate>Tue, 29 Aug 2023 10:10:06 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Gdscript</category> - <category>Godot</category> - <description>Some notes I took regarding Godot's confusing collision layers and masks.</description> - <content:encoded><![CDATA[<p>The first time I learned about Godot’s collision layers and masks (will refer to them just as layers) I thought I understood them only to find out that they’re a bit confusing when trying to figure out interactions between objects that are supposed to detect each other. On my last entry where I <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_2.html">ported the FlappyBird clone to <em>Godot 4.1</em></a> I stumbled upon an issue with the bird not colliding properly with the pipes and the ceiling detector not… well, detecting.</p> -<p>At the end of the day the issue wasn’t that the layers weren’t properly setup but rather that the API to change the state of the collision layers changed between <em>Godot 3</em> and <em>Godot 4</em>: when calling <code>set_collision_layer_value</code> (or <code>.._mask</code>) instead of specifying the <code>bit</code> which starts at <code>0</code>, the <code>layer_number</code> is required that happens to start at <code>1</code>. This was a headache for like an hour and made me realise that I didn’t understand layers that well or else I would’ve picked the error almost instantly.</p> -<p>While researching I found two really good short explainations that helped me grasp the concepts better in the same <a href="https://ask.godotengine.org/4010/whats-difference-between-collision-layers-collision-masks">post</a>, the first a bit technical (by <a href="https://ask.godotengine.org/user/Bojidar+Marinov">Bojidar Marinov</a>):</p> -<ul> -<li>If enemy’s mask and object’s mask are set to <code>0</code> (i.e. no layers), they will still collide with the player, because the player’s mask still includes their respective layers.</li> -<li>Overall, if the objects are <code>A</code> and <code>B</code>, the check for collision is <code>A.mask & B.layers || B.mask & A.layers</code>, where <code>&</code> is bitwise-and, and <code>||</code> is the or operator. I.e. it takes the layers that correspond to the other object’s mask, and checks if any of them is on in both places. It will then proceed to check it the other way around, and if any of the two tests passes, it would report the collision.</li> -</ul> -<p>And the second, shorter and less technical but still powerful (in the same post linking back to <a href="https://kidscancode.org/blog/2018/02/godot3_kinematic2d/">Godot 3.0: Using KinematicBody2D</a>):</p> -<ul> -<li><code>collision_layer</code> describes the layers that the object appears in. By default, all bodies are on layer <code>1</code>.</li> -<li><code>collision_mask</code> describes what layers the body will scan for collisions. If an object isn’t in one of the mask layers, the body will ignore it. By default, all bodies scan layer <code>1</code>.</li> -</ul> -<p>While the complete answer is the first, as that is how layers work, the second can be used like a rule: 1) the <code>layer</code> is where the object lives, while 2) the <code>mask</code> is what the object will detect.</p>]]></content:encoded> - </item> - <item> - <title>Porting the FlappyBird clone to Godot 4.1 devlog 2</title> - <link>https://blog.luevano.xyz/g/flappybird_godot_devlog_2.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/flappybird_godot_devlog_2.html</guid> - <pubDate>Sun, 27 Aug 2023 23:28:10 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Gdscript</category> - <category>Godot</category> - <description>Notes on porting my FlappyBird clone to Godot 4.1, as well as notes on the improvements and changes made overall.</description> - <content:encoded><![CDATA[<p>As stated in my <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html">FlappyBird devlog 1</a> entry I originally started the clone in <em>Godot 4</em>, then backported back to <em>Godot 3</em> because of HTML5 support, and now I’m porting it back again to <em>Godot 4</em> as there is support again and I want to start getting familiar with it for future projects.</p> -<p>The source code can be found at <a href="https://github.com/luevano/flappybirdgodot">luevano/flappybirdgodot</a> (<code>main</code> branch). Playable at <a href="https://lorentzeus.itch.io/flappybirdgodot">itch.io</a>:</p> -<p style="text-align:center"><iframe src="https://itch.io/embed/1551015?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/flappybirdgodot">FlappyBirdGodot by Lorentzeus</a></iframe></p> - -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#porting-to-godot-4">Porting to Godot 4</a><ul> -<li><a href="#general-changes">General changes</a></li> -<li><a href="#player">Player</a></li> -<li><a href="#world">World</a><ul> -<li><a href="#scene">Scene</a></li> -<li><a href="#script">Script</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#changes-and-improvements">Changes and improvements</a><ul> -<li><a href="#audio">Audio</a></li> -<li><a href="#event-bus">Event bus</a></li> -<li><a href="#ui">UI</a></li> -<li><a href="#misc">Misc</a></li> -</ul> -</li> -</ul> -</div> -<h2 id="porting-to-godot-4">Porting to Godot 4<a class="headerlink" href="#porting-to-godot-4" title="Permanent link">¶</a></h2> -<p><strong>Disclaimer:</strong> I started the port back in <em>Godot 4.0</em> something and left the project for a while, then opened the project again in <em>Godot 4.1</em>, and it didn’t ask to convert anything so probably nowadays the conversion is better. Godot’s documentation is pretty useful, I looked at the <a href="https://docs.godotengine.org/en/stable/tutorials/scripting/gdscript/gdscript_basics.html">GDScript reference</a> and <a href="https://docs.godotengine.org/en/stable/tutorials/scripting/gdscript/gdscript_exports.html">GDScript exports</a> and that helped a lot.</p> -<h3 id="general-changes">General changes<a class="headerlink" href="#general-changes" title="Permanent link">¶</a></h3> -<p>These include the first changes for fixing some of the conflicting code to at least make it run (no gameplay) as well as project settings adjustments.</p> -<ul> -<li>Changing the <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html#default-import-settings">default import settings</a> for pixel art no longer works (though it’s worth to double check as they changed from <code>Texture</code> to a <code>Texture2D</code>). The important parameter to <a href="https://ask.godotengine.org/122518/how-to-import-pixel-art-in-godot-4">change is the <em>Filter</em> for the textures</a>.<ul> -<li>Change by going to the <em>Project Settings</em> at <em>General -> Rendering -> Textures</em> and set the <em>Canvas Textures -> Default Texture Filter</em> to “Nearest”.</li> -</ul> -</li> -<li>Re-set the <em>InputMap</em> as the system probably changed from <em>Godot 3</em> to <em>Godot 4</em>.</li> -<li>For <code>SavedData</code> singleton, change from <code>File</code> to <code>ConfigFile</code> and refactor. This is really not needed for making it run, but I changed this right away.</li> -<li>Remove <code>velocity</code> property of <code>Player</code> which is already included in <code>CharacterBody2D</code>.</li> -<li>Disable all <code>TileMap</code> related code as tile maps have changed drastically, they need more effort to covnert.</li> -<li>Change <code>String(int)</code> to <code>str(int)</code>.</li> -</ul> -<h3 id="player">Player<a class="headerlink" href="#player" title="Permanent link">¶</a></h3> -<p>Now that the game at least runs, next thing is to make it “playable”:</p> -<ul> -<li><code>AnimatedSprite</code> changed to <code>AnimatedSprite2D</code> (with the inclusion of <code>AnimatedSprite3D</code>). This node type changed with the automatic conversion.<ul> -<li>Instead of checking if an animation is playing with the the <code>playing</code> property, the method <code>is_playing()</code> needs to be used.</li> -</ul> -</li> -<li>The <code>default_gravity</code> from the <code>ProjectSettings</code> no longer needs to be multiplied by <code>10</code> to have reasonable numbers. The default is now <code>980</code> instead of <code>98</code>. I later changed this when refactoring the code and fine-tuning the feel of the movement.</li> -<li>The <em>Collision mask</em> can be changed programatically with the <code>set_collision_mask_value</code> (and similar with the layer). Before, the mask/layer was specified by the <code>bit</code> which started from <code>0</code>, but now it is accessed by the <code>layer_number</code> which starts from <code>1</code>.</li> -</ul> -<h3 id="world">World<a class="headerlink" href="#world" title="Permanent link">¶</a></h3> -<p>This is the most challenging part as the <code>TileMap</code> system changed drastically, it is basically a <em>from the ground up redesign</em>, luckily the <code>TileMap</code>s I use are really simple. Since this is not intuitive from the get-go, I took some notes on the steps I took to set up the world <code>TileMap</code>.</p> -<h4 id="scene">Scene<a class="headerlink" href="#scene" title="Permanent link">¶</a></h4> -<p>Instead of using one scene per <code>TileMap</code> only one <code>TileMap</code> can be used with multiple <em>Atlas</em> in the <code>TileSet</code>. Multiple physics layers can now be used per <code>TileSet</code> so you can separate the physics collisions on a per <em>Atlas</em> or <em>Tile</em> basis. The inclusion of <em>Tile patterns</em> also helps when working with multiple <em>Tiles</em> for a single cell “placement”. How I did it:</p> -<ol> -<li>Created one scene with one <code>TileMap</code> node, called <code>WorldTileMap.tscn</code>, with only one <code>TileSet</code> as multiple <em>Atlas</em>‘ can be used (this would be a single <code>TileSet</code> in <em>Godot 3</em>).<ul> -<li>To add a <code>TileSet</code>, select the <code>WorldTileMap</code> and go to <em>Inspector -> TileMap -> TileSet</em> then click on “<empty>” and then “New TileSet” button.</li> -<li>To manipulate a <code>TileSet</code>, it needs to be selected, either by clicking in the <em>Inspector</em> section or on the bottom of the screen (by default) to the left of <em>TileMap</em>, as shown in the image below.</li> -</ul> -</li> -</ol> -<figure id="__yafg-figure-48"> -<img alt="TileMap's TileSet selection highlighted in red, "Add" button in green." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_tileset_selected_add_atlas.png.png "TileMap's TileSet selection highlighted in red, "Add" button in green"."> -<figcaption></figcaption> -</figure> -<ol start="2"> -<li>Add two <em>Atlas</em> to the <code>TileSet</code> (one for the ground tiles and another for the pipes) by clicking on the “Add” button (as shown in the image above) and then on “Atlas”.</li> -<li>By selecting an atlas and having the “Setup” selected, change the <em>Name</em> to something recognizable like <code>ground</code> and add the texture atlas (the spritesheet) by dragging and dropping in the “<empty>” <em>Texture</em> field, as shown in the image below. Take a not of the <em>ID</em>, they start from <code>0</code> and increment for each atlas, but if they’re not <code>0</code> and <code>1</code> change them.</li> -</ol> -<figure id="__yafg-figure-49"> -<img alt="TileSet atlas setup selection highlighted in red, atlas name and id in green." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_atlas_setup_selection.png" title="TileSet atlas setup selection highlighted in red, atlas name and id in green."> -<figcaption>TileSet atlas setup selection highlighted in red, atlas name and id in green.</figcaption> -</figure> -<ol start="4"> -<li>I also like to delete unnecessary tiles (for now) by selecting the atlas “Setup” and the “Eraser” tool, as shown in the image below. Then to erase tiles just select them and they’ll be highlighted in black, once deleted they will be grayed out. If you want to activate tiles again just deselect the “Eraser” tool and select wanted tiles.</li> -</ol> -<figure id="__yafg-figure-50"> -<img alt="Atlas setup erase tiles. "Setup" selection and "Eraser" tool highlighted in red and green, respectively." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_atlas_setup_selection.png" title="Atlas setup erase tiles. "Setup" selection and "Eraser" tool highlighted in red and green, respectively."> -<figcaption>Atlas setup erase tiles. “Setup” selection and “Eraser” tool highlighted in red and green, respectively.</figcaption> -</figure> -<ol start="5"> -<li>For the pipes it is a good idea to modify the “tile width” for horizontal <code>1x2</code> tiles. This can be acomplished by removing all tiles except for one, then going to the “Select” section of the atlas, selecting a tile and extending it either graphically by using the yellow circles or by using the properties, as shown in the image below.</li> -</ol> -<figure id="__yafg-figure-51"> -<img alt="Atlas resize tile. "Select" selection and "Size in Atlas" highlighted in red and green, respectively." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_atlas_pipe_wide.png" title="Atlas resize tile. "Select" selection and "Size in Atlas" highlighted in red and green, respectively."> -<figcaption>Atlas resize tile. “Select” selection and “Size in Atlas” highlighted in red and green, respectively.</figcaption> -</figure> -<ol start="6"> -<li>Add physics (collisions) by selecting the <code>WorldTileMap</code>‘s <code>TileSet</code> and clicking on “Add Element” at the <em>TileMap -> TileSet -> Physics Layer</em> twice, one physics layer per atlas. Then set the collision’s layers and masks accordingly (ground on layer 2, pipe on 3). In my case, based on my already set <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html#layers">layers</a>.<ul> -<li>This will enable physics properties on the tiles when selecting them (by selecting the atlas, being in the correct “Select” section and selecting a tile) and start drawing a polygon with the tools provided. This part is hard to explain in text, but below is an image of how it looks once the polygon is set.</li> -</ul> -</li> -</ol> -<figure id="__yafg-figure-52"> -<img alt="Tile add physics polygon in Physics Layer 0." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_tileset_selected_add_atlas.png" title="Tile add physics polygon on physics layer 0."> -<figcaption>Tile add physics polygon on physics layer 0.</figcaption> -</figure> -<pre><code>- Notice that the polygon is drawn in *Physics Layer 0*. Using the grid option to either `1` or `2` is useful when drawing the polygon, make sure the polygon closes itself or it wont be drawn. -</code></pre> -<ol start="8"> -<li>Create a tile pattern by drawing the tiles wanted in the editor and then going to the <em>Patterns</em> tab (to the right of <em>Tiles</em>) in the <em>TileMap</em>, selecting all tiles wanted in the pattern and dragging the tiles to the <em>Patterns</em> window. Added patterns will show in this window as shown in the image below, and assigned with IDs starting from <code>0</code>.</li> -</ol> -<figure id="__yafg-figure-53"> -<img alt="Tileset pattern." src="https://static.luevano.xyz/images/g/flappybird_godot4/godot4_tileset_pattern.png" title="Tileset pattern."> -<figcaption>Tileset pattern.</figcaption> -</figure> -<h4 id="script">Script<a class="headerlink" href="#script" title="Permanent link">¶</a></h4> -<p>Basically merged all 3 scripts (<code>ground_tile_map.gd</code>, <code>pipe_tile_map.gd</code> and <code>world_tiles.gd</code>) into one (<code>world_tile_map.gd</code>) and immediatly was able to delete a lot of signal calls between those 3 scripts and redundant code.</p> -<p>The biggest change in the scripting side are the functions to place tiles. For <em>Godot 3</em>:</p> -<pre><code class="language-gdscript"># place single tile in specific cell -void set_cell(x: int, y: int, tile: int, flip_x: bool = false, flip_y: bool = false, transpose: bool = false, autotile_coord: Vector2 = Vector2( 0, 0 )) -void set_cellv(position: Vector2, tile: int, flip_x: bool = false, flip_y: bool = false, transpose: bool = false, autotile_coord: Vector2 = Vector2( 0, 0 )) -</code></pre> -<p>Whereas in <em>Godot 4</em>:</p> -<pre><code class="language-gdscript"># place single tile in specific cell -void set_cell(layer: int, coords: Vector2i, source_id: int = -1, atlas_coords: Vector2i = Vector2i(-1, -1), alternative_tile: int = 0) -# erase tile at specific cell -void erase_cell(layer: int, coords: Vector2i) -</code></pre> -<p>How to use these functions in <em>Godot 4</em> (new properties or differences/changes):</p> -<ul> -<li><code>layer</code>: for my case I only use 1 layer so it is always set to <code>0</code>.</li> -<li><code>coords</code>: would be the equivalent to <code>position</code> for <code>set_cellv</code> in <em>Godot 3</em>.</li> -<li><code>source_id</code>: which atlas to use (ground: <code>0</code> or pipe <code>1</code>).</li> -<li><code>atlas_coords</code>: tile to use in the atlas. This would be the equivalent to <code>tile</code> in <em>Godot 3</em>.</li> -<li><code>alternative_tile</code>: for tiles that have alternatives such as mirrored or rotated tiles, not required in my case.</li> -</ul> -<p>Setting <code>source_id=-1</code>, <code>atlas_coords=Vector21(-1,-1)</code> or <code>alternative_tile=-1</code> will delete the tile at <code>coords</code>, similar to just using <code>erase_cell</code>.</p> -<p>With the addition to <em>Tile patterns</em> (to place multiple tiles), there is a new function:</p> -<pre><code class="language-gdscript"># place pattern -void set_pattern(layer: int, position: Vector2i, pattern: TileMapPattern) -</code></pre> -<p>Where <code>position</code> has the same meaning as <code>coords</code> in <code>set_cell</code>/<code>erase_cell</code>, not sure why it has a different name. The <code>pattern</code> can be obtained by using <code>get_pattern</code> method on the <code>tile_set</code> property of the <code>TileMap</code>. Something like:</p> -<pre><code class="language-gdscript">var pattern: TileMapPattern = tile_set.get_pattern(index) -</code></pre> -<p>Other than that, <code>Vector2</code> needs to be changed to <code>Vector2i</code>.</p> -<h2 id="changes-and-improvements">Changes and improvements<a class="headerlink" href="#changes-and-improvements" title="Permanent link">¶</a></h2> -<p>General changes and additions that have nothing to do with porting to <em>Godot 4</em>, things I wanted to add regardless of the version.</p> -<h3 id="audio">Audio<a class="headerlink" href="#audio" title="Permanent link">¶</a></h3> -<p>The audio in the <em>Godot 3</em> version was added in the last minute and it was blasting by default with no option to decrease the volume or mute it. To deal with this:</p> -<ol> -<li>Refactored the code into a single scene/script to have better control.</li> -<li>Added a volume control slider by following <a href="https://www.gdquest.com/tutorial/godot/audio/volume-slider/">this GDQuest guide</a>.</li> -<li>Added a mute button, following the same principle as with the volume control.</li> -</ol> -<p>The basic code required for these features is the following:</p> -<pre><code class="language-gdscript"># get audio bus index -var audio_bus_name: String = "Master" -var _bus: int = AudioServer.get_bus_index(audio_bus_name) - -# change the volume -var linear_volume: float = 0.5 # 50%, needs to be between 0.0 and 1.0 -var db_volume: float = linear_to_db(linear_volume) -AudioServer.set_bus_volume_db(_bus, db_volume) - -# mute -AudioServer.set_bus_mute(_bus, true) # false to unmute -</code></pre> -<p>Just be careful with how the <code>linear_volume</code> is set (from a button or slider) as it has to be between <code>0.0</code> and <code>1.0</code>.</p> -<h3 id="event-bus">Event bus<a class="headerlink" href="#event-bus" title="Permanent link">¶</a></h3> -<p>Moved all the signal logic into an event bus to get rid of the coupling I had. This is accomplished by:</p> -<ol> -<li>Creating a singleton (autoload) script which I called <code>event.gd</code> and can be accessed with <code>Event</code>.</li> -<li>All the signals are now defined in <code>event.gd</code>.</li> -<li>When a signal needs to be emited instead of emitting the signal from any particular script, emit it from the event bus with <code>Event.<signal_name>.emit(<optional_args>)</code>.</li> -<li>When connecting to a signal instead of taking a reference to where the signal is defined, simply connect it with with <code>Event.<signal_name>.connect(<callable>[.bind(<optional_args>)])</code><ul> -<li>For signals that already send arguments to the callable, they do not need to be specified in <code>bind</code>, only extras are needed here.</li> -</ul> -</li> -</ol> -<h3 id="ui">UI<a class="headerlink" href="#ui" title="Permanent link">¶</a></h3> -<p>Really the only UI I had before was for rendering fonts, and the way fonts work changed a bit. Before, 3 resources were needed as <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html#fonts">noted in my previous entry</a>:</p> -<ol> -<li>Font file itself (<code>.ttf</code> for example).</li> -<li><code>DynamicFontData</code>: used to point to a font file (<code>.ttf</code>) and then used as base resource.</li> -<li><code>DynamicFont</code>: usable in godot control nodes which holds the <code>DynamicFontData</code> and configuration such as size.</li> -</ol> -<p>Now only 1 resource is needed: <code>FontFile</code> which is the <code>.ttf</code> file itself or a godot-created resource. There is also a <code>FontVariation</code> option, which takes a <code>FontFile</code> and looks like its used to create fallback options for fonts. The configuration (such as size) is no longer held in the font resource, but rather in the parent control node (like a <code>Label</code>). Double clicking on the <code>.ttf</code> file and disabling antialiasing and compression is something that might be needed. Optionally create a <code>FontLabelSettings</code> which will hold the <code>.ttf</code> file and used as base for <code>Label</code>s. Use “Make Unique” for different sizes. Another option is to use <em>Themes</em> and <em>Variations</em>.</p> -<p>I also created the respective volume button and slider UI for the added audio functionality as well as creating a base <code>Label</code> to avoid repeating configuration on each <code>Label</code> node.</p> -<h3 id="misc">Misc<a class="headerlink" href="#misc" title="Permanent link">¶</a></h3> -<p>Small changes that don’t affect much:</p> -<ul> -<li>Updated <code>@export</code> to <code>@export_range</code>. The auto conversion didn’t use the correct annotation and instead used a comment.</li> -<li>Refactored the <code>game_scale</code> methodolgy as it was inconsistent. Now only one size is used as base and everything else is just scaled with the <code>root</code> <code>Window</code>.</li> -<li>Got rid of the FPS monitoring, was only using it for debugging purposes back then.</li> -</ul>]]></content:encoded> - </item> - <item> - <title>Set up a pastebin alternative with PrivateBin and YOURLS</title> - <link>https://blog.luevano.xyz/a/pastebin_alt_with_privatebin.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/pastebin_alt_with_privatebin.html</guid> - <pubDate>Sun, 20 Aug 2023 09:46:33 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a pastebin alternative with PrivateBin and YOURLS as shortener, on Arch.</description> - <content:encoded><![CDATA[<p>I learned about PrivateBin a few weeks back and ever since I’ve been looking into installing it, along with a URL shortener (a service I wanted to self host since forever). It took me a while as I ran into some problems while experimenting and documenting all the necessary bits in here.</p> -<p>My setup is exposed to the public, and as always is heavily based on previous entries as described in <a href="#prerequisites">Prerequisites</a>. Descriptions on setting up MariaDB (preferred MySQL replacement for Arch) and PHP are written in this entry as this is the first time I’ve needed them.</p> -<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#mariadb">MariaDB</a><ul> -<li><a href="#create-usersdatabases">Create users/databases</a></li> -</ul> -</li> -<li><a href="#php">PHP</a><ul> -<li><a href="#configuration">Configuration</a></li> -<li><a href="#nginx">Nginx</a></li> -</ul> -</li> -<li><a href="#yourls">YOURLS</a><ul> -<li><a href="#configuration_1">Configuration</a></li> -<li><a href="#nginx_1">Nginx</a><ul> -<li><a href="#ssl-certificate">SSL certificate</a></li> -</ul> -</li> -<li><a href="#usage">Usage</a></li> -</ul> -</li> -<li><a href="#privatebin">PrivateBin</a><ul> -<li><a href="#configuration_2">Configuration</a><ul> -<li><a href="#yourls-integration">YOURLS integration</a></li> -</ul> -</li> -<li><a href="#nginx_2">Nginx</a><ul> -<li><a href="#ssl-certificate_1">SSL certificate</a></li> -</ul> -</li> -</ul> -</li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>If you want to expose to a (sub)domain, then similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> for the reverse proxy plus certificates):</p> -<ul> -<li><code>nginx</code> for the reverse proxy.</li> -<li><code>certbot</code> for the SSL certificates.</li> -<li><code>yay</code> to install AUR packages.<ul> -<li>I briefly mention how to install and use it on <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html#yay">Manga server with Komga: yay</a>.</li> -</ul> -</li> -<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>privatebin</code> and <code>yourls</code> (or whatever you want to call them).</li> -</ul> -<h2 id="mariadb">MariaDB<a class="headerlink" href="#mariadb" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/MariaDB">MariaDB</a> is a drop-in replacement of <a href="https://wiki.archlinux.org/title/MySQL">MySQL</a>.</p> -<p>Install the <code>mariadb</code> package:</p> -<pre><code class="language-sh">pacman -S mariadb -</code></pre> -<p>Before starting/enabling the systemd service run:</p> -<pre><code class="language-sh">mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql -</code></pre> -<p><code>start</code>/<code>enable</code> the <code>mariadb.service</code>:</p> -<pre><code class="language-sh">systemctl start mariadb.service -systemctl enable mariadb.service -</code></pre> -<p>Run and follow the secure installation script before proceding any further:</p> -<pre><code class="language-sh">mariadb-secure-installation -</code></pre> -<p>Change the binding address so the service listens on <code>localhost</code> only by modifying <code>/etc/my.cnf.d/server.cnf</code>:</p> -<pre><code class="language-ini">[mariadb] -bind-address = localhost -</code></pre> -<h3 id="create-usersdatabases">Create users/databases<a class="headerlink" href="#create-usersdatabases" title="Permanent link">¶</a></h3> -<p>To use <code>mariadb</code> simply run the command and it will try to login with the corresponding linux user running it. The general login command is:</p> -<pre><code class="language-sh">mariadb -u <username> -p <database_name> -</code></pre> -<p>The <code>database_name</code> is optional. It will prompt a password input field.</p> -<p>Using <code>mariadb</code> as root, create users with their respective database if needed with the following queries:</p> -<pre><code class="language-sql">MariaDB> CREATE USER '<username>'@'localhost' IDENTIFIED BY '<password>'; -MariaDB> CREATE DATABASE <database_name>; -MariaDB> GRANT ALL PRIVILEGES ON <database_name>.* TO '<username>'@'localhost'; -MariaDB> quit -</code></pre> -<p>The <code>database_name</code> will depend on how YOURLS and PrivateBin are configured, that is if the services use a separate database and/or table prefixes are used.</p> -<h2 id="php">PHP<a class="headerlink" href="#php" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/PHP">PHP</a> is a general-purpose scripting language that is usually used for web development, which was supposed to be ass for a long time but it seems to be a misconseption from the <em>old times</em>.</p> -<p>Install the <code>php</code>, <code>php-fpm</code>, <code>php-gd</code> packages:</p> -<pre><code class="language-sh">pacman -S php php-fpm php-gd -</code></pre> -<p><code>start</code>/<code>enable</code> the <code>php-fpm.service</code>:</p> -<pre><code class="language-sh">systemctl start php-fpm.service -systemctl enable php-fpm.service -</code></pre> -<h3 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h3> -<p>Only showing changes needed, main config file is located at <code>/etc/php/php.ini</code>, or drop-in files can be placed at <code>/etc/php/conf.d/</code> instead.</p> -<p>Set timezone (<a href="https://www.php.net/manual/en/timezones.php">list of timezones</a>):</p> -<pre><code class="language-ini">date.timezone = Europe/Berlin -</code></pre> -<p>Enable the <code>gd</code> and <code>mysql</code> extensions:</p> -<pre><code class="language-ini">extension=gd -extension=pdo_mysql -extension=mysqli -</code></pre> -<h3 id="nginx">Nginx<a class="headerlink" href="#nginx" title="Permanent link">¶</a></h3> -<p>Create a PHP specific config that can be reusable at <code>/etc/nginx/php_fastcgi.conf</code>:</p> -<pre><code class="language-nginx">location ~ \.php$ { - # required for yourls - add_header Access-Control-Allow-Origin $http_origin; - - # 404 - try_files $fastcgi_script_name =404; - - # default fastcgi_params - include fastcgi_params; - - # fastcgi settings - fastcgi_pass unix:/run/php-fpm/php-fpm.sock; - fastcgi_index index.php; - fastcgi_buffers 8 16k; - fastcgi_buffer_size 32k; - - # fastcgi params - fastcgi_param DOCUMENT_ROOT $realpath_root; - fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name; - #fastcgi_param PHP_ADMIN_VALUE "open_basedir=$base/:/usr/lib/php/:/tmp/"; -} -</code></pre> -<p>This then can be imported by any <code>server</code> directive that needs it.</p> -<h2 id="yourls">YOURLS<a class="headerlink" href="#yourls" title="Permanent link">¶</a></h2> -<p><a href="https://yourls.org/">YOURLS</a> is a self-hosted URL shortener that is supported by PrivateBin.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S yourls -</code></pre> -<p>Create a new user and database as described in <a href="#create-usersdatabases">MariaDB: Create users/databases</a>.</p> -<h3 id="configuration_1">Configuration<a class="headerlink" href="#configuration_1" title="Permanent link">¶</a></h3> -<p>The default configuration file is self explanatory, it is located at <code>/etc/webapps/yourls/config.php</code>. Make sure to correctly set the user/database YOURLS will use and either create a cookie or get one from <a href="http://yourls.org/cookie">URL provided</a>.</p> -<p>It is important to change the <code>$yours_user_passwords</code> variable, YOURLS will hash the passwords on login so it is not stored in plaintext. Password hashing can be disabled with:</p> -<pre><code class="language-php">define( 'YOURLS_NO_HASH_PASSWORD', true ); -</code></pre> -<p>I also changed the “shortening method” to <code>62</code> to include more characters:</p> -<pre><code class="language-php">define( 'YOURLS_URL_CONVERT', 62 ); -</code></pre> -<p>The <code>$yourls_reserved_URL</code> variable will need more blacklisted words depending on the use-case. Make sure the <code>YOURLS_PRIVATE</code> variable is set to <code>true</code> (default) if the service will be exposed to the public.</p> -<h3 id="nginx_1">Nginx<a class="headerlink" href="#nginx_1" title="Permanent link">¶</a></h3> -<p>Create a <code>yourls.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p> -<pre><code class="language-nginx">server { - listen 80; - root /usr/share/webapps/yourls/; - server_name short.example.com; - index index.php; - - location / { - try_files $uri $uri/ /yourls-loader.php$is_args$args; - } - - include /etc/nginx/php_fastcgi.conf; -} -</code></pre> -<p>Make sure the following header is included in the <code>php</code>‘s <code>nginx</code> location block described in <a href="#nginx">YOURLS: Nginx</a>:</p> -<pre><code class="language-nginx">add_header Access-Control-Allow-Origin $http_origin; -</code></pre> -<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4> -<p>Create/extend the certificate by running:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>Restart the <code>nginx</code> service for changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="usage">Usage<a class="headerlink" href="#usage" title="Permanent link">¶</a></h3> -<p>The admin area is located at <code>https://short.example.com/admin/</code>, login with any of the configured users set with the <code>$yours_user_passwords</code> in the config. Activate plugins by going to the “Manage Plugins” page (located at the top left) and clicking in the respective “Activate” button by hovering the “Actin” column, as shown below:</p> -<figure id="__yafg-figure-1"> -<img alt="YOURLS: Activate plugin" src="https://static.luevano.xyz/images/b/yourls/yourls_activate_plugin.png" title="YOURLS: Activate plugin"> -<figcaption>YOURLS: Activate plugin</figcaption> -</figure> -<p>I personally activated the “Random ShortURLs” and “Allow Hyphens in Short URLs”. Once the “Random ShortURLs” plugin is activated it can be configured by going to the “Random ShortURLs Settings” page (located at the top left, right below “Manage Plugins”), only config available is “Random Keyword Length”.</p> -<p>The main admin area can be used to manually shorten any link provided, by using the automatic shortening or by providing a custom short URL.</p> -<p>Finally, the “Tools” page (located at the top left) conains the <code>signature</code> token, used for <a href="https://yourls.org/docs/guide/advanced/passwordless-api">YOURLS: Passwordless API</a> as well as useful bookmarklets for URL shortening while browsing.</p> -<h2 id="privatebin">PrivateBin<a class="headerlink" href="#privatebin" title="Permanent link">¶</a></h2> -<p><a href="https://privatebin.info/">PrivateBin</a> is a minimalist self-hosted alternative to <a href="https://pastebin.com/">pastebin</a>.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S privatebin -</code></pre> -<p>Create a new user and database as described in <a href="#create-usersdatabases">MariaDB: Create users/databases</a>.</p> -<h3 id="configuration_2">Configuration<a class="headerlink" href="#configuration_2" title="Permanent link">¶</a></h3> -<p>This heavily depends on personal preference, all defaults are fine. Make a copy of the sample config template:</p> -<pre><code class="language-sh">cp /etc/webapps/privatebin/conf.sample.php /etc/webapps/privatebin/conf.php -</code></pre> -<p>The most important changes needed are <code>basepath</code> according to the <code>privatebin</code> URL and the <code>[model]</code> and <code>[model_options]</code> to use MySQL instead of plain filesystem files:</p> -<pre><code class="language-php">[model] -; example of DB configuration for MySQL -class = Database -[model_options] -dsn = "mysql:host=localhost;dbname=privatebin;charset=UTF8" -tbl = "privatebin_" ; table prefix -usr = "privatebin" -pwd = "<password>" -opt[12] = true ; PDO::ATTR_PERSISTENT -</code></pre> -<p>Any other <code>[model]</code> or <code>[model_options]</code> needs to be commented out (for example, the default filesystem setting).</p> -<h4 id="yourls-integration">YOURLS integration<a class="headerlink" href="#yourls-integration" title="Permanent link">¶</a></h4> -<p>I recommend creating a separate user for <code>privatebin</code> in <code>yourls</code> by modifying the <code>$yours_user_passwords</code> variable in <code>yourls</code> config file. Then login with this user and get the <code>signature</code> from the “Tools” section in the admin page, for more: <a href="https://yourls.org/docs/guide/advanced/passwordless-api">YOURLS: Passwordless API</a>.</p> -<p>For a “private” <code>yourls</code> installation (that needs username/pasword), set <code>urlshortener</code>:</p> -<pre><code class="language-php">urlshortener = "https://short.example.com/yourls-api.php?signature=xxxxxxxxxx&action=shorturl&format=json&url=" -</code></pre> -<p><mark>Note that this will expose the <code>signature</code> in the HTTP requests and anybody with the signature can use it to shorten external URLs.</mark></p> -<h3 id="nginx_2">Nginx<a class="headerlink" href="#nginx_2" title="Permanent link">¶</a></h3> -<p>To deny access to some bots/crawlers, PrivateBin provides a sample <code>.htaccess</code>, which is used in Apache. We need an Nginx version, which I found <a href="https://gist.github.com/benediktg/948a70136e2104c8601da7d355061323">here</a>.</p> -<p>Add the following at the beginning of the <code>http</code> block of the <code>/etc/nginx/nginx.conf</code> file:</p> -<pre><code class="language-nginx">http { - map $http_user_agent $pastebin_badagent { - ~*bot 1; - ~*spider 1; - ~*crawl 1; - ~https?:// 1; - WhatsApp 1; - SkypeUriPreview 1; - facebookexternalhit 1; - } - - #... -} -</code></pre> -<p>Create a <code>privatebin.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p> -<pre><code class="language-nginx">server { - listen 80; - root //usr/share/webapps/privatebin/; - server_name bin.example.com; - index index.php; - - if ($pastebin_badagent) { - return 403; - } - - location / { - try_files $uri $uri/ /index.php$is_args$args; - } - - include /etc/nginx/php_fastcgi.conf; -} -</code></pre> -<h4 id="ssl-certificate_1">SSL certificate<a class="headerlink" href="#ssl-certificate_1" title="Permanent link">¶</a></h4> -<p>Create/extend the certificate by running:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>Restart the <code>nginx</code> service for changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre>]]></content:encoded> - </item> - <item> - <title>Set up a media server with Jellyfin, Sonarr and Radarr</title> - <link>https://blog.luevano.xyz/a/jellyfin_server_with_sonarr_radarr.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/jellyfin_server_with_sonarr_radarr.html</guid> - <pubDate>Mon, 24 Jul 2023 04:30:14 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a media server with Jellyfin, Sonarr and Radarr, on Arch. With Bazarr, too.</description> - <content:encoded><![CDATA[<p>Second part of my self hosted media server. This is a direct continuation of <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html">Set up qBitTorrent with Jackett for use with Starr apps</a>, which will be mentioned as “first part” going forward. Sonarr, Radarr, Bazarr (Starr apps) and Jellyfin setups will be described in this part. Same introduction applies to this entry, regarding the use of documentation and configuration.</p> -<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p> -<p><mark>Kindly note that I do not condone the use of BitTorrent for illegal activities. I take no responsibility for what you do when setting up anything shown here. It is for you to check your local laws before using automated downloaders such as Sonarr and Radarr.</mark></p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#radarr">Radarr</a><ul> -<li><a href="#reverse-proxy">Reverse proxy</a></li> -<li><a href="#start-using-radarr">Start using Radarr</a><ul> -<li><a href="#configuration">Configuration</a><ul> -<li><a href="#media-management">Media Management</a></li> -<li><a href="#quality">Quality</a></li> -<li><a href="#custom-formats">Custom Formats</a></li> -<li><a href="#profiles">Profiles</a></li> -<li><a href="#download-clients">Download clients</a></li> -<li><a href="#indexers">Indexers</a></li> -</ul> -</li> -<li><a href="#download-content">Download content</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#sonarr">Sonarr</a><ul> -<li><a href="#reverse-proxy_1">Reverse proxy</a></li> -<li><a href="#start-using-sonarr">Start using Sonarr</a><ul> -<li><a href="#configuration_1">Configuration</a><ul> -<li><a href="#media-management_1">Media Management</a></li> -<li><a href="#quality_1">Quality</a></li> -<li><a href="#profiles_1">Profiles</a></li> -<li><a href="#download-clients_1">Download clients</a></li> -<li><a href="#indexers_1">Indexers</a></li> -</ul> -</li> -<li><a href="#download-content_1">Download content</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#jellyfin">Jellyfin</a><ul> -<li><a href="#reverse-proxy_2">Reverse proxy</a><ul> -<li><a href="#ssl-certificate">SSL certificate</a></li> -</ul> -</li> -<li><a href="#start-using-jellyfin">Start using Jellyfin</a><ul> -<li><a href="#plugins">Plugins</a></li> -<li><a href="#transcoding">Transcoding</a><ul> -<li><a href="#nvidia-drivers">NVIDIA drivers</a></li> -<li><a href="#enable-hardware-acceleration">Enable hardware acceleration</a></li> -</ul> -</li> -</ul> -</li> -</ul> -</li> -<li><a href="#bazarr">Bazarr</a><ul> -<li><a href="#reverse-proxy_3">Reverse proxy</a></li> -<li><a href="#start-using-bazarr">Start using Bazarr</a><ul> -<li><a href="#configuration_2">Configuration</a><ul> -<li><a href="#providers">Providers</a></li> -</ul> -</li> -</ul> -</li> -</ul> -</li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>Same prerequisites as with the <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#prerequisites">First part: Prerequisites</a> plus:</p> -<ul> -<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>jellyfin</code>. Only if you want to expose Jellyfin to a subdomain.</li> -</ul> -<p>The <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">First part: Directory structure</a> is the same here. The <code>servarr</code> user and group should be available, too.</p> -<p><mark>It is assumed that the first part was followed.</mark></p> -<h2 id="radarr">Radarr<a class="headerlink" href="#radarr" title="Permanent link">¶</a></h2> -<p><a href="https://radarr.video/">Radarr</a> is a movie collection manager that can be used to download movies via torrents. This is actually a fork of Sonarr, so they’re pretty similar, I just wanted to set up movies first.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S radarr -</code></pre> -<p><mark>Add the <code>radarr</code> user to the <code>servarr</code> group:</mark></p> -<pre><code class="language-sh">gpasswd -a radarr servarr -</code></pre> -<p>The default port that Radarr uses is <code>7878</code> for http (the one you need for the reverse proxy).</p> -<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3> -<p>Add the following <code>location</code> blocks into the <code>isos.conf</code> with whatever subdirectory name you want, I’ll leave it as <code>radarr</code>:</p> -<pre><code class="language-nginx">location /radarr/ { - proxy_pass http://127.0.0.1:7878/radarr/; # change port if needed - proxy_http_version 1.1; - - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Host $host; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $http_connection; - - proxy_redirect off; -} -# Allow the API External Access via NGINX -location /radarr/api { - auth_basic off; - proxy_pass http://127.0.0.1:7878/radarr/api; # change port if needed -} -</code></pre> -<p>This is taken from <a href="https://wiki.servarr.com/radarr/installation#nginx">Radarr Nginx reverse proxy configuration</a>. Restart the <code>nginx</code> service for the changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-radarr">Start using Radarr<a class="headerlink" href="#start-using-radarr" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>radarr.service</code>:</p> -<pre><code class="language-sh">systemctl enable radarr.service -systemctl start radarr.service -</code></pre> -<p>This will start the service and create the default configs under <code>/var/lib/radarr</code>. You need to change the <code>URLBase</code> as the reverse proxy is under a subdirectory (<code>/radarr</code>). Edit <code>/var/lib/radarr/config.xml</code>:</p> -<pre><code class="language-xml">... -<UrlBase>/radarr</UrlBase> -... -</code></pre> -<p>Then restart the <code>radarr</code> service:</p> -<pre><code class="language-sh">systemctl restart radarr.service -</code></pre> -<p>Now <code>https://isos.yourdomain.com/radarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials.</p> -<p>Note that if you want to have an anime movies library, it is recommended to run a second instance of Radarr for this as shown in <a href="https://wiki.servarr.com/radarr/installation#linux-multiple-instances">Radarr: Linux multiple instances</a> and follow <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles-anime/">TRaSH: How to setup quality profiles anime</a> if an anime instance is what you want.</p> -<h4 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h4> -<p>Will be following the official <a href="https://wiki.servarr.com/radarr/quick-start-guide">Radarr: Quick start guide</a> as well as the recommendations by <a href="https://trash-guides.info/Radarr/">TRaSH: Radarr</a>.</p> -<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p> -<h5 id="media-management">Media Management<a class="headerlink" href="#media-management" title="Permanent link">¶</a></h5> -<ul> -<li><strong>File Management</strong>:<ul> -<li><em>Propers and Repacks</em>: set it to “Do Not Prefer” and instead you’ll use the <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#repackproper">Repack/Proper</a> <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats">custom format by TRaSH</a>.</li> -</ul> -</li> -</ul> -<h5 id="quality">Quality<a class="headerlink" href="#quality" title="Permanent link">¶</a></h5> -<p>This is personal preference and it dictates your preferred file sizes. You can follow <a href="https://trash-guides.info/Radarr/Radarr-Quality-Settings-File-Size/">TRaSH: Quality settings</a> to maximize the quality of the downloaded content and restrict low quality stuff.</p> -<p>Personally, I think TRaSH’s quality settings are a bit elitist and first world-y. I’m fine with whatever and the tracker I’m using has the quality I want anyways. I did, however, set it to a minimum of <code>0</code> and maximum of <code>400</code> for the qualities shown in TRaSH’s guide. Configuring anything below <code>720p</code> shouldn’t be necessary anyways.</p> -<h5 id="custom-formats">Custom Formats<a class="headerlink" href="#custom-formats" title="Permanent link">¶</a></h5> -<p>Again, this is also completely a personal preference selection and depends on the quality and filters you want. My custom format selections are mostly based on <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles/#hd-bluray-webA">TRaSH: HD Bluray + WEB quality profile</a>.</p> -<p>The only <em>Unwanted</em> format that I’m not going to use is the Low Quality (<a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#lq">LQ</a>) as it blocks one of the sources I’m using to download a bunch of movies. The reasoning behind the LQ custom format is that these release groups don’t care much about quality (they keep low file sizes) and name tagging, which I understand but I’m fine with this as I can upgrade movies individually whenever I want (I want a big catalog of content that I can quickly watch).</p> -<h5 id="profiles">Profiles<a class="headerlink" href="#profiles" title="Permanent link">¶</a></h5> -<p>As mentioned in <a href="#custom-formats">Custom Formats</a> and <a href="#quality">Quality</a> this is completly a personal preference. I’m going to go for “Low Quality” downloads by still following some of the conventions from TRaSH. I’m using the <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles/#hd-bluray-webA">TRaSH: HD Bluray + WEB quality profile</a> with the exclusion of the <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#lq">LQ</a> profile.</p> -<p>I set the name to “HD Bluray + WEB”. I’m also not upgrading the torrents for now. Language set to “Original”.</p> -<h5 id="download-clients">Download clients<a class="headerlink" href="#download-clients" title="Permanent link">¶</a></h5> -<p>Pretty straight forward, just click on the giant “+” button and click on the qBitTorrent option. Then configure:</p> -<ul> -<li>Name: can be anything, just an identifier.</li> -<li>Enable: enable it.</li> -<li>Host: use <code>127.0.0.1</code>. For some reason I can’t make it work with the reverse proxied qBitTorrent.</li> -<li>Port: the port number you chose, <code>30000</code> in my case.</li> -<li>Url Base: leave blank as even though you have it exposed under <code>/qbt</code>, the service itself is not.</li> -<li>Username: the Web UI username, <code>admin</code> by default.</li> -<li>Password: the Web UI username, <code>adminadmin</code> by default (you should’ve changed it if you have the service exposed).</li> -<li>Category: <code>movies</code>.</li> -</ul> -<p>Everything else can be left as default, but maybe change <em>Completed Download Handling</em> if you’d like. Same goes for the general <em>Failed Download Handling</em> download clients’ option.</p> -<h5 id="indexers">Indexers<a class="headerlink" href="#indexers" title="Permanent link">¶</a></h5> -<p>Also easy to set up, also just click on the giant “+” button and click on the <em>custom</em> Torznab option (you can also use the <em>preset -> Jackett</em> Torznab option). Then configure:</p> -<ul> -<li>Name: can be anything, just an identifier. I like to do “Jackett - INDEXER”, where “INDEXER” is just an identifier.</li> -<li>URL: <code>http://127.0.0.1:9117/jack/api/v2.0/indexers/YOURINDEXER/results/torznab/</code>, where <code>YOURINDEXER</code> is specific to each indexer (<code>yts</code>, <code>nyaasi</code>, etc.). Can be directly copied from the indexer’s “Copy Torznab Feed” button on the Jackett Web UI.</li> -<li>API Path: <code>/api</code>, leave as is.</li> -<li>API Key: this can be found at the top right corner in Jackett’s Web UI.</li> -<li>Categories: which categories to use when searching, these are generic categories until you test/add the indexer. After you add the indexer you can come back and select your prefered categories (like just toggling the movies categories).</li> -<li>Tags: I like to add a tag for the indexer name like <code>yts</code> or <code>nyaa</code>. This is useful to control which indexers to use when adding new movies.</li> -</ul> -<p>Everything else on default. <em>Download Client</em> can also be set, which can be useful to keep different categories per indexer or something similar. <em>Seed Ratio</em> and <em>Seed Time</em> can also be set and are used to manage when to stop the torrent, this can also be set globally on the qBitTorrent Web UI, this is a personal setting.</p> -<h4 id="download-content">Download content<a class="headerlink" href="#download-content" title="Permanent link">¶</a></h4> -<p>You can now start to download content by going to <em>Movies -> Add New</em>. Basically just follow the <a href="https://wiki.servarr.com/radarr/quick-start-guide#how-to-add-a-movie">Radarr: How to add a movie</a> guide. The screenshots from the guide are a bit outdated but it contains everything you need to know.</p> -<p>I personally use:</p> -<ul> -<li>Monitor: Movie Only.</li> -<li>Minimum Availability: Released.</li> -<li>Quiality Profile: “HD Bluray + WEB”, the one configured in this entry.</li> -<li>Tags: the indexer name I want to use to download the movie, usually just <code>yts</code> for me (remember this is a “LQ” release group, so if you have that custom format disable it) as mentioned in <a href="#indexers">Indexers</a>. If you don’t specify a tag it will only use indexers that don’t have a tag set.</li> -<li>Start search for missing movie: toggled on. Immediatly start searching for the movie and start the download.</li> -</ul> -<p>Once you click on “Add Movie” it will add it to the <em>Movies</em> section and start searching and selecting the best torrent it finds, according to the “filters” (quality settings, profile and indexer(s)).</p> -<p>When it selects a torrent it sends it to qBitTorrent and you can even go ahead and monitor it over there. Else you can also monitor at <em>Activity -> Queue</em>.</p> -<p>After the movie is downloaded and processed by Radarr, it will create the appropriate hardlinks to the <code>media/movies</code> directory, as set in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">First part: Directory structure</a>.</p> -<p>Optionally, you can add subtitles using <a href="#bazarr">Bazarr</a>.</p> -<h2 id="sonarr">Sonarr<a class="headerlink" href="#sonarr" title="Permanent link">¶</a></h2> -<p><a href="https://sonarr.tv/">Sonarr</a> is a TV series collection manager that can be used to download series via torrents. Most of the install process, configuration and whatnot is going to be basically the same as with Radarr.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S sonarr -</code></pre> -<p><mark>Add the <code>sonarr</code> user to the <code>servarr</code> group:</mark></p> -<pre><code class="language-sh">gpasswd -a sonarr servarr -</code></pre> -<p>The default port that Radarr uses is <code>8989</code> for http (the one you need for the reverse proxy).</p> -<h3 id="reverse-proxy_1">Reverse proxy<a class="headerlink" href="#reverse-proxy_1" title="Permanent link">¶</a></h3> -<p>Basically the same as with <a href="#reverse-proxy">Radarr: Reverse proxy</a>, <mark>except that the <code>proxy_set_header</code> changes from <code>$proxy_host</code> to <code>$host</code>.</mark></p> -<p>Add the following <code>location</code> blocks into the <code>isos.conf</code>, I’ll leave it as <code>sonarr</code>:</p> -<pre><code class="language-nginx">location /sonarr/ { - proxy_pass http://127.0.0.1:8989/sonarr/; # change port if needed - proxy_http_version 1.1; - - proxy_set_header Host $proxy_host; # this differs from the radarr reverse proxy - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Host $host; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection $http_connection; - - proxy_redirect off; -} -# Allow the API External Access via NGINX -location /sonarr/api { - auth_basic off; - proxy_pass http://127.0.0.1:8989/sonarr/api; # change port if needed -} -</code></pre> -<p>This is taken from <a href="https://wiki.servarr.com/sonarr/installation#nginx">Sonarr: Nginx reverse proxy configuration</a>. Restart the <code>nginx</code> service for the changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-sonarr">Start using Sonarr<a class="headerlink" href="#start-using-sonarr" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>sonarr.service</code>:</p> -<pre><code class="language-sh">systemctl enable sonarr.service -systemctl start sonarr.service -</code></pre> -<p>This will start the service and create the default configs under <code>/var/lib/sonarr</code>. You need to change the <code>URLBase</code> as the reverse proxy is under a subdirectory (<code>/sonarr</code>). Edit <code>/var/lib/sonarr/config.xml</code>:</p> -<pre><code class="language-xml">... -<UrlBase>/sonarr</UrlBase> -... -</code></pre> -<p>Then restart the <code>sonarr</code> service:</p> -<pre><code class="language-sh">systemctl restart sonarr.service -</code></pre> -<p>Now <code>https://isos.yourdomain.com/sonarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials.</p> -<p>Similar to <a href="#radarr">Radarr</a> if you want to have an anime library, it is recommended to run a second instance of Sonarr for this as shown in <a href="https://wiki.servarr.com/sonarr/installation#linux-multiple-instances">Sonarr: Linux multiple instances</a> and follow <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx-Anime/">TRaSH: Release profile regex (anime)</a> and the <a href="https://trash-guides.info/Sonarr/Sonarr-recommended-naming-scheme/#anime-episode-format">TRaSH: Anime recommended naming scheme</a> if an anime instance is what you want.</p> -<h4 id="configuration_1">Configuration<a class="headerlink" href="#configuration_1" title="Permanent link">¶</a></h4> -<p>Will be following the official <a href="https://wiki.servarr.com/sonarr/quick-start-guide">Sonarr: Quick start guide</a> as well as the recommendations by <a href="https://trash-guides.info/Sonarr/">TRaSH: Sonarr</a>.</p> -<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p> -<h5 id="media-management_1">Media Management<a class="headerlink" href="#media-management_1" title="Permanent link">¶</a></h5> -<ul> -<li><strong>File Management</strong>:<ul> -<li><em>Propers and Repacks</em>: set it to “Do Not Prefer” and instead you’ll use the <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#propers-and-repacks">Propers and Repacks</a> release profile and fill with <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#p2p-groups-repackproper">P2P Groups + Repack/Proper</a>.</li> -</ul> -</li> -</ul> -<h5 id="quality_1">Quality<a class="headerlink" href="#quality_1" title="Permanent link">¶</a></h5> -<p>Similar to <a href="#quality">Radarr: Quality</a> this is personal preference and it dictates your preferred file sizes. You can follow <a href="https://trash-guides.info/Sonarr/Sonarr-Quality-Settings-File-Size/">TRaSH: Quality settings</a> to maximize the quality of the downloaded content and restrict low quality stuff.</p> -<p>Will basically do the same as in <a href="#quality">Radarr: Quality</a>: set minimum of <code>0</code> and maximum of <code>400</code> for everything <code>720p</code> and above.</p> -<h5 id="profiles_1">Profiles<a class="headerlink" href="#profiles_1" title="Permanent link">¶</a></h5> -<p>This is a bit different than with <a href="#radarr">Radarr</a>, the way it is configured is by setting “Release profiles”. I took the profiles from <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/">TRaSH: WEB-DL Release profile regex</a>. The only possible change I’ll do is disable the Low Quality Groups and/or the “Golden rule” filter (for <code>x265</code> encoded video).</p> -<p>For me it ended up looking like this:</p> -<figure id="__yafg-figure-5"> -<img alt="Sonarr: Release profiles" src="https://static.luevano.xyz/images/b/sonarr/sonarr_release_profiles.png" title="Sonarr: Release profiles"> -<figcaption>Sonarr: Release profiles</figcaption> -</figure> -<p>But yours can differ as its mostly personal preference. For the “Quality profile” I’ll be using the default “HD-1080p” most of the time, but I also created a “HD + WEB (720/1080)” which works best for some.</p> -<h5 id="download-clients_1">Download clients<a class="headerlink" href="#download-clients_1" title="Permanent link">¶</a></h5> -<p>Exactly the same as with <a href="#download-clients">Radarr: Download clients</a> only change is the category from <code>movies</code> to <code>tv</code> (or whatever you want), click on the giant “+” button and click on the qBitTorrent option. Then configure:</p> -<ul> -<li>Name: can be anything, just an identifier.</li> -<li>Enable: enable it.</li> -<li>Host: use <code>127.0.0.1</code>.</li> -<li>Port: the port number you chose, <code>30000</code> in my case.</li> -<li>Url Base: leave blank as even though you have it exposed under <code>/qbt</code>, the service itself is not.</li> -<li>Username: the Web UI username, <code>admin</code> by default.</li> -<li>Password: the Web UI username, <code>adminadmin</code> by default (you should’ve changed it if you have the service exposed).</li> -<li>Category: <code>tv</code>.</li> -</ul> -<p>Everything else can be left as default, but maybe change <em>Completed Download Handling</em> if you’d like. Same goes for the general <em>Failed Download Handling</em> download clients’ option.</p> -<h5 id="indexers_1">Indexers<a class="headerlink" href="#indexers_1" title="Permanent link">¶</a></h5> -<p>Also exactly the same as with <a href="#indexers">Radarr: Indexers</a>, click on the giant “+” button and click on the <em>custom</em> Torznab option (this doesn’t have the Jackett preset). Then configure:</p> -<ul> -<li>Name: can be anything, just an identifier. I like to do “Jackett - INDEXER”, where “INDEXER” is just an identifier.</li> -<li>URL: <code>http://127.0.0.1:9117/jack/api/v2.0/indexers/YOURINDEXER/results/torznab/</code>, where <code>YOURINDEXER</code> is specific to each indexer (<code>eztv</code>, <code>nyaasi</code>, etc.). Can be directly copied from the indexer’s “Copy Torznab Feed” button on the Jackett Web UI.</li> -<li>API Path: <code>/api</code>, leave as is.</li> -<li>API Key: this can be found at the top right corner in Jackett’s Web UI.</li> -<li>Categories: which categories to use when searching, these are generic categories until you test/add the indexer. After you add the indexer you can come back and select your prefered categories (like just toggling the TV categories).</li> -<li>Tags: I like to add a tag for the indexer name like <code>eztv</code> or <code>nyaa</code>. This is useful to control which indexers to use when adding new series.</li> -</ul> -<p>Everything else on default. <em>Download Client</em> can also be set, which can be useful to keep different categories per indexer or something similar. <em>Seed Ratio</em> and <em>Seed Time</em> can also be set and are used to manage when to stop the torrent, this can also be set globally on the qBitTorrent Web UI, this is a personal setting.</p> -<h4 id="download-content_1">Download content<a class="headerlink" href="#download-content_1" title="Permanent link">¶</a></h4> -<p>Almost the same as with <a href="#download-content">Radarr: Download content</a>, but I’ve been personally selecting the torrents I want to download for each season/episode so far, as the indexers I’m using are all over the place and I like consistencies. Will update if I find a (near) 100% automation process, but I’m fine with this anyways as I always monitor that everything is going fine.</p> -<p>Add by going to <em>Series -> Add New</em>. Basically just follow the <a href="https://wiki.servarr.com/sonarr/library#add-new">Sonarr: Library add new</a> guide. Adding series needs a bit more options that movies in Radarr, but it’s straight forward.</p> -<p>I personally use:</p> -<ul> -<li>Monitor: All Episodes.</li> -<li>Quiality Profile: “HD + WEB (720/1080)”. This depends on what I want for that how, lately I’ve been experimenting with this one.</li> -<li>Series Type: Standard. For now I’m just downloading shows, but it has an Anime option.</li> -<li>Tags: the “indexer_name” I want to use to download the movie, I’ve been using all indexers so I just use all tags as I’m experimenting and trying multiple options.</li> -<li>Season Folder: enabled. I like as much organization as possible.</li> -<li>Start search for missing episodes: disabled. Depends on you, due to my indexers, I prefer to check manually the season packs, for example.</li> -<li>Start search for cutoff unmet episodes: disabled. Honestly don’t really know what this is.</li> -</ul> -<p>Once you click on “Add X” it will add it to the <em>Series</em> section and will start as monitored. So far I haven’t noticed that it immediately starts downloading (because of the “Start search for missing episodes” setting) but I always click on unmonitor the series, so I can manually check (again, due to the low quality of my indexers).</p> -<p>When it automatically starts to download an episode/season it will send it to qBitTorrent and you can monitor it over there. Else you can also monitor at <em>Activity -> Queue</em>. Same thing goes if you download manually each episode/season via the interactive search.</p> -<p>To interactively search episodes/seasons go to <em>Series</em> and then click on any series, then click either on the interactive search button for the episode or the season, it is an icon of a person as shown below:</p> -<figure id="__yafg-figure-6"> -<img alt="Sonarr: Interactive search button" src="https://static.luevano.xyz/images/b/sonarr/sonarr_interactive_search_button.png" title="Sonarr: Interactive search button"> -<figcaption>Sonarr: Interactive search button</figcaption> -</figure> -<p>Then it will bring a window with the search results, where it shows the indexer it got the result from, the size of the torrent, peers, language, quality, the score it received from the configured release profiles an alert in case that the torrent is “bad” and the download button to manually download the torrent you want. An example shown below:</p> -<figure id="__yafg-figure-7"> -<img alt="Sonarr: Interactive search results" src="https://static.luevano.xyz/images/b/sonarr/sonarr_interactive_search_results.png" title="Sonarr: Interactive search results"> -<figcaption>Sonarr: Interactive search results</figcaption> -</figure> -<p>After the movie is downloaded and processed by Sonarr, it will create the appropriate hardlinks to the <code>media/tv</code> directory, as set in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">Directory structure</a>.</p> -<p>Optionally, you can add subtitles using <a href="#bazarr">Bazarr</a>.</p> -<h2 id="jellyfin">Jellyfin<a class="headerlink" href="#jellyfin" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Jellyfin">Jellyfin</a> is a media server “manager”, usually used to manage and organize video content (movies, TV series, etc.) which could be compared with <a href="https://wiki.archlinux.org/title/plex">Plex</a> or <a href="https://wiki.archlinux.org/title/Emby">Emby</a> for example (take them as possible alternatives).</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S jellyfin-bin -</code></pre> -<p>I’m installing the pre-built binary instead of building it as I was getting a lot of errors and the server was even crashing. You can try installing <code>jellyfin</code> instead.</p> -<p><mark>Add the <code>jellyfin</code> user to the <code>servarr</code> group:</mark></p> -<pre><code class="language-sh">gpasswd -a jellyfin servarr -</code></pre> -<p>You can already <code>start</code>/<code>enable</code> the <code>jellyfin.service</code> which will start at <code>http://127.0.0.1:8096/</code> by default where you need to complete the initial set up. But let’s create the reverse proxy first then start everything and finish the set up.</p> -<h3 id="reverse-proxy_2">Reverse proxy<a class="headerlink" href="#reverse-proxy_2" title="Permanent link">¶</a></h3> -<p>I’m going to have my <code>jellyfin</code> instance under a subdomain with an <code>nginx</code> reverse proxy as shown in the <a href="https://wiki.archlinux.org/title/Jellyfin#Nginx_reverse_proxy">Arch wiki</a>. For that, create a <code>jellyfin.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p> -<pre><code class="language-nginx">server { - listen 80; - server_name jellyfin.yourdomain.com; # change accordingly to your wanted subdomain and domain name - set $jellyfin 127.0.0.1; # jellyfin is running at localhost (127.0.0.1) - - # Security / XSS Mitigation Headers - add_header X-Frame-Options "SAMEORIGIN"; - add_header X-XSS-Protection "1; mode=block"; - add_header X-Content-Type-Options "nosniff"; - - # Content Security Policy - # See: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP - # Enforces https content and restricts JS/CSS to origin - # External Javascript (such as cast_sender.js for Chromecast) must be whitelisted. - add_header Content-Security-Policy "default-src https: data: blob: http://image.tmdb.org; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' https://www.gstatic.com/cv/js/sender/v1/cast_sender.js https://www.youtube.com blob:; worker-src 'self' blob:; connect-src 'self'; object-src 'none'; frame-ancestors 'self'"; - - location = / { - return 302 https://$host/web/; - } - - location / { - # Proxy main Jellyfin traffic - proxy_pass http://$jellyfin:8096; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Protocol $scheme; - proxy_set_header X-Forwarded-Host $http_host; - - # Disable buffering when the nginx proxy gets very resource heavy upon streaming - proxy_buffering off; - } - - # location block for /web - This is purely for aesthetics so /web/#!/ works instead of having to go to /web/index.html/#!/ - location = /web/ { - # Proxy main Jellyfin traffic - proxy_pass http://$jellyfin:8096/web/index.html; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Protocol $scheme; - proxy_set_header X-Forwarded-Host $http_host; - } - - location /socket { - # Proxy Jellyfin Websockets traffic - proxy_pass http://$jellyfin:8096; - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Protocol $scheme; - proxy_set_header X-Forwarded-Host $http_host; - } -} -</code></pre> -<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4> -<p>Create/extend the certificate by running:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>Similarly to the <code>isos</code> subdomain, that will autodetect the new subdomain and extend the existing certificate(s). Restart the <code>nginx</code> service for changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-jellyfin">Start using Jellyfin<a class="headerlink" href="#start-using-jellyfin" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>jellyfin.service</code> if you haven’t already:</p> -<pre><code class="language-sh">systemctl enable jellyfin.service -systemctl start jellyfin.service -</code></pre> -<p>Then navigate to <code>https://jellyfin.yourdomain.com</code> and either continue with the set up wizard if you didn’t already or continue with the next steps to configure your libraries.</p> -<p>The initial setup wizard makes you create an user (will be the admin for now) and at least one library, though these can be done later. For more check <a href="https://jellyfin.org/docs/general/quick-start/">Jellyfin: Quick start</a>.</p> -<p>Remember to use the configured directory as mentioned in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">Directory structure</a>. Any other configuration (like adding users or libraries) can be done at the dashboard: click on the 3 horizontal lines on the top left of the Web UI then navigate to <em>Administration -> Dashboard</em>. I didn’t configure much other than adding a couple of users for me and friends, I wouldn’t recommend using the admin account to watch (personal preference).</p> -<p>Once there is at least one library it will show at <em>Home</em> along with the latest movies (if any) similar to the following (don’t judge, these are just the latest I added due to friend’s requests):</p> -<figure id="__yafg-figure-8"> -<img alt="Jellyfin: Home libraries" src="https://static.luevano.xyz/images/b/jellyfin/jellyfin_home_libraries.png" title="Jellyfin: Home libraries"> -<figcaption>Jellyfin: Home libraries</figcaption> -</figure> -<p>And inside the “Movies” library you can see the whole catalog where you can filter or just scroll as well as seeing <em>Suggestions</em> (I think this starts getting populated after a while) and <em>Genres</em>:</p> -<figure id="__yafg-figure-9"> -<img alt="Jellyfin: Library catalog options" src="https://static.luevano.xyz/images/b/jellyfin/jellyfin_library_catalog_options.png" title="Jellyfin: Library catalog options"> -<figcaption>Jellyfin: Library catalog options</figcaption> -</figure> -<h4 id="plugins">Plugins<a class="headerlink" href="#plugins" title="Permanent link">¶</a></h4> -<p>You can also install/activate <a href="https://jellyfin.org/docs/general/server/plugins/">plugins</a> to get extra features. Most of the plugins you might want to use are already available in the official repositories and can be found in the “Catalog”. There are a lot of plugins that are focused around anime and TV metadata, as well as an Open Subtitles plugin to automatically download missing subtitles (though this is managed with <a href="#bazarr">Bazarr</a>).</p> -<p>To activate plugins click on the 3 horizontal lines on the top left of the Web UI then navigate to <em>Administration -> Dashboard -> Advanced -> Plugins</em> and click on the <em>Catalog</em> tab (top of the Web UI). Here you can select the plugins you want to install. By default only the official ones are shown, for more you can add more <a href="https://jellyfin.org/docs/general/server/plugins/#repositories">repositories</a>.</p> -<p>The only plugin I’m using is the “Playback Reporting”, to get a summary of what is being used in the instance. But I might experiment with some anime-focused plugins when the time comes.</p> -<h4 id="transcoding">Transcoding<a class="headerlink" href="#transcoding" title="Permanent link">¶</a></h4> -<p>Although not recommended and explicitly set to not download any <code>x265</code>/<code>HEVC</code> content (by using the <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#golden-rule">Golden rule</a>) there might be cases where the only option you have is to download such content. If that is the case and you happen to have a way to do <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/">Hardware Acceleration</a>, for example by having an NVIDIA graphics card (in my case) then you should enable it to avoid using lots of resources on your system.</p> -<p>Using hardware acceleration will leverage your GPU to do the transcoding and save resources on your CPU. I tried streaming <code>x265</code> content and it basically used 70-80% on all cores of my CPU, while on the other hand using my GPU it used the normal amount on the CPU (70-80% on a single core).</p> -<p>This will be the steps to install on an <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/nvidia/">NVIDIA</a> graphics card, specifically a GTX 1660 Ti. But more info and guides can be found at <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/">Jellyfin: Hardware Acceleration</a> for other acceleration methods.</p> -<h5 id="nvidia-drivers">NVIDIA drivers<a class="headerlink" href="#nvidia-drivers" title="Permanent link">¶</a></h5> -<p>Ensure you have the NVIDIA drivers and utils installed. I’ve you’ve done this in the past then you can skip this part, else you might be using the default <code>nouveau</code> drivers. Follow the next steps to set up the NVIDIA drivers, which basically is a summary of <a href="https://wiki.archlinux.org/title/NVIDIA#Installation">NVIDIA: Installation</a> for my setup, so <mark>double check the wiki in case you have an older NVIDIA graphics card</mark>.</p> -<p>Install the <code>nvidia</code> and <code>nvidia-utils</code> packages:</p> -<pre><code class="language-sh">pacman -S nvidia nvidia-utils -</code></pre> -<p>Modify <code>/etc/mkinitcpio.conf</code> to remove <code>kms</code> from the <code>HOOKS</code> array. It should look like this (commented line is how it was for me before the change):</p> -<pre><code class="language-sh">... -# HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block filesystems fsck) -HOOKS=(base udev autodetect modconf keyboard keymap consolefont block filesystems fsck) -... -</code></pre> -<p><a href="https://wiki.archlinux.org/title/Mkinitcpio#Image_creation_and_activation">Regenerate the initramfs</a> by executing:</p> -<pre><code class="language-sh">mkinitcpio -P -</code></pre> -<p>Finally, reboot the system. After the reboot you should be able to check your GPU info and processes being run with the GPU by executing <code>nvidia-smi</code>.</p> -<h5 id="enable-hardware-acceleration">Enable hardware acceleration<a class="headerlink" href="#enable-hardware-acceleration" title="Permanent link">¶</a></h5> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S jellyfin-ffmpeg6-bin -</code></pre> -<p>This provides the <code>jellyfin-ffmpeg</code> executable, which is necessary for Jellyfin to do hardware acceleration, it needs to be this specific one.</p> -<p>Then in the Jellyfin go to the transcoding settings by clicking on the 3 horizontal lines on the top left of the Web UI and navigating to <em>Administration -> Dashboard -> Playback -> Transcoding</em> and:</p> -<ul> -<li>Change the <em>Hardware acceleration</em> from “None” to “Nvidia NVENC”.</li> -<li>Some other options will pop up, just make sure you enable “HEVC” (which is <code>x265</code>) in the list of <em>Enable hardware encoding for:</em>.</li> -<li>Scroll down and specify the <code>ffmpeg</code> path, which is <code>/usr/lib/jellyfin-ffmpeg/ffmpeg</code>.</li> -</ul> -<p>Don’t forget to click “Save” at the bottom of the Web UI, it will ask if you want to enable hardware acceleration.</p> -<h2 id="bazarr">Bazarr<a class="headerlink" href="#bazarr" title="Permanent link">¶</a></h2> -<p><a href="https://www.bazarr.media/">Bazarr</a> is a companion for Sonarr and Radarr that manages and downloads subtitles.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S bazarr -</code></pre> -<p><mark>Add the <code>bazarr</code> user to the <code>servarr</code> group:</mark></p> -<pre><code class="language-sh">gpasswd -a bazarr servarr -</code></pre> -<p>The default port that Bazarr uses is <code>6767</code> for http (the one you need for the reverse proxy), and it has pre-configured the default ports for Radarr and Sonarr.</p> -<h3 id="reverse-proxy_3">Reverse proxy<a class="headerlink" href="#reverse-proxy_3" title="Permanent link">¶</a></h3> -<p>Basically the same as with <a href="#reverse-proxy">Radarr: Reverse proxy</a> and <a href="#reverse-proxy-1">Sonarr: Reverse proxy</a>.</p> -<p>Add the following setting in the <code>server</code> block of the <code>isos.conf</code>:</p> -<pre><code class="language-nginx">server { - # server_name and other directives - ... - - # Increase http2 max sizes - large_client_header_buffers 4 16k; - - # some other blocks like location blocks - ... -} -</code></pre> -<p>Then add the following <code>location</code> blocks in the <code>isos.conf</code>, where I’ll keep it as <code>/bazarr/</code>:</p> -<pre><code class="language-nginx">location /bazarr/ { - proxy_pass http://127.0.0.1:6767/bazarr/; # change port if needed - proxy_http_version 1.1; - - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header Host $http_host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "Upgrade"; - - proxy_redirect off; -} -# Allow the Bazarr API through if you enable Auth on the block above -location /bazarr/api { - auth_request off; - proxy_pass http://127.0.0.1:6767/bazarr/api; -} -</code></pre> -<p>This is taken from <a href="https://wiki.bazarr.media/Additional-Configuration/Reverse-Proxy-Help/">Bazarr: Reverse proxy help</a>. Restart the <code>nginx</code> service for the changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-bazarr">Start using Bazarr<a class="headerlink" href="#start-using-bazarr" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>bazarr.service</code> if you haven’t already:</p> -<pre><code class="language-sh">systemctl start bazarr.service -systemctl enable bazarr.service -</code></pre> -<p>This will start the service and create the default configs under <code>/var/lib/bazarr</code>. You need to change the <code>base_url</code> for the necessary services as they’re running under a reverse proxy and under subdirectories. Edit <code>/var/lib/bazarr/config/config.ini</code>:</p> -<pre><code class="language-ini">[general] -port = 6767 -base_url = /bazarr - -[sonarr] -port = 8989 -base_url = /sonarr - -[radarr] -port = 7878 -base_url = /radarr -</code></pre> -<p>Then restart the <code>bazarr</code> service:</p> -<pre><code class="language-sh">systemctl restart bazarr.service -</code></pre> -<p>Now <code>https://isos.yourdomain.com/bazarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials. I also disabled <em>Settings -> General -> Updates -> Automatic</em>.</p> -<h4 id="configuration_2">Configuration<a class="headerlink" href="#configuration_2" title="Permanent link">¶</a></h4> -<p>Will be following the official <a href="https://wiki.bazarr.media/Getting-Started/setup-guide/">Bazarr: Setup guide</a> as well as the recommendations by <a href="https://trash-guides.info/Bazarr/">TRaSH: Bazarr</a>.</p> -<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p> -<h5 id="providers">Providers<a class="headerlink" href="#providers" title="Permanent link">¶</a></h5> -<p>This doesn’t require much thinking and its up to personal preference, but I’ll list the ones I added:</p> -<ul> -<li><a href="https://www.opensubtitles.com/">OpenSubtitles.com</a>: requires an account (the <code>.org</code> option is deprecated).<ul> -<li>For a free account it only lets you download around 20 subtitles per day, and they contain ads. You could pay for a VIP account ($3 per month) and that will give you 1000 subtitles per day and no ads. But if you’re fine with 20 ads per day you can try to get rid of the ads by running an automated script. Such option can be found at <a href="https://github.com/brianspilner01/media-server-scripts/blob/master/sub-clean.sh">brianspilner01/media-server-scripts: sub-clean.sh</a>.</li> -</ul> -</li> -<li>YIFY Subtitles</li> -<li>Subscenter</li> -<li>Supersubtitles</li> -<li>TVSubtitles</li> -<li>Subtitulamos.tv: Spanish subtitles provider.</li> -<li>Argenteam: LATAM Spanish subtitles provider.</li> -<li>Subdivx: LATAM Spanish / Spanish subtitles provider.</li> -</ul> -<p>I’ve tested this setup for the following languages (with all default settings as stated in the guides):</p> -<ul> -<li>English</li> -<li>Spanish</li> -</ul> -<p>I tried with “Latin American Spanish” but they’re hard to find, those two work pretty good.</p> -<p>None of these require an <a href="https://anti-captcha.com/">Anti-Captcha</a> account (which is a paid service), but I created one anyways in case I need it. Though you need to add credits to it (pretty cheap though) if you ever use it.</p>]]></content:encoded> - </item> - <item> - <title>Set up qBitTorrent with Jackett for use with Starr apps</title> - <link>https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html</guid> - <pubDate>Mon, 24 Jul 2023 02:06:24 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a torrenting solution with qBitTorrent in preparation for a media server with Jellyfin and Starr apps, on Arch. With Jackett and flaresolverr, too.</description> - <content:encoded><![CDATA[<p>Riding on my excitement of having a good internet connection and having setup my <em>home server</em> now it’s time to self host a media server for movies, series and anime. I’ll setup qBitTorrent as the downloader, Jackett for the trackers, the <em>Starr apps</em> for the automatic downloading and Jellyfin as the media server manager/media viewer. This was going to be a single entry but it ended up being a really long one so I’m splitting it, this being the first part.</p> -<p>I’ll be exposing my stuff on a subdomain only so I can access it while out of home and for SSL certificates (not required), but shouldn’t be necessary and instead you can use a VPN (<a href="https://blog.luevano.xyz/a/vpn_server_with_openvpn.html">how to set up</a>). For your reference, whenever I say “Starr apps” (*arr apps) I mean the family of apps such as Sonarr, Radarr, Bazarr, Readarr, Lidarr, etc..</p> -<p>Most of my config is based on the <a href="https://trash-guides.info/">TRaSH-Guides</a> (will be mentioned as “TRaSH” going forward). Specially get familiar with the <a href="https://trash-guides.info/Hardlinks/How-to-setup-for/Native/">TRaSH: Native folder structure</a> and with the <a href="https://trash-guides.info/Hardlinks/Hardlinks-and-Instant-Moves/">TRaSH: Hardlinks and instant moves</a>. Will also use the default configurations based on the respective documentation for each Starr app and service, except when stated otherwise.</p> -<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p> -<p><mark>Kindly note that I do not condone the use of torrenting for illegal activities. I take no responsibility for what you do when setting up anything shown here. It is for you to check your local laws before using automated downloaders such as Sonarr and Radarr.</mark></p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a><ul> -<li><a href="#directory-structure">Directory structure</a></li> -</ul> -</li> -<li><a href="#jackett">Jackett</a><ul> -<li><a href="#reverse-proxy">Reverse proxy</a><ul> -<li><a href="#ssl-certificate">SSL certificate</a></li> -</ul> -</li> -<li><a href="#start-using-jackett">Start using Jackett</a><ul> -<li><a href="#indexers">Indexers</a></li> -</ul> -</li> -<li><a href="#flaresolverr">FlareSolverr</a></li> -</ul> -</li> -<li><a href="#qbittorrent">qBitTorrent</a><ul> -<li><a href="#reverse-proxy_1">Reverse proxy</a></li> -<li><a href="#start-using-qbittorrent">Start using qBitTorrent</a><ul> -<li><a href="#configuration">Configuration</a></li> -<li><a href="#trackers">Trackers</a></li> -</ul> -</li> -</ul> -</li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>The specific programs are mostly recommendations, if you’re familiar with something else or want to change things around, feel free to do so but everything will be written with them in mind.</p> -<p>If you want to expose to a (sub)domain, then similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> for the reverse proxy plus certificates):</p> -<ul> -<li><code>nginx</code> for the reverse proxy.</li> -<li><code>certbot</code> for the SSL certificates.</li> -<li><code>ufw</code> for the firewall.</li> -<li><code>yay</code> to install AUR packages.<ul> -<li>I mentioned how to install and use it on my previous entry: <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html#yay">Manga server with Komga: yay</a>.</li> -</ul> -</li> -<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>isos</code> (or whatever you want to call it).<ul> -<li>For automation software (qBitTorrent, Jackett, Starr apps, etc.). One subdomain per service can be used instead.</li> -</ul> -</li> -</ul> -<p><mark>Note: I’m using the explicit <code>127.0.0.1</code> ip instead of <code>localhost</code> in the reverse proxies/services config as <code>localhost</code> resolves to <code>ipv6</code> sometimes which is not configured on my server correctly.</mark> If you have it configured you can use <code>localhost</code> without any issue.</p> -<h3 id="directory-structure">Directory structure<a class="headerlink" href="#directory-structure" title="Permanent link">¶</a></h3> -<p>Basically following <a href="https://trash-guides.info/Hardlinks/How-to-setup-for/Native/">TRaSH: Native folder structure</a> except for the directory permissions part, I’ll do the same as with my <a href="https://blog.luevano.xyz/a/manga_server_with_komga#set-default-directory-permissions.html">Komga setup guide</a> to stablish default group permissions.</p> -<p>The desired behaviour is: set <code>servarr</code> as group ownership, set write access to group and whenever a new directory/file is created, inherit these permission settings. <code>servarr</code> is going to be a service user and I’ll use the root of a mounted drive at <code>/mnt/a</code>.</p> -<ol> -<li>Create a service user called <code>servarr</code> (it could just be a group, too):</li> -</ol> -<pre><code class="language-sh">useradd -r -s /usr/bin/nologin -M -c "Servarr applications" servarr -</code></pre> -<ol start="2"> -<li>Create the <code>torrents</code> directory and set default permissions:</li> -</ol> -<pre><code class="language-sh">cd /mnt/a # change this according to your setup -mkdir torrents -chown servarr:servarr torrents -chmod g+w torrents -chmod g+s torrents -setfacl -d -m g::rwx torrents -setfacl -d -m o::rx torrents -</code></pre> -<ol start="3"> -<li>Check that the permissions are set correctly (<code>getfacl torrents</code>)</li> -</ol> -<pre><code># file: torrents/ -# owner: servarr -# group: servarr -# flags: -s- -user::rwx -group::rwx -other::r-x -default:user::rwx -default:group::rwx -default:other::r-x -</code></pre> -<ol start="4"> -<li>Create the subdirectories you want with any user (I’ll be using <code>servarr</code> personally):</li> -</ol> -<pre><code class="language-sh">mkdir torrents/{tv,movies,anime} -chown -R servarr: torrents -</code></pre> -<ol start="5"> -<li>Finally repeat steps 2 - 4 for the <code>media</code> directory.</li> -</ol> -<p>The final directory structure should be the following:</p> -<pre><code>root_dir -├── torrents -│ ├── movies -│ ├── music -│ └── tv -└── media - ├── movies - ├── music - └── tv -</code></pre> -<p>Where <code>root_dir</code> is <code>/mnt/a</code> in my case. This is going to be the reference for the following applications set up.</p> -<p>Later, add the necessary users to the <code>servarr</code> group if they need write access, by executing:</p> -<pre><code class="language-sh">gpasswd -a <USER> servarr -</code></pre> -<h2 id="jackett">Jackett<a class="headerlink" href="#jackett" title="Permanent link">¶</a></h2> -<p><a href="https://github.com/Jackett/Jackett">Jackett</a> is a “proxy server” (or “middle-ware”) that translates queries from apps (such as the Starr apps in this case) into tracker-specific http queries. Note that there is an alternative called <a href="https://github.com/Prowlarr/Prowlarr">Prowlarr</a> that is better integrated with most if not all Starr apps, requiring less maintenance; I’ll still be sticking with Jackett, though.</p> -<p>Install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S jackett -</code></pre> -<p>I’ll be using the default <code>9117</code> port, but change accordingly if you decide on another one.</p> -<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3> -<p>I’m going to have most of the services under the same subdomain, with different subdirectories. Create the config file <code>isos.conf</code> at the usual <code>sites-available/enabled</code> path for <code>nginx</code>:</p> -<pre><code class="language-nginx">server { - listen 80; - server_name isos.yourdomain.com; - - location /jack { # you can change this to jackett or anything you'd like, but it has to match the jackett config on the next steps - proxy_pass http://127.0.0.1:9117; # change the port according to what you want - - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Host $http_host; - proxy_redirect off; - } -} -</code></pre> -<p>This is the basic reverse proxy config as shown in <a href="https://github.com/Jackett/Jackett#running-jackett-behind-a-reverse-proxy">Jackett: Running Jackett behind a reverse proxy</a>. The rest of the services will be added under different <code>location</code> block on their respective steps.</p> -<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4> -<p>Create/extend the certificate by running:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>That will automatically detect the new subdomain config and create/extend your existing certificate(s). Restart the <code>nginx</code> service for changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-jackett">Start using Jackett<a class="headerlink" href="#start-using-jackett" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>jackett.service</code>:</p> -<pre><code class="language-sh">systemctl enable jackett.service -systemctl start jackett.service -</code></pre> -<p>It will autocreate the default configuration under <code>/var/lib/jackett/ServerConfig.json</code>, which you need to edit at least to change the <code>BasePathOverride</code> to match what you used in the <code>nginx</code> config:</p> -<pre><code class="language-json">{ - "Port": 9117, - "SomeOtherConfigs": "some_other_values", - "BasePathOverride": "/jack", - "MoreConfigs": "more_values", -} -</code></pre> -<p>Also modify the <code>Port</code> if you changed it. Restart the <code>jackett</code> service:</p> -<pre><code class="language-sh">systemctl restart jackett.service -</code></pre> -<p>It should now be available at <code>https://isos.yourdomain.com/jack</code>. <mark>Add an admin password right away</mark> by scroll down and until the first config setting; don’t forget to click on “Set Password”. Change any other config you want from the Web UI, too (you’ll need to click on the blue “Apply server settings” button).</p> -<p>Note that you need to set the “Base URL override” to <code>http://127.0.0.1:9117</code> (or whatever port you used) so that the “Copy Torznab Feed” button works for each indexer.</p> -<h4 id="indexers">Indexers<a class="headerlink" href="#indexers" title="Permanent link">¶</a></h4> -<p>For Jackett, an indexer is just a configured tracker for some of the commonly known torrent sites. Jackett comes with a lot of pre-configured public and private indexers which usually have multiple URLs (mirrors) per indexer, useful when the main torrent site is down. Some indexers come with extra features/configuration depending on what the site specializes on.</p> -<p>To add an indexer click on the “+ Add Indexer” at the top of the Web UI and look for indexers you want, then click on the “+” icon on the far-most right for each indexer or select the ones you want (clicking on the checkbox on the far-most left of the indexer) and scroll all the way to the bottom to click on “Add Selected”. They then will show as a list with some available actions such as “Copy RSS Feed”, “Copy Torznab Feed”, “Copy Potato Feed”, a button to search, configure, delete and test the indexer, as shown below:</p> -<figure id="__yafg-figure-2"> -<img alt="Jacket: configured indexers" src="https://static.luevano.xyz/images/b/jack/jack_configured_indexers.png" title="Jackett: configured indexers"> -<figcaption>Jackett: configured indexers</figcaption> -</figure> -<p>You can manually test the indexers by doing a basic search to see if they return anything, either by searching on each individual indexer by clicking on the magnifying glass icon on the right of the indexer or clicking on “Manual Search” button which is next to the “+ Add Indexer” button at the top right.</p> -<p>Explore each indexer’s configuration in case there is stuff you might want to change.</p> -<h3 id="flaresolverr">FlareSolverr<a class="headerlink" href="#flaresolverr" title="Permanent link">¶</a></h3> -<p><a href="https://github.com/FlareSolverr/FlareSolverr">FlareSolverr</a> is used to bypass <em>certain</em> protection that some torrent sites have. This is not 100% necessary and only needed for some trackers sometimes, it also doesn’t work 100%.</p> -<p>You could install from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S flaresolverr-bin -</code></pre> -<p>At the time of writing, the <code>flaresolverr</code> package didn’t work for me because of <code>python-selenium</code>. <code>flaresolverr-bin</code> was updated around the time I was writing this, so that is what I’m using and what’s working fine so far, it contains almost everything needed (it has self contained libraries) except for <code>xfvb</code>.</p> -<p>Install dependencies via <code>pacman</code>:</p> -<pre><code class="language-sh">pacman -S xorg-server-xvfb -</code></pre> -<p>You can now <code>start</code>/<code>enable</code> the <code>flaresolverr.service</code>:</p> -<pre><code class="language-sh">systemctl enable flaresolverr.service -systemctl start flaresolverr.service -</code></pre> -<p>Verify that the service started correctly by checking the logs:</p> -<pre><code class="language-sh">journalctl -fxeu flaresolverr -</code></pre> -<p>It should show “Test successful” and “Serving on http://0.0.0.0:8191” (which is the default). Jackett now needs to be configured by adding <code>http://127.0.0.1:8191</code> almost at the end in the “FlareSolverr API URL” field, then click on the blue “Apply server settings” button at the beginning of the config section. This doesn’t need to be exposed or anything, it’s just an internal API that Jackett (or anything you want) will use.</p> -<h2 id="qbittorrent">qBitTorrent<a class="headerlink" href="#qbittorrent" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/QBittorrent">qBitTorrent</a> is a fast, stable and light BitTorrent client that comes with many features and in my opinion it’s a really user friendly client and my personal choice for years now. But you can choose whatever client you want, there are more lightweight alternatives such as <a href="https://wiki.archlinux.org/title/transmission">Transmission</a>.</p> -<p>Install the <code>qbittorrent-nox</code> package (“nox” as in “no X server”):</p> -<pre><code class="language-sh">pacman -S qbittorrent-nox -</code></pre> -<p>By default the package doesn’t create any (service) user, but it is recommended to have one so you can run the service under it. Create the user, I’ll call it <code>qbittorrent</code> and leave it with the default homedir (<code>/home</code>):</p> -<pre><code class="language-sh">useradd -r -m qbittorrent -</code></pre> -<p><mark>Add the <code>qbittorrent</code> user to the <code>servarr</code> group:</mark></p> -<pre><code class="language-sh">gpasswd -a qbittorrent servarr -</code></pre> -<p>Decide a port number you’re going to run the service on for the next steps, the default is <code>8080</code> but I’ll use <code>30000</code>; it doesn’t matter much, as long as it matches for all the config. This is the <code>qbittorrent</code> service port, it is used to connect to the instance itself through the Web UI or via API, <mark>you also need to open a port for listening to peer connections.</mark> Choose any port you want, for example <code>50000</code> and open it with your firewall, <code>ufw</code> in my case:</p> -<pre><code class="language-sh">ufw allow 50000/tcp comment "qBitTorrent - Listening port" -</code></pre> -<h3 id="reverse-proxy_1">Reverse proxy<a class="headerlink" href="#reverse-proxy_1" title="Permanent link">¶</a></h3> -<p>Add the following <code>location</code> block into the <code>isos.conf</code> with whatever subdirectory name you want, I’ll call it <code>qbt</code>:</p> -<pre><code class="language-nginx">location /qbt/ { - proxy_pass http://localhost:30000/; # change port to whatever number you want - proxy_http_version 1.1; - - proxy_set_header Host $host; - proxy_set_header X-Forwarded-Host $http_host; - proxy_set_header X-Forwarded-For $remote_addr; - - proxy_cookie_path / "/; Secure"; - proxy_read_timeout 600s; - proxy_send_timeout 600s; -} -</code></pre> -<p>This is taken from <a href="https://github.com/qbittorrent/qBittorrent/wiki/NGINX-Reverse-Proxy-for-Web-UI">qBitTorrent: Nginx reverse proxy for Web UI</a>. Restart the <code>nginx</code> service for the changes to take effect:</p> -<pre><code class="language-sh">systemctl restart nginx.service -</code></pre> -<h3 id="start-using-qbittorrent">Start using qBitTorrent<a class="headerlink" href="#start-using-qbittorrent" title="Permanent link">¶</a></h3> -<p>You can now <code>start</code>/<code>enable</code> the <code>qbittorrent-nox@.service</code> using the service account created (<code>qbittorrent</code>):</p> -<pre><code class="language-sh">systemctl enable `qbittorrent-nox@qbittorrent.service -systemctl start `qbittorrent-nox@qbittorrent.service -</code></pre> -<p>This will start <code>qbittorrent</code> using default config. You need to change the port (in my case to <code>30000</code>) and set <code>qbittorrent</code> to restart on exit (the Web UI has a close button). I guess this can be done before enabling/starting the service, but either way let’s create a “drop-in” file by “editing” the service:</p> -<pre><code class="language-sh">systemctl edit `qbittorrent-nox@qbittorrent.service -</code></pre> -<p>Which will bring up a file editing mode containing the service unit and a space where you can add/override anything, add:</p> -<pre><code class="language-ini">[Service] -Environment="QBT_WEBUI_PORT=30000" # or whatever port number you want -Restart=on-success -RestartSec=5s -</code></pre> -<p>When exiting from the file (if you wrote anything) it will create the override config. Restart the service for changes to take effect (you might be asked to reload the systemd daemon):</p> -<pre><code class="language-sh">systemctl restart `qbittorrent-nox@qbittorrent.service -</code></pre> -<p>You can now head to <code>https://isos.yourdomain.com/qbt/</code> and login with user <code>admin</code> and password <code>adminadmin</code> (by default). <mark>Change the default password right away</mark> by going to <em>Tools -> Options -> Web UI -> Authentication</em>. The Web UI is basically the same as the normal desktop UI so if you’ve used it it will feel familiar. From here you can threat it as a normal torrent client and even start using for other stuff other than the specified here.</p> -<h4 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h4> -<p>It should be usable already but you can go further and fine tune it, specially to some kind of “convention” as shown in <a href="https://trash-guides.info/Downloaders/qBittorrent/Basic-Setup/">TRaSH: qBitTorrent basic setup</a> and subsequent <code>qbittorrent</code> guides.</p> -<p>I use all the suggested settings by <em>TRaSH</em>, where the only “changes” are for personal paths, ports, and in general connection settings that depend on my setup. The only super important setting I noticed that can affect our setup (meaning what is described in this entry) is the <em>Web UI -> Authentication</em> for the “Bypass authentication for clients on localhost”. This will be an issue because the reverse proxy is accessing <code>qbittorrent</code> via <code>localhost</code>, so this will make the service open to the world, experiment at your own risk.</p> -<p>Finally, add categories by following <a href="https://trash-guides.info/Downloaders/qBittorrent/How-to-add-categories/">TRaSH: qBitTorrent how to add categories</a>, basically right clicking on <em>Categories -> All (x)</em> (located at the left of the Web UI) and then on “Add category”; I use the same “Category” and “Save Path” (<code>tv</code> and <code>tv</code>, for example), where the “Save Path” will be a subdirectory of the configured global directory for torrents (<a href="https://trash-guides.info/Downloaders/qBittorrent/How-to-add-categories/#paths-and-categories-breakdown">TRaSH: qBitTorent paths and categories breakdown</a>). I added 3: <code>tv</code>, <code>movies</code> and <code>anime</code>.</p> -<h4 id="trackers">Trackers<a class="headerlink" href="#trackers" title="Permanent link">¶</a></h4> -<p>Often some of the trackers that come with torrents are dead or just don’t work. You have the option to add extra trackers to torrents either by:</p> -<ul> -<li>Automatically add a predefined list on new torrents: configure at <em>Tools -> Options -> BitTorrent</em>, enable the last option “Automatically add these trackers to new downloads” then add the list of trackers. This should only be done on public torrents as private ones might ban you or something.</li> -<li>Manually add a list of trackers to individual torrents: configure by selecting a torrent, clicking on <em>Trackers</em> on the bottom of the Web UI, right clicking on an empty space and selecting “Add trackers…” then add the list of trackers.</li> -</ul> -<p>On both options, the list of trackers need to have at least one new line in between each new tracker. You can find trackers from the following sources:</p> -<ul> -<li><a href="https://newtrackon.com/list">List of stable trackers</a></li> -<li><a href="https://github.com/ngosang/trackerslist">ngosang/trackerslist</a><ul> -<li>It also mentions <a href="https://github.com/ngosang/trackerslist#third-party-tools">Third party tools</a> to automate this process.</li> -</ul> -</li> -</ul> -<p>Both sources maintain an updated list of trackers. You also might need to enable an advanced option so all the new trackers are contacted (<a href="https://github.com/qbittorrent/qBittorrent/issues/7882">Only first tracker contacted</a>): configure at <em>Tools -> Options -> Advanced -> libtorrent Section</em> and enable both “Always announce to all tiers” and “Always announce to all trackers in a tier”.</p>]]></content:encoded> - </item> - <item> - <title>Configure system logs on Arch to avoid filled up disk</title> - <link>https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</guid> - <pubDate>Thu, 15 Jun 2023 10:22:20 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Short</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to configure the system logs, mostly journald, from filling up the disk, on Arch.</description> - <content:encoded><![CDATA[<p>It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.</p> -<p>Turns out that out of the box, Arch has a default config for <code>systemd</code>‘s <code>journald</code> that keeps a persistent <code>journal</code> log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal <code>journal</code> directory, <code>nginx</code> directory and my now unused <code>prosody</code> instance.</p> -<p>For <code>prosody</code> it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space. -For <code>journal</code> I did a combination of configuring <code>SystemMaxUse</code> and creating a <em>Namespace</em> for all “email” related services as mentioned in the <a href="https://wiki.archlinux.org/title/Systemd/Journal#Per_unit_size_limit_by_a_journal_namespace">Arch wiki: systemd/Journal</a>; basically just configuring <code>/etc/systemd/journald.conf</code> (and <code>/etc/systemd/journald@email.conf</code> with the comment change) with:</p> -<pre><code class="language-ini">[Journal] -Storage=persistent -SystemMaxUse=100MB # 50MB for the "email" Namespace -</code></pre> -<p>And then for each service that I want to use this “email” <em>Namespace</em> I add:</p> -<pre><code class="language-ini">[Service] -LogNamespace=email -</code></pre> -<p>Which can be changed manually or by executing <code>systemctl edit service_name.service</code> and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running <code>systemctl daemon-reload</code> and <code>systemctl restart service_name.service</code> (probably also restart <code>systemd-journald</code>).</p> -<p>I also disabled the logging for <code>ufw</code> by running <code>ufw logging off</code> as it logs everything to the <code>kernel</code> “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal <code>[UFW BLOCK]</code> log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.</p> -<p>Finally to clean up the <code>nginx</code> logs, you need to install <code>logrotate</code> (<code>pacman -S logrotate</code>) as that is what is used to clean up the <code>nginx</code> log directory. <code>nginx</code> already “installs” a config file for <code>logrotate</code> which is located at <code>/etc/logrotate.d/</code>, I just added a few lines:</p> -<pre><code>/var/log/nginx/*log { - rotate 7 - weekly - dateext - dateformat -%Y-%m-%d - missingok - notifempty - create 640 http log - sharedscripts - compress - postrotate - test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid` - endscript -} -</code></pre> -<p>Once you’re ok with your config, it’s just a matter of running <code>logrotate -v -f /etc/logrotate.d/nginx</code> which forces the run of the rule for <code>nginx</code>. After this, <code>logrotate</code> will be run daily if you <code>enable</code> the <code>logrotate</code> timer: <code>systemctl enable logrotate.timer</code>.</p>]]></content:encoded> - </item> - <item> - <title>Set up a manga server with Komga and mangal</title> - <link>https://blog.luevano.xyz/a/manga_server_with_komga.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/manga_server_with_komga.html</guid> - <pubDate>Sat, 10 Jun 2023 19:36:07 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a manga server with Komga as media server and mangal for downloading manga, on Arch. Tachiyomi integration is available thanks to Komga.</description> - <content:encoded><![CDATA[<p>I’ve been wanting to set up a manga media server to hoard some mangas/comics and access them via Tachiyomi, but I didn’t have enough space in my vultr VPS. Now that I have symmetric fiber optic at home and my spare PC to use as a server I decided to go ahead and create one. As always, <mark>i use arch btw</mark> so these instructions are specifically for it, I’m not sure how easier/harder it is for other distros, I’m just too comfortable with arch honestly.</p> -<p>I’m going to run it as an exposed service using a subdomain of my own, so the steps are taking that into account, if you want to run it locally (or on a LAN/VPN) then it is going to be easier/with less steps (you’re on your own). Also, as you might notice I don’t like to use D*ck*r images or anything (ew).</p> -<p><mark>At the time of editing this entry (06-28-2023) Komga has already upgraded to <code>v.1.0.0</code> and it introduces some breaking changes if you already had your instance set up. Read more <a href="https://komga.org/installation/upgrade.html#prepare-for-v1-0-0">here</a>.</mark> The only change I did here was changing the port to the new default.</p> -<p>As always, all commands are run as root unless stated otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#yay">yay</a><ul> -<li><a href="#install">Install</a></li> -<li><a href="#usage">Usage</a></li> -</ul> -</li> -<li><a href="#mangal">mangal</a><ul> -<li><a href="#install-from-source">Install from source</a></li> -<li><a href="#configuration">Configuration</a></li> -<li><a href="#usage_1">Usage</a><ul> -<li><a href="#headless-browser">Headless browser</a></li> -<li><a href="#tui">TUI</a></li> -<li><a href="#inline">Inline</a></li> -<li><a href="#automation">Automation</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#komga">Komga</a><ul> -<li><a href="#reverse-proxy">Reverse proxy</a><ul> -<li><a href="#ssl-certificate">SSL certificate</a></li> -</ul> -</li> -<li><a href="#start-using-komga">Start using Komga</a></li> -<li><a href="#library-creation">Library creation</a><ul> -<li><a href="#set-default-directory-permissions">Set default directory permissions</a></li> -<li><a href="#populate-manga-library">Populate manga library</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#alternative-downloaders">Alternative downloaders</a></li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>Similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries, if you want it as a subdomain:</p> -<ul> -<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>komga</code> (or whatever you want).</li> -<li>An SSL certificate, if you’re following the other entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> entry), add a <code>komga.conf</code> and run <code>certbot --nginx</code> (or similar) to extend/create the certificate. More details below: <a href="#reverse-proxy">Reverse proxy</a> and <a href="#ssl-certificate">SSL certificate</a>.</li> -</ul> -<h2 id="yay">yay<a class="headerlink" href="#yay" title="Permanent link">¶</a></h2> -<p>This is the first time I mention the <strong>AUR</strong> (and <code>yay</code>) in my entries, so I might as well just write a bit about it.</p> -<p>The <a href="https://aur.archlinux.org/">AUR</a> is the <strong>A</strong>rch Linux <strong>U</strong>ser <strong>R</strong>epository and it’s basically like an extension of the official one which is supported by the community, the only thing is that it requires a different package manager. The one I use (and I think everyone does, too) is <code>yay</code>, which as far as I know is like a wrapper of <code>pacman</code>.</p> -<h3 id="install">Install<a class="headerlink" href="#install" title="Permanent link">¶</a></h3> -<p>To install and use <code>yay</code> we need a normal account with sudo access, <mark>all the commands related to <code>yay</code> are run as normal user and then it asks for sudo password</mark>. <a href="https://github.com/Jguer/yay#installation">Installation</a> its straight forward: clone <code>yay</code> repo and install. Only dependencies are <code>git</code> and <code>base-devel</code>:</p> -<p>Install dependencies:</p> -<pre><code class="language-sh">sudo pacman -S git base-devel -</code></pre> -<p>Clone <code>yay</code> and install it (I also like to delete the cloned git repo):</p> -<pre><code class="language-sh">git clone git@github.com:Jguer/yay.git -cd yay -makepkg -si -cd .. -sudo rm -r yay -</code></pre> -<h3 id="usage">Usage<a class="headerlink" href="#usage" title="Permanent link">¶</a></h3> -<p><code>yay</code> is used basically the same as <code>pacman</code> with the difference that it is run as normal user (then later requiring sudo password) and that it asks extra input when installing something, such as if we want to build the package from source or if we want to show package diffs.</p> -<p>To install a package (for example Komga in this blog entry), run:</p> -<pre><code class="language-sh">yay -S komga -</code></pre> -<h2 id="mangal">mangal<a class="headerlink" href="#mangal" title="Permanent link">¶</a></h2> -<p><a href="https://github.com/metafates/mangal">mangal</a> is a CLI/TUI manga downloader with anilist integration and custom Lua scrapers.</p> -<p>You could install it from the AUR with <code>yay</code>:</p> -<pre><code class="language-sh">yay -S mangal-bin -</code></pre> -<p>But I’ll use my <a href="https://github.com/luevano/mangal">fork</a> as it contains some fixes and extra stuff.</p> -<h3 id="install-from-source">Install from source<a class="headerlink" href="#install-from-source" title="Permanent link">¶</a></h3> -<p>As I mentioned in my past <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a> I had to <a href="https://github.com/luevano/mangal">fork</a> <code>mangal</code> and related repositories to fix/change a few things. Currently the major fix I did in <code>mangal</code> is for the built in <a href="https://mangadex.org/">MangaDex</a> scraper which had really annoying bug in the chunking of the manga chapter listing.</p> -<p>So instad of installing with <code>yay</code> we’ll build it from source. We need to have <code>go</code> installed:</p> -<pre><code class="language-sh">pacman -S go -</code></pre> -<p>Then clone my fork of <code>mangal</code> and <code>install</code> it:</p> -<pre><code class="language-sh">git clone https://github.com/luevano/mangal.git # not sure if you can use SSH to clone -cd mangal -make install # or just `make build` and then move the binary to somewhere in your $PATH -</code></pre> -<p>This will use <code>go install</code> so it will install to a path specified by the <code>go</code> environment variables, for more run <code>go help install</code>. It was installed to <code>$HOME/.local/bin/go/mangal</code> for me because my env vars, then just make sure this is included in your <code>PATH</code>.</p> -<p>Check it was correctly installed by running <code>mangal version</code>, which should print something like:</p> -<pre><code>▇▇▇ mangal - - Version ... - Git Commit ... - Build Date ... - Built By ... - Platform ... -</code></pre> -<h3 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h3> -<p>I’m going to do everything with a normal user (<code>manga-dl</code>) which I created just to download manga. So all of the commands will be run without sudo/root privileges.</p> -<p>Change some of the configuration options:</p> -<pre><code class="language-sh">mangal config set -k downloader.path -v "/mnt/d/mangal" # downloads to current dir by default -mangal config set -k formats.use -v "cbz" # downloads as pdf by default -mangal config set -k installer.user -v "luevano" # points to my scrapers repository which contains a few extra scrapers and fixes, defaults to metafates' one; this is important if you're using my fork, don't use otherwise as it uses extra stuff I added -mangal config set -k logs.write -v true # I like to get logs for what happens -</code></pre> -<p><strong>Note</strong>: For testing purposes (if you want to explore <code>mangal</code>) set <code>downloader.path</code> once you’re ready to start to populate the Komga library directory (at <a href="#populate-manga-library">Komga: populate manga library</a>).</p> -<p>For more configs and to read what they’re for:</p> -<pre><code class="language-sh">mangal config info -</code></pre> -<p>Also install the custom Lua scrapers by running:</p> -<pre><code class="language-sh">mangal sources install -</code></pre> -<p>And install whatever you want, it picks up the sources/scrapers from the configured repository (<code>installer.<key></code> config), if you followed, it will show my scrapers.</p> -<h3 id="usage_1">Usage<a class="headerlink" href="#usage_1" title="Permanent link">¶</a></h3> -<p>Two main ways of using <code>mangal</code>: </p> -<ul> -<li><strong>TUI</strong>: for initial browsing/downloading and testing things out. If the manga finished publishing, this should be enough.</li> -<li><strong>inline</strong>: for automation on manga that is still publishing and I need to check/download every once in a while.</li> -</ul> -<h4 id="headless-browser">Headless browser<a class="headerlink" href="#headless-browser" title="Permanent link">¶</a></h4> -<p>Before continuing, I gotta say I went through some bullshit while trying to use the custom Lua scrapers that use the <em>headless</em> browser (actually just a wrapper of <a href="https://github.com/go-rod/rod">go-rod/rod</a>, and honestly it is not really a “headless” browser, <code>mangal</code> “documentation” is just wrong). For more on my rant check out my last <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a>.</p> -<p>There is no concrete documentation on the “headless” browser, only that it is automatically set up and ready to use… but it doesn’t install any library/dependency needed. I discovered the following libraries that were missing on my Arch minimal install:</p> -<ul> -<li>library -> arch package containing it</li> -<li>libnss3.so -> nss</li> -<li>libatk-1.0.so.0 -> at-spi2-core</li> -<li>libcups.so.2 -> libcups</li> -<li>libdrm.so.2 -> libdrm</li> -<li>libXcomposite.so.1 -> libxcomposite</li> -<li>libXdamage.so.1 -> libxdamage</li> -<li>libXrandr.so.2 -> libxrandr</li> -<li>libgbm.so.1 -> mesa</li> -<li>libxkbcommon.so.0 -> libxkbcommon</li> -<li>libpango-1.0.so.0 -> pango</li> -<li>libasound.so.2 -> alsa-lib</li> -</ul> -<p>To install them::</p> -<pre><code class="language-sh">pacman -S nss at-spi2-core libcups libdrm libxcomposite libxdamage libxrandr mesa libxkbcommon pango alsa-lib -</code></pre> -<p>I can’t guarantee that those are all the packages needed, those are the ones I happen to discover (had to <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the lua libs and add some logging because the error message was too fucking generic).</p> -<p>These dependencies are probably met by installing either <code>chromedriver</code> or <code>google-chrome</code> from the AUR (for what I could see on the package dependencies).</p> -<h4 id="tui">TUI<a class="headerlink" href="#tui" title="Permanent link">¶</a></h4> -<p>Use the TUI by running</p> -<pre><code class="language-sh">mangal -</code></pre> -<p>Download manga using the TUI by selecting the source/scrapper, search the manga/comic you want and then you can select each chapter to download (use <code>tab</code> to select all). This is what I use when downloading manga that already finished publishing, or when I’m just searching and testing out how it downloads the manga (directory name, and manga information).</p> -<p>Note that some scrapters will contain duplicated chapters, as they have multiple uploaded chapters from the community, usually for different <em>scanlation groups</em>. This happens a lot with <a href="https://mangadex.org/">MangaDex</a>.</p> -<h4 id="inline">Inline<a class="headerlink" href="#inline" title="Permanent link">¶</a></h4> -<p>The inline mode is a single terminal command meant to be used to automate stuff or for more advanced options. You can peek a bit into the “<a href="https://github.com/metafates/mangal/wiki/Inline-mode#command-examples">documentation</a>” which honestly it’s ass because it doesn’t explain much. The minimal command for inline according to the <code>mangal help</code> is:</p> -<pre><code class="language-sh">mangal inline --manga <option> --query <manga-title> -</code></pre> -<p>But this will not produce anything because it also needs <code>--source</code> (or set the default using the config key <code>downloader.default_sources</code>) and either <code>--json</code> which basically just does the search and returns the result in <code>json</code> format or <code>--download</code> to actually download whatever is found; I recommend to do <code>--json</code> first to check that the correct manga will be downloaded then do <code>--download</code>.</p> -<p>Something not mentioned anywhere is the <code>--manga</code> flag options (found it at the source code), it has 3 available options:</p> -<ul> -<li><code>first</code>: first manga entry found for the search.</li> -<li><code>last</code>: last manga entry found for the search.</li> -<li><code>exact</code>: exact manga title match. This is the one I use.</li> -</ul> -<p>Similar to <code>--chapters</code>, there are a few options not explained (that I found at the source code, too). I usually just use <code>all</code> but other options:</p> -<ul> -<li><code>all</code>: all chapters found in the chapter list.</li> -<li><code>first</code>: first chapter found in the chapter list.</li> -<li><code>last</code>: last chapter found in the chapter list</li> -<li><code>[from]-[to]</code>: selector for the chapters found in the chapter list, index starts at 0.<ul> -<li>If the selectors (<code>from</code> or <code>to</code>) exceed the amount of chapters in the chapterlist it just adjusts to he maximum available.</li> -<li>I had to fix this at the source code because if you wanted <code>to</code> to be the last chapter, it did <code>to + 1</code> and it failed due to index out of range.</li> -</ul> -</li> -<li><code>@[sub]@</code>: not sure how this works exactly, my understanding is that it’s for “named” chapters.</li> -</ul> -<p>That said, I’ll do an example by using <a href="https://mangapill.com">Mangapill</a> as source, and will search for <a href="https://mangapill.com/manga/2285/kimetsu-no-yaiba">Demon Slayer: Kimetsu no Yaiba</a>:</p> -<ol> -<li>Search first and make sure my command will pull the manga I want:</li> -</ol> -<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --json | jq # I use jq to pretty format the output -</code></pre> -<ol start="2"> -<li>I make sure the json output contains the correct manga information: name, url, etc..</li> -</ol> -<ul> -<li>You can also include the flag <code>--include-anilist-manga</code> to include anilist information (if any) so you can check that the correct anilist id is attached. If the correct one is not attached (and it exists) then you can bind the <code>--query</code> (search term) to a specific anilist id by running:</li> -</ul> -<pre><code class="language-sh">mangal inline anilist set --name "Kimetsu no Yaiba" --id 101922 -</code></pre> -<ol start="3"> -<li>If I’m okay with the outputs, then I change <code>--json</code> for <code>--download</code> to actually download:</li> -</ol> -<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --download -</code></pre> -<ol start="4"> -<li>Check if the manga is downloaded correctly. I do this by going to my download directory and checking the directory name (I’m picky with this stuff), that all chapters where downloaded, that it includes a correct <code>series.json</code> file and it contains a <code>cover.<img-ext></code>; this usually means it correctly pulled information from anilist and that it will contain metadata Komga will be able to use.</li> -</ol> -<h4 id="automation">Automation<a class="headerlink" href="#automation" title="Permanent link">¶</a></h4> -<p>The straight forward approach for automation is just to bundle a bunch of <code>mangal inline</code> commands in a shell script and schedule it’s execution either via <a href="https://wiki.archlinux.org/title/cron">cron</a> or <a href="https://wiki.archlinux.org/title/systemd/Timers">systemd/Timers</a>. But, as always, I overcomplicated/overengineered my approach, which is the following:</p> -<ol> -<li>Group manga names per source.</li> -<li>Configure anything that should always be set before executing <code>mangal</code>, this includes anilist bindings.</li> -<li>Have a way to track the changes/updates on each run.</li> -<li>Use that tracker to know where to start downloading chapters from.<ul> -<li>This is optional, as you can just do <code>--chapters "all"</code> and it will work but I do it mostly to keep the logs/output cleaner/shorter.</li> -</ul> -</li> -<li>Download/update each manga using <code>mangal inline</code>.</li> -<li>Wrap everything in a <code>systemd</code> <code>service</code> and <code>timer</code>.</li> -</ol> -<p>Manga list example:</p> -<pre><code class="language-sh">mangapill="Berserk|Chainsaw Man|Dandadan|Jujutsu Kaisen|etc..." -</code></pre> -<p>Function that handles the download per manga in the list:</p> -<pre><code class="language-sh">mangal_src_dl () { - source_name=$1 - manga_list=$(echo "$2" | tr '|' '\n') - - while IFS= read -r line; do - # By default download all chapters - chapters="all" - last_chapter_n=$(grep -e "$line" "$TRACKER_FILE" | cut -d'|' -f2 | grep -v -e '^$' | tail -n 1) - if [ -n "${last_chapter_n}" ]; then - chapters="$last_chapter_n-9999" - echo "Downloading [${last_chapter_n}-] chapters for $line from $source_name..." - else - echo "Downloading all chapters for $line from $source_name..." - fi - dl_output=$(mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "$chapters" -d) - - if [ $? -ne 0 ]; then - echo "Failed to download chapters for $line." - continue - fi - - line_count=$(echo "$dl_output" | grep -v -e '^$' | wc -l) - if [ $line_count -gt 0 ]; then - echo "Downloaded $line_count chapters for $line:" - echo "$dl_output" - new_last_chapter_n=$(echo "$dl_output" | tail -n 1 | cut -d'[' -f2 | cut -d']' -f1) - # manga_name|last_chapter_number|downloaded_chapters_on_this_update|manga_source - echo "$line|$new_last_chapter_n|$line_count|$source_name" >> $TRACKER_FILE - else - echo "No new chapters for $line." - fi - done <<< "$manga_list" -} -</code></pre> -<p>Where <code>$TRACKER_FILE</code> is just a variable holding a path to some file where you can store the tracking and <code>$DOWNLOAD_FORMAT</code> the format for the mangas, for me it’s <code>cbz</code>. Then the usage would be something like <code>mangal_src_dl "Mangapill" "$mangapill"</code>, meaning that it is a function call per source.</p> -<p>A simpler function without “tracking” would be:</p> -<pre><code class="language-sh">mangal_src_dl () { - source_name=$1 - manga_list=$(echo "$2" | tr '|' '\n') - - while IFS= read -r line; do - echo "Downloading all chapters for $line from $source_name..." - mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "all" -d - if [ $? -ne 0 ]; then - echo "Failed to download chapters for $line." - continue - fi - echo "Finished downloading chapters for $line." - done <<< "$manga_list" -} -</code></pre> -<p>The tracker file would have a format like follows:</p> -<pre><code># Updated: 06/10/23 10:53:15 AM CST -Berserk|0392|392|Mangapill -Dandadan|0110|110|Mangapill -... -</code></pre> -<p>And note that if you already had manga downloaded and you run the script for the first time, then it will show as if it downloaded everything from the first chapter, but that’s just how <code>mangal</code> works, it will actually just discover downloaded chapters and only download anything missing.</p> -<p>Any configuration the downloader/updater might need needs to be done before the <code>mangal_src_dl</code> calls. I like to configure mangal for download path, format, etc.. I found that it is needed to clear the <code>mangal</code> and <code>rod</code> browser cache (headless browser used in some custom sources) from personal experience and from others: <a href="https://github.com/metafates/mangal/issues/170">mangal#170</a> and <a href="https://github.com/oae/kaizoku/issues/89">kaizoku#89</a>.</p> -<p>Also you should set any anilist binding necessary for the downloading (as the cache was cleared). An example of an anilist binding I had to do is for Mushoku Tensei, as it has both a <a href="https://anilist.co/manga/85470/Mushoku-Tensei-Jobless-Reincarnation/">light novel</a> and <a href="https://anilist.co/manga/85564/Mushoku-Tensei-Jobless-Reincarnation/">manga</a> version, which for me it’s the following binding:</p> -<pre><code class="language-sh">mangal inline anilist set --name "Mushoku Tensei - Isekai Ittara Honki Dasu" --id 85564 -</code></pre> -<p>Finally is just a matter of using your prefered way of scheduling, I’ll use <code>systemd/Timers</code> but anything is fine. You could make the downloader script more sophisticated and only running every week on which each manga gets (usually) released but that’s too much work; I’ll just run it once daily probably.</p> -<p>A feature I want to add and probably will is sending notifications (probably through email) on a summary for manga downloaded or failed to download so I’m on top of the updates. For now this is good enough and it’s been working so far.</p> -<h2 id="komga">Komga<a class="headerlink" href="#komga" title="Permanent link">¶</a></h2> -<p><a href="https://komga.org/">Komga</a> is a comics/mangas media server.</p> -<p>Install from the AUR:</p> -<pre><code class="language-sh">yay -S komga -</code></pre> -<p>This <code>komga</code> package creates a <code>komga</code> (service) user and group which is tied to the also included <code>komga.service</code>.</p> -<p>Configure it by editing <code>/etc/komga.conf</code>:</p> -<pre><code class="language-sh">SERVER_PORT=25600 -SERVER_SERVLET_CONTEXT_PATH=/ # this depends a lot of how it's going to be served (domain, subdomain, ip, etc) - -KOMGA_LIBRARIES_SCAN_CRON="0 0 * * * ?" -KOMGA_LIBRARIES_SCAN_STARTUP=false -KOMGA_LIBRARIES_SCAN_DIRECTORY_EXCLUSIONS='#recycle,@eaDir,@Recycle' -KOMGA_FILESYSTEM_SCANNER_FORCE_DIRECTORY_MODIFIED_TIME=false -KOMGA_REMEMBERME_KEY=USE-WHATEVER-YOU-WANT-HERE -KOMGA_REMEMBERME_VALIDITY=2419200 - -KOMGA_DATABASE_BACKUP_ENABLED=true -KOMGA_DATABASE_BACKUP_STARTUP=true -KOMGA_DATABASE_BACKUP_SCHEDULE="0 0 */8 * * ?" -</code></pre> -<p>My changes (shown above):</p> -<ul> -<li><code>cron</code> schedules.<ul> -<li>It’s not actually <code>cron</code> but rather a <code>cron</code>-like syntax used by <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronSequenceGenerator.html">Spring</a> as stated in the <a href="https://komga.org/installation/configuration.html#optional-configuration">Komga config</a>.</li> -</ul> -</li> -<li>Added the remember me key.</li> -<li>For more check out <a href="https://komga.org/installation/configuration.html">Komga: Configuration options</a>.</li> -</ul> -<p>If you’re going to run it locally (or LAN/VPN) you can start the <code>komga.service</code> and access it via IP at <code>http://<your-server-ip>:<port>(/base_url)</code> as stated at <a href="https://komga.org/installation/webui.html">Komga: Accessing the web interface</a>, then you can continue with the <a href="#mangal">mangal</a> section, else continue with the next steps for the reverse proxy and certificate.</p> -<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3> -<p>Create the reverse proxy configuration (this is for <code>nginx</code>). In my case I’ll use a subdomain, so this is a new config called <code>komga.conf</code> at the usual <code>sites-available/enabled</code> path:</p> -<pre><code class="language-nginx">server { - listen 80; - server_name komga.yourdomain.com; # change accordingly to your wanted subdomain and domain name - - location / { - proxy_pass http://localhost:25600; # change port if needed - proxy_http_version 1.1; - - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - - proxy_read_timeout 600s; - proxy_send_timeout 600s; - } -} -</code></pre> -<p>If it’s going to be used as a subdir on another domain then just change the <code>location</code> with <code>/subdir</code> instead of <code>/</code>; be careful with the <code>proxy_pass</code> directive, it has to match what you configured at <code>/etc/komga.conf</code> for the <code>SERVER_SERVLET_CONTEXT_PATH</code> regardless of the <code>/subdir</code> you selected at <code>location</code>.</p> -<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4> -<p>If using a subdir then the same certificate for the subdomain/domain should work fine and no extra stuff is needed, else if following along me then we can create/extend the certificate by running:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>That will automatically detect the new subdomain config and create/extend your existing certificate(s). In my case I manage each certificate’s subdomain:</p> -<pre><code class="language-sh">certbot --nginx -d domainname.com -d subdomain.domainname.com -d komga.domainname.com -</code></pre> -<h3 id="start-using-komga">Start using Komga<a class="headerlink" href="#start-using-komga" title="Permanent link">¶</a></h3> -<p>We can now <code>start</code>/<code>enable</code> the <code>komga.service</code>:</p> -<pre><code class="language-sh">systemctl enable komga.service -systemctl start komga.service -</code></pre> -<p>And access the web interface at <code>https://komga.domainname.com</code> which should show the login page for Komga. The first time it will ask to create an account as shown in <a href="https://komga.org/installation/webui.html#create-user-account">Komga: Create user account</a>, this will be an admin account. Fill in the email and password (can be changed later). The email doesn’t have to be an actual email, for now it’s just for management purposes.</p> -<p>Next thing would be to add any extra account (for read-only/download manga permissions), add/import libraries, etc.. For now I’ll leave it here until we start downloading manga on the next steps.</p> -<h3 id="library-creation">Library creation<a class="headerlink" href="#library-creation" title="Permanent link">¶</a></h3> -<p>Creating a library is as simple as creating a directory somewhere and point to it in Komga. The following examples are for my use case, change accordingly. I’ll be using <code>/mnt/d/mangal</code> for my library (as stated in the <a href="#configuration">mangal: configuration</a> section):</p> -<pre><code class="language-sh">mkdir /mnt/d/mangal -</code></pre> -<p>Where I chose the name <code>mangal</code> as its the name of the downloader/scrapper, it could be anything, this is just how I like to organize stuff.</p> -<p>For the most part, the permissions don’t matter much (as long as it’s readable by the <code>komga</code> user) unless you want to delete some manga, then <code>komga</code> user also needs write permissions.</p> -<p>Then just create the library in Komga web interface (the <code>+</code> sign next to <em>Libraries</em>), choose a name <em>“Mangal”</em> and point to the root folder <code>/mnt/d/mangal</code>, then just click <em>Next</em>, <em>Next</em> and <em>Add</em> for the defaults (that’s how I’ve been using it so far). This is well explained at <a href="https://komga.org/guides/libraries.html">Komga: Libraries</a>.</p> -<p>The real important part (for me) is the permissions of the <code>/mnt/d/mangal</code> directory, as I want to have write access for <code>komga</code> so I can manage from the web interface itself. It looks like it’s just a matter of giving ownership to the <code>komga</code> user either for owner or for group (or to all for that matter), but since I’m going to use a separate user to download manga then I need to choose carefully.</p> -<h4 id="set-default-directory-permissions">Set default directory permissions<a class="headerlink" href="#set-default-directory-permissions" title="Permanent link">¶</a></h4> -<p>The desired behaviour is: set <code>komga</code> as group ownership, set write access to group and whenever a new directory/file is created, inherit these permission settings. I found out via <a href="https://unix.stackexchange.com/a/1315">this</a> stack exchange answer how to do it. So, for me:</p> -<pre><code class="language-sh">chown manga-dl:komga /mnt/d/mangal # required for group ownership for komga -chmod g+s /mnt/d/mangal # required for group permission inheritance -setfacl -d -m g::rwx /mnt/d/mangal # default permissions for group -setfacl -d -m o::rx /mnt/d/mangal # default permissions for other (as normal, I think this command can be excluded) -</code></pre> -<p>Where <code>manga-dl</code> is the user I created to download manga with. Optionally add <code>-R</code> flag to those 4 commands in case it already has subdirectories/files (this might mess file permissions, but it’s not an issue as far as I konw).</p> -<p>Checking that the permissions are set correctly (<code>getfacl /mnt/d/mangal</code>):</p> -<pre><code>getfacl: Removing leading '/' from absolute path names -# file: mnt/d/mangal -# owner: manga-dl -# group: komga -# flags: -s- -user::rwx -group::rwx -other::r-x -default:user::rwx -default:group::rwx -default:other::r-x -</code></pre> -<p>You can then check by creating a new subdirectory (in <code>/mnt/d/mangal</code>) and it should have the same group permissions.</p> -<h4 id="populate-manga-library">Populate manga library<a class="headerlink" href="#populate-manga-library" title="Permanent link">¶</a></h4> -<p>You can now start downloading manga using <code>mangal</code> either manually or by running the <code>cron</code>/<code>systemd/Timers</code> and it will be detected by Komga automatically when it scans the library (once every hour according to my config). You can manually scan the library, though, by clicking on the 3 dots to the right of the library name (in Komga) and click on “Scan library files”.</p> -<p>Then you can check that the metadata is correct (once the manga is fully indexed and metadata finished building), such as title, summary, chapter count, language, tags, genre, etc., which honestly it never works fine as <code>mangal</code> creates the <code>series.json</code> with the <code>comicId</code> field with an upper case <code>I</code> and Komga expects it to be a lower case <code>i</code> (<code>comicid</code>) so it falls back to using the info from the first chapter. I’ll probably will fix this on <code>mangal</code> side, and see how it goes.</p> -<p>So, what I do is manually edit the metadata for the manga, by changing whatever it’s wrong or add what’s missing (I like adding anilist and MyAnimeList links) and then leave it as is. This is up to you.</p> -<h2 id="alternative-downloaders">Alternative downloaders<a class="headerlink" href="#alternative-downloaders" title="Permanent link">¶</a></h2> -<p>Just for the record, here is a list of downloaders/scrapers I considered before starting to use <code>mangal</code>:</p> -<ul> -<li><a href="https://github.com/oae/kaizoku">kaizoku</a>: NodeJS web server that uses <code>mangal</code> for its “backend” and honestly since I liked <code>mangal</code> so much I should use it, the only reason I don’t is because I’m a bitch and I don’t want to use a D*ck*r image and NodeJS (ew) (in general is pretty bloated in my opinion). If I get tired of my solution with pure <code>mangal</code> I might as well just migrate to it as It’s a more automatic solution.</li> -<li><a href="https://github.com/manga-py/manga-py">manga-py</a>: Python CLI application that’s a really good option as far as I’ve explored it, I’m just not using it yet as <code>mangal</code> has been really smooth and has everything I need, but will definitely explore it in the future if I need to. The cool thing out of the box is the amount of sources it can scrape from (somethign lacking from <code>mangal</code>).</li> -<li><a href="https://github.com/mylar3/mylar3">mylar3</a>: Python web server that should be the easier way to download manga with once correctly set up, but I guess I’m too dumb and don’t know how to configure it. Looks like you need to have access to specific private torrent trackers or whatever the other ways to download are, I just couldn’t figure out how to set it up and for public torrent stuff everything will be all over the place, so this was no option for me at the end.</li> -</ul> -<p>Others:</p> -<ul> -<li><a href="https://hakuneko.download/">HakuNeku</a>: It looks pretty easy to use and future rich, only thing is that it’s not designed for headless servers, just a normal app. So this is also not an option for me. You could use it on your computer and <code>rsync</code> to your server or use some other means to upload to your server (a <em>nono</em> for me).</li> -<li><a href="https://github.com/riderkick/FMD">FMD</a>: No fucking idea on how to use it and it’s not been updated since 2019, just listing it here as an option if it interests you.</li> -</ul>]]></content:encoded> - </item> - <item> - <title>Updated the how-to entries titles</title> - <link>https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html</guid> - <pubDate>Sat, 03 Jun 2023 03:46:44 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Update</category> - <description>Just a small update on the title for some old entries.</description> - <content:encoded><![CDATA[<p>One of the main reasons I started “blogging” was basically just to document how I set up stuff up so I can reference them later in the future if I ever needed to replicate the steps or just to show somebody, and these entries had helped to do so multiple times. I’ll keep creating these entries but after a while the <em>Creating a</em> title started to feel weird, because we’re not <em>creating</em> anything really, it is just a set up/configuration/how-to/etc. So I think that using <em>Set up a</em> for the titles is better and makes more sense; probably using <em>How to set up a</em> is better for the SEO bullshit.</p> -<p>Anyways, so I’ll start using <em>Set up a</em> instead of <em>Creating a</em> and will retroactively change the titles for these entries (by this entry the change should be applied already). This might impact some RSS feeds as they keep up a cache of the feed and might duplicate the entries, heads up if for some reason somebody is using it.</p>]]></content:encoded> - </item> - <item> - <title>I had to learn Go and Lua the hard way</title> - <link>https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html</guid> - <pubDate>Sat, 03 Jun 2023 03:32:17 GMT</pubDate> - <category>English</category> - <category>Rant</category> - <category>Short</category> - <category>Tools</category> - <description>Thanks to the issues of a program (mangal) I'm starting to use for my manga media server, I had to learn Go and Lua the hard way so that I can fix it and use it.</description> - <content:encoded><![CDATA[<p><mark>TL;DR</mark>: I learned Go and Lua the hard way by forking (for fixing):</p> -<ul> -<li><a href="https://github.com/luevano/mangal">mangal</a>: main manga scraper written in Go.</li> -<li><a href="https://github.com/luevano/mangal-lua-libs">mangal-lua-libs</a>: <a href="https://github.com/yuin/gopher-lua">gopher-lua</a> libraries for mangal.</li> -<li><a href="https://github.com/luevano/mangal-scrapers">mangal-scrapers</a>: custom mangal scrapers written in Lua.</li> -</ul> -<p>In the last couple of days I’ve been setting up a <a href="https://komga.org/">Komga</a> server for manga downloaded using <a href="https://github.com/metafates/mangal">metafates/mangal</a> (upcoming set up entry about it) and everything was fine so far until I tried to download One Piece from <a href="https://mangadex.org/">MangaDex</a> of which <code>mangal</code> has a built-in scraper. Long story short the issue was that MangaDex’s API only allows requesting manga chapters on chunks of 500 and the way that was being handled was completely wrong, specifics can be found on my <a href="https://github.com/luevano/mangal/commit/395bce96e439ee828d0180328a5cf9204bfd818a">commit</a> (and the subsequent minor fix <a href="https://github.com/luevano/mangal/commit/6bf709fe9b333ec9d4375ed80f9b055d07a40c1c">commit</a>).</p> -<p>I tried to do a PR, but the project hasn’t been active since Feb 2023 (same reason I didn’t even try to do PRs on the other repos) so I closed it and will start working on my own <a href="https://github.com/luevano/mangal">fork</a>, probaly just merging everything <a href="https://github.com/Belphemur">Belphemur</a>‘s <a href="https://github.com/Belphemur/mangal">fork</a> has to offer, as he’s been working on <code>mangal</code> actively. I could probably just fork from him and/or just submit PR requests to him, but I think I saw some changes I didn’t really like, will have to look more into it.</p> -<p>Also, while trying to use some of the custom scrapers I ran into issues with the headless chrome explorer implementation where it didn’t close on each manga chapter download, causig my CPU and Mem usage to get maxed out and losing control of the system, so I had to also <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the <a href="https://github.com/metafates/mangal-lua-libs">metafates/mangal-lua-libs</a> and “fixed” (I say fixed because that wasn’t the issue at the end, it was how the custom scrapers where using it, shitty documentation) the issue by adding the <code>browser.Close()</code> function to the <code>headless</code> Lua API (<a href="https://github.com/luevano/mangal-lua-libs/commit/97fba97ab23efe88278dfacbeed2dd83c5472de0">commit</a>) and merged some commits from the original <a href="https://github.com/vadv/gopher-lua-libs">vadv/gopher-lua-libs</a> just to include any features added to the Lua libs needed.</p> -<p>Finally I <a href="https://github.com/luevano/mangal-scrapers">forked</a> the <a href="https://github.com/metafates/mangal-scrapers">metafates/mangal-scrapers</a> (which I actually forked <a href="https://github.com/NotPhantomX">NotPhantomX</a>‘s <a href="https://github.com/NotPhantomX/mangal-scrapers">fork</a> as they had included more scrapers from some pull requests) to be able to have updated custom Lua scrapers (in which I also fixed the <code>headless</code> bullshit) and use them on my <code>mangal</code>.</p> -<p>So, I went into the rabbit hole of manga scrapping because I wanted to set up my Komga server, and more importantly I had to quickly learn Go and Lua (Lua was easier) and I have to say that Go is super convoluted on the module management, all research I did lead me to totally different responses, but it is just because of different Go versions and the year of the responses.</p>]]></content:encoded> - </item> - <item> - <title>Al fin tengo fibra ópticona</title> - <link>https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html</guid> - <pubDate>Tue, 09 May 2023 08:59:00 GMT</pubDate> - <category>Rant</category> - <category>Short</category> - <category>Spanish</category> - <category>Update</category> - <description>Por fin pude contratar fibra óptica simétrica y ya no sufro con el cobre de cierta compañía horrible.</description> - <content:encoded><![CDATA[<p>Quienes me conocen sabrán que llevo como 2 años intentando contratar internet de fibra óptica (específicamente el de T*lm*x). El problema es que nunca había <em>nodos/terminales</em> disponibles o, la verdad, que los técnicos ni querían hacer su jale porque están acostumbrados a que les debes soltar una feria para que te la instalen.</p> -<p>Pues bueno, el punto es que me tocó estar aguantando la compañía horrible de *zz*, que sólo tiene cobre; el servicio es malo y a cada rato le suben de precio. Por esto último volví a checar precios de otras compañías para comparar y resulta que me estaban cobrando como $100 - $150 pesos extra con el mismo paquete que ya tenía/tengo. Hasta ahí estaba encabronado, y no ayudó nada que intenté hablar con los muy incompetentes de <em>soporte</em> y no pudieron digamos “resolverme”, porque ¿cómo es posible que siendo cliente de como 5 años ni si quiera pueden avisarme que ya tienen mejores paquetes (que la neta es el mismo paquete pero más barato)?</p> -<p>Intenté pedirles que me cambien al paquete actual (mismo todo, única diferencia el precio), pero resulta que me meterían a plazo forzoso. Obviamente esto me prendió un cuete en la cola y por eso chequé con T*lm*x, que a mi sorpresa salía que sí había fibra óptica disponible en mi cantón. Inicié el proceso de portabilidad y me dijeron que en como dos semanas me la instalaban, pero resulta que el basado del técnico me marcó al día siguiente para decirme que <mark>YA ESTABA AFUERA DE MI CASA</mark> para instalarlo. Gané.</p> -<p>Resulta que ahora sí hay <em>nodos/terminales</em>, de hecho instalaron 3 nuevos y están completamente vacíos, me tocó muy buena suerte y el muy basado del técnico se lo aventó en medio segundo sin ningún pedo, no me pidió nada más que detalles de dónde quería el módem. No tenía efectivo si no le soltaba un varo, se portó muy chingón.</p>]]></content:encoded> - </item> - <item> - <title>Updated pyssg to include pymdvar and the website</title> - <link>https://blog.luevano.xyz/a/updated_pyssg_pymdvar_and_website.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/updated_pyssg_pymdvar_and_website.html</guid> - <pubDate>Sat, 06 May 2023 12:39:14 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Tools</category> - <category>Update</category> - <description>Worked on another update of pyssg which now includes my extension pymdvar and updated the website overall.</description> - <content:encoded><![CDATA[<p>Again, I’ve updated <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> to add a bit of unit-testing as well as to include my extension <a href="https://github.com/luevano/pymdvar"><code>pymdvar</code></a> which is used to convert <code>${some_variables}</code> into their respective <code>values</code> based on a config file and/or environment variables. With this I also updated a bit of the CSS of the site as well as basically all the entries and base templates, a much needed update (for me, because externally doesn’t look like much). Along with this I also added a “return to top” button, once you scroll enough on the site, a new button appears on the bottom right to get back to the top, also added table of contents to entries taht could use them (as well as a bit of CSS to them).</p> -<p>This update took a long time because I had a fundamental issue with how I was managing the “static” website, where I host all assets such as CSS, JS, images, etc.. Because I was using the <code><base></code> HTML tag. The issue is that this tag affects everything and there is no “opt-out” on some body tags, meaning that I would have to write the whole URL for all static assets. So I tried looking into changing how the image extension for <a href="https://python-markdown.github.io/"><code>python-markdown</code></a> works, so that it includes this “base” URL I needed. But it was too much hassle, so I ended up developing my own extension mentioned earlier. Just as a side note, I noticed that my extension doesn’t cover all my needs, so probably it wont cover yours, if you end up using it just test it out a bit yourself and then go ahead, PRs are welcomed.</p> -<p>One thing led to another so I ended up changing a lot of stuff, and with changes comes tireness and eded up leaving the project for a while (again). This also led to not wanting to write or add anything else to the site until I sorted things out. But I’m again reviving it I guess, and up to the next cycle.</p> -<p>The next things I’ll be doing are continuing with my <a href="https://blog.luevano.xyz/tag/@gamedev">@gamedev</a> journey and probably upload some drawings if I feel like doing some.</p>]]></content:encoded> - </item> - <item> - <title>Rewrote pyssg again</title> - <link>https://blog.luevano.xyz/a/rewrote_pyssg_again.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/rewrote_pyssg_again.html</guid> - <pubDate>Tue, 20 Dec 2022 04:31:05 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Tools</category> - <category>Update</category> - <description>Rewrote pyssg to make it more flexible and to work with YAML configuration files.</description> - <content:encoded><![CDATA[<p>I’ve been wanting to change the way <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> reads config files and generates <code>HTML</code> files so that it is more flexible and I don’t need to have 2 separate build commands and configs (for <a href="https://blog.luevano.xyz">blog</a> and <a href="https://art.luevano.xyz">art</a>), and also to handle other types of “sites”; because <code>pyssg</code> was built with blogging in mind, so it was a bit limited to how it could be used. So I had to kind of <em>rewrite</em> <code>pyssg</code>, and with the latest version I can now generate the whole site and use the same templates for everything, quite neat for my use case.</p> -<p>Anyways, so I bought a new domain for all <code>pyssg</code> related stuff, mostly because I wanted somewhere to test live builds while developing, it is of course <a href="https://pyssg.xyz">pyssg.xyz</a>; as of now it is the same template, CSS and scripts that I use here, probably will change in the future. I’ll be testing new features and anything <code>pyssg</code> related stuff.</p> -<p>I should start pointing all links to <code>pyssg</code> to the actual site instead of the github repository (or my <a href="https://git.luevano.xyz">git</a> repository), but I haven’t decided how to handle everything.</p>]]></content:encoded> - </item> - <item> - <title>Creating my Go Godot Jam 3 entry using Godot 3.5 devlog 1</title> - <link>https://blog.luevano.xyz/g/gogodot_jam3_devlog_1.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/gogodot_jam3_devlog_1.html</guid> - <pubDate>Fri, 10 Jun 2022 09:17:05 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Gamejam</category> - <category>Gdscript</category> - <category>Godot</category> - <description>Details on the implementation for the game I created for the Go Godot Jam 3, which theme is Evolution.</description> - <content:encoded><![CDATA[<p>The jam’s theme is Evolution and all the details are listed <a href="https://itch.io/jam/go-godot-jam-3">here</a>. <del>This time I’m logging as I go, so there might be some changes to the script or scenes along the way.</del> <ins>I couldn’t actually do this, as I was running out of time</ins>. Note that I’m not going to go into much details, the obvious will be ommitted.</p> -<p>I wanted to do a <em>Snake</em> clone, and I’m using this jam as an excuse to do it and add something to it. The features include:</p> -<ul> -<li>Snakes will pass their stats in some form to the next snakes.</li> -<li>Non-grid snake movement. I just hate the grid constraint, so I wanted to make it move in any direction.</li> -<li>Depending on the food you eat, you’ll gain new mutations/abilities <del>and the more you eat the more that mutation develops</del> <ins>didn’t have time to add this feature, sad</ins>.</li> -<li>Procedural map creation.</li> -</ul> -<p>I created this game using <em>Godot 3.5-rc3</em>. You can find the source code in my GitHub <a href="https://github.com/luevano/gogodot_jam3">here</a> which at the time of writing this it doesn’t contain any exported files, for that you can go ahead and play it in your browser at <a href="https://lorentzeus.itch.io/snake-tronic">itch.io</a>, which you can find below:</p> -<p style="text-align:center"><iframe src="https://itch.io/embed/1562701?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/snake-tronic">Snake-tronic by Lorentzeus</a></iframe></p> - -<p>You can also find the jam entry <a href="https://itch.io/jam/go-godot-jam-3/rate/1562701">here</a>.</p> -<p>Similarly with the my FlappyBird clone, I plan to update this to a better state.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#initial-setup">Initial setup</a></li> -<li><a href="#assets">Assets</a></li> -<li><a href="#the-snake">The snake</a><ul> -<li><a href="#basic-movement">Basic movement</a></li> -<li><a href="#setting-up-path-following">Setting up path following</a></li> -<li><a href="#define-body-parts-for-the-snake">Define body parts for the snake</a></li> -<li><a href="#adding-body-parts">Adding body parts</a></li> -<li><a href="#fix-on-body-segments-following-head">Fix on body segments following head</a></li> -</ul> -</li> -<li><a href="#the-food">The food</a></li> -<li><a href="#za-warudo-the-world">Za warudo! (The world)</a><ul> -<li><a href="#food-placement">Food placement</a></li> -</ul> -</li> -<li><a href="#stats-clas-and-loadingsaving-data">Stats clas and loading/saving data</a><ul> -<li><a href="#stats-class">Stats class</a></li> -<li><a href="#loadsave-data">Load/save data</a></li> -</ul> -</li> -<li><a href="#scoring">Scoring</a></li> -<li><a href="#snake-redesigned-with-the-state-machine-pattern">Snake redesigned with the state machine pattern</a></li> -<li><a href="#other-minor-stuff">Other minor stuff</a></li> -<li><a href="#final-notes">Final notes</a></li> -</ul> -</div> -<h2 id="initial-setup">Initial setup<a class="headerlink" href="#initial-setup" title="Permanent link">¶</a></h2> -<p>Again, similar to the <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html">FlappyBird</a> clone I created, I’m using the directory structure I wrote about on <a href="https://blog.luevano.xyz/g/godot_project_structure.html">Godot project structure</a> with slight modifications to test things out. Also using similar <em>Project settings</em> as those from the <em>FlappyBird</em> clone like the pixel art texture imports, keybindings, layers, etc..</p> -<p>I’ve also setup <a href="https://github.com/bram-dingelstad/godot-gifmaker">GifMaker</a>, with slight modifications as the <em>AssetLib</em> doesn’t install it correctly and contains unnecessry stuff: moved necessary files to the <code>res://addons</code> directory, deleted test scenes and files in general, and copied the license to the <code>res://docs</code> directory. Setting this up was a bit annoying because the tutorial it’s bad (with all due respect). I might do a separate entry just to explain how to set it up, because I couldn’t find it anywhere other than by inspecting some of the code/scenes. <ins>I ended up leaving this disabled in the game as it hit the performance by a lot, but it’s an option I’ll end up researching more</ins>.</p> -<p>This time I’m also going to be using an <a href="https://www.gdquest.com/docs/guidelines/best-practices/godot-gdscript/event-bus/">Event bus</a> singleton (which I’m going to just call <em>Event</em>) as managing signals was pretty annoying on my last project; as well as a <em>Global</em> singleton for essential stuff so I don’t have to do as many cross references between nodes/scenes.</p> -<h2 id="assets">Assets<a class="headerlink" href="#assets" title="Permanent link">¶</a></h2> -<p>This time I’ll be creating my own assets in <a href="https://www.aseprite.org/">Aseprite</a>, wont be that good, but enough to prototype and get things going.</p> -<p>Other than that I used few key sprites from <a href="https://vryell.itch.io/">vryell</a>: <a href="https://vryell.itch.io/controller-keyboard-icons">Controller & Keyboard Icons</a> and a font from <a href="https://datagoblin.itch.io/">datagoblin</a>: <a href="https://datagoblin.itch.io/monogram">Monogram</a>.</p> -<h2 id="the-snake">The snake<a class="headerlink" href="#the-snake" title="Permanent link">¶</a></h2> -<p>This is the most challenging part in my opinion as making all the body parts follow the head in a user defined path it’s kinda hard. I tried with like 4-5 options and the one I’m detailing here is the only one that worked as I wanted for me. This time the directory structure I’m using is the following:</p> -<figure id="__yafg-figure-39"> -<img alt="FileSystem - Snake dir structure" src="https://static.luevano.xyz/images/g/gogodot_jam3/file_system_snake_dir_structure.png" title="FileSystem - Snake dir structure"> -<figcaption>FileSystem - Snake dir structure</figcaption> -</figure> -<h3 id="basic-movement">Basic movement<a class="headerlink" href="#basic-movement" title="Permanent link">¶</a></h3> -<p>The most basic thing is to move the head, this is what we have control of. Create a scene called <code>Head.tscn</code> and setup the basic <em>KinematicBody2D</em> with it’s own <em>Sprite</em> and <em>CollisionShape2D</em> (I used a small circle for the tip of the head), and set the <em>Collision Layer/Mask</em> accordingly, for now just <code>layer = bit 1</code>. And all we need to do, is keep moving the snake forwards and be able to rotate left or right. Created a new script called <code>head.gd</code> attached to the root (<em>KinematicBody2D</em>) and added:</p> -<pre><code class="language-gdscript">extends KinematicBody2D - -enum { - LEFT=-1, - RIGHT=1 -} - -var velocity: Vector2 = Vector2.ZERO -var _direction: Vector2 = Vector2.UP - - -func _physics_process(delta: float) -> void: - if Input.is_action_pressed("move_left"): - _rotate_to(LEFT) - if Input.is_action_pressed("move_right"): - _rotate_to(RIGHT) - - velocity = _direction * Global.SNAKE_SPEED - - velocity = move_and_slide(velocity) - _handle_time_elapsed(delta) - - -func _rotate_to(direction: int) -> void: - rotate(deg2rad(direction * Global.SNAKE_ROT_SPEED * get_physics_process_delta_time())) - _direction = _direction.rotated(deg2rad(direction * Global.SNAKE_ROT_SPEED * get_physics_process_delta_time())) -</code></pre> -<p>After tunning all the necessary parameters you should get something like this:</p> -<figure id="__yafg-figure-40"> -<img alt="Snake - Basic movement (left and right controls)" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement.gif" title="Snake - Basic movement (left and right controls)"> -<figcaption>Snake - Basic movement (left and right controls)</figcaption> -</figure> -<h3 id="setting-up-path-following">Setting up path following<a class="headerlink" href="#setting-up-path-following" title="Permanent link">¶</a></h3> -<p>To move other snake parts by following the snake head the only solution I found was to use the <em>Path2D</em> and <em>PathFollow2D</em> nodes. <em>Path2D</em> basically just handles the curve/path that <em>PathFollow2D</em> will use to move its child node; and I say “child node” in singular… as <em>PathFollow2D</em> can only handle one damn child, all the other ones will have weird transformations and/or rotations. So, the next thing to do is to setup a way to compute (and draw so we can validate) the snake’s path/curve.</p> -<p>Added the signal <code>snake_path_new_point(coordinates)</code> to the <em>Event</em> singleton and then add the following to <code>head.gd</code>:</p> -<pre><code class="language-gdscript">var _time_elapsed: float = 0.0 - -# using a timer is not recommended for < 0.01 -func _handle_time_elapsed(delta: float) -> void: - if _time_elapsed >= Global.SNAKE_POSITION_UPDATE_INTERVAL: - Event.emit_signal("snake_path_new_point", global_position) - _time_elapsed = 0.0 - _time_elapsed += delta -</code></pre> -<p>This will be pinging the current snake head position every <code>0.01</code> seconds (defined in <em>Global</em>). Now create a new scene called <code>Snake.tscn</code> which will contain a <em>Node2D</em>, a <em>Path2D</em> and an instance of <em>Head</em> as its childs. Create a new script called <code>snake.gd</code> attached to the root (<em>Node2D</em>) with the following content:</p> -<pre><code class="language-gdscript">class_name Snake -extends Node2D - -onready var path: Path2D = $Path - -func _ready(): - Event.connect("snake_path_new_point", self, "_on_Head_snake_path_new_point") - - -func _draw() -> void: - if path.curve.get_baked_points().size() >= 2: - draw_polyline(path.curve.get_baked_points(), Color.aquamarine, 1, true) - - -func _on_Head_snake_path_new_point(coordinates: Vector2) -> void: - path.curve.add_point(coordinates) - # update call is to draw curve as there are new points to the path's curve - update() -</code></pre> -<p>With this, we’re now populating the <em>Path2D</em> curve points with the position of the snake head. You should be able to see it because of the <code>_draw</code> call. If you run it you should see something like this:</p> -<figure id="__yafg-figure-41"> -<img alt="Snake - Basic movement with path" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_with_path.gif" title="Snake - Basic movement with path"> -<figcaption>Snake - Basic movement with path</figcaption> -</figure> -<h3 id="define-body-parts-for-the-snake">Define body parts for the snake<a class="headerlink" href="#define-body-parts-for-the-snake" title="Permanent link">¶</a></h3> -<p>At this point the only thing to do is to add the corresponding next body parts and tail of the snake. To do so, we need a <em>PathFollow2D</em> to use the live-generating <em>Path2D</em>, the only caveat is that we need one of these per body part/tail (this took me hours to figure out, <em>thanks documentation</em>).</p> -<p>Create a new scene called <code>Body.tscn</code> with a <em>PathFollow2D</em> as its root and an <em>Area2D</em> as its child, then just add the necessary <em>Sprite</em> and <em>CollisionShap2D</em> for the <em>Area2D</em>, I’m using <code>layer = bit 2</code> for its collision. Create a new script called <code>generic_segment.gd</code> with the following code:</p> -<pre><code class="language-gdscript">extends PathFollow2D - -export(String, "body", "tail") var TYPE: String = "body" - - -func _physics_process(delta: float) -> void: - offset += Global.SNAKE_SPEED * delta -</code></pre> -<p>And this can be attached to the <em>Body</em>‘s root node (<em>PathFollow2D</em>), no extra setup needed. Repeat the same steps for creating the <code>Tail.tscn</code> scene and when attaching the <code>generic_segment.gd</code> script just configure the <code>Type</code> parameter to <code>tail</code> in the GUI (by selecting the node with the script attached and editing in the <em>Inspector</em>).</p> -<h3 id="adding-body-parts">Adding body parts<a class="headerlink" href="#adding-body-parts" title="Permanent link">¶</a></h3> -<p>Now it’s just a matter of handling when to add new body parts in the <code>snake.gd</code> script. For now I’ve only setup for adding body parts to fulfill the initial length of the snake (this doesn’t include the head or tail). The extra code needed is the following:</p> -<pre><code class="language-gdscript">export(PackedScene) var BODY_SEGMENT_NP: PackedScene -export(PackedScene) var TAIL_SEGMENT_NP: PackedScene - -var current_body_segments: int = 0 -var max_body_segments: int = 1 - - -func _add_initial_segment(type: PackedScene) -> void: - if path.curve.get_baked_length() >= (current_body_segments + 1.0) * Global.SNAKE_SEGMENT_SIZE: - var _temp_body_segment: PathFollow2D = type.instance() - path.add_child(_temp_body_segment) - current_body_segments += 1 - - -func _on_Head_snake_path_new_point(coordinates: Vector2) -> void: - path.curve.add_point(coordinates) - # update call is to draw curve as there are new points to the path's curve - update() - - # add the following lines - if current_body_segments < max_body_segments: - _add_initial_segment(BODY_SEGMENT_NP) - elif current_body_segments == max_body_segments: - _add_initial_segment(TAIL_SEGMENT_NP) -</code></pre> -<p>Select the <em>Snake</em> node and add the <em>Body</em> and <em>Tail</em> scene to the parameters, respectively. Then when running you should see something like this:</p> -<figure id="__yafg-figure-42"> -<img alt="Snake - Basic movement with all body parts" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_added_body_parts.gif" title="Snake - Basic movement with all body parts"> -<figcaption>Snake - Basic movement with all body parts</figcaption> -</figure> -<p>Now, we need to handle adding body parts after the snake is complete and already moved for a bit, this will require a queue so we can add part by part in the case that we eat multiple pieces of food in a short period of time. For this we need to add some signals: <code>snake_adding_new_segment(type)</code>, <code>snake_added_new_segment(type)</code>, <code>snake_added_initial_segments</code> and use them when makes sense. Now we need to add the following:</p> -<pre><code class="language-gdscript">var body_segment_stack: Array -var tail_segment: PathFollow2D -# didn't konw how to name this, basically holds the current path lenght -# whenever the add body segment, and we use this stack to add body parts -var body_segment_queue: Array -</code></pre> -<p>As well as updating <code>_add_initial_segment</code> with the following so it adds the new segment on the specific variable:</p> -<pre><code class="language-gdscript">if _temp_body_segment.TYPE == "body": - body_segment_stack.append(_temp_body_segment) -else: - tail_segment = _temp_body_segment -</code></pre> -<p>Now that it’s just a matter of creating the segment queue whenever a new segment is needed, as well as adding each segment in a loop whenever we have items in the queue and it’s a good distance to place the segment on. These two things can be achieved with the following code:</p> -<pre><code class="language-gdscript"># this will be called in _physics_process -func _add_new_segment() -> void: - var _path_length_threshold: float = body_segment_queue[0] + Global.SNAKE_SEGMENT_SIZE - if path.curve.get_baked_length() >= _path_length_threshold: - var _removed_from_queue: float = body_segment_queue.pop_front() - var _temp_body_segment: PathFollow2D = BODY_SEGMENT_NP.instance() - var _new_body_offset: float = body_segment_stack.back().offset - Global.SNAKE_SEGMENT_SIZE - - _temp_body_segment.offset = _new_body_offset - body_segment_stack.append(_temp_body_segment) - path.add_child(_temp_body_segment) - tail_segment.offset = body_segment_stack.back().offset - Global.SNAKE_SEGMENT_SIZE - - current_body_segments += 1 - - -func _add_segment_to_queue() -> void: - # need to have the queues in a fixed separation, else if the eating functionality - # gets spammed, all next bodyparts will be spawned almost at the same spot - if body_segment_queue.size() == 0: - body_segment_queue.append(path.curve.get_baked_length()) - else: - body_segment_queue.append(body_segment_queue.back() + Global.SNAKE_SEGMENT_SIZE) -</code></pre> -<p>With everything implemented and connected accordingly then we can add segments on demand (for testing I’m adding with a key press), it should look like this:</p> -<figure id="__yafg-figure-43"> -<img alt="Snake - Basic movement with dynamic addition of new segments" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_with_dynamic_segments.gif" title="Snake - Basic movement with dynamic addition of new segments"> -<figcaption>Snake - Basic movement with dynamic addition of new segments</figcaption> -</figure> -<p>For now, this should be enough, I’ll add more stuff as needed as I go. Last thing is that after finished testing that the movement felt ok, I just added a way to stop the snake whenever it collides with itself by using the following code (and the signal <code>snake_segment_body_entered(body)</code>) in a <code>main.gd</code> script that is the entry point for the game:</p> -<pre><code class="language-gdscript">func _snake_disabled(on_off: bool) -> void: - _snake.propagate_call("set_process", [on_off]) - _snake.propagate_call("set_process_internal", [on_off]) - _snake.propagate_call("set_physics_process", [on_off]) - _snake.propagate_call("set_physics_process_internal", [on_off]) - _snake.propagate_call("set_process_input", [on_off]) -</code></pre> -<p>Which will stop the snake node and all children.</p> -<h3 id="fix-on-body-segments-following-head">Fix on body segments following head<a class="headerlink" href="#fix-on-body-segments-following-head" title="Permanent link">¶</a></h3> -<p>After a while of testing and developing, I noticed that sometimes the head “detaches” from the body when a lot of rotations happen (moving the snake left or right), because of how imprecise the <em>Curve2D</em> is. To do this I just send a signal (<code>snake_rotated</code>) whenever the snake rotates and make a small correction (in <code>generic_segment.gd</code>):</p> -<pre><code class="language-gdscript">func _on_snake_rotated() -> void: - offset -= 0.75 * Global.SNAKE_SPEED * pow(get_physics_process_delta_time(), 2) -</code></pre> -<p>This is completely random, I tweaked it manually after a lot of iterations.</p> -<h2 id="the-food">The food<a class="headerlink" href="#the-food" title="Permanent link">¶</a></h2> -<p>For now I just decided to setup a simple system to see everything works fine. The idea is to make some kind of generic food node/scene and a “food manager” to spawn them, for now in totally random locations. For this I added the following signals: <code>food_placing_new_food(type)</code>, <code>food_placed_new_food(type)</code> and <code>food_eaten(type)</code>.</p> -<p>First thing is creating the <code>Food.tscn</code> which is just an <em>Area2D</em> with its necessary children with an attached script called <code>food.gd</code>. The script is really simple:</p> -<pre><code class="language-gdscript">class_name Food # needed to access Type enum outside of the script, this registers this script as a node -extends Area2D - -enum Type { - APPLE -} - -var _type_texture: Dictionary = { - Type.APPLE: preload("res://entities/food/sprites/apple.png") -} - -export(Type) var TYPE -onready var _sprite: Sprite = $Sprite - - -func _ready(): - connect("body_entered", self, "_on_body_entered") - _sprite.texture = _type_texture[TYPE] - - -func _on_body_entered(body: Node) -> void: - Event.emit_signal("food_eaten", TYPE) - queue_free() -</code></pre> -<p>Then this <code>food_eaten</code> signal is received in <code>snake.gd</code> to add a new segment to the queue.</p> -<p>Finally, for the food manager I just created a <code>FoodManager.tscn</code> with a <em>Node2D</em> with an attached script called <code>food_manager.gd</code>. To get a random position:</p> -<pre><code class="language-gdscript">func _get_random_pos() -> Vector2: - var screen_size: Vector2 = get_viewport().get_visible_rect().size - var temp_x: float = randf() * screen_size.x - screen_size.x / 2.0 - var temp_y: float = randf() * screen_size.y - screen_size.y / 2.0 - - return Vector2(temp_x, temp_y) -</code></pre> -<p>Which gets the job done, but later I’ll have to add a way to check that the position is valid. And to actually place the food:</p> -<pre><code class="language-gdscript">func _place_new_food() -> void: - var food: Area2D = FOOD.instance() - var position: Vector2 = _get_random_pos() - food.global_position = position - add_child(food) -</code></pre> -<p>And this is used in <code>_process</code> to place new food whenever needed. For now I added a condition to add food until 10 pieces are in place, and keep adding whenever the food is is lower than 10. After setting everything up, this is the result:</p> -<figure id="__yafg-figure-44"> -<img alt="Snake - Food basic interaction" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_food_basic_interaction.gif" title="Snake - Food basic interaction"> -<figcaption>Snake - Food basic interaction</figcaption> -</figure> -<h2 id="za-warudo-the-world">Za warudo! (The world)<a class="headerlink" href="#za-warudo-the-world" title="Permanent link">¶</a></h2> -<p>It just happend that I saw a video to create random maps by using a method called <a href="https://www.mit.edu/~kardar/teaching/projects/chemotaxis(AndreaSchmidt)/random.htm">random walks</a>, this video was made by <a href="https://www.youtube.com/c/NADLABS">NAD LABS</a>: <a href="https://www.youtube.com/watch?v=ppP2Doq3p7s">Nuclear Throne Like Map Generation In Godot</a>. It’s a pretty simple but powerful script, he provided the source code from which I based my random walker, just tweaked a few things and added others. Some of the maps than can be generated with this method (already aded some random sprites):</p> -<figure id="__yafg-figure-45"> -<img alt="World map generator - Random map 1" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_1.png" title="World map generator - Random map 1"> -<figcaption>World map generator - Random map 1</figcaption> -</figure> -<figure id="__yafg-figure-46"> -<img alt="World map generator - Random map 2" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_2.png" title="World map generator - Random map 2"> -<figcaption>World map generator - Random map 2</figcaption> -</figure> -<figure id="__yafg-figure-47"> -<img alt="World map generator - Random map 3" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_3.png" title="World map generator - Random map 3"> -<figcaption>World map generator - Random map 3</figcaption> -</figure> -<p>It started with just black and white tiles, but I ended up adding some sprites as it was really harsh to the eyes. My implementation is basically the same as <em>NAD LABS</em>‘ with few changes, most importantly: I separated the generation in 2 diferent tilemaps (floor and wall) to have better control as well as wrapped everything in a single scene with a “main” script with the following important functions:</p> -<pre><code class="language-gdscript">func get_valid_map_coords() -> Array: - var safe_area: Array = walker_head.get_cells_around() - var cells_used: Array = ground_tilemap.get_used_cells() - for location in safe_area: - cells_used.erase(location) - return cells_used - - -func get_centered_world_position(location: Vector2) -> Vector2: - return ground_tilemap.map_to_world(location) + Vector2.ONE * Global.TILE_SIZE / 2.0 -</code></pre> -<p>Where <code>get_cells_around</code> is just a function that gets the safe cells around the origin. And this <code>get_valid_map_coords</code> just returns used cells minus the safe cells, to place food. <code>get_centered_world_position</code> is so we can center the food in the tiles.</p> -<p>Some signals I used for the world gen: <code>world_gen_walker_started(id)</code>, <code>world_gen_walker_finished(id)</code>, <code>world_gen_walker_died(id)</code> and <code>world_gen_spawn_walker_unit(location)</code>.</p> -<h3 id="food-placement">Food placement<a class="headerlink" href="#food-placement" title="Permanent link">¶</a></h3> -<p>The last food algorithm doesn’t check anything related to the world, and thus the food could spawn in the walls and outside the map.</p> -<p>First thing is I generalized the food into a single script and added basic food and special food which inherit from base food. The most important stuff for the base food is to be able to set all necessary properties at first:</p> -<pre><code class="language-gdscript">func update_texture() -> void: - _sprite.texture = texture[properties["type"]] - - -func set_properties(pos: Vector2, loc: Vector2, special: bool, type: int, points: int=1, special_points: int=1, ttl: float = -1.0) -> void: - properties["global_position"] = pos - global_position = pos - properties["location"] = loc - properties["special"] = special - properties["type"] = type - - properties["points"] = points - properties["special_points"] = special_points - properties["ttl"] = ttl - if properties["ttl"] != -1.0: - timer.wait_time = properties["ttl"] - timer.start() -</code></pre> -<p>Where the <code>update_texture</code> needs to be a separate function, because we need to create the food first, set properties, add as a child and then update the sprite; we also need to keep track of the global position, location (in tilemap coordinates) and identifiers for the type of food.</p> -<p>Then basic/special food just extend base food, define a <code>Type</code> enum and preloads the necessary textures, for example:</p> -<pre><code class="language-gdscript">enum Type { - APPLE, - BANANA, - RAT -} - - -func _ready(): - texture[Type.APPLE] = preload("res://entities/food/sprites/apple.png") - texture[Type.BANANA] = preload("res://entities/food/sprites/banana.png") - texture[Type.RAT] = preload("res://entities/food/sprites/rat.png") -</code></pre> -<p>Now, some of the most important change to <code>food_manager.gd</code> is to get an actual random valid position:</p> -<pre><code class="language-gdscript">func _get_random_pos() -> Array: - var found_valid_loc: bool = false - var index: int - var location: Vector2 - - while not found_valid_loc: - index = randi() % possible_food_locations.size() - location = possible_food_locations[index] - if current_basic_food.find(location) == -1 and current_special_food.find(location) == -1: - found_valid_loc = true - - return [world_generator.get_centered_world_position(location), location] -</code></pre> -<p>Other than that, there are some differences between placing normal and special food (specially the signal they send, and if an extra “special points” property is set). Some of the signals that I used that might be important: <code>food_placing_new_food(type)</code>, <code>food_placed_new_food(type, location)</code> and <code>food_eaten(type, location)</code>.</p> -<h2 id="stats-clas-and-loadingsaving-data">Stats clas and loading/saving data<a class="headerlink" href="#stats-clas-and-loadingsaving-data" title="Permanent link">¶</a></h2> -<p>I got the idea of saving the current stats (points, max body segments, etc.) in a separate <code>Stats</code> class for easier load/save data. This option I went with didn’t work as I would liked it to work, as it was a pain in the ass to setup and each time a new property is added you have to manually setup the load/save helper functions… so not the best option. This option I used was json but saving a Node directly could work better or using resources (saving <code>tres</code> files).</p> -<h3 id="stats-class">Stats class<a class="headerlink" href="#stats-class" title="Permanent link">¶</a></h3> -<p>The <code>Stats</code> “class” is just a script that extends from <em>Node</em> called <code>stats.gd</code>. It needs to define the <code>class_name</code> as <code>Stats</code>. The main content:</p> -<pre><code class="language-gdscript"># main -var points: int = 0 -var segments: int = 0 - -# track of trait points -var dash_points: int = 0 -var slow_points: int = 0 -var jump_points: int = 0 - -# times trait achieved -var dash_segments: int = 0 -var slow_segments: int = 0 -var jump_segments: int = 0 - -# trait properties -var dash_percentage: float = 0.0 -var slow_percentage: float = 0.0 -var jump_lenght: float = 0.0 - -# trait active -var trait_dash: bool = false -var trait_slow: bool = false -var trait_jump: bool = false -</code></pre> -<p>And with the ugliest functions:</p> -<pre><code class="language-gdscript">func get_stats() -> Dictionary: - return { - "points": points, - "segments": segments, - "dash_points": dash_points, - "dash_segments": dash_segments, - "dash_percentage": dash_percentage, - "slow_points": slow_points, - "slow_segments": slow_segments, - "slow_percentage": slow_percentage, - "jump_points": jump_points, - "jump_segments": jump_segments, - "jump_lenght": jump_lenght, - "trait_dash": trait_dash, - "trait_slow": trait_slow, - "trait_jump": trait_jump - } - - -func set_stats(stats: Dictionary) -> void: - points = stats["points"] - segments = stats["segments"] - dash_points = stats["dash_points"] - slow_points = stats["slow_points"] - jump_points = stats["jump_points"] - dash_segments = stats["dash_segments"] - slow_segments = stats["slow_segments"] - jump_segments = stats["jump_segments"] - dash_percentage = stats["dash_percentage"] - slow_percentage = stats["slow_percentage"] - jump_lenght = stats["jump_lenght"] - trait_dash = stats["trait_dash"] - trait_slow = stats["trait_slow"] - trait_jump = stats["trait_jump"] -</code></pre> -<p>And this is not scalable at all, but I had to do this at the end of the jam so no way of optimizing and/or doing it correctly, sadly.</p> -<h3 id="loadsave-data">Load/save data<a class="headerlink" href="#loadsave-data" title="Permanent link">¶</a></h3> -<p>The load/save function is pretty standard. It’s a singleton/autoload called <code>SavedData</code> with a script that extends from <em>Node</em> called <code>save_data.gd</code>:</p> -<pre><code class="language-gdscript">const DATA_PATH: String = "user://data.save" - -var _stats: Stats - - -func _ready() -> void: - _load_data() - - -# called when setting "stats" and thus saving -func save_data(stats: Stats) -> void: - _stats = stats - var file: File = File.new() - file.open(DATA_PATH, File.WRITE) - file.store_line(to_json(_stats.get_stats())) - file.close() - - -func get_stats() -> Stats: - return _stats - - -func _load_data() -> void: - # create an empty file if not present to avoid error while loading settings - _handle_new_file() - - var file = File.new() - file.open(DATA_PATH, File.READ) - _stats = Stats.new() - _stats.set_stats(parse_json(file.get_line())) - file.close() - - -func _handle_new_file() -> void: - var file: File = File.new() - if not file.file_exists(DATA_PATH): - file.open(DATA_PATH, File.WRITE) - _stats = Stats.new() - file.store_line(to_json(_stats.get_stats())) - file.close() -</code></pre> -<p>It uses json as the file format, but I might end up changing this in the future to something else more reliable and easier to use (<code>Stats</code> class related issues).</p> -<h2 id="scoring">Scoring<a class="headerlink" href="#scoring" title="Permanent link">¶</a></h2> -<p>For this I created a scoring mechanisms and just called it <code>ScoreManager</code> (<code>score_manager.gd</code>) which just basically listens to <code>food_eaten</code> signal and adds points accordingly to the current <em>Stats</em> object loaded. The main function is:</p> -<pre><code class="language-gdscript">func _on_food_eaten(properties: Dictionary) -> void: - var is_special: bool = properties["special"] - var type: int = properties["type"] - var points: int = properties["points"] - var special_points: int = properties["special_points"] - var location: Vector2 = properties["global_position"] - var amount_to_grow: int - var special_amount_to_grow: int - - amount_to_grow = _process_points(points) - _spawn_added_score_text(points, location) - _spawn_added_segment_text(amount_to_grow) - - if is_special: - special_amount_to_grow = _process_special_points(special_points, type) - # _spawn_added_score_text(points, location) - _spawn_added_special_segment_text(special_amount_to_grow, type) - _check_if_unlocked(type) -</code></pre> -<p>Where the most important function is:</p> -<pre><code class="language-gdscript">func _process_points(points: int) -> int: - var score_to_grow: int = (stats.segments + 1) * Global.POINTS_TO_GROW - stats.points - var amount_to_grow: int = 0 - var growth_progress: int - stats.points += points - if points >= score_to_grow: - amount_to_grow += 1 - points -= score_to_grow - # maybe be careful with this - amount_to_grow += points / Global.POINTS_TO_GROW - stats.segments += amount_to_grow - Event.emit_signal("snake_add_new_segment", amount_to_grow) - - growth_progress = Global.POINTS_TO_GROW - ((stats.segments + 1) * Global.POINTS_TO_GROW - stats.points) - Event.emit_signal("snake_growth_progress", growth_progress) - return amount_to_grow -</code></pre> -<p>Which will add the necessary points to <code>Stats.points</code> and return the amount of new snake segments to grow. After this <code>_spawn_added_score_segment</code> and <code>_spawn_added_segment_text</code> just spawn a <em>Label</em> with the info on the points/segments gained; this is custom UI I created, nothing fancy.</p> -<p>Last thing is taht in <code>_process_points</code> there is a check at the end, where if the food eaten is “special” then a custom variation of the last 3 functions are executed. These are really similar, just specific to each kind of food.</p> -<p>This <code>ScoreManager</code> also handles the calculation for the <code>game_over</code> signal, to calculte progress, set necessary <code>Stats</code> values and save the data:</p> -<pre><code class="language-gdscript">func _on_game_over() -> void: - var max_stats: Stats = _get_max_stats() - SaveData.save_data(max_stats) - Event.emit_signal("display_stats", initial_stats, stats, mutation_stats) - - -func _get_max_stats() -> Stats: - var old_stats_dict: Dictionary = initial_stats.get_stats() - var new_stats_dict: Dictionary = stats.get_stats() - var max_stats: Stats = Stats.new() - var max_stats_dict: Dictionary = max_stats.get_stats() - var bool_stats: Array = [ - "trait_dash", - "trait_slow", - "trait_jump" - ] - - for i in old_stats_dict: - if bool_stats.has(i): - max_stats_dict[i] = old_stats_dict[i] or new_stats_dict[i] - else: - max_stats_dict[i] = max(old_stats_dict[i], new_stats_dict[i]) - max_stats.set_stats(max_stats_dict) - return max_stats -</code></pre> -<p>Then this sends a signal <code>display_stats</code> to activate UI elements that shows the progression.</p> -<p>Naturally, the saved <code>Stats</code> are loaded whenever needed. For example, for the <code>Snake</code>, we load the stats and setup any value needed from there (like a flag to know if any ability is enabled), and since we’re saving the new <code>Stats</code> at the end, then on restart we load the updated one.</p> -<h2 id="snake-redesigned-with-the-state-machine-pattern">Snake redesigned with the state machine pattern<a class="headerlink" href="#snake-redesigned-with-the-state-machine-pattern" title="Permanent link">¶</a></h2> -<p>I redesigned the snake code (the head, actually) to use the state machine pattern by following <a href="https://gdscript.com/solutions/godot-state-machine/">this guide</a> which is definitely a great guide, straight to the point and easy to implement.</p> -<p>Other than what is shown in the guide, I implemented some important functions in the <code>state_machine.gd</code> script itself, to be used by each of the states as needed:</p> -<pre><code class="language-gdscript">func rotate_on_input() -> void: - if Input.is_action_pressed("move_left"): - player.rotate_to(player.LEFT) - if Input.is_action_pressed("move_right"): - player.rotate_to(player.RIGHT) - - -func slow_down_on_collisions(speed_backup: float): - if player.get_last_slide_collision(): - Global.SNAKE_SPEED = player.velocity.length() - else: - Global.SNAKE_SPEED = speed_backup - - -func handle_slow_speeds() -> void: - if Global.SNAKE_SPEED <= Global.SNAKE_SPEED_BACKUP / 4.0: - Global.SNAKE_SPEED = Global.SNAKE_SPEED_BACKUP - Event.emit_signal("game_over") -</code></pre> -<p>And then in the <code>StateMachine</code>‘s <code>_process</code>:</p> -<pre><code class="language-gdscript">func _physics_process(delta: float) -> void: - # state specific code, move_and_slide is called here - if state.has_method("physics_process"): - state.physics_process(delta) - - handle_slow_speeds() - player.handle_time_elapsed(delta) -</code></pre> -<p>And now it’s just a matter of implementing the necessary states. I used 4: <code>normal_stage.gd</code>, <code>slow_state.gd</code>, <code>dash_state.gd</code> and <code>jump_state.gd</code>.</p> -<p>The <code>normal_state.gd</code> contains what the original <code>head.gd</code> code contained:</p> -<pre><code class="language-gdscript">func physics_process(delta: float) -> void: - fsm.rotate_on_input() - fsm.player.velocity = fsm.player.direction * Global.SNAKE_SPEED - fsm.player.velocity = fsm.player.move_and_slide(fsm.player.velocity) - - fsm.slow_down_on_collisions(Global.SNAKE_SPEED_BACKUP) - - -func input(event: InputEvent) -> void: - if fsm.player.can_dash and event.is_action_pressed("dash"): - exit("DashState") - if fsm.player.can_slow and event.is_action_pressed("slow"): - exit("SlowState") - if fsm.player.can_jump and event.is_action_pressed("jump"): - exit("JumpState") -</code></pre> -<p>Here, the <code>exit</code> method is basically to change to the next state. And lastly, I’m only gonna show the <code>dash_state.gd</code> as the other ones are pretty similar:</p> -<pre><code class="language-gdscript">func enter(): - if fsm.DEBUG: - print("Got inside %s." % name) - Event.emit_signal("snake_started_dash") - Global.SNAKE_SPEED = Global.SNAKE_DASH_SPEED - yield(get_tree().create_timer(Global.SNAKE_DASH_TIME), "timeout") - exit() - - -func exit(): - Event.emit_signal("snake_finished_dash") - Global.SNAKE_SPEED = Global.SNAKE_SPEED_BACKUP - fsm.back() - - -func physics_process(delta: float) -> void: - fsm.rotate_on_input() - fsm.player.velocity = fsm.player.direction * Global.SNAKE_SPEED - fsm.player.velocity = fsm.player.move_and_slide(fsm.player.velocity) - - fsm.slow_down_on_collisions(Global.SNAKE_DASH_SPEED) -</code></pre> -<p>Where the important parts happen in the <code>enter</code> and <code>exit</code> functions. We need to change the <code>Global.SNAKE_SPEED</code> with the <code>Global.SNAKE_DASH_SPEED</code> on <code>start</code>and start the timer for how long should the dash last. And on the <code>exit</code> we reset the <code>Global.SNAKE_SPEED</code> back to normal. There is probably a better way of updating the <code>Global.SNAKE_SPEED</code> but this works just fine.</p> -<p>For the other ones is the same. Only difference with the <code>jump_state.gd</code> is that the collision from head to body is disabled, and no rotation is allowed (by not calling the <code>rotate_on_input</code> function).</p> -<h2 id="other-minor-stuff">Other minor stuff<a class="headerlink" href="#other-minor-stuff" title="Permanent link">¶</a></h2> -<p>Not as important but worth mentioning:</p> -<ul> -<li>Added restartability function.</li> -<li>Added signals for game control: <code>game_over</code> and <code>game_start</code>, but ended not using them.</li> -<li>Fixed issue where the <em>Curve2D</em> stayed the same even when restarting by just setting an empty curve on starting the node.</li> -<li>Added a debug mode for drawing of the <em>Curve2D</em> instead of always drawing.</li> -<li>Tweaked the tracking of the snake size.</li> -<li>Tweaked the food system to contain more attributes and use a base food node.</li> -<li>Added a HUD with mini snake sprites.</li> -<li>Added a HUD for growth progress on snake body segments and abilities.</li> -<li>Refactored the nodes to make it work with <code>change_scene_to</code>, and added a main menu.</li> -<li>Added GUI for dead screen, showing the progress.</li> -</ul> -<h2 id="final-notes">Final notes<a class="headerlink" href="#final-notes" title="Permanent link">¶</a></h2> -<p>I actually didn’t finish this game (as how I visualized it), but I got it in a <em>semi-playable</em> state which is good. My big learning during this jam is the time management that it requires to plan and design a game. I lost a lot of time trying to implement some mechanics because I was facing many issues, because of my lack of practice (which was expected) as well as trying to blog and create the necessary sprites myself. Next time I should just get an asset pack and do something with it, as well as keeping the scope of my game shorter.</p> -<p>For exporting and everything else, I went with what I did for my <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1#final-notes-and-exporting">FlappyBird Godot clone: final notes and exporting</a></p>]]></content:encoded> - </item> - <item> - <title>Creating a FlappyBird clone in Godot 3.5 devlog 1</title> - <link>https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html</guid> - <pubDate>Sun, 29 May 2022 03:38:43 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Gdscript</category> - <category>Godot</category> - <description>Since I'm starting to get more into gamedev stuff, I'll start blogging about it just to stay consistent.</description> - <content:encoded><![CDATA[<p>I just have a bit of experience with <em>Godot</em> and with gamedev in general, so I started with this game as it is pretty straight forward. On a high level the main characteristics of the game are:</p> -<ul> -<li>Literally just one sprite going up and down.</li> -<li>Constant horizontal move of the world/player.</li> -<li>If you go through the gap in the pipes you score a point.</li> -<li>If you touch the pipes, the ground or go past the “ceiling” you lose.</li> -</ul> -<p>The game was originally developed with <em>Godot 4.0 alpha 8</em>, but it didn’t support HTML5 (webassembly) export… so I backported to <em>Godot 3.5 rc1</em>.</p> -<p><ins><mark>Note:</mark> I’ve updated the game to <em>Godot 4</em> and documented it on my <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_2.html">FlappyBird devlog 2</a> entry.</ins></p> -<p>Not going to specify all the details, only the needed parts and what could be confusing, as the source code is available and can be inspected; also this assumes minimal knowledge of <em>Godot</em> in general. Usually when I mention that a set/change of something it usually it’s a property and it can be found under the <em>Inspector</em> on the relevant node, unless stated otherwise; also, all scripts attached have the same name as the scenes, but in <em>snake_case</em> (scenes/nodes in <em>PascalCase</em>).</p> -<p>One thing to note, is that I started writing this when I finished the game, so it’s hard to go part by part, and it will be hard to test individual parts when going through this as everything is depending on each other. For the next devlog, I’ll do it as I go and it will include all the changes to the nodes/scripts as I was finding them, probably better idea and easier to follow.</p> -<p>The source code can be found at <a href="https://github.com/luevano/flappybirdgodot/tree/godot-3.5">luevano/flappybirdgodot#godot-3.5</a> (<code>godot-3.5</code> branch), it also contains the exported versions for HTML5, Windows and Linux (<del>be aware that the sound might be too high and I’m too lazy to make it configurable, it was the last thing I added</del> <ins>on the latest version this is fixed and audio level is configurable now</ins>). Playable on <a href="https://lorentzeus.itch.io/flappybirdgodot">itch.io</a> (<em>Godot 4</em> version):</p> -<p style="text-align:center"><iframe src="https://itch.io/embed/1551015?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/flappybirdgodot">FlappyBirdGodot by Lorentzeus</a></iframe></p> - -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#initial-setup">Initial setup</a><ul> -<li><a href="#directory-structure">Directory structure</a></li> -<li><a href="#config">Config</a><ul> -<li><a href="#default-import-settings">Default import settings</a></li> -<li><a href="#general-settings">General settings</a></li> -<li><a href="#keybindings">Keybindings</a></li> -<li><a href="#layers">Layers</a></li> -</ul> -</li> -</ul> -</li> -<li><a href="#assets">Assets</a><ul> -<li><a href="#importing">Importing</a></li> -</ul> -</li> -<li><a href="#scenes">Scenes</a><ul> -<li><a href="#tilemaps">TileMaps</a><ul> -<li><a href="#default-ground-tiles">Default ground tiles</a></li> -</ul> -</li> -<li><a href="#player">Player</a></li> -<li><a href="#other">Other</a></li> -<li><a href="#game">Game</a></li> -<li><a href="#ui">UI</a><ul> -<li><a href="#fonts">Fonts</a></li> -<li><a href="#scene-setup">Scene setup</a></li> -</ul> -</li> -<li><a href="#main">Main</a></li> -</ul> -</li> -<li><a href="#scripting">Scripting</a><ul> -<li><a href="#player_1">Player</a></li> -<li><a href="#worlddetector">WorldDetector</a></li> -<li><a href="#worldtiles">WorldTiles</a><ul> -<li><a href="#groundtilemap">GroundTileMap</a></li> -<li><a href="#pipetilemap">PipeTileMap</a></li> -</ul> -</li> -<li><a href="#saved-data">Saved data</a></li> -<li><a href="#game_1">Game</a></li> -<li><a href="#ui_1">UI</a></li> -<li><a href="#main_1">Main</a></li> -</ul> -</li> -<li><a href="#final-notes-and-exporting">Final notes and exporting</a><ul> -<li><a href="#preparing-the-files">Preparing the files</a></li> -<li><a href="#exporting">Exporting</a></li> -</ul> -</li> -</ul> -</div> -<h2 id="initial-setup">Initial setup<a class="headerlink" href="#initial-setup" title="Permanent link">¶</a></h2> -<h3 id="directory-structure">Directory structure<a class="headerlink" href="#directory-structure" title="Permanent link">¶</a></h3> -<p>I’m basically going with what I wrote on <a href="https://blog.luevano.xyz/g/godot_project_structure.html">Godot project structure</a> recently, and probably with minor changes depending on the situation.</p> -<h3 id="config">Config<a class="headerlink" href="#config" title="Permanent link">¶</a></h3> -<h4 id="default-import-settings">Default import settings<a class="headerlink" href="#default-import-settings" title="Permanent link">¶</a></h4> -<p>Since this is just pixel art, the importing settings for textures needs to be adjusted so the sprites don’t look blurry. Go to <em>Project -> Project settings… -> Import defaults</em> and on the drop down select <code>Texture</code>, untick everything and make sure <em>Compress/Mode</em> is set to <code>Lossless</code>.</p> -<figure id="__yafg-figure-11"> -<img alt="Project settings - Import defaults - Texture settings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_import_texture.png" title="Project settings - Import defaults - Texture settings"> -<figcaption>Project settings - Import defaults - Texture settings</figcaption> -</figure> -<h4 id="general-settings">General settings<a class="headerlink" href="#general-settings" title="Permanent link">¶</a></h4> -<p>It’s also a good idea to setup some config variables project-wide. To do so, go to <em>Project -> Project settings… -> General</em>, select <em>Application/config</em> and add a new property (there is a text box at the top of the project settings window) for game scale: <code>application/config/game_scale</code> for the type use <code>float</code> and then click on add; configure the new property to <code>3.0</code>; On the same window, also add <code>application/config/version</code> as a <code>string</code>, and make it <code>1.0.0</code> (or whatever number you want).</p> -<figure id="__yafg-figure-12"> -<img alt="Project settings - General - Game scale and version properties" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_config_properties.png" title="Project settings - General - Game scale and version properties"> -<figcaption>Project settings - General - Game scale and version properties</figcaption> -</figure> -<p>For my personal preferences, also disable some of the <em>GDScript</em> debug warnings that are annoying, this is done at <em>Project -> Project settings… -> General</em>, select <em>Debug/GDScript</em> and toggle off <code>Unused arguments</code>, <code>Unused signal</code> and <code>Return value discarded</code>, and any other that might come up too often and don’t want to see.</p> -<figure id="__yafg-figure-13"> -<img alt="Project settings - General - GDScript debug warnings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_debug_gdscript.png" title="Project settings - General - GDScript debug warnings"> -<figcaption>Project settings - General - GDScript debug warnings</figcaption> -</figure> -<p>Finally, set the initial window size in <em>Project -> Project settings… -> General</em>, select <em>Display/Window</em> and set <em>Size/Width</em> and <em>Size/Height</em> to <code>600</code> and <code>800</code>, respectively. As well as the <em>Stretch/Mode</em> to <code>viewport</code> , and <em>Stretch/Aspect</em> to <code>keep</code>:</p> -<figure id="__yafg-figure-14"> -<img alt="Project settings - General - Initial window size" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_window_settings.png" title="Project settings - General - Initial window size"> -<figcaption>Project settings - General - Initial window size</figcaption> -</figure> -<h4 id="keybindings">Keybindings<a class="headerlink" href="#keybindings" title="Permanent link">¶</a></h4> -<p>I only used 3 actions (keybindings): jump, restart and toggle_debug (optional). To add custom keybindings (so that the <code>Input.something()</code> API can be used), go to <em>Project -> Project settings… -> Input Map</em> and on the text box write <code>jump</code> and click add, then it will be added to the list and it’s just a matter of clicking the <code>+</code> sign to add a <em>Physical key</em>, press any key you want to be used to jump and click ok. Do the same for the rest of the actions.</p> -<figure id="__yafg-figure-15"> -<img alt="Project settings - Input Map - Adding necessary keybindings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_input_map.png" title="Project settings - Input Map - Adding necessary keybindings"> -<figcaption>Project settings - Input Map - Adding necessary keybindings</figcaption> -</figure> -<h4 id="layers">Layers<a class="headerlink" href="#layers" title="Permanent link">¶</a></h4> -<p>Finally, rename the physics layers so we don’t lose track of which layer is which. Go to <em>Project -> Layer Names -> 2d Physics</em> and change the first 5 layer names to (in order): <code>player</code>, <code>ground</code>, <code>pipe</code>, <code>ceiling</code> and <code>score</code>.</p> -<figure id="__yafg-figure-16"> -<img alt="Project settings - Layer Names - 2D Physics" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_layer_names_2d_physics.png" title="Project settings - Layer Names - 2D Physics"> -<figcaption>Project settings - Layer Names - 2D Physics</figcaption> -</figure> -<h2 id="assets">Assets<a class="headerlink" href="#assets" title="Permanent link">¶</a></h2> -<p>For the assets I found out about a pack that contains just what I need: <a href="https://megacrash.itch.io/flappy-bird-assets">flappy-bird-assets</a> by <a href="https://megacrash.itch.io/">MegaCrash</a>; I just did some minor modifications on the naming of the files. For the font I used <a href="https://poppyworks.itch.io/silver">Silver</a>, and for the sound the resources from <a href="https://github.com/meeq/FlappyBird-N64">FlappyBird-N64</a> (which seems to be taken from <a href="https://www.101soundboards.com/boards/10178-flappy-bird-sounds">101soundboards.com</a> which the orignal copyright holder is <a href="https://dotgears.com/">.Gears</a> anyways).</p> -<h3 id="importing">Importing<a class="headerlink" href="#importing" title="Permanent link">¶</a></h3> -<p>Create the necessary directories to hold the respective assets and it’s just a matter of dragging and dropping, I used directories: <code>res://entities/actors/player/sprites/</code>, <code>res://fonts/</code>, <code>res://levels/world/background/sprites/</code>, <code>res://levels/world/ground/sprites/</code>, <code>res://levels/world/pipe/sprites/</code>, <code>res://sfx/</code>. For the player sprites, the -<em>FileSystem</em> window looks like this (<code>entities/actor</code> directories are really not necessary):</p> -<figure id="__yafg-figure-17"> -<img alt="FileSystem - Player sprite imports" src="https://static.luevano.xyz/images/g/flappybird_godot/player_sprite_imports.png" title="FileSystem - Player sprite imports"> -<figcaption>FileSystem - Player sprite imports</figcaption> -</figure> -<p>It should look similar for other directories, except maybe for the file extensions. For example, for the sfx:</p> -<figure id="__yafg-figure-18"> -<img alt="FileSystem - SFX imports" src="https://static.luevano.xyz/images/g/flappybird_godot/sfx_imports.png" title="FileSystem - SFX imports"> -<figcaption>FileSystem - SFX imports</figcaption> -</figure> -<h2 id="scenes">Scenes<a class="headerlink" href="#scenes" title="Permanent link">¶</a></h2> -<p>Now it’s time to actually create the game, by creating the basic scenes that will make up the game. The hardest part and the most confusing is going to be the <em>TileMaps</em>, so that goes first.</p> -<h3 id="tilemaps">TileMaps<a class="headerlink" href="#tilemaps" title="Permanent link">¶</a></h3> -<p>I’m using a scene called <code>WorldTiles</code> with a <em>Node2D</em> node as root called the same. With 2 different <em>TileMap</em> nodes as children named <code>GroundTileMap</code> and <code>PipeTileMap</code> (these are their own scene); yes 2 different <em>TileMaps</em> because we need 2 different physics colliders (in <em>Godot 4.0</em> you can have a single <em>TileMap</em> with different physics colliders in it). Each node has its own script. It should look something like this:</p> -<figure id="__yafg-figure-19"> -<img alt="Scene - WorldTiles (TileMaps)" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_world_tiles.png" title="Scene - WorldTiles (TileMaps)"> -<figcaption>Scene - WorldTiles (TileMaps)</figcaption> -</figure> -<p>I used the following directory structure:</p> -<figure id="__yafg-figure-20"> -<img alt="Scene - WorldTiles - Directory structure" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_world_tiles_directory_structure.png" title="Scene - WorldTiles - Directory structure"> -<figcaption>Scene - WorldTiles - Directory structure</figcaption> -</figure> -<p>To configure the <code>GroundTileMap</code>, select the node and click on <code>(empty)</code> on the <em>TileMap/Tile set</em> property and then click on <code>New TileSet</code>, then click where the <code>(empty)</code> used to be, a new window should open on the bottom:</p> -<figure id="__yafg-figure-21"> -<img alt="TileSet - Configuration window" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_config_window.png" title="TileSet - Configuration window"> -<figcaption>TileSet - Configuration window</figcaption> -</figure> -<p>Click on the plus on the bottom left and you can now select the specific tile set to use. Now click on the yellow <code>+ New Single Tile</code>, activate the grid and select any of the tiles. Should look like this:</p> -<figure id="__yafg-figure-22"> -<img alt="TileSet - New single tile" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_new_single_tile.png" title="TileSet - New single tile"> -<figcaption>TileSet - New single tile</figcaption> -</figure> -<p>We need to do this because for some reason we can’t change the snap options before selecting a tile. After selecting a random tile, set up the <em>Snap Options/Step</em> (in the <em>Inspector</em>) and set it to <code>16x16</code> (or if using a different tile set, to it’s tile size):</p> -<figure id="__yafg-figure-23"> -<img alt="TileSet - Tile - Step snap options" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_tile_step_snap_options.png" title="TileSet - Tile - Step snap options"> -<figcaption>TileSet - Tile - Step snap options</figcaption> -</figure> -<p>Now you can select the actual single tile. Once selected click on <code>Collision</code>, use the rectangle tool and draw the rectangle corresponding to that tile’s collision:</p> -<figure id="__yafg-figure-24"> -<img alt="TileSet - Tile - Selection and collision" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_tile_selection_collision.png" title="TileSet - Tile - Selection and collision"> -<figcaption>TileSet - Tile - Selection and collision</figcaption> -</figure> -<p>Do the same for the other 3 tiles. If you select the <em>TileMap</em> itself again, it should look like this on the right (on default layout it’s on the left of the <em>Inspector</em>):</p> -<figure id="__yafg-figure-25"> -<img alt="TileSet - Available tiles" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_available_tiles.png" title="TileSet - Available tiles"> -<figcaption>TileSet - Available tiles</figcaption> -</figure> -<p>The ordering is important only for the “underground tile”, which is the filler ground, it should be at the end (index 3); if this is not the case, repeat the process (it’s possible to rearrange them but it’s hard to explain as it’s pretty weird).</p> -<p>At this point the tilemap doesn’t have any physics and the cell size is wrong. Select the <code>GroundTileMap</code>, set the <em>TileMap/Cell/Size</em> to <code>16x16</code>, the <em>TileMap/Collision/Layer</em> set to <code>bit 2</code> only (ground layer) and disable any <em>TileMap/Collision/Mask</em> bits. Should look something like this:</p> -<figure id="__yafg-figure-26"> -<img alt="TileMap - Cell size and collision configuration" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_map_cell_collision_configuration.png" title="TileMap - Cell size and collision configuration"> -<figcaption>TileMap - Cell size and collision configuration</figcaption> -</figure> -<p>Now it’s just a matter of repeating the same for the pipes (<code>PipeTileMap</code>), only difference is that when selecting the tiles you need to select 2 tiles, as the pipe is 2 tiles wide, or just set the <em>Snap Options/Step</em> to <code>32x16</code>, for example, just keep the cell size to <code>16x16</code>.</p> -<h4 id="default-ground-tiles">Default ground tiles<a class="headerlink" href="#default-ground-tiles" title="Permanent link">¶</a></h4> -<p>I added few default ground tiles to the scene, just for testing purposes but I left them there. These could be place programatically, but I was too lazy to change things. On the <code>WorldTiles</code> scene, while selecting the <code>GroundTileMap</code>, you can select the tiles you want to paint with, and left click in the grid to paint with the selected tile. Need to place tiles from <code>(-8, 7)</code> to <code>(10, 7)</code> as well as the tile below with the filler ground (the tile position/coordinates show at the bottom left, refer to the image below):</p> -<figure id="__yafg-figure-27"> -<img alt="Scene - WorldTiles - Default ground tiles" src="https://static.luevano.xyz/images/g/flappybird_godot/world_tiles_default_tiles.png" title="Scene - WorldTiles - Default ground tiles"> -<figcaption>Scene - WorldTiles - Default ground tiles</figcaption> -</figure> -<h3 id="player">Player<a class="headerlink" href="#player" title="Permanent link">¶</a></h3> -<p>On a new scene called <code>Player</code> with a <em>KinematicBody2D</em> node named <code>Player</code> as the root of the scene, then for the children: <em>AnimatedSprite</em> as <code>Sprite</code>, <em>CollisionShape2D</em> as <code>Collision</code> (with a circle shape) and 3 <em>AudioStreamPlayers</em> for <code>JumpSound</code>, <code>DeadSound</code> and <code>HitSound</code>. Not sure if it’s a good practice to have the audio here, since I did that at the end, pretty lazy. Then, attach a script to the <code>Player</code> node and then it should look like this:</p> -<figure id="__yafg-figure-28"> -<img alt="Scene - Player - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_node_setup.png" title="Scene - Player - Node setup"> -<figcaption>Scene - Player - Node setup</figcaption> -</figure> -<p>Select the <code>Player</code> node and set the <em>CollisionShape2D/Collision/Layer</em> to <code>1</code> and the <em>CollisionObject2D/Collision/Mask</em> to <code>2</code> and <code>3</code> (ground and pipe).</p> -<p>For the <code>Sprite</code> node, when selecting it click on the <code>(empty)</code> for the <em>AnimatedSprite/Frames</em> property and click <code>New SpriteFrames</code>, click again where the <code>(empty)</code> used to be and ane window should open on the bottom:</p> -<figure id="__yafg-figure-29"> -<img alt="Scene - Player - SpriteFrames window" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_spriteframes_window.png" title="Scene - Player - SpriteFrames window"> -<figcaption>Scene - Player - SpriteFrames window</figcaption> -</figure> -<p>Right off the bat, set the <code>Speed</code> to <code>10 FPS</code> (bottom left) and rename <code>default</code> to <code>bird_1</code>. With the <code>bird_1</code> selected, click on the <code>Add frames from a Sprite Sheet</code>, which is the second button under <code>Animation Frames:</code> which looks has an icon of a small grid (next to the folder icon), a new window will popup where you need to select the respective sprite sheet to use and configure it for importing. On the <code>Select Frames</code> window, change the <code>Vertical</code> to <code>1</code>, and then select all 4 frames (<em>Ctrl + Scroll</em> wheel to zoom in):</p> -<figure id="__yafg-figure-30"> -<img alt="Scene - Player - Sprite sheet importer" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_sprite_sheet_importer.png" title="Scene - Player - Sprite sheet importer"> -<figcaption>Scene - Player - Sprite sheet importer</figcaption> -</figure> -<p>After that, the <em>SpriteFrames</em> window should look like this:</p> -<figure id="__yafg-figure-31"> -<img alt="Scene - Player - SpriteFrames window with sprite sheet configured" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_spriteframes_window_with_sprite_sheet.png" title="Scene - Player - SpriteFrames window with sprite sheet configured"> -<figcaption>Scene - Player - SpriteFrames window with sprite sheet configured</figcaption> -</figure> -<p>Finally, make sure the <code>Sprite</code> node has the <em>AnimatedSprite/Animation</em> is set to <code>bird_1</code> and that the <code>Collision</code> node is configured correctly for its size and position (I just have it as a radius of <code>7</code>). As well as dropping the SFX files into the corresponding <em>AudioStreamPlayer</em> (into the <em>AudioStreamPlayer/Stream</em> property).</p> -<h3 id="other">Other<a class="headerlink" href="#other" title="Permanent link">¶</a></h3> -<p>These are really simple scenes that don’t require much setup:</p> -<ul> -<li><code>CeilingDetector</code>: just an <em>Area2D</em> node with a <em>CollisionShape2D</em> in the form of a rectangle (<em>CollisionShape2D/Shape/extents</em> to <code>(120, 10)</code>), stretched horizontally so it fits the whole screen. <em>CollisionObject2D/Collision/Layer</em> set to <code>bit 4</code> (ceiling) and <em>CollisionObject2D/Collision/Mask</em> set to <code>bit 1</code> (player).</li> -<li><code>ScoreDetector</code>: similar to the <code>CeilingDetector</code>, but vertical (<em>CollisionShape2D/Shape/extents</em> to <code>(2.5, 128)</code>) and <em>CollisionObject2D/Collision/Layer</em> set to <code>bit 1</code> (player).</li> -<li><code>WorldDetector</code>: <em>Node2D</em> with a script attached, and 3 <em>RayCast2D</em> as children:<ul> -<li><code>NewTile</code>: <em>Raycast2D/Enabled</em> to true (checked), <em>Raycast2D/Cast To</em> <code>(0, 400)</code>, <em>Raycast2D/Collision Mask</em> to <code>bit 2</code> (ground) and <em>Node2D/Transform/Position</em> to <code>(152, -200)</code></li> -<li><code>OldTile</code>: same as “NewTile”, except for the <em>Node2D/Transform/Position</em>, set it to <code>(-152, -200)</code>.</li> -<li><code>OldPipe</code>: same as “OldTile”, except for the <em>Raycast2D/Collision Mask</em>, set it to <code>bit 3</code> (pipe).</li> -</ul> -</li> -</ul> -<h3 id="game">Game<a class="headerlink" href="#game" title="Permanent link">¶</a></h3> -<p>This is the actual <code>Game</code> scene that holds all the playable stuff, here we will drop in all the previous scenes; the root node is a <em>Node2D</em> and also has an attached script. Also need to add 2 additional <em>AudioStreamPlayers</em> for the “start” and “score” sounds, as well as a <em>Sprite</em> for the background (<em>Sprite/Offset/Offset</em> set to <code>(0, 10)</code>) and a <em>Camera2D</em> (<em>Camera2D/Current</em> set to true (checked)). It should look something like this:</p> -<figure id="__yafg-figure-32"> -<img alt="Scene - Game - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_game_node_setup.png" title="Scene - Game - Node setup"> -<figcaption>Scene - Game - Node setup</figcaption> -</figure> -<p>The scene viewport should look something like the following:</p> -<figure id="__yafg-figure-33"> -<img alt="Scene - Game - Viewport" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_game_viewport.png" title="Scene - Game - Viewport"> -<figcaption>Scene - Game - Viewport</figcaption> -</figure> -<h3 id="ui">UI<a class="headerlink" href="#ui" title="Permanent link">¶</a></h3> -<h4 id="fonts">Fonts<a class="headerlink" href="#fonts" title="Permanent link">¶</a></h4> -<p>We need some font <code>Resources</code> to style the <em>Label</em> fonts. Under the <em>FileSystem</em> window, right click on the fonts directory (create one if needed) and click on <code>New Resource...</code> and select <em>DynamicFontData</em>, save it in the “fonts” directory as <code>SilverDynamicFontData.tres</code> (<code>Silver</code> as it is the font I’m using) then double click the just created resource and set the <em>DynamicFontData/Font Path</em> to the actual <code>Silver.ttf</code> font (or whatever you want).</p> -<p>Then create a new resource and this time select <em>DynamicFont</em>, name it <code>SilverDynamicFont.tres</code>, then double click to edit and add the <code>SilverDynamicFontData.tres</code> to the <em>DynamicFont/Font/Font Data</em> property (and I personally toggled off the <em>DynamicFont/Font/Antialiased</em> property), now just set the <em>DynamicFont/Settings/(Size, Outline Size, Outline Color)</em> to <code>32</code>, <code>1</code> and <code>black</code>, respectively (or any other values you want). It should look something like this:</p> -<figure id="__yafg-figure-34"> -<img alt="Resource - DynamicFont - Default font" src="https://static.luevano.xyz/images/g/flappybird_godot/resource_dynamic_font.png" title="Resource - DynamicFont - Default font"> -<figcaption>Resource - DynamicFont - Default font</figcaption> -</figure> -<p>Do the same for another <em>DynamicFont</em> which will be used for the score label, named <code>SilverScoreDynamicFont.tres</code>. Only changes are <em>Dynamic/Settings/(Size, Outline Size)</em> which are set to <code>128</code> and <code>2</code>, respectively. The final files for the fonts should look something like this:</p> -<figure id="__yafg-figure-35"> -<img alt="Resource - Dynamicfont - Directory structure" src="https://static.luevano.xyz/images/g/flappybird_godot/resource_dynamic_font_directory_structure.png" title="Resource - Dynamicfont - Directory structure"> -<figcaption>Resource - Dynamicfont - Directory structure</figcaption> -</figure> -<h4 id="scene-setup">Scene setup<a class="headerlink" href="#scene-setup" title="Permanent link">¶</a></h4> -<p>This has a bunch of nested nodes, so I’ll try to be concise here. The root node is a <em>CanvasLayer</em> named <code>UI</code> with its own script attached, and for the children:</p> -<ul> -<li><code>MarginContainer</code>: <em>MarginContainer</em> with <em>Control/Margin/(Left, Top)</em> set to <code>10</code> and <em>Control/Margin/(Right, Bottom)</em> set to <code>-10</code>.<ul> -<li><code>InfoContainer</code>: <em>VBoxContainer</em> with <em>Control/Theme Overrides/Constants/Separation</em> set to <code>250</code>.<ul> -<li><code>ScoreContainer</code>: <em>VBoxContainer</em>.<ul> -<li><code>Score</code>: <em>Label</em> with <em>Label/Align</em> set to <code>Center</code>, <em>Control/Theme Overrides/Fonts/Font</em> to the <code>SilverScoreDynamicFont.tres</code>, if needed adjust the <em>DynamicFont</em> settings.</li> -<li><code>HighScore</code>: same as <code>Score</code>, escept for the <em>Control/Theme Overrides/Fonts/Font</em> which is set to <code>SilverDynamicFont.tres</code>.</li> -</ul> -</li> -<li><code>StartGame</code>: Same as <code>HighScore</code>.</li> -</ul> -</li> -<li><code>DebugContainer</code>: <em>VBoxContainer</em>.<ul> -<li><code>FPS</code>: <em>Label</em>.</li> -</ul> -</li> -<li><code>VersionContainer</code>: <em>VBoxContainer</em> with <em>BoxContainer/Alignment</em> set to <code>Begin</code>.<ul> -<li><code>Version</code>: <em>Label</em> with <em>Label/Align</em> set to <code>Right</code>.</li> -</ul> -</li> -</ul> -</li> -</ul> -<p>The scene ends up looking like this:</p> -<figure id="__yafg-figure-36"> -<img alt="Scene - UI - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_ui.png" title="Scene - UI - Node setup"> -<figcaption>Scene - UI - Node setup</figcaption> -</figure> -<h3 id="main">Main<a class="headerlink" href="#main" title="Permanent link">¶</a></h3> -<p>This is the final scene where we connect the <code>Game</code> and the <code>UI</code>. It’s made of a <em>Node2D</em> with it’s own script attached and an instance of <code>Game</code> and <code>UI</code> as it’s children.</p> -<p>This is a good time to set the default scene when we run the game by going to <em>Project -> Project settings… -> General</em> and in <em>Application/Run</em> set the <em>Main Scene</em> to the <code>Main.tscn</code> scene.</p> -<h2 id="scripting">Scripting<a class="headerlink" href="#scripting" title="Permanent link">¶</a></h2> -<p>I’m going to keep this scripting part to the most basic code blocks, as it’s too much code, for a complete view you can head to the <a href="https://github.com/luevano/flappybird_godot/tree/godot-3.5">source code</a>.</p> -<p>As of now, the game itself doesn’t do anything if we hit play. The first thing to do so we have something going on is to do the minimal player scripting.</p> -<h3 id="player_1">Player<a class="headerlink" href="#player_1" title="Permanent link">¶</a></h3> -<p>The most basic code needed so the bird goes up and down is to just detect <code>jump</code> key presses and add a negative jump velocity so it goes up (<code>y</code> coordinate is reversed in godot…), we also check the velocity sign of the <code>y</code> coordinate to decide if the animation is playing or not.</p> -<pre><code class="language-gdscript">class_name Player -extends KinematicBody2D - -export(float, 1.0, 1000.0, 1.0) var JUMP_VELOCITY: float = 380.0 - -onready var sprite: AnimatedSprite = $Sprite - -var gravity: float = 10 * ProjectSettings.get_setting("physics/2d/default_gravity") -var velocity: Vector2 = Vector2.ZERO - - -func _physics_process(delta: float) -> void: - velocity.y += gravity * delta - - if Input.is_action_just_pressed("jump"): - velocity.y = -JUMP_VELOCITY - - if velocity.y < 0.0: - sprite.play() - else: - sprite.stop() - - velocity = move_and_slide(velocity) -</code></pre> -<p>You can play it now and you should be able to jump up and down, and the bird should stop on the ground (although you can keep jumping). One thing to notice is that when doing <code>sprite.stop()</code> it stays on the last frame, we can fix that using the code below (and then change <code>sprite.stop()</code> for <code>_stop_sprite()</code>):</p> -<pre><code class="language-gdscript">func _stop_sprite() -> void: - if sprite.playing: - sprite.stop() - if sprite.frame != 0: - sprite.frame = 0 -</code></pre> -<p>Where we just check that the last frame has to be the frame 0.</p> -<p>Now just a matter of adding other needed code for moving horizontally, add sound by getting a reference to the <em>AudioStreamPlayers</em> and doing <code>sound.play()</code> when needed, as well as handling death scenarios by adding a <code>signal died</code> at the beginning of the script and handle any type of death scenario using the below function:</p> -<pre><code class="language-gdscript">func _emit_player_died() -> void: - # bit 2 corresponds to pipe (starts from 0) - set_collision_mask_bit(2, false) - dead = true - SPEED = 0.0 - emit_signal("died") - # play the sounds after, because yield will take a bit of time, - # this way the camera stops when the player "dies" - velocity.y = -DEATH_JUMP_VELOCITY - velocity = move_and_slide(velocity) - hit_sound.play() - yield(hit_sound, "finished") - dead_sound.play() -</code></pre> -<p>Finally need to add the actual checks for when the player dies (like collision with ground or pipe) as well as a function that listens to a signal for when the player goes to the ceiling.</p> -<h3 id="worlddetector">WorldDetector<a class="headerlink" href="#worlddetector" title="Permanent link">¶</a></h3> -<p>The code is pretty simple, we just need a way of detecting if we ran out of ground and send a signal, as well as sending as signal when we start detecting ground/pipes behind us (to remove it) because the world is being generated as we move. The most basic functions needed are:</p> -<pre><code class="language-gdscript">func _was_colliding(detector: RayCast2D, flag: bool, signal_name: String) -> bool: - if detector.is_colliding(): - return true - if flag: - emit_signal(signal_name) - return false - return true - - -func _now_colliding(detector: RayCast2D, flag: bool, signal_name: String) -> bool: - if detector.is_colliding(): - if not flag: - emit_signal(signal_name) - return true - return false -</code></pre> -<p>We need to keep track of 3 “flags”: <code>ground_was_colliding</code>, <code>ground_now_colliding</code> and <code>pipe_now_colliding</code> (and their respective signals), which are going to be used to do the checks inside <code>_physics_process</code>. For example for checking for new ground: <code>ground_now_colliding = _now_colliding(old_ground, ground_now_colliding, "ground_started_colliding")</code>.</p> -<h3 id="worldtiles">WorldTiles<a class="headerlink" href="#worldtiles" title="Permanent link">¶</a></h3> -<p>This script is what handles the <code>GroundTileMap</code> as well as the <code>PipeTileMap</code> and just basically functions as a “Signal bus” connecting a bunch of signals from the <code>WorldDetector</code> with the <em>TileMaps</em> and just tracking how many pipes have been placed:</p> -<pre><code class="language-gdscript">export(int, 2, 20, 2) var PIPE_SEP: int = 6 -var tiles_since_last_pipe: int = PIPE_SEP - 1 - - -func _on_WorldDetector_ground_stopped_colliding() -> void: - emit_signal("place_ground") - - tiles_since_last_pipe += 1 - if tiles_since_last_pipe == PIPE_SEP: - emit_signal("place_pipe") - tiles_since_last_pipe = 0 - - -func _on_WorldDetector_ground_started_colliding() -> void: - emit_signal("remove_ground") - - -func _on_WorldDetector_pipe_started_colliding() -> void: - emit_signal("remove_pipe") -</code></pre> -<h4 id="groundtilemap">GroundTileMap<a class="headerlink" href="#groundtilemap" title="Permanent link">¶</a></h4> -<p>This is the node that actually places the ground tiles upong receiving a signal. In general, what you want is to keep track of the newest tile that you need to place (empty spot) as well as the last tile that is in the tilemap (technically the first one if you count from left to right). I was experimenting with <code>enum</code>s so I used them to define the possible <code>Ground</code> tiles:</p> -<pre><code class="language-gdscript">enum Ground { - TILE_1, - TILE_2, - TILE_3, - TILE_DOWN_1, -} -</code></pre> -<p>This way you can just select the tile by doing <code>Ground.TILE_1</code>, which will correspond to the <code>int</code> value of <code>0</code>. So most of the code is just:</p> -<pre><code class="language-gdscript"># old_tile is the actual first tile, whereas the new_tile_position -# is the the next empty tile; these also correspond to the top tile -const _ground_level: int = 7 -const _initial_old_tile_x: int = -8 -const _initial_new_tile_x: int = 11 -var old_tile_position: Vector2 = Vector2(_initial_old_tile_x, _ground_level) -var new_tile_position: Vector2 = Vector2(_initial_new_tile_x, _ground_level) - - -func _place_new_ground() -> void: - set_cellv(new_tile_position, _get_random_ground()) - set_cellv(new_tile_position + Vector2.DOWN, Ground.TILE_DOWN_1) - new_tile_position += Vector2.RIGHT - - -func _remove_first_ground() -> void: - set_cellv(old_tile_position, -1) - set_cellv(old_tile_position + Vector2.DOWN, -1) - old_tile_position += Vector2.RIGHT -</code></pre> -<p>Where you might notice that the <code>_initial_new_tile_x</code> is <code>11</code>, instead of <code>10</code>, refer to <a href="#default-ground-tiles">Default ground tiles</a> where we placed tiles from <code>-8</code> to <code>10</code>, so the next empty one is <code>11</code>. These <code>_place_new_ground</code> and <code>_remove_first_ground</code> functions are called upon receiving the signal.</p> -<h4 id="pipetilemap">PipeTileMap<a class="headerlink" href="#pipetilemap" title="Permanent link">¶</a></h4> -<p>This is really similar to the <code>GroundTileMap</code> code, instead of defining an <code>enum</code> for the ground tiles, we define it for the pipe patterns (because each pipe is composed of multiple pipe tiles). If your pipe tile set looks like this (notice the index):</p> -<figure id="__yafg-figure-37"> -<img alt="PipeTileMap - Tile set indexes" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_pipes_indexes.png" title="PipeTileMap - Tile set indexes"> -<figcaption>PipeTileMap - Tile set indexes</figcaption> -</figure> -<p>Then you can use the following “pipe patterns”:</p> -<pre><code class="language-gdscript">var pipe: Dictionary = { - PipePattern.PIPE_1: [0, 1, 2, 2, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2], - PipePattern.PIPE_2: [0, 1, 2, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2], - PipePattern.PIPE_3: [0, 1, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2], - PipePattern.PIPE_4: [0, 1, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2], - PipePattern.PIPE_5: [0, 1, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2, 2], - PipePattern.PIPE_6: [0, 1, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2, 2, 2] -} -</code></pre> -<p>Now, the pipe system requires a bit more of tracking as we need to instantiate a <code>ScoreDetector</code> here, too. I ended up keeping track of the placed pipes/detectors by using a “pipe stack” (and “detector stack”) which is just an array of placed objects from which I pop the first when deleting them:</p> -<pre><code class="language-gdscript">onready var _pipe_sep: int = get_parent().PIPE_SEP -const _pipe_size: int = 16 -const _ground_level: int = 7 -const _pipe_level_y: int = _ground_level - 1 -const _initial_new_pipe_x: int = 11 -var new_pipe_starting_position: Vector2 = Vector2(_initial_new_pipe_x, _pipe_level_y) -var pipe_stack: Array - -# don't specify type for game, as it results in cyclic dependency, -# as stated here: https://godotengine.org/qa/39973/cyclic-dependency-error-between-actor-and-actor-controller -onready var game = get_parent().get_parent() -var detector_scene: PackedScene = preload("res://levels/detectors/score_detector/ScoreDetector.tscn") -var detector_offset: Vector2 = Vector2(16.0, -(_pipe_size / 2.0) * 16.0) -var detector_stack: Array -</code></pre> -<p>The <code>detector_offset</code> is just me being picky. For placing a new pipe, we get the starting position (bottom pipe tile) and build upwards, then instantiate a new <code>ScoreDetector</code> (<code>detector_scene</code>) and set it’s position to the pipe starting position plus the offset, so it’s centered in the pipe, then just need to connect the <code>body_entered</code> signal from the detector with the game, so we keep track of the scoring. Finally just add the placed pipe and detector to their corresponding stacks:</p> -<pre><code class="language-gdscript">func _place_new_pipe() -> void: - var current_pipe: Vector2 = new_pipe_starting_position - for tile in pipe[_get_random_pipe()]: - set_cellv(current_pipe, tile) - current_pipe += Vector2.UP - - var detector: Area2D = detector_scene.instance() - detector.position = map_to_world(new_pipe_starting_position) + detector_offset - detector.connect("body_entered", game, "_on_ScoreDetector_body_entered") - detector_stack.append(detector) - add_child(detector) - - pipe_stack.append(new_pipe_starting_position) - new_pipe_starting_position += _pipe_sep * Vector2.RIGHT -</code></pre> -<p>For removing pipes, it’s really similar but instead of getting the position from the next tile, we pop the first element from the (pipe/detector) stack and work with that. To remove the cells we just set the index to <code>-1</code>:</p> -<pre><code class="language-gdscript">func _remove_old_pipe() -> void: - var current_pipe: Vector2 = pipe_stack.pop_front() - var c: int = 0 - while c < _pipe_size: - set_cellv(current_pipe, -1) - current_pipe += Vector2.UP - c += 1 - - var detector: Area2D = detector_stack.pop_front() - remove_child(detector) - detector.queue_free() -</code></pre> -<p>These functions are called when receiving the signal to place/remove pipes.</p> -<h3 id="saved-data">Saved data<a class="headerlink" href="#saved-data" title="Permanent link">¶</a></h3> -<p>Before proceeding, we require a way to save/load data (for the high scores). We’re going to use the <em>ConfigFile</em> node that uses a custom version of the <code>ini</code> file format. Need to define where to save the data:</p> -<pre><code class="language-gdscript">const DATA_PATH: String = "user://data.cfg" -const SCORE_SECTION: String = "score" -var _data: ConfigFile -</code></pre> -<p>Note that <code>user://</code> is a OS specific path in which the data can be stored on a per user basis, for more: <a href="https://docs.godotengine.org/en/stable/tutorials/io/data_paths.html">File paths</a>. Then, a way to load the save file:</p> -<pre><code class="language-gdscript">func _load_data() -> void: - # create an empty file if not present to avoid error while loading settings - var file: File = File.new() - if not file.file_exists(DATA_PATH): - file.open(DATA_PATH, file.WRITE) - file.close() - - _data = ConfigFile.new() - var err: int = _data.load(DATA_PATH) - if err != OK: - print("[ERROR] Cannot load data.") -</code></pre> -<p>A way to save the data:</p> -<pre><code class="language-gdscript">func save_data() -> void: - var err: int = _data.save(DATA_PATH) - if err != OK: - print("[ERROR] Cannot save data.") -</code></pre> -<p>And of course, a way to get and set the high score:</p> -<pre><code class="language-gdscript">func set_new_high_score(high_score: int) -> void: - _data.set_value(SCORE_SECTION, "high_score", high_score) - - -func get_high_score() -> int: - return _data.get_value(SCORE_SECTION, "high_score") -</code></pre> -<p>Then, whenever this script is loaded we load the data and if it’s a new file, then add the default high score of 0:</p> -<pre><code class="language-gdscript">func _ready() -> void: - _load_data() - - if not _data.has_section(SCORE_SECTION): - set_new_high_score(0) - save_data() -</code></pre> -<p>Now, this script in particular will need to be a <a href="https://docs.godotengine.org/en/stable/tutorials/scripting/singletons_autoload.html">Singleton (AutoLoad)</a>, which means that there will be only one instance and will be available across all scripts. To do so, go to <em>Project -> Project settings… -> AutoLoad</em> and select this script in the <code>Path:</code> and add a <code>Node Name:</code> (I used <code>SavedData</code>, if you use something else, be careful while following this devlog) which will be the name we’ll use to access the singleton. Toggle on <code>Enable</code> if needed, it should look like this:</p> -<figure id="__yafg-figure-38"> -<img alt="Project settings - AutoLoad - SavedData singleton" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_autoload_saved_data.png" title="Project settings - AutoLoad - SavedData singleton"> -<figcaption>Project settings - AutoLoad - SavedData singleton</figcaption> -</figure> -<h3 id="game_1">Game<a class="headerlink" href="#game_1" title="Permanent link">¶</a></h3> -<p>The game script it’s also like a “Signal bus” in the sense that it connects all its childs’ signals together, and also has the job of starting/stopping the <code>_process</code> and <code>_physics_process</code> methods from the childs as needed. First, we need to define the signals and and references to all child nodes:</p> -<pre><code class="language-gdscript">signal game_started -signal game_over -signal new_score(score, high_score) - -onready var player: Player = $Player -onready var background: Sprite= $Background -onready var world_tiles: WorldTiles = $WorldTiles -onready var ceiling_detector: Area2D = $CeilingDetector -onready var world_detector: Node2D = $WorldDetector -onready var camera: Camera2D = $Camera -onready var start_sound: AudioStreamPlayer = $StartSound -onready var score_sound: AudioStreamPlayer = $ScoreSound -</code></pre> -<p>It’s important to get the actual “player speed”, as we’re using a scale to make the game look bigger (remember, pixel art), to do so we need a reference to the <code>game_scale</code> we setup at the beginning and compute the <code>player_speed</code>:</p> -<pre><code class="language-gdscript">var _game_scale: float = ProjectSettings.get_setting("application/config/game_scale") -var player_speed: float - - -func _ready() -> void: - scale = Vector2(_game_scale, _game_scale) - # so we move at the actual speed of the player - player_speed = player.SPEED / _game_scale -</code></pre> -<p>This <code>player_speed</code> will be needed as we need to move all the nodes (<code>Background</code>, <code>Camera</code>, etc.) in the <code>x</code> axis as the player is moving. This is done in the <code>_physics_process</code>:</p> -<pre><code class="language-gdscript">func _physics_process(delta: float) -> void: - ceiling_detector.move_local_x(player_speed * delta) - world_detector.move_local_x(player_speed * delta) - background.move_local_x(player_speed * delta) - camera.move_local_x(player_speed * delta) -</code></pre> -<p>We also need a way to start and stop the processing of all the nodes:</p> -<pre><code class="language-gdscript">func _set_processing_to(on_off: bool, include_player: bool = true) -> void: - set_process(on_off) - set_physics_process(on_off) - if include_player: - player.set_process(on_off) - player.set_physics_process(on_off) - world_tiles.set_process(on_off) - world_tiles.set_physics_process(on_off) - ceiling_detector.set_process(on_off) - ceiling_detector.set_physics_process(on_off) -</code></pre> -<p>Where the <code>player</code> is a special case, as when the player dies, it should still move (only down), else it would just freeze in place. In <code>_ready</code> we connect all the necessary signals as well as initially set the processing to <code>false</code> using the last function. To start/restart the game we need to keep a flag called <code>is_game_running</code> initially set to <code>false</code> and then handle the (re)startability in <code>_input</code>:</p> -<pre><code class="language-gdscript">func _input(event: InputEvent) -> void: - if not is_game_running and event.is_action_pressed("jump"): - _set_processing_to(true) - is_game_running = true - emit_signal("game_started") - start_sound.play() - - if event.is_action_pressed("restart"): - get_tree().reload_current_scene() -</code></pre> -<p>Then we handle two specific signals:</p> -<pre><code class="language-gdscript">func _on_Player_died() -> void: - _set_processing_to(false, false) - emit_signal("game_over") - - -func _on_ScoreDetector_body_entered(body: Node2D) -> void: - score += 1 - if score > high_score: - high_score = score - SavedData.set_new_high_score(high_score) - SavedData.save_data() - emit_signal("new_score", score, high_score) - score_sound.play() -</code></pre> -<p>When the <code>player</code> dies, we set all processing to <code>false</code>, except for the player itself (so it can drop all the way to the ground). Also, when receiving a “scoring” signal, we manage the current score, as well as saving the new high score when applicable, note that we need to read the <code>high_score</code> at the beginning by calling <code>SavedData.get_high_score()</code>. This signal we emit will be received by the <code>UI</code> so it updates accordingly.</p> -<h3 id="ui_1">UI<a class="headerlink" href="#ui_1" title="Permanent link">¶</a></h3> -<p>First thing is to get a reference to all the child <em>Labels</em>, an initial reference to the high score as well as the version defined in the project settings:</p> -<pre><code class="language-gdscript">onready var fps_label: Label = $MarginContainer/DebugContainer/FPS -onready var version_label: Label = $MarginContainer/VersionContainer/Version -onready var score_label: Label = $MarginContainer/InfoContainer/ScoreContainer/Score -onready var high_score_label: Label = $MarginContainer/InfoContainer/ScoreContainer/HighScore -onready var start_game_label: Label = $MarginContainer/InfoContainer/StartGame - -onready var _initial_high_score: int = SavedData.get_high_score() - -var _version: String = ProjectSettings.get_setting("application/config/version") -</code></pre> -<p>Then set the initial <em>Label</em> values as well as making the <code>fps_label</code> invisible:</p> -<pre><code class="language-gdscript">func _ready() -> void: - fps_label.visible = false - version_label.set_text("v%s" % _version) - high_score_label.set_text("High score: %s" % _initial_high_score) -</code></pre> -<p>Now we need to handle the <code>fps_label</code> update and toggle:</p> -<pre><code class="language-gdscript">func _input(event: InputEvent) -> void: - if event.is_action_pressed("toggle_debug"): - fps_label.visible = !fps_label.visible - - -func _process(delta: float) -> void: - if fps_label.visible: - fps_label.set_text("FPS: %d" % Performance.get_monitor(Performance.TIME_FPS)) -</code></pre> -<p>Finally the signal receiver handlers which are straight forward:</p> -<pre><code class="language-gdscript">func _on_Game_game_started() -> void: - start_game_label.visible = false - high_score_label.visible = false - - -func _on_Game_game_over() -> void: - start_game_label.set_text("Press R to restart") - start_game_label.visible = true - high_score_label.visible = true - - -func _on_Game_new_score(score: int, high_score: int) -> void: - score_label.set_text(String(score)) - high_score_label.set_text("High score: %s" % high_score) -</code></pre> -<h3 id="main_1">Main<a class="headerlink" href="#main_1" title="Permanent link">¶</a></h3> -<p>This is the shortest script, it just connects the signals between the <code>Game</code> and the <code>UI</code>:</p> -<pre><code class="language-gdscript">onready var game: Game = $Game -onready var ui: UI = $UI - -var _game_over: bool = false - - -func _ready() -> void: - game.connect("game_started", ui, "_on_Game_game_started") - game.connect("game_over", ui, "_on_Game_game_over") - game.connect("new_score", ui, "_on_Game_new_score") -</code></pre> -<h2 id="final-notes-and-exporting">Final notes and exporting<a class="headerlink" href="#final-notes-and-exporting" title="Permanent link">¶</a></h2> -<p>At this point the game should be fully playable (if any detail missing feel free to look into the source code linked at the beginning). Only thing missing is an icon for the game; I did one pretty quicly with the assets I had.</p> -<h3 id="preparing-the-files">Preparing the files<a class="headerlink" href="#preparing-the-files" title="Permanent link">¶</a></h3> -<p>If you followed the directory structure I used, then only thing needed is to transform the icon to a native Windows <code>ico</code> format (if exporting to Windows, else ignore this part). For this you need <a href="https://imagemagick.org/index.php">ImageMagick</a> or some other program that can transform <code>png</code> (or whatever file format you used for the icon) to <code>ico</code>. I used [Chocolatey][https://chocolatey.org/] to install <code>imagemagick</code>, then to convert the icon itself used: <code>magick convert icon.png -define icon:auto-resize=256,128,64,48,32,16 icon.ico</code> as detailed in <em>Godot</em>‘s <a href="https://docs.godotengine.org/en/stable/tutorials/export/changing_application_icon_for_windows.html">Changing application icon for Windows</a>.</p> -<h3 id="exporting">Exporting<a class="headerlink" href="#exporting" title="Permanent link">¶</a></h3> -<p>You need to download the templates for exporting as detailed in <em>Godot</em>‘s <a href="https://docs.godotengine.org/en/stable/tutorials/export/exporting_projects.html">Exporting projects</a>. Basically you go to <em>Editor -> Manage Export Templates…</em> and download the latest one specific to your <em>Godot</em> version by clicking on <code>Download and Install</code>.</p> -<p>If exporting for Windows then you also need to download <code>rcedit</code> from <a href="https://github.com/electron/rcedit/releases/latest">here</a>. Just place it wherever you want (I put it next to the <em>Godot</em> executable).</p> -<p>Then go to <em>Project -> Export…</em> and the Window should be empty, add a new template by clicking on <code>Add...</code> at the top and then select the template you want. I used HTML5, Windows Desktop and Linux/X11. Really the only thing you need to set is the “Export Path” for each template, which is te location of where the executable will be written to, and in the case of the <em>Windows Desktop</em> template you could also setup stuff like <code>Company Name</code>, <code>Product Name</code>, <code>File/Product Version</code>, etc..</p> -<p>Once the templates are setup, select any and click on <code>Export Project</code> at the bottom, and make sure to untoggle <code>Export With Debug</code> in the window that pops up, this checkbox should be at the bottom of the new window.</p>]]></content:encoded> - </item> - <item> - <title>General Godot project structure</title> - <link>https://blog.luevano.xyz/g/godot_project_structure.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/godot_project_structure.html</guid> - <pubDate>Sun, 22 May 2022 01:16:10 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Godot</category> - <category>Short</category> - <description>Details on the project structure I'm using for Godot, based on preference and some research I did.</description> - <content:encoded><![CDATA[<p>One of my first issues when starting a project is how to structure everything. So I had to spend some time researching best practices and go with what I like the most and after trying some of them I wanted to write down somewhere what I’m sticking with.</p> -<p>The first place to look for is, of course, the official <em>Godot</em> documentation on <a href="https://docs.godotengine.org/en/stable/tutorials/best_practices/project_organization.html">Project organization</a>; along with project structure discussion, also comes with best practices for code style and what-not. I don’t like this project/directory structure that much, just because it tells you to bundle everything under the same directory but it’s a really good starting point, for example it tells you to use:</p> -<ul> -<li>/models/town/house/<ul> -<li>house.dae</li> -<li>window.png</li> -<li>door.png</li> -</ul> -</li> -</ul> -<p>Where I would prefer to have more modularity, for example:</p> -<ul> -<li>/levels/structures/town/house (or /levels/town/structures/house)<ul> -<li>window/<ul> -<li>window.x</li> -<li>window.y</li> -<li>window.z</li> -</ul> -</li> -<li>door/<ul> -<li>…</li> -</ul> -</li> -<li>house.x</li> -<li>house.y</li> -<li>house.z</li> -</ul> -</li> -</ul> -<p>It might look like it’s more work, but I prefer it like this. I wish <a href="https://www.braindead.bzh/entry/creating-a-game-with-godot-engine-ep-2-project-organization">this site</a> was still available, as I got most of my ideas from there and was a pretty good resource, but apparently the owner is not maintaining his site anymore; but there is <a href="https://www.reddit.com/r/godot/comments/7786ee/comment/dojuzuf/?utm_source=share&utm_medium=web2x&context=3">this excelent comment on reddit</a> which shows a project/directory structure more in line with what I’m currently using (and similr to the site that is down that I liked). I ended up with:</p> -<ul> -<li>/.git</li> -<li>/assets (raw assets/editable assets/asset packs)</li> -<li>/releases (executables ready to publish)</li> -<li>/src (the actual godot project)<ul> -<li>.godot/</li> -<li>actors/ (or entities)<ul> -<li>player/<ul> -<li>sprites/</li> -<li>player.x</li> -<li>…</li> -</ul> -</li> -<li>enemy/ (this could be a dir with subdirectories for each type of enemy for example…)<ul> -<li>sprites/</li> -<li>enemy.x</li> -<li>…</li> -</ul> -</li> -<li>actor.x</li> -<li>…</li> -</ul> -</li> -<li>levels/ (or scenes)<ul> -<li>common/<ul> -<li>sprites/</li> -<li>…</li> -</ul> -</li> -<li>main/<ul> -<li>…</li> -</ul> -</li> -<li>overworld/<ul> -<li>…</li> -</ul> -</li> -<li>dugeon/<ul> -<li>…</li> -</ul> -</li> -<li>Game.tscn (I’m considering the “Game” as a level/scene)</li> -<li>game.gd</li> -</ul> -</li> -<li>objects/<ul> -<li>box/<ul> -<li>…</li> -</ul> -</li> -<li>…</li> -</ul> -</li> -<li>screens/<ul> -<li>main_menu/<ul> -<li>…</li> -</ul> -</li> -<li>…</li> -</ul> -</li> -<li>globals/ (singletons/autoloads)</li> -<li>ui/<ul> -<li>menus/<ul> -<li>…</li> -</ul> -</li> -<li>…</li> -</ul> -</li> -<li>sfx/<ul> -<li>…</li> -</ul> -</li> -<li>vfx/<ul> -<li>…</li> -</ul> -</li> -<li>etc/<ul> -<li>…</li> -</ul> -</li> -<li>Main.tscn (the entry point of the game)</li> -<li>main.gd</li> -<li>icon.png (could also be on a separate “icons” directory)</li> -<li>project.godot</li> -<li>…</li> -</ul> -</li> -<li>\<any other repository related files></li> -</ul> -<p>And so on, I hope the idea is clear. I’ll probably change my mind on the long run, but for now this has been working fine.</p>]]></content:encoded> - </item> - <item> - <title>Will start blogging about gamedev</title> - <link>https://blog.luevano.xyz/g/starting_gamedev_blogging.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/g/starting_gamedev_blogging.html</guid> - <pubDate>Tue, 17 May 2022 05:19:54 GMT</pubDate> - <category>English</category> - <category>Gamedev</category> - <category>Godot</category> - <category>Short</category> - <category>Update</category> - <description>Since I'm starting to get more into gamedev stuff, I'll start blogging about it just to keep consistent.</description> - <content:encoded><![CDATA[<p>I’ve been wanting to get into gamedev for a while now, but it’s always a pain to stay consistent. I just recently started to get into it again, and this time I’m trying to actually do stuff.</p> -<p>So, the plan is to blog about my progress and clone some simple games just to get started. I’m thinking on sticking with <a href="https://godotengine.org/">Godot</a> just because I like that it’s open source, it’s getting better and better overtime (big rewrite happening right now) and I already like how the engine works. <del>Specifically I’ll start using <em>Godot 4</em> even though it’s not done yet, to get used to the new features, specifically pumped for <a href="https://godotengine.org/article/gdscript-progress-report-feature-complete-40">GDScript 2.0</a>.</del> <ins>Actually… (for the small clones/ripoffs) I’ll need to use <em>Godot 3.X</em> (probably 3.5), as <em>Godot 4</em> doesn’t have support to export to webassembly (HTML5) yet, and I want that to publish to <a href="https://itch.io/">itch.io</a> and my website. I’ll continue to use <em>Godot 4</em> for bigger projects, as they will take longer and I hope that by the time I need to publish, there’s no issues to export.</ins></p> -<p>For a moment I almost started a new subdomain just for gamedev stuff, but decided to just use a different directory for subtleness; this directory and use of tags should be enough. I’ll be posting the entry about the first rip-off I’m <em>developing</em> (FlappyBird L O L) shortly.</p> -<p><strong>Update</strong>: <a href="https://godotengine.org/article/godot-4-0-sets-sail/">Godot 4</a> already released and it now has HTML5 support, so back to the original plan.</p>]]></content:encoded> - </item> - <item> - <title>My setup for a password manager and MFA authenticator</title> - <link>https://blog.luevano.xyz/a/password_manager_authenticator_setup.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/password_manager_authenticator_setup.html</guid> - <pubDate>Sun, 15 May 2022 22:40:34 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Tools</category> - <description>A short description on my personal setup regarding a password manager and alternatives to G\*\*gl\* authenticator.</description> - <content:encoded><![CDATA[<p><strong>Disclaimer</strong>: I won’t go into many technical details here of how to install/configure/use the software, this is just supposed to be a short description on my setup.</p> -<p>It’s been a while since I started using a password manager at all, and I’m happy that I started with <a href="https://keepassxc.org/">KeePassXC</a> (open source, multiplatform password manager that it’s completely offline) as a direct recommendation from <a href="https://www.lmcj.xyz/"><mark>EL ELE EME</mark></a>; before this I was using the same password for everything (like a lot of people), which is a well know privacy issue as noted in detail by <a href="https://askleo.com/different-passwords-for-everything/">Leo</a> (I don’t personally recommed LastPass as Leo does). Note that you will still need a <em>master password</em> to lock/unlock your password database (you can additionally use a hardware key and a key file).</p> -<p>Anyways, setting up <em>keepass</em> is pretty simple, as there is a client for almost any device; note that <em>keepass</em> is basically just the format and the base for all of the clients, as its common with pretty much any open source software. In my case I’m using <a href="https://keepassxc.org/">KeePassXC</a> in my computer and <a href="https://www.keepassdx.com/">KeePassDX</a> in my phone (Android). The only concern is keeping everything in sync because <em>keepass</em> doesn’t have any automatic method of synchronizing between devices because of security reasons (as far as I know), meaning that you have to manage that yourself.</p> -<p>Usually you can use something like G**gl* drive, dropbox, mega, nextcloud, or any other cloud solution that you like to sync your <em>keepass</em> database between devices; I personally prefer to use <a href="https://syncthing.net/">Syncthing</a> as it’s open source, it’s really easy to setup and has worked wonders for me since I started using it, also it keeps versions of your files that can serve as backups in any scenario where the database gets corrupted or something.</p> -<p>Finally, when I went through the issue with the micro SD and the <em>adoptable storage</em> bullshit (you can find the rant <a href="https://blog.luevano.xyz/a/devs_android_me_trozaron.html">here</a>, in spanish) I had to also migrate from <em>G**gl* authenticator</em> (<em>gauth</em>) to something else for the simple reason that <em>gauth</em> doesn’t even let you do backups, nor it’s synched with your account… nothing, it is just standalone and if you ever lose your phone you’re fucked; so I decided to go with <a href="https://getaegis.app/">Aegis authenticator</a>, as it is open source, you have control over all your secret keys, you can do backups directly to the filesystem, you can secure your database with an extra password, etc., etc.. In general <em>aegis</em> is the superior MFA authenticator (at least compared with <em>gauth</em>) and everything that’s compatible with <em>gauth</em> is compatible with <em>aegis</em> as the format is a standard (as a matter of fact, <em>keepass</em> also has this MFA feature which is called TOPT and is also compatible, but I prefer to have things separate). I also use <em>syncthing</em> to keep a backup of my <em>aegis</em> database.</p> -<p><strong>TL;DR</strong>:</p> -<ul> -<li><a href="https://syncthing.net/">Syncthing</a> to sync files between devices (for the password databases).</li> -<li><a href="https://keepassxc.org/">KeePassXC</a> for the password manager in my computer.</li> -<li><a href="https://www.keepassdx.com/">KeePassDX</a> for the password manager in my phone.</li> -<li><a href="https://getaegis.app/">Aegis authenticator</a> for the universal MFA authenticator.</li> -</ul>]]></content:encoded> - </item> - <item> - <title>Los devs de Android/MIUI me trozaron</title> - <link>https://blog.luevano.xyz/a/devs_android_me_trozaron.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/devs_android_me_trozaron.html</guid> - <pubDate>Sun, 15 May 2022 09:51:04 GMT</pubDate> - <category>Rant</category> - <category>Spanish</category> - <category>Update</category> - <description>Perdí un día completo resolviendo un problema muy estúpido, por culpa de los devs de Android/MIUI.</description> - <content:encoded><![CDATA[<p>Llevo dos semanas posponiendo esta entrada porque andaba bien enojado (todavía, pero ya se anda pasando) y me daba <em>zzz</em>. Pero bueno, antes que nada este pex ocupa un poco de contexto sobre dos cositas:</p> -<ul> -<li><a href="https://tachiyomi.org/">Tachiyomi</a>: Una aplicación de android que uso para descargar y leer manga. Lo importante aquí es que por default se guardan los mangas con cada página siendo una sola imagen, por lo que al mover el manga de un lado a otro tarda mucho tiempo.</li> -<li><a href="https://source.android.com/devices/storage/adoptable">Adoptable storage</a>: Un <em>feature</em> de android que básicamente te deja usar una micro SD (mSD) externa como si fuera interna, encriptando y dejando la mSD inutilizable en cualquier otro dispositivo. La memoria interna se <em>pierde</em> o algo por el estilo (bajo mi experiencia), por lo que parece es bastante útil cuando la capacidad de la memoria interna es baja.</li> -</ul> -<p>Ahora sí vamonos por partes, primero que nada lo que sucedió fue que ordené una mSD con más capacidad que la que ya tenía (64 GB -> 512 GB, poggies), porque últimamente he estado bajando y leyendo mucho manga entonces me estaba quedando sin espacio. Ésta llegó el día de mi cumpleaños lo cuál estuvo chingón, me puse a hacer backup de la mSD que ya tenía y preparando todo, muy bonito, muy bonito.</p> -<p>Empecé a tener problemas, porque al estar moviendo tanto archivo pequeño (porque recordemos que el <em>tachiyomi</em> trata a cada página como una sola imagen), la conexión entre el celular y mi computadora se estaba corte y corte por alguna razón; en general muchos pedos. Por lo que mejor le saqué la nueva mSD y la metí directo a mi computadora por medio de un adaptador para batallar menos y que fuera más rápido.</p> -<p>Hacer este pedo de mover archivos directamente en la mSD puede llevar a corromper la memoria, no se los detalles pero pasa (o quizá estoy meco e hice algo mal). Por lo que al terminar de mover todo a la nueva mSD y ponerla en el celular, éste se emputó que porque no la detectaba y que quería tirar un formateo a la mSD. A este punto no me importaba mucho, sólo era questión de volvera mover archivos y ser más cuidadoso; “<em>no issues from my end</em>” diría en mis <em>standups</em>.</p> -<p>Todo valió <strong>vergota</strong> porque en cierto punto al elegir sí formatear la mSD mi celular me daba la opción de “<em>usar la micro SD para el celular</em>” o “<em>usar la micro SD como memoria portátil</em>” (o algo entre esas líneas), y yo, estúpidamente, elegí la primera, porque me daba sentido: “no, pues simón, voy a usar esta memoria para este celular”.</p> -<p>Pues mamé, resulta que esa primera opción lo que realmente quería decir es que se iba a usar la micro SD como interna usando el pex este de <em>adoptable storage</em>. Entonces básicamente <em>perdí</em> mi capacidad de memoria interna (128 GB aprox.), y toda la mSD nueva se usó como memoria interna. Todo se juntó, si intentaba sacar la mSD todo se iba a la mierda y no podía usar muchas aplicaciones. “<em>No hay pedo</em>”, pensé, “<em>nada más es cuestión de desactivar esta mamada de adoptable storage</em>”.</p> -<p>Ni madres dijeron los devs de Android, este pedo nada más es un <em>one-way</em>: puedes activar <em>adoptable storage</em> pero para desactivarlo <strong>ocupas, a huevo, formatear tu celular a estado de fábrica</strong>. Chingué a mi madre, comí mierda, perdí.</p> -<p>Pues eso fue lo que hice, ni modo. Hice backup de todo lo que se me ocurrió (también me di cuenta que G**gl* authenticator es cagada ya que no te deja hacer backup, entre otras cosas, mejor usen <a href="https://getaegis.app/">Aegis authenticator</a>), desactivé todo lo que se tenía que desactivar y tocó hacer <em>factory reset</em>, ni modo. Pero como siempre las cosas salen mal y tocó comer mierda del banco porque me bloquearon la tarjeta, perdí credenciales necesarias para el trabajo (se resolvió rápido), etc., etc.. Ya no importa, ya casi todo está resuelto, sólo queda ir al banco a resolver lo de la tarjeta bloqueada (esto es para otro <em>rant</em>, pinches apps de bancos piteras, ocupan hacer una sola cosa y la hacen mal).</p> -<p>Al final del día, la causa del problema fueron los malditos mangas (por andar queriendo <em>backupearlos</em>), que terminé bajando de nuevo manualmente y resultó mejor porque aparentemente <em>tachiyomi</em> agregó la opción de “<em>zippear</em>” los mangas en formato <a href="https://docs.fileformat.com/ebook/cbz/">CBZ</a>, por lo que ya son más fácil de mover de un lado para otro, el fono no se queda pendejo, etc., etc..</p> -<p>Por último, quiero decir que los devs de Android son unos pendejos por no hacer reversible la opción de <em>adoptable storage</em>, y los de MIUI son todavía más por no dar detalles de lo que significan sus opciones de formateo, especialmente si una opción es tan chingadora que para revertirla necesitas formatear a estado de fábrica tu celular; más que nada es culpa de los de MIUI, todavía que ponen un chingo de A(i)DS en todas sus apps, no pueden poner una buena descripción en sus opciones. <strong>REEEE</strong>.</p>]]></content:encoded> - </item> - <item> - <title>Volviendo a usar la página</title> - <link>https://blog.luevano.xyz/a/volviendo_a_usar_la_pagina.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/volviendo_a_usar_la_pagina.html</guid> - <pubDate>Thu, 28 Apr 2022 03:21:02 GMT</pubDate> - <category>Short</category> - <category>Spanish</category> - <category>Update</category> - <description>Actualización en el estado de la página, después de mucho tiempo de ausencia.</description> - <content:encoded><![CDATA[<p>Después de mucho tiempo de estar luchando con querer volver a usar este pex (maldita <em>d</em> word y demás), ya me volví a acomodar el setup para agregar nuevas entradas.</p> -<p>Entre las cosas que tuve que hacer fue actualizar el <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> porque no lo podía usar de una como estaba; y de pasado le agregue una que otra feature nueva. Luego quiero agregarle más funcionalidad para poder <em>buildear</em> la página completa; por ahora se hace en segmentos: todo lo de <a href="https://luevano.xyz">luevano.xyz</a> está hecho manual, mientras que <a href="https://blog.luevano.xyz">blog</a> y <a href="https://art.luevano.xyz">art</a> usan <a href="https://github.com/luevano/pyssg">pyssg</a>.</p> -<p>Otra cosa es que quizá me devuelva a editar alguans entradas nada más para homogeneizar las entradas específicas a <em>Create a…</em> (tiene más sentido que sean <em>Setup x…</em> o algo similar).</p> -<p>En otras noticias, estoy muy agusto en el jale que tengo actualmente aunque lleve alrededor de 3 semanas de un infierno en el jale. Debo pensar en si debo omitir cosas personales o del trabajo aquí, ya que quién sabe quién se pueda llegar a topar con esto <em>*thinking emoji*</em>.</p>]]></content:encoded> - </item> - <item> - <title>Set up a VPN server with OpenVPN</title> - <link>https://blog.luevano.xyz/a/vpn_server_with_openvpn.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/vpn_server_with_openvpn.html</guid> - <pubDate>Sun, 01 Aug 2021 09:27:02 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a VPN server using OpenVPN on a server running Nginx, on Arch. Only for IPv4.</description> - <content:encoded><![CDATA[<p>I’ve been wanting to do this entry, but had no time to do it since I also have to set up the VPN service as well to make sure what I’m writing makes sense, today is the day.</p> -<p>Like with any other of my entries I based my setup on the <a href="https://wiki.archlinux.org/title/OpenVPN">Arch Wiki</a>, <a href="https://github.com/Nyr/openvpn-install">this install script</a> and <a href="https://github.com/graysky2/ovpngen">this profile generator script</a>.</p> -<p>This will be installed and working alongside the other stuff I’ve wrote about on other posts (see the <a href="https://blog.luevano.xyz/tag/@server.html">server</a> tag). All commands here are executes as root unless specified otherwise. Also, this is intended only for IPv4 (it’s not that hard to include IPv6, but meh). As always, all commands are executed as root unless stated otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#create-pki-from-scratch">Create PKI from scratch</a></li> -<li><a href="#openvpn">OpenVPN</a><ul> -<li><a href="#enable-forwarding">Enable forwarding</a></li> -<li><a href="#create-client-configurations">Create client configurations</a></li> -</ul> -</li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>Pretty simple:</p> -<ul> -<li>Working server with root access, and with <code>ufw</code> as the firewall.</li> -<li>Open port <code>1194</code> (default), or as a fallback on <code>443</code> (click <a href="https://openvpn.net/vpn-server-resources/advanced-option-settings-on-the-command-line/">here</a> for more). I will do mine on port <code>1194</code> but it’s just a matter of changing 2 lines of configuration and one <code>ufw</code> rule.</li> -</ul> -<h2 id="create-pki-from-scratch">Create PKI from scratch<a class="headerlink" href="#create-pki-from-scratch" title="Permanent link">¶</a></h2> -<p>PKI stands for <em>Public Key Infrastructure</em> and basically it’s required for certificates, private keys and more. This is supposed to work between two servers and one client: a server in charge of creating, signing and verifying the certificates, a server with the OpenVPN service running and the client making the request.</p> -<p>In a nutshel, this is supposed to work something like: 1) a client wants to use the VPN service, so it creates a requests and sends it to the signing server, 2) this server checks the requests and signs the request, returning the certificates to both the VPN service and the client and 3) the client can now connect to the VPN service using the signed certificate which the OpenVPN server knows about.</p> -<p>That’s how the it should be st up… but, to be honest, all of this is a hassle and (in my case) I want something simple to use and manage. So I’m gonna do all on one server and then just give away the configuration file for the clients, effectively generating files that anyone can run and will work, meaning that you need to be careful who you give this files (it also comes with a revoking mechanism, so no worries).</p> -<p>This is done with <a href="https://wiki.archlinux.org/title/Easy-RSA">Easy-RSA</a>.</p> -<p>Install the <code>easy-rsa</code> package:</p> -<pre><code class="language-sh">pacman -S easy-rsa -</code></pre> -<p>Initialize the PKI and generate the CA keypair:</p> -<pre><code class="language-sh">cd /etc/easy-rsa -easyrsa init-pki -easyrsa build-ca nopass -</code></pre> -<p>Create the server certificate and private key (while in the same directory):</p> -<pre><code class="language-sh">EASYRSA_CERT_EXPIRE=3650 easyrsa build-server-full server nopass -</code></pre> -<p>Where <code>server</code> is just a name to identify your server certificate keypair, I just use <code>server</code> but could be anything (like <code>luevano.xyz</code> in my case).</p> -<p>Create the client revocation list AKA CRL (will be used later, but might as well have it now):</p> -<pre><code class="language-sh">EASYRSA_CRL_DAYS=3650 easyrsa gen-crl -</code></pre> -<p>After this we should have 6 new files:</p> -<pre><code>/etc/easy-rsa/pki/ca.crt -/etc/easy-rsa/pki/private/ca.key -/etc/easy-rsa/pki/issued/server.crt -/etc/easy-rsa/pki/reqs/server.req -/etc/easy-rsa/pki/private/server.key -/etc/easy-rsa/pki/crl.pem -</code></pre> -<p>It is recommended to copy some of these files over to the <code>openvpn</code> directory, but I prefer to keep them here and just change some of the permissions:</p> -<pre><code class="language-sh">chmod o+rx pki -chmod o+rx pki/ca.crt -chmod o+rx pki/issued -chmod o+rx pki/issued/server.crt -chmod o+rx pki/private -chmod o+rx pki/private/server.key -chown nobody:nobody pki/crl.pem -chmod o+r pki/crl.pem -</code></pre> -<p>Finally, go to the <code>openvpn</code> directory and create the required files there:</p> -<pre><code class="language-sh">cd /etc/openvpn/server -openssl dhparam -out dh.pem 2048 -openvpn --genkey secret ta.key -</code></pre> -<h2 id="openvpn">OpenVPN<a class="headerlink" href="#openvpn" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/OpenVPN">OpenVPN</a> is a robust and highly flexible VPN daemon, that’s pretty complete feature-wise.</p> -<p>Install the <code>openvpn</code> package:</p> -<pre><code class="language-sh">pacman -S openvpn -</code></pre> -<p>Now, most of the stuff is going to be handled by (each, if you have more than one) server configuration. This might be the hardest thing to configure, but I’ve used a basic configuration file that worked a lot to me, which is a compilation of stuff that I found on the internet while configuring the file a while back.</p> -<pre><code># Server ip addres (ipv4). -local 1.2.3.4 # your server public ip - -# Port. -port 1194 # Might want to change it to 443 - -# TCP or UDP. -;proto tcp -proto udp # If ip changes to 443, you should change this to tcp, too - -# "dev tun" will create a routed IP tunnel, -# "dev tap" will create an ethernet tunnel. -;dev tap -dev tun - -# Server specific certificates and more. -ca /etc/easy-rsa/pki/ca.crt -cert /etc/easy-rsa/pki/issued/server.crt -key /etc/easy-rsa/pki/private/server.key # This file should be kept secret. -dh /etc/openvpn/server/dh.pem -auth SHA512 -tls-crypt /etc/openvpn/server/ta.key 0 # This file is secret. -crl-verify /etc/easy-rsa/pki/crl.pem - -# Network topology. -topology subnet - -# Configure server mode and supply a VPN subnet -# for OpenVPN to draw client addresses from. -server 10.8.0.0 255.255.255.0 - -# Maintain a record of client <-> virtual IP address -# associations in this file. -ifconfig-pool-persist ipp.txt - -# Push routes to the client to allow it -# to reach other private subnets behind -# the server. -;push "route 192.168.10.0 255.255.255.0" -;push "route 192.168.20.0 255.255.255.0" - -# If enabled, this directive will configure -# all clients to redirect their default -# network gateway through the VPN, causing -# all IP traffic such as web browsing and -# and DNS lookups to go through the VPN -push "redirect-gateway def1 bypass-dhcp" - -# Certain Windows-specific network settings -# can be pushed to clients, such as DNS -# or WINS server addresses. -# Google DNS. -;push "dhcp-option DNS 8.8.8.8" -;push "dhcp-option DNS 8.8.4.4" - -# The keepalive directive causes ping-like -# messages to be sent back and forth over -# the link so that each side knows when -# the other side has gone down. -keepalive 10 120 - -# The maximum number of concurrently connected -# clients we want to allow. -max-clients 5 - -# It's a good idea to reduce the OpenVPN -# daemon's privileges after initialization. -user nobody -group nobody - -# The persist options will try to avoid -# accessing certain resources on restart -# that may no longer be accessible because -# of the privilege downgrade. -persist-key -persist-tun - -# Output a short status file showing -# current connections, truncated -# and rewritten every minute. -status openvpn-status.log - -# Set the appropriate level of log -# file verbosity. -# -# 0 is silent, except for fatal errors -# 4 is reasonable for general usage -# 5 and 6 can help to debug connection problems -# 9 is extremely verbose -verb 3 - -# Notify the client that when the server restarts so it -# can automatically reconnect. -# Only usable with udp. -explicit-exit-notify 1 -</code></pre> -<p><code>#</code> and <code>;</code> are comments. Read each and every line, you might want to change some stuff (like the logging), specially the first line which is your server public IP.</p> -<h4 id="enable-forwarding">Enable forwarding<a class="headerlink" href="#enable-forwarding" title="Permanent link">¶</a></h4> -<p>Now, we need to enable <em>packet forwarding</em> (so we can access the web while connected to the VPN), which can be enabled on the interface level or globally (you can check the different options with <code>sysctl -a | grep forward</code>). I’ll do it globally, run:</p> -<pre><code class="language-sh">sysctl net.ipv4.ip_forward=1 -</code></pre> -<p>And create/edit the file <code>/etc/sysctl.d/30-ipforward.conf</code>:</p> -<pre><code>net.ipv4.ip_forward=1 -</code></pre> -<p>Now we need to configure <code>ufw</code> to forward traffic through the VPN. Append the following to <code>/etc/default/ufw</code> (or edit the existing line):</p> -<pre><code>... -DEFAULT_FORWARD_POLICY="ACCEPT" -... -</code></pre> -<p>And change the <code>/etc/ufw/before.rules</code>, appending the following lines after the header <strong>but before the *filter line</strong>:</p> -<pre><code>... -# NAT (Network Address Translation) table rules -*nat -:POSTROUTING ACCEPT [0:0] - -# Allow traffic from clients to the interface --A POSTROUTING -s 10.8.0.0/24 -o interface -j MASQUERADE - -# do not delete the "COMMIT" line or the NAT table rules above will not be processed -COMMIT - -# Don't delete these required lines, otherwise there will be errors -*filter -... -</code></pre> -<p>Where <code>interface</code> must be changed depending on your system (in my case it’s <code>ens3</code>, another common one is <code>eth0</code>); I always check this by running <code>ip addr</code> which gives you a list of interfaces (the one containing your server public IP is the one you want, or whatever interface your server uses to connect to the internet):</p> -<pre><code>... -2: ens3: <SOMETHING,SOMETHING> bla bla - link/ether bla:bla - altname enp0s3 - inet my.public.ip.addr bla bla -... -</code></pre> -<p>And also make sure the <code>10.8.0.0/24</code> matches the subnet mask specified in the <code>server.conf</code> file (in this example it matches). You should check this very carefully, because I just spent a good 2 hours debugging why my configuration wasn’t working, and this was te reason (I could connect to the VPN, but had no external connection to the web).</p> -<p>Finally, allow the OpenVPN port you specified (in this example its <code>1194/udp</code>) and reload <code>ufw</code>:</p> -<pre><code class="language-sh">ufw allow 1194/udp comment "OpenVPN" -ufw reload -</code></pre> -<p>At this point, the server-side configuration is done and you can start and enable the service:</p> -<pre><code class="language-sh">systemctl start openvpn-server@server.service -systemctl enable openvpn-server@server.service -</code></pre> -<p>Where the <code>server</code> after <code>@</code> is the name of your configuration, <code>server.conf</code> without the <code>.conf</code> in my case.</p> -<h3 id="create-client-configurations">Create client configurations<a class="headerlink" href="#create-client-configurations" title="Permanent link">¶</a></h3> -<p>You might notice that I didn’t specify how to actually connect the VPN. For that we need a configuration file similar to the <code>server.conf</code> file that we created.</p> -<p>The real way of doing this would be to run similar steps as the ones with <code>easy-rsa</code> locally, send them to the server, sign them, and retrieve them. Fuck all that, we’ll just create all configuration files on the server as I was mentioning earlier.</p> -<p>Also, the client configuration file has to match the server one (to some degree), to make this easier you can create a <code>client-common</code> file in <code>/etc/openvpn/server</code> with the following content:</p> -<pre><code>client -dev tun -remote 1.2.3.4 1194 udp # change this to match your ip and port -resolv-retry infinite -nobind -persist-key -persist-tun -remote-cert-tls server -auth SHA512 -verb 3 -</code></pre> -<p>Where you should make any changes necessary, depending on your configuration.</p> -<p>Now, we need a way to create and revoke new configuration files. For this I created a script, heavily based on one of the links I mentioned at the beginning. You can place these scripts anywhere you like, and you should take a look before running them because you’ll be running them with elevated privileges (sudo).</p> -<p>In a nutshell, what it does is: generate a new client certificate keypair, update the CRL and create a new <code>.ovpn</code> configuration file that consists on the <code>client-common</code> data and all of the required certificates; or, revoke an existing client and refresh the CRL. The file is placed under <code>~/ovpn</code>.</p> -<p>Create a new file with the following content (name it whatever you like) and don’t forget to make it executable (<code>chmod +x vpn_script</code>):</p> -<pre><code class="language-sh">#!/bin/sh -# Client ovpn configuration creation and revoking. -MODE=$1 -if [ ! "$MODE" = "new" -a ! "$MODE" = "rev" ]; then - echo "$1 is not a valid mode, using default 'new'" - MODE=new -fi - -CLIENT=${2:-guest} -if [ -z $2 ];then - echo "there was no client name passed as second argument, using 'guest' as default" -fi - -# Expiration config. -EASYRSA_CERT_EXPIRE=3650 -EASYRSA_CRL_DAYS=3650 - -# Current PWD. -CPWD=$PWD -cd /etc/easy-rsa/ - -if [ "$MODE" = "rev" ]; then - easyrsa --batch revoke $CLIENT - - echo "$CLIENT revoked." -elif [ "$MODE" = "new" ]; then - easyrsa build-client-full $CLIENT nopass - - # This is what actually generates the config file. - { - cat /etc/openvpn/server/client-common - echo "<ca>" - cat /etc/easy-rsa/pki/ca.crt - echo "</ca>" - echo "<cert>" - sed -ne '/BEGIN CERTIFICATE/,$ p' /etc/easy-rsa/pki/issued/$CLIENT.crt - echo "</cert>" - echo "<key>" - cat /etc/easy-rsa/pki/private/$CLIENT.key - echo "</key>" - echo "<tls-crypt>" - sed -ne '/BEGIN OpenVPN Static key/,$ p' /etc/openvpn/server/ta.key - echo "</tls-crypt>" - } > "$(eval echo ~${SUDO_USER:-$USER}/ovpn/$CLIENT.ovpn)" - - eval echo "~${SUDO_USER:-$USER}/ovpn/$CLIENT.ovpn file generated." -fi - -# Finish up, re-generates the crl -easyrsa gen-crl -chown nobody:nobody pki/crl.pem -chmod o+r pki/crl.pem -cd $CPWD -</code></pre> -<p>And the way to use is to run <code>bash vpn_script <mode> <client_name></code> where <code>mode</code> is <code>new</code> or <code>rev</code> (revoke) as sudo (when revoking, it doesn’t actually delete the <code>.ovpn</code> file in <code>~/ovpn</code>). Again, this is a little script that I put together, so you should check it out, it may need tweaks (specially depending on your directory structure for <code>easy-rsa</code>).</p> -<p>Now, just get the <code>.ovpn</code> file generated, import it to OpenVPN in your client of preference and you should have a working VPN service.</p>]]></content:encoded> - </item> - <item> - <title>Hoy me tocó desarrollo de personaje</title> - <link>https://blog.luevano.xyz/a/hoy_toco_desarrollo_personaje.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/hoy_toco_desarrollo_personaje.html</guid> - <pubDate>Wed, 28 Jul 2021 06:10:55 GMT</pubDate> - <category>Rant</category> - <category>Spanish</category> - <category>Update</category> - <description>Una breve historia sobre cómo estuvo mi día, porque me tocó desarrollo de personaje y lo quiero sacar del coraje que traigo.</description> - <content:encoded><![CDATA[<p>Sabía que hoy no iba a ser un día tan bueno, pero no sabía que iba a estar tan horrible; me tocó desarrollo de personaje y saqué el <em>bad ending</em>.</p> -<p>Básicamente tenía que cumplir dos misiones hoy: ir al banco a un trámite y vacunarme contra el Covid-19. Muy sencillas tareas.</p> -<p>Primero que nada me levanté de una pesadilla horrible en la que se puede decir que se me subió el muerto al querer despertar, esperé a que fuera casi la hora de salida de mi horario de trabajo, me bañé y fui directo al banco primero. Todo bien hasta aquí.</p> -<p>En el camino al banco, durante la plática con el conductor del Uber salió el tema del horario del banco. Yo muy tranquilo dije “pues voy algo tarde, pero sí alcanzo, cierran a las 5, ¿no?” a lo que me respondió el conductor “nel jefe, a las 4, y se van media hora antes”; quedé. Chequé y efectivamente cerraban a las 4. Entonces le dije que le iba a cambiar la ruta directo a donde me iba a vacunar, pero ya era muy tarde y quedaba para la dirección opuesta.”Ni pedo, ahí déjame y pido otro viaje, no te apures”, le dije y como siempre pues me deseó que se compusiera mi día; <strong>afortunadamente</strong> el banco sí estaba abierto para lo que tenía que hacer, así que fue un buen giro. Me puse muy feliz y asumí que sería un buen día, como me lo dijo mi conductor; <strong>literalmente NO SABÍA</strong>.</p> -<p>Salí feliz de poder haber completado esa misión y poder irme a vacunar. Pedí otro Uber a donde tenía que ir y todo bien. Me tocó caminar mucho porque la entrada estaba en punta de la chingada de donde me dejó el conductor, pero no había rollo, era lo de menos. Me desanimé cuando vi que había una cantidad estúpida de gente, era una fila que abarcaba todo el estacionamiento y daba demasiadas vueltas; “ni pedo”, dije, “si mucho me estaré aquí una hora, hora y media”… otra vez, <strong>literalmente NO SABÍA</strong>.</p> -<p>Pasó media hora y había avanzado lo que parecía ser un cuarto de la fila, entonces todo iba bien. Pues nel, había avanzado el equivalente a un octavo de la fila, este pedo no iba a salir en una hora-hora y media. Para acabarla de chingar era todo bajo el tan amado sol de Chiwawa. “No hay pedo, me entretengo tirando chal con alguien en el wasap”, pues no, aparentemente no cargué el celular y ya tenía 15-20% de batería… volví a quedar.</p> -<p>Se me acabó la pila, ya había pasado una hora y parecía que la fila era infinita, simplemente avanzábamos demasiado lento, a pesar de que los que venían atrás de mí repetían una y otra vez “mira, avanza bien rápido, ya mero llegamos”, ilusos. Duré aproximadamente 3 horas formado, aguantando conversaciones estúpidas a mi alrededor, gente quejándose por estar parada (yo también me estaba quejando pero dentro de mi cabeza), y por alguna razón iban familias completas de las cuales al final del día sólo uno o dos integrantes de la familia entraban a vacunarse.</p> -<p>En fin que se acabó la tortura y ya tocaba irse al cantón, todo bien. “No hay pedo, no me tocó irme en Uber, aquí agarro un camíon” pensé. Pero no, ningún camión pasó durante la hora que estuve esperando y de los 5 taxis que intenté parar <strong>NINGUNO</strong> se detuvo. Decidí irme caminado, ya qué más daba, en ese punto ya nada más era hacer corajes <em>dioquis</em>.</p> -<p>En el camino vi un Oxxo y decidí desviarme para comprar algo de tomar porque andaba bien deshidratado. En el mismo segundo que volteé para ir hacia el Oxxo pasó un camión volando y lo único que pensaba era que el conductor me decía “Jeje ni pedo:)”. Exploté, me acabé, simplemente perdí, saqué el <em>bad ending</em>.</p> -<p>Ya estaba harto y hasta iba a comprar un cargador para ya irme rápido, estaba cansado del día, simplemente ahí terminó la quest, había sacado el peor final. Lo bueno es que se me ocurrió pedirle al cajero un cargador y que me <em>tirara paro</em>. Todo bien, pedí mi Uber y llegué a mi casa sano y a salvo, pero con la peor rabia que me había dado en mucho tiempo. Simplemente ¿mi culo? explotado. Este día me tocó un desarrollo de personaje muy cabrón, se mamó el D*****o.</p> -<p>Lo único rescatable fue que había una (más bien como 5) chica muy guapa en la fila, lástima que los <em>stats</em> de mi personaje me tienen bloqueadas las conversaciones con desconocidos.</p> -<p>Y pues ya, este pex ya me sirvió para desahogarme, una disculpa por la redacción tan <em>pitera</em>. Sobres.</p>]]></content:encoded> - </item> - <item> - <title>Tenía este pex algo descuidado</title> - <link>https://blog.luevano.xyz/a/tenia_esto_descuidado.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/tenia_esto_descuidado.html</guid> - <pubDate>Sun, 18 Jul 2021 07:51:50 GMT</pubDate> - <category>Short</category> - <category>Spanish</category> - <category>Update</category> - <description>Nada más un update en el estado del blog y lo que he andado haciendo.</description> - <content:encoded><![CDATA[<p>Así es, tenía un poco descuidado este pex, siendo la razón principal que andaba ocupado con cosas de <em>la vida profesional</em>, ayay. Pero ya que ando un poco más despejado y menos estresado voy a seguir usando el blog y a ver qué más hago.</p> -<p>Tengo unas entradas pendientes que quiero hacer del estilo de “tutorial” o “how-to”, pero me lo he estado debatiendo, porque Luke ya empezó a hacerlo más de verdad en <a href="https://landchad.net/">landchad.net</a>, lo cual recomiendo bastante pues igual yo empecé a hacer esto por él (y por <a href="https://lmcj.xyz/"><mark>EL ELE EME</mark></a>); aunque la verdad pues es muy específico a como él hace las cosas y quizá sí puede haber diferencias, pero ya veré en estos días. La próxima que quiero hacer es sobre el VPN, porque no lo he <em>setupeado</em> desde que reinicié El Página Web y La Servidor, entonces acomodaré el VPN de nuevo y de pasada tiro entrada de eso.</p> -<p>También dejé un dibujo pendiente, que la neta lo dejé por 2 cosas: está bien cabrón (porque también lo quiero colorear) y porque estaba ocupado; de lo cuál ya sólo queda el <em>está bien cabrón</em> pero no he tenido el valor de retomarlo. Lo triste es que ya pasó el tiempo del hype y ya no tengo mucha motivación para terminarlo más que el hecho de que cuando lo termine empezaré a usar Clip Studio Paint en vez de Krita, porque compré una licencia ahora que estuvo en 50% de descuento.</p> -<p>Algo bueno es que me he estado sintiendo muy bien conmigo mismo últimamente, aunque casi no hable de eso. Sí hay una razón en específico, pero es una razón algo tonta. Espero así siga.</p> -<p><del>Ah, y también quería acomodarme una sección de comentarios, pero como siempre, todas las opciones están bien <em>bloated</em>, entonces pues me voy a hacer una en corto seguramente en Python para <em>el back</em>, MySQL para la base de datos y Javascript para la conexión acá en <em>el front</em>, algo tranqui.</del> <ins>Nel, siempre no ocupo esto, pa’ qué.</ins></p> -<p>Sobres pues.</p>]]></content:encoded> - </item> - <item> - <title>Set up an XMPP server with Prosody compatible with Conversations and Movim</title> - <link>https://blog.luevano.xyz/a/xmpp_server_with_prosody.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/xmpp_server_with_prosody.html</guid> - <pubDate>Wed, 09 Jun 2021 05:24:30 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up an XMPP server using Prosody on a server running Nginx, on Arch. This server will be compatible with at least Conversations and Movim.</description> - <content:encoded><![CDATA[<p><strong>Update</strong>: I no longer host this XMPP server as it consumed a lot of resources and I wasn’t using it that much. I’ll probably re-create it in the future, though.</p> -<p>Recently I set up an <a href="https://xmpp.org/">XMPP</a> server (and a Matrix one, too) for my personal use and for friends if they want one; made one for <a href="https://lmcj.xyz"><mark>EL ELE EME</mark></a> for example. So, here are the notes on how I set up the server that is compatible with the <a href="https://conversations.im/">Conversations</a> app and the <a href="https://movim.eu/">Movim</a> social network. You can see my addresses at <a href="https://luevano.xyz/contact.html">contact</a> and the XMPP compliance/score of the server.</p> -<p>One of the best resources I found that helped me a lot was <a href="https://community.hetzner.com/tutorials/prosody-debian9">Installing and Configuring Prosody XMPP Server on Debian 9</a>, the <a href="https://wiki.archlinux.org/title/Prosody">Arch Wiki</a> and the <a href="https://prosody.im/">oficial documentation</a>.</p> -<p>As with my other entries, this is under a server running Arch Linux, with the Nginx web server and Certbot certificates. And all commands here are executed as root, unless specified otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#prosody">Prosody</a></li> -<li><a href="#nginx-configuration-file">Nginx configuration file</a></li> -<li><a href="#coturn">Coturn</a></li> -<li><a href="#wrapping-up">Wrapping up</a></li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>Same as with my other entries (<a href="https://luevano.xyz/a/website_with_nginx.html">website</a>, <a href="https://blog.luevano.xyz/a/mail_server_with_postfix.html">mail</a> and <a href="https://blog.luevano.xyz/a/git_server_with_cgit.html">git</a>) plus:</p> -<ul> -<li><strong>A</strong> and (optionally) <strong>AAA</strong> DNS records for:<ul> -<li><code>xmpp</code>: the actual XMPP server and the file upload service.</li> -<li><code>muc</code> (or <code>conference</code>): for multi-user chats.</li> -<li><code>pubsub</code>: the publish-subscribe service.</li> -<li><code>proxy</code>: a proxy in case one of the users needs it.</li> -<li><code>vjud</code>: user directory.</li> -</ul> -</li> -<li>(Optionally, but recommended) the following <strong>SRV</strong> DNS records; make sure it is pointing to an <strong>A</strong> or <strong>AAA</strong> record (matching the records from the last point, for example):<ul> -<li><code>_xmpp-client._tcp.{your.domain}.</code> for port <code>5222</code> pointing to <code>xmpp.{your.domain}.</code></li> -<li><code>_xmpp-server._tcp.{your.domain}.</code> for port <code>5269</code> pointing to <code>xmpp.{your.domain}.</code></li> -<li><code>_xmpp-server._tcp.muc.{your.domain}.</code> for port <code>5269</code> pointing to <code>xmpp.{your.domain}.</code></li> -</ul> -</li> -<li>SSL certificates for the previous subdomains; similar that with my other entries just create the appropriate <code>prosody.conf</code> (where <code>server_name</code> will be all the subdomains defined above) file and run <code>certbot --nginx</code>. You can find the example configuration file almost at the end of this entry.</li> -<li>Email addresses for <code>admin</code>, <code>abuse</code>, <code>contact</code>, <code>security</code>, etc. Or use your own email for all of them, doesn’t really matter much as long as you define them in the configuration and are valid, I have aliases so those emails are forwarded to me.</li> -<li>Allow ports <code>5000</code>, <code>5222</code>, <code>5269</code>, <code>5280</code> and <code>5281</code> for <a href="https://prosody.im/doc/ports">Prosody</a> and, <code>3478</code> and <code>5349</code> for <a href="https://webrtc.org/getting-started/turn-server">Turnserver</a> which are the defaults for <code>coturn</code>.</li> -</ul> -<h2 id="prosody">Prosody<a class="headerlink" href="#prosody" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Prosody">Prosody</a> is an implementation of the XMPP protocol that is flexible and extensible.</p> -<p>Install the <code>prosody</code> package (with optional dependencies) and the <code>mercurial</code> package:</p> -<pre><code class="language-sh">pacman -S prosody, mercurial, lua52-sec, lua52-dbi, lua52-zlib -</code></pre> -<p>We need mercurial to be able to download and update the extra modules needed to make the server compliant with <code>conversations.im</code> and <code>mov.im</code>. Go to <code>/var/lib/prosody</code>, clone the latest Prosody modules repository and prepare the directories:</p> -<pre><code class="language-sh">cd /var/lib/prosody -hg clone https://hg.prosody.im/prosody-modules modules-available -mkdir modules-enabled -</code></pre> -<p>You can see that I follow a similar approach that I used with Nginx and the server configuration, where I have all the modules available in a directory, and make a symlink to another to keep track of what is being used. You can update the repository by running <code>hg pull --update</code> while inside the <code>modules-available</code> directory (similar to Git).</p> -<p>Make symbolic links to the following modules:</p> -<pre><code>ln -s /var/lib/prosody/modules-available/{module_name} /var/lib/prosody/modules-enabled/ -... -</code></pre> -<ul> -<li>Modules (<code>{module_name}</code>):<ul> -<li><code>mod_bookmarks</code></li> -<li><code>mod_cache_c2s_caps</code></li> -<li><code>mod_checkcerts</code></li> -<li><code>mod_cloud_notify</code></li> -<li><code>mod_csi_battery_saver</code></li> -<li><code>mod_default_bookmarks</code></li> -<li><code>mod_external_services</code></li> -<li><code>mod_http_avatar</code></li> -<li><code>mod_http_pep_avatar</code></li> -<li><code>mod_http_upload</code></li> -<li><code>mod_http_upload_external</code></li> -<li><code>mod_idlecompat</code></li> -<li><code>mod_muc_limits</code></li> -<li><code>mod_muc_mam_hints</code></li> -<li><code>mod_muc_mention_notifications</code></li> -<li><code>mod_presence_cache</code></li> -<li><code>mod_pubsub_feeds</code></li> -<li><code>mod_pubsub_text_interface</code></li> -<li><code>mod_smacks</code></li> -<li><code>mod_strict_https</code></li> -<li><code>mod_vcard_muc</code></li> -<li><code>mod_vjud</code></li> -<li><code>mod_watchuntrusted</code></li> -</ul> -</li> -</ul> -<p>And add other modules if needed, but these work for the apps that I mentioned. You should also change the permissions for these files:</p> -<pre><code class="language-sh">chown -R prosody:prosody /var/lib/prosody -</code></pre> -<p>Now, configure the server by editing the <code>/etc/prosody/prosody.cfg.lua</code> file. It’s a bit tricky to configure, so here is my configuration file (lines starting with <code>--</code> are comments). Make sure to change according to your domain, and maybe preferences. Read each line and each comment to know what’s going on, It’s easier to explain it with comments in the file itself than strip it in a lot of pieces.</p> -<p>And also, note that the configuration file has a “global” section and a per “virtual server”/”component” section, basically everything above all the VirtualServer/Component sections are global, and bellow each VirtualServer/Component, corresponds to that section.</p> -<pre><code>-- important for systemd -daemonize = true -pidfile = "/run/prosody/prosody.pid" - --- or your account, not that this is an xmpp jid, not email -admins = { "admin@your.domain" } - -contact_info = { - abuse = { "mailto:abuse@your.domain", "xmpp:abuse@your.domain" }; - admin = { "mailto:admin@your.domain", "xmpp:admin@your.domain" }; - admin = { "mailto:feedback@your.domain", "xmpp:feedback@your.domain" }; - security = { "mailto:security@your.domain" }; - support = { "mailto:support@your.domain", "xmpp:support@muc.your.domain" }; -} - --- so prosody look up the plugins we added -plugin_paths = { "/var/lib/prosody/modules-enabled" } - -modules_enabled = { - -- Generally required - "roster"; -- Allow users to have a roster. Recommended ;) - "saslauth"; -- Authentication for clients and servers. Recommended if you want to log in. - "tls"; -- Add support for secure TLS on c2s/s2s connections - "dialback"; -- s2s dialback support - "disco"; -- Service discovery - -- Not essential, but recommended - "carbons"; -- Keep multiple clients in sync - "pep"; -- Enables users to publish their avatar, mood, activity, playing music and more - "private"; -- Private XML storage (for room bookmarks, etc.) - "blocklist"; -- Allow users to block communications with other users - "vcard4"; -- User profiles (stored in PEP) - "vcard_legacy"; -- Conversion between legacy vCard and PEP Avatar, vcard - "limits"; -- Enable bandwidth limiting for XMPP connections - -- Nice to have - "version"; -- Replies to server version requests - "uptime"; -- Report how long server has been running - "time"; -- Let others know the time here on this server - "ping"; -- Replies to XMPP pings with pongs - "register"; -- Allow users to register on this server using a client and change passwords - "mam"; -- Store messages in an archive and allow users to access it - "csi_simple"; -- Simple Mobile optimizations - -- Admin interfaces - "admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands - --"admin_telnet"; -- Opens telnet console interface on localhost port 5582 - -- HTTP modules - "http"; -- Explicitly enable http server. - "bosh"; -- Enable BOSH clients, aka "Jabber over HTTP" - "websocket"; -- XMPP over WebSockets - "http_files"; -- Serve static files from a directory over HTTP - -- Other specific functionality - "groups"; -- Shared roster support - "server_contact_info"; -- Publish contact information for this service - "announce"; -- Send announcement to all online users - "welcome"; -- Welcome users who register accounts - "watchregistrations"; -- Alert admins of registrations - "motd"; -- Send a message to users when they log in - --"legacyauth"; -- Legacy authentication. Only used by some old clients and bots. - --"s2s_bidi"; -- not yet implemented, have to wait for v0.12 - "bookmarks"; - "checkcerts"; - "cloud_notify"; - "csi_battery_saver"; - "default_bookmarks"; - "http_avatar"; - "idlecompat"; - "presence_cache"; - "smacks"; - "strict_https"; - --"pep_vcard_avatar"; -- not compatible with this version of pep, wait for v0.12 - "watchuntrusted"; - "webpresence"; - "external_services"; - } - --- only if you want to disable some modules -modules_disabled = { - -- "offline"; -- Store offline messages - -- "c2s"; -- Handle client connections - -- "s2s"; -- Handle server-to-server connections - -- "posix"; -- POSIX functionality, sends server to background, enables syslog, etc. -} - -external_services = { - { - type = "stun", - transport = "udp", - host = "proxy.your.domain", - port = 3478 - }, { - type = "turn", - transport = "udp", - host = "proxy.your.domain", - port = 3478, - -- you could decide this now or come back later when you install coturn - secret = "YOUR SUPER SECRET TURN PASSWORD" - } -} - ---- general global configuration -http_ports = { 5280 } -http_interfaces = { "*", "::" } - -https_ports = { 5281 } -https_interfaces = { "*", "::" } - -proxy65_ports = { 5000 } -proxy65_interfaces = { "*", "::" } - -http_default_host = "xmpp.your.domain" -http_external_url = "https://xmpp.your.domain/" --- or if you want to have it somewhere else, change this -https_certificate = "/etc/prosody/certs/xmpp.your.domain.crt" - -hsts_header = "max-age=31556952" - -cross_domain_bosh = true ---consider_bosh_secure = true -cross_domain_websocket = true ---consider_websocket_secure = true - -trusted_proxies = { "127.0.0.1", "::1", "192.169.1.1" } - -pep_max_items = 10000 - --- this is disabled by default, and I keep it like this, depends on you ---allow_registration = true - --- you might want this options as they are -c2s_require_encryption = true -s2s_require_encryption = true -s2s_secure_auth = false ---s2s_insecure_domains = { "insecure.example" } ---s2s_secure_domains = { "jabber.org" } - --- where the certificates are stored (/etc/prosody/certs by default) -certificates = "certs" -checkcerts_notify = 7 -- ( in days ) - --- rate limits on connections to the server, these are my personal settings, because by default they were limited to something like 30kb/s -limits = { - c2s = { - rate = "2000kb/s"; - }; - s2sin = { - rate = "5000kb/s"; - }; - s2sout = { - rate = "5000kb/s"; - }; -} - --- again, this could be yourself, it is a jid -unlimited_jids = { "admin@your.domain" } - -authentication = "internal_hashed" - --- if you don't want to use sql, change it to internal and comment the second line --- since this is optional, i won't describe how to setup mysql or setup the user/database, that would be out of the scope for this entry -storage = "sql" -sql = { driver = "MySQL", database = "prosody", username = "prosody", password = "PROSODY USER SECRET PASSWORD", host = "localhost" } - -archive_expires_after = "4w" -- configure message archive -max_archive_query_results = 20; -mam_smart_enable = true -default_archive_policy = "roster" -- archive only messages from users who are in your roster - --- normally you would like at least one log file of certain level, but I keep all of them, the default is only the info = "*syslog" one -log = { - info = "*syslog"; - warn = "prosody.warn"; - error = "prosody.err"; - debug = "prosody.debug"; - -- "*console"; -- Needs daemonize=false -} - --- cloud_notify -push_notification_with_body = false -- Whether or not to send the message body to remote pubsub node -push_notification_with_sender = false -- Whether or not to send the message sender to remote pubsub node -push_max_errors = 5 -- persistent push errors are tolerated before notifications for the identifier in question are disabled -push_max_devices = 5 -- number of allowed devices per user - --- by default every user on this server will join these muc rooms -default_bookmarks = { - { jid = "room@muc.your.domain", name = "The Room" }; - { jid = "support@muc.your.domain", name = "Support Room" }; -} - --- could be your jid -untrusted_fail_watchers = { "admin@your.domain" } -untrusted_fail_notification = "Establishing a secure connection from $from_host to $to_host failed. Certificate hash: $sha1. $errors" - ------------ Virtual hosts ----------- -VirtualHost "your.domain" - name = "Prosody" - http_host = "xmpp.your.domain" - -disco_items = { - { "your.domain", "Prosody" }; - { "muc.your.domain", "MUC Service" }; - { "pubsub.your.domain", "Pubsub Service" }; - { "proxy.your.domain", "SOCKS5 Bytestreams Service" }; - { "vjud.your.domain", "User Directory" }; -} - - --- Multi-user chat -Component "muc.your.domain" "muc" - name = "MUC Service" - modules_enabled = { - --"bob"; -- not compatible with this version of Prosody - "muc_limits"; - "muc_mam"; -- message archive in muc, again, a placeholder - "muc_mam_hints"; - "muc_mention_notifications"; - "vcard_muc"; - } - - restrict_room_creation = false - - muc_log_by_default = true - muc_log_presences = false - log_all_rooms = false - muc_log_expires_after = "1w" - muc_log_cleanup_interval = 4 * 60 * 60 - - --- Upload -Component "xmpp.your.domain" "http_upload" - name = "Upload Service" - http_host= "xmpp.your.domain" - -- you might want to change this, these are numbers in bytes, so 10MB and 100MB respectively - http_upload_file_size_limit = 1024*1024*10 - http_upload_quota = 1024*1024*100 - - --- Pubsub -Component "pubsub.your.domain" "pubsub" - name = "Pubsub Service" - pubsub_max_items = 10000 - modules_enabled = { - "pubsub_feeds"; - "pubsub_text_interface"; - } - - -- personally i don't have any feeds configured - feeds = { - -- The part before = is used as PubSub node - --planet_jabber = "http://planet.jabber.org/atom.xml"; - --prosody_blog = "http://blog.prosody.im/feed/atom.xml"; - } - - --- Proxy -Component "proxy.your.domain" "proxy65" - name = "SOCKS5 Bytestreams Service" - proxy65_address = "proxy.your.domain" - - --- Vjud, user directory -Component "vjud.your.domain" "vjud" - name = "User Directory" - vjud_mode = "opt-in" -</code></pre> -<p>You <mark>HAVE</mark> to read all of the configuration file, because there are a lot of things that you need to change to make it work with your server/domain. Test the configuration file with:</p> -<pre><code class="language-sh">luac5.2 -p /etc/prosody/prosody.cfg.lua -</code></pre> -<p>Notice that by default <code>prosody</code> will look up certificates that look like <code>sub.your.domain</code>, but if you get the certificates like I do, you’ll have a single certificate for all subdomains, and by default it is in <code>/etc/letsencrypt/live</code>, which has some strict permissions. So, to import it you can run:</p> -<pre><code class="language-sh">prosodyctl --root cert import /etc/letsencrypt/live -</code></pre> -<p>Ignore the complaining about not finding the subdomain certificates and note that you will have to run that command on each certificate renewal, to automate this, add the <code>--deploy-hook</code> flag to your automated Certbot renewal system; for me it’s a <code>systemd</code> timer with the following <code>certbot.service</code>:</p> -<pre><code class="language-ini">[Unit] -Description=Let's Encrypt renewal - -[Service] -Type=oneshot -ExecStart=/usr/bin/certbot renew --quiet --agree-tos --deploy-hook "systemctl reload nginx.service && prosodyctl --root cert import /etc/letsencrypt/live" -</code></pre> -<p>And if you don’t have it already, the <code>certbot.timer</code>:</p> -<pre><code class="language-ini">[Unit] -Description=Twice daily renewal of Let's Encrypt's certificates - -[Timer] -OnCalendar=0/12:00:00 -RandomizedDelaySec=1h -Persistent=true - -[Install] -WantedBy=timers.target -</code></pre> -<p>Also, go to the <code>certs</code> directory and make the appropriate symbolic links:</p> -<pre><code class="language-sh">cd /etc/prosody/certs -ln -s your.domain.crt SUBDOMAIN.your.domain.crt -ln -s your.domain.key SUBDOMAIN.your.domain.key -... -</code></pre> -<p>That’s basically all the configuration that needs Prosody itself, but we still have to configure Nginx and Coturn before starting/enabling the <code>prosody</code> service.</p> -<h2 id="nginx-configuration-file">Nginx configuration file<a class="headerlink" href="#nginx-configuration-file" title="Permanent link">¶</a></h2> -<p>Since this is not an ordinary configuration file I’m going to describe this, too. Your <code>prosody.conf</code> file should have the following location blocks under the main server block (the one that listens to HTTPS):</p> -<pre><code class="language-nginx"># HTTPS server block -server { - root /var/www/prosody/; - server_name xmpp.luevano.xyz muc.luevano.xyz pubsub.luevano.xyz vjud.luevano.xyz proxy.luevano.xyz; - index index.html; - - # for extra https discovery (XEP-0256) - location /.well-known/acme-challenge { - allow all; - } - - # bosh specific - location /http-bind { - proxy_pass https://localhost:5281/http-bind; - - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_buffering off; - tcp_nodelay on; - } - - # websocket specific - location /xmpp-websocket { - proxy_pass https://localhost:5281/xmpp-websocket; - - proxy_http_version 1.1; - proxy_set_header Connection "Upgrade"; - proxy_set_header Upgrade $http_upgrade; - - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_read_timeout 900s; - } - - # general proxy - location / { - proxy_pass https://localhost:5281; - - proxy_set_header Host $host; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Real-IP $remote_addr; - } - ... - # Certbot stuff -} -# HTTP server block (the one that certbot creates) -server { - ... -} -</code></pre> -<p>Also, you need to add the following to your actual <code>your.domain</code> (this cannot be a subdomain) configuration file:</p> -<pre><code class="language-nginx">server { - ... - location /.well-known/host-meta { - default_type 'application/xrd+xml'; - add_header Access-Control-Allow-Origin '*' always; - } - - location /.well-known/host-meta.json { - default_type 'application/jrd+json'; - add_header Access-Control-Allow-Origin '*' always; - } - ... -} -</code></pre> -<p>And you will need the following <code>host-meta</code> and <code>host-meta.json</code> files inside the <code>.well-known/acme-challenge</code> directory for <code>your.domain</code> (following my nomenclature: <code>/var/www/yourdomaindir/.well-known/acme-challenge/</code>).</p> -<p>For <code>host-meta</code> file:</p> -<pre><code class="language-xml"><?xml version='1.0' encoding='utf-8'?> -<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'> - <Link rel="urn:xmpp:alt-connections:xbosh" - href="https://xmpp.your.domain:5281/http-bind" /> - <Link rel="urn:xmpp:alt-connections:websocket" - href="wss://xmpp.your.domain:5281/xmpp-websocket" /> -</XRD> -</code></pre> -<p>And <code>host-meta.json</code> file:</p> -<pre><code class="language-json">{ - "links": [ - { - "rel": "urn:xmpp:alt-connections:xbosh", - "href": "https://xmpp.your.domain:5281/http-bind" - }, - { - "rel": "urn:xmpp:alt-connections:websocket", - "href": "wss://xmpp.your.domain:5281/xmpp-websocket" - } - ] -} -</code></pre> -<p>Remember to have your <code>prosody.conf</code> file symlinked (or discoverable by Nginx) to the <code>sites-enabled</code> directory. You can now test and restart your <code>nginx</code> service (and test the configuration, optionally):</p> -<pre><code class="language-sh">nginx -t -systemctl restart nginx.service -</code></pre> -<h2 id="coturn">Coturn<a class="headerlink" href="#coturn" title="Permanent link">¶</a></h2> -<p><a href="https://github.com/coturn/coturn">Coturn</a> is the implementation of TURN and STUN server, which in general is for (at least in the XMPP world) voice support and external service discovery.</p> -<p>Install the <code>coturn</code> package:</p> -<pre><code class="language-sh">pacman -S coturn -</code></pre> -<p>You can modify the configuration file (located at <code>/etc/turnserver/turnserver.conf</code>) as desired, but at least you need to make the following changes (uncomment or edit):</p> -<pre><code class="language-ini">use-auth-secret -realm=proxy.your.domain -static-auth-secret=YOUR SUPER SECRET TURN PASSWORD -</code></pre> -<p>I’m sure there is more configuration to be made, like using SQL to store data and whatnot, but for now this is enough for me. Note that you may not have some functionality that’s needed to create dynamic users to use the TURN server, and to be honest I haven’t tested this since I don’t use this feature in my XMPP clients, but if it doesn’t work, or you know of an error or missing configuration don’t hesitate to <a href="https://luevano.xyz/contact.html">contact me</a>.</p> -<p>Start/enable the <code>turnserver</code> service:</p> -<pre><code class="language-sh">systemctl start turnserver.service -systemctl enable turnserver.service -</code></pre> -<p>You can test if your TURN server works at <a href="https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/">Trickle ICE</a>. You may need to add a user in the <code>turnserver.conf</code> to test this.</p> -<h2 id="wrapping-up">Wrapping up<a class="headerlink" href="#wrapping-up" title="Permanent link">¶</a></h2> -<p>At this point you should have a working XMPP server, start/enable the <code>prosody</code> service now:</p> -<pre><code class="language-sh">systemctl start prosody.service -systemctl enable prosody.service -</code></pre> -<p>And you can add your first user with the <code>prosodyctl</code> command (it will prompt you to add a password):</p> -<pre><code class="language-sh">prosodyctl adduser user@your.domain -</code></pre> -<p>You may want to add a <code>compliance</code> user, so you can check if your server is set up correctly. To do so, go to <a href="https://compliance.conversations.im/add/">XMPP Compliance Tester</a> and enter the <code>compliance</code> user credentials. It should have similar compliance score to mine:</p> -<p><a href='https://compliance.conversations.im/server/luevano.xyz'><img src='https://compliance.conversations.im/badge/luevano.xyz'></a></p> -<p>Additionally, you can test the security of your server in <a href="https://xmpp.net/index.php">IM Observatory</a>, here you only need to specify your <code>domain.name</code> (not <code>xmpp.domain.name</code>, if you set up the <strong>SRV</strong> DNS records correctly). Again, it should have a similar score to mine:</p> -<p><a href='https://xmpp.net/result.php?domain=luevano.xyz&type=client'><img src='https://xmpp.net/badge.php?domain=luevano.xyz' alt='xmpp.net score' /></a></p> -<p>You can now log in into your XMPP client of choice, if it asks for the server it should be <code>xmpp.your.domain</code> (or <code>your.domain</code> for some clients) and your login credentials <code>you@your.domain</code> and the password you chose (which you can change in most clients).</p> -<p>That’s it, send me a message at <a href="xmpp:david@luevano.xyz">david@luevano.xyz</a> if you were able to set up the server successfully.</p>]]></content:encoded> - </item> - <item> - <title>Al fin ya me acomodé la página pa' los dibujos</title> - <link>https://blog.luevano.xyz/a/acomodada_la_pagina_de_arte.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/acomodada_la_pagina_de_arte.html</guid> - <pubDate>Sun, 06 Jun 2021 19:06:09 GMT</pubDate> - <category>Short</category> - <category>Spanish</category> - <category>Update</category> - <description>Actualización en el estado de la página, en este caso sobre la existencia de una nueva página para los dibujos y arte en general.</description> - <content:encoded><![CDATA[<p>Así es, ya quedó acomodado el sub-dominio <code>art.luevano.xyz</code> pos pal <a href="https://art.luevano.xyz">arte</a> veda. Entonces pues ando feliz por eso.</p> -<p>Este pedo fue gracias a que me reescribí la forma en la que <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> maneja los templates, ahora uso el sistema de <a href="https://jinja.palletsprojects.com/en/3.1.x/"><code>jinja</code></a> en vez del cochinero que hacía antes.</p> -<p>Y pues nada más eso, aquí está el <a href="https://art.luevano.xyz/a/elephant_octopus.html">primer post</a> y por supuesto acá está el link del RSS <a href="https://art.luevano.xyz/rss.xml">https://art.luevano.xyz/rss.xml</a>.</p>]]></content:encoded> - </item> - <item> - <title>Así nomás está quedando el página</title> - <link>https://blog.luevano.xyz/a/asi_nomas_esta_quedando.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/asi_nomas_esta_quedando.html</guid> - <pubDate>Fri, 04 Jun 2021 08:24:03 GMT</pubDate> - <category>Short</category> - <category>Spanish</category> - <category>Update</category> - <description>Actualización en el estado de la página, el servidor de XMPP y Matrix que me acomodé y próximas cosas que quiero hacer.</description> - <content:encoded><![CDATA[<p>Estuve acomodando un poco más el <em>sItIo</em>, al fin agregué la “sección” de <a href="https://luevano.xyz/contact.html">contact</a> y de <a href="https://luevano.xyz/donate.html">donate</a> por si hay algún loco que quiere tirar varo.</p> -<p>También me puse a acomodar un servidor de <a href="https://xmpp.org/">XMPP</a> el cual, en pocas palabras, es un protocolo de mensajería instantánea (y más) descentralizado, por lo cual cada quien puede hacer una cuenta en el servidor que quiera y conectarse con cuentas creadas en otro servidor… exacto, como con los correos electrónicos. Y esto está perro porque si tú tienes tu propio server, así como con uno de correo electrónico, puedes controlar qué características tiene, quiénes pueden hacer cuenta, si hay <em>end-to-end encryption</em> (o mínimo <em>end-to-server</em>), entre un montón de otras cosas.</p> -<p>Ahorita este server es SUMISO (<em>compliant</em> en español, jeje) para jalar con la app <a href="https://conversations.im/">conversations</a> y con la red social <a href="https://movim.eu/">movim</a>, pero realmente funcionaría con casi cualquier cliente de XMPP, amenos que ese cliente implemente algo que no tiene mi server. Y también acomodé un server de <a href="https://matrix.org/">Matrix</a> que es muy similar pero es bajo otro protocolo y se siente más como un discord/slack (al menos en el <a href="https://element.io/">element</a>), muy chingón también.</p> -<p>Si bien aún quedan cosas por hacer sobre estos dos servers que me acomodé (además de hacerles unas entradas para documentar cómo lo hice), quiero moverme a otra cosa que sería acomodar una sección de dibujos, lo cual en teoría es bien sencillo, pero como quiero poder automatizar la publicación de estos, quiero modificar un poco el <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> para que jale chido para este pex.</p> -<p>Ya por último también quiero moverle un poco al CSS, porque lo dejé en un estado muy culerón y quiero meterle/ajustar unas cosas para que quede más limpio y medianamente bonito… <em>dentro de lo que cabe porque evidentemente me vale verga si se ve como una página del 2000</em>.</p> -<p><strong>Actualización</strong>: Ya tumbé el servidor de XMPP porque consumía bastantes recursos y no lo usaba tanto, si en un futuro consigo un mejor servidor podría volver a hostearlo.</p>]]></content:encoded> - </item> - <item> - <title>I'm using a new blogging system</title> - <link>https://blog.luevano.xyz/a/new_blogging_system.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/new_blogging_system.html</guid> - <pubDate>Fri, 28 May 2021 03:21:39 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Tools</category> - <category>Update</category> - <description>I created a new blogging system called pyssg, which is based on what I was using but, to be honest, better.</description> - <content:encoded><![CDATA[<p>So, I was tired of working with <code>ssg</code> (and then <code>sbg</code> which was a modified version of <code>ssg</code> that I “wrote”), for one general reason: not being able to extend it as I would like; and not just dumb little stuff, I wanted to be able to have more control, to add tags (which another tool that I found does: <code>blogit</code>), and even more in a future.</p> -<p>The solution? Write a new program “from scratch” in <em>pYtHoN</em>. Yes it is bloated, yes it is in its early stages, but it works just as I want it to work, and I’m pretty happy so far with the results and have with even more ideas in mind to “optimize” and generally clean my wOrKfLoW to post new blog entries. I even thought of using it for posting into a “feed” like gallery for drawings or pictures in general.</p> -<p>I called it <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, because it sounds nice and it wasn’t taken in the PyPi. It is just a terminal program that reads either a configuration file or the options passed as flags when calling the program.</p> -<p>It still uses Markdown files because I find them very easy to work with. And instead of just having a “header” and a “footer” applied to each parsed entry, you will have templates (generated with the program) for each piece that I thought made sense (idea taken from <code>blogit</code>): the common header and footer, the common header and footer for each entry and, header, footer and list elements for articles and tags. When parsing the Markdown file these templates are applied and stitched together to make a single HTML file. Also generates an RSS feed and the <code>sitemap.xml</code> file, which is nice.</p> -<p>It might sound convoluted, but it works pretty well, with of course room to improve; I’m open to suggestions, issue reporting or direct contributions <a href="https://github.com/luevano/pyssg">here</a>. For now, it is only tested on Linux (and don’t think on making it work on windows, but feel free to do PR for the compatibility).</p> -<p>That’s it for now, the new RSS feed is available here: <a href="https://blog.luevano.xyz/rss.xml">https://blog.luevano.xyz/rss.xml</a>.</p> -<p><strong>Update</strong>: Since writing this entry, <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> has evolved quite a bit, so not everything described here is still true. For the latest updates check the newest entries or the git repository itself.</p>]]></content:encoded> - </item> - <item> - <title>Set up a Git server and cgit front-end</title> - <link>https://blog.luevano.xyz/a/git_server_with_cgit.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/git_server_with_cgit.html</guid> - <pubDate>Sun, 21 Mar 2021 19:00:29 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to create a Git server using cgit on a server running Nginx, on Arch. This is a follow up on post about creating a website with Nginx and Certbot.</description> - <content:encoded><![CDATA[<p>My git server is all I need to setup to actually <em>kill</em> my other server (I’ve been moving from servers on these last 2-3 blog entries), that’s why I’m already doing this entry. I’m basically following <a href="https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server">git’s guide on setting up a server</a> plus some specific stuff for <mark>btw i use</mark> Arch Linux (<a href="https://wiki.archlinux.org/index.php/Git_server#Web_interfaces">Arch Linux Wiki: Git server</a> and <a href="https://miracoin.wordpress.com/2014/11/25/step-by-step-guide-on-setting-up-git-server-in-arch-linux-pushable/">Step by step guide on setting up git server in arch linux (pushable)</a>).</p> -<p>Note that this is mostly for personal use, so there’s no user/authentication control other than that of normal <code>ssh</code>. And as with the other entries, most if not all commands here are run as root unless stated otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#git">Git</a></li> -<li><a href="#cgit">Cgit</a><ul> -<li><a href="#cgits-file-rendering">Cgit’s file rendering</a></li> -</ul> -</li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>I might get tired of saying this (it’s just copy paste, basically)… but you will need the same prerequisites as before (check my <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> and <a href="https://blog.luevano.xyz/a/mail_server_with_postfix.html">mail</a> entries), with the extras:</p> -<ul> -<li>(Optional, if you want a “front-end”) A <strong>CNAME</strong> for “git” and (optionally) “www.git”, or some other name for your sub-domains.</li> -<li>An SSL certificate, if you’re following the other entries, add a <code>git.conf</code> and run <code>certbot --nginx</code> to extend the certificate.</li> -</ul> -<h2 id="git">Git<a class="headerlink" href="#git" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/git">Git</a> is a version control system.</p> -<p>If not installed already, install the <code>git</code> package:</p> -<pre><code class="language-sh">pacman -S git -</code></pre> -<p>On Arch Linux, when you install the <code>git</code> package, a <code>git</code> user is automatically created, so all you have to do is decide where you want to store the repositories, for me, I like them to be on <code>/home/git</code> like if <code>git</code> was a “normal” user. So, create the <code>git</code> folder (with corresponding permissions) under <code>/home</code> and set the <code>git</code> user’s home to <code>/home/git</code>:</p> -<pre><code class="language-sh">mkdir /home/git -chown git:git /home/git -usermod -d /home/git git -</code></pre> -<p>Also, the <code>git</code> user is “expired” by default and will be locked (needs a password), change that with:</p> -<pre><code class="language-sh">chage -E -1 git -passwd git -</code></pre> -<p>Give it a strong one and remember to use <code>PasswordAuthentication no</code> for <code>ssh</code> (as you should). Create the <code>.ssh/authorized_keys</code> for the <code>git</code> user and set the permissions accordingly:</p> -<pre><code class="language-sh">mkdir /home/git/.ssh -chmod 700 /home/git/.ssh -touch /home/git/.ssh/authorized_keys -chmod 600 /home/git/.ssh/authorized_keys -chown -R git:git /home/git -</code></pre> -<p>Now is a good idea to copy over your local SSH public keys to this file, to be able to push/pull to the repositories. Do it by either manually copying it or using <code>ssh</code>‘s built in <code>ssh-copy-id</code> (for that you may want to check your <code>ssh</code> configuration in case you don’t let people access your server with user/password).</p> -<p>Next, and almost finally, we need to edit the <code>git-daemon</code> service, located at <code>/usr/lib/systemd/system/</code> (called <code>git-daemon@.service</code>):</p> -<pre><code class="language-ini">... -ExecStart=-/usr/lib/git-core/git-daemon --inetd --export-all --base-path=/home/git --enable=receive-pack -... -</code></pre> -<p>I just appended <code>--enable=receive-pack</code> and note that I also changed the <code>--base-path</code> to reflect where I want to serve my repositories from (has to match what you set when changing <code>git</code> user’s home).</p> -<p>Now, go ahead and start and enable the <code>git-daemon</code> socket:</p> -<pre><code class="language-sh">systemctl start git-daemon.socket -systemctl enable git-daemon.socket -</code></pre> -<p>You’re basically done. Now you should be able to push/pull repositories to your server… except, you haven’t created any repository in your server, that’s right, they’re not created automatically when trying to push. To do so, you have to run (while inside <code>/home/git</code>):</p> -<pre><code class="language-sh">git init --bare {repo_name}.git -chown -R git:git {repo_name}.git -</code></pre> -<p><mark>Those two lines above will need to be run each time you want to add a new repository to your server</mark>. There are options to “automate” this but I like it this way.</p> -<p>After that you can already push/pull to your repository. I have my repositories (locally) set up so I can push to more than one remote at the same time (my server, GitHub, GitLab, etc.); to do so, check <a href="https://gist.github.com/rvl/c3f156e117e22a25f242">this gist</a>.</p> -<h2 id="cgit">Cgit<a class="headerlink" href="#cgit" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Cgit">Cgit</a> is a fast web interface for git.</p> -<p>This is optionally since it’s only for the web application.</p> -<p>Install the <code>cgit</code> and <code>fcgiwrap</code> packages:</p> -<pre><code class="language-sh">pacman -S cgit fcgiwrap -</code></pre> -<p>Now, just start and enable the <code>fcgiwrap</code> socket:</p> -<pre><code class="language-sh">systemctl start fcgiwrap.socket -systemctl enable fcgiwrap.socket -</code></pre> -<p>Next, create the <code>git.conf</code> as stated in my <a href="https://blog.luevano.xyz/a/website_with_nginx.html">nginx setup entry</a>. Add the following lines to your <code>git.conf</code> file:</p> -<pre><code class="language-nginx">server { - listen 80; - listen [::]:80; - root /usr/share/webapps/cgit; - server_name {yoursubdomain}.{yourdomain}; - try_files $uri @cgit; - - location @cgit { - include fastcgi_params; - fastcgi_param SCRIPT_FILENAME $document_root/cgit.cgi; - fastcgi_param PATH_INFO $uri; - fastcgi_param QUERY_STRING $args; - fastcgi_param HTTP_HOST $server_name; - fastcgi_pass unix:/run/fcgiwrap.sock; - } -} -</code></pre> -<p>Where the <code>server_name</code> line depends on you, I have mine setup to <code>git.luevano.xyz</code> and <code>www.git.luevano.xyz</code>. Optionally run <code>certbot --nginx</code> to get a certificate for those domains if you don’t have already.</p> -<p>Now, all that’s left is to configure <code>cgit</code>. Create the configuration file <code>/etc/cgitrc</code> with the following content (my personal options, pretty much the default):</p> -<pre><code class="language-apache">css=/cgit.css -logo=/cgit.png - -enable-http-clone=1 -# robots=noindex, nofollow -virtual-root=/ - -repo.url={url} -repo.path={dir_path} -repo.owner={owner} -repo.desc={short_description} -... -</code></pre> -<p>Where you can uncomment the <code>robots</code> line to not let web crawlers (like Google’s) to index your <code>git</code> web app. And at the end keep all your repositories (the ones you want to make public), for example for my <a href="https://git.luevano.xyz/.dots"><em>dotfiles</em></a> I have:</p> -<pre><code class="language-apache">... -repo.url=.dots -repo.path=/home/git/.dots.git -repo.owner=luevano -repo.desc=These are my personal dotfiles. -... -</code></pre> -<p>Otherwise you could let <code>cgit</code> to automatically detect your repositories (you have to be careful if you want to keep “private” repos) using the option <code>scan-path</code> and setup <code>.git/description</code> for each repository. For more, you can check <a href="https://man.archlinux.org/man/cgitrc.5">cgitrc(5)</a>.</p> -<h3 id="cgits-file-rendering">Cgit’s file rendering<a class="headerlink" href="#cgits-file-rendering" title="Permanent link">¶</a></h3> -<p>By default you can’t see the files on the site, you need a highlighter to render the files, I use <code>highlight</code>. Install the <code>highlight</code> package:</p> -<pre><code class="language-sh">pacman -S highlight -</code></pre> -<p>Copy the <code>syntax-highlighting.sh</code> script to the corresponding location (basically adding <code>-edited</code> to the file):</p> -<pre><code class="language-sh">cp /usr/lib/cgit/filters/syntax-highlighting.sh /usr/lib/cgit/filters/syntax-highlighting-edited.sh -</code></pre> -<p>And edit it to use the version 3 and add <code>--inline-css</code> for more options without editing <code>cgit</code>‘s CSS file:</p> -<pre><code class="language-sh">... -# This is for version 2 -# exec highlight --force -f -I -X -S "$EXTENSION" 2>/dev/null - -# This is for version 3 -exec highlight --force --inline-css -f -I -O xhtml -S "$EXTENSION" 2>/dev/null -... -</code></pre> -<p>Finally, enable the filter in <code>/etc/cgitrc</code> configuration:</p> -<pre><code class="language-apache">source-filter=/usr/lib/cgit/filters/syntax-highlighting-edited.sh -</code></pre> -<p>That would be everything. If you need support for more stuff like compressed snapshots or support for markdown, check the optional dependencies for <code>cgit</code>.</p>]]></content:encoded> - </item> - <item> - <title>Set up a Mail server with Postfix, Dovecot, SpamAssassin and OpenDKIM</title> - <link>https://blog.luevano.xyz/a/mail_server_with_postfix.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/mail_server_with_postfix.html</guid> - <pubDate>Sun, 21 Mar 2021 04:05:59 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a Mail server using Postfix, Dovecot, SpamAssassin and OpenDKIM, on Arch. This is a follow up on post about creating a website with Nginx and Certbot.</description> - <content:encoded><![CDATA[<p>The entry is going to be long because it’s a <em>tedious</em> process. This is also based on <a href="https://github.com/LukeSmithxyz/emailwiz">Luke Smith’s script</a>, but adapted to Arch Linux (his script works on debian-based distributions). This entry is mostly so I can record all the notes required while I’m in the process of installing/configuring the mail server on a new VPS of mine; <del>also I’m going to be writing a script that does everything in one go (for Arch Linux), that will be hosted <a href="https://git.luevano.xyz/server_scripts.git">here</a>.</del> <ins>I haven’t had time to do the script so nevermind this, if I ever do it I’ll make a new entry regarding it.</ins></p> -<p>This configuration works for local users (users that appear in <code>/etc/passwd</code>), and does not use any type of SQL database. Do note that I’m not running Postfix in a chroot, which can be a problem if you’re following my steps as noted by <mark><a href="https://bojanmilevski.com/">Bojan</a></mark>; in the case that you want to run in chroot then add the steps chown in the Arch wiki: <a href="https://wiki.archlinux.org/title/postfix#Postfix_in_a_chroot_jail">Postfix in a chroot jail</a>; the issue faced if following my steps and using a chroot is that there will be issues resolving the hostname due to <code>/etc/hosts</code> or <code>/etc/hostname</code> not being available in the chroot.</p> -<p>All commands executed here are run with root privileges, unless stated otherwise.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#postfix">Postfix</a></li> -<li><a href="#dovecot">Dovecot</a></li> -<li><a href="#opendkim">OpenDKIM</a><ul> -<li><a href="#opendkim-dns-txt-records">OpenDKIM DNS TXT records</a></li> -</ul> -</li> -<li><a href="#spamassassin">SpamAssassin</a></li> -<li><a href="#wrapping-up">Wrapping up</a></li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>Basically the same as with the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website with Nginx and Certbot</a>, with the extras:</p> -<ul> -<li>You will need a <strong>CNAME</strong> for “mail” and (optionally) “www.mail”, or whatever you want to call the sub-domains (although the <a href="https://tools.ietf.org/html/rfc2181#section-10.3">RFC 2181</a> states that it NEEDS to be an <strong>A</strong> record, fuck the police).</li> -<li>An SSL certificate. You can use the SSL certificate obtained following my last post using <code>certbot</code> (just create a <code>mail.conf</code> and run <code>certbot --nginx</code> again).</li> -<li>Ports <code>25</code>, <code>587</code> (SMTP), <code>465</code> (SMTPS), <code>143</code> (IMAP) and <code>993</code> (IMAPS) open on the firewall (I use <code>ufw</code>).</li> -</ul> -<h2 id="postfix">Postfix<a class="headerlink" href="#postfix" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/postfix">Postfix</a> is a “mail transfer agent” which is the component of the mail server that receives and sends emails via SMTP.</p> -<p>Install the <code>postfix</code> package:</p> -<pre><code class="language-sh">pacman -S postfix -</code></pre> -<p>We have two main files to configure (inside <code>/etc/postfix</code>): <code>master.cf</code> (<a href="https://man.archlinux.org/man/master.5">master(5)</a>) and <code>main.cf</code> (<a href="https://man.archlinux.org/man/postconf.5">postconf(5)</a>). We’re going to edit <code>main.cf</code> first either by using the command <code>postconf -e 'setting'</code> or by editing the file itself (I prefer to edit the file).</p> -<p>Note that the default file itself has a lot of comments with description on what each thing does (or you can look up the manual, linked above), I used what Luke’s script did plus some other settings that worked for me.</p> -<p>Now, first locate where your website cert is, mine is at the default location <code>/etc/letsencrypt/live/</code>, so my <code>certdir</code> is <code>/etc/letsencrypt/live/luevano.xyz</code>. Given this information, change <code>{yourcertdir}</code> on the corresponding lines. The configuration described below has to be appended in the <code>main.cf</code> configuration file.</p> -<p>Certificates and ciphers to use for authentication and security:</p> -<pre><code class="language-apache">smtpd_tls_key_file = {yourcertdir}/privkey.pem -smtpd_tls_cert_file = {yourcertdir}/fullchain.pem -smtpd_use_tls = yes -smtpd_tls_auth_only = yes -smtp_tls_security_level = may -smtp_tls_loglevel = 1 -smtp_tls_CAfile = {yourcertdir}/cert.pem -smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 -smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 -smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 -smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 -tls_preempt_cipherlist = yes -smtpd_tls_exclude_ciphers = aNULL, LOW, EXP, MEDIUM, ADH, AECDH, MD5, - DSS, ECDSA, CAMELLIA128, 3DES, CAMELLIA256, - RSA+AES, eNULL - -smtp_tls_CApath = /etc/ssl/certs -smtpd_tls_CApath = /etc/ssl/certs - -smtpd_relay_restrictions = permit_sasl_authenticated, permit_mynetworks, defer_unauth_destination -</code></pre> -<p>Also, for the <em>connection</em> with <code>dovecot</code>, append the next few lines (telling postfix that <code>dovecot</code> will use user/password for authentication):</p> -<pre><code class="language-apache">smtpd_sasl_auth_enable = yes -smtpd_sasl_type = dovecot -smtpd_sasl_path = private/auth -smtpd_sasl_security_options = noanonymous, noplaintext -smtpd_sasl_tls_security_options = noanonymous -</code></pre> -<p>Specify the mailbox home, this is going to be a directory inside your user’s home containing the actual mail files, for example it will end up being<code>/home/david/Mail/Inbox</code>:</p> -<pre><code class="language-apache">home_mailbox = Mail/Inbox/ -</code></pre> -<p>Pre-configuration to work seamlessly with <code>dovecot</code> and <code>opendkim</code>:</p> -<pre><code class="language-apache">myhostname = {yourdomainname} -mydomain = localdomain -mydestination = $myhostname, localhost.$mydomain, localhost - -milter_default_action = accept -milter_protocol = 6 -smtpd_milters = inet:127.0.0.1:8891 -non_smtpd_milters = inet:127.0.0.1:8891 -mailbox_command = /usr/lib/dovecot/deliver -</code></pre> -<p>Where <code>{yourdomainname}</code> is <code>luevano.xyz</code> in my case. Lastly, if you don’t want the sender’s IP and user agent (application used to send the mail), add the following line:</p> -<pre><code class="language-apache">smtp_header_checks = regexp:/etc/postfix/smtp_header_checks -</code></pre> -<p>And create the <code>/etc/postfix/smtp_header_checks</code> file with the following content:</p> -<pre><code class="language-coffee">/^Received: .*/ IGNORE -/^User-Agent: .*/ IGNORE -</code></pre> -<p>That’s it for <code>main.cf</code>, now we have to configure <code>master.cf</code>. This one is a bit more tricky.</p> -<p>First look up lines (they’re uncommented) <code>smtp inet n - n - - smtpd</code>, <code>smtp unix - - n - - smtp</code> and <code>-o syslog_name=postfix/$service_name</code> and either delete or uncomment them… or just run <code>sed -i "/^\s*-o/d;/^\s*submission/d;/\s*smtp/d" /etc/postfix/master.cf</code> as stated in Luke’s script.</p> -<p>Lastly, append the following lines to complete postfix setup and pre-configure for <code>spamassassin</code>.</p> -<pre><code class="language-txt">smtp unix - - n - - smtp -smtp inet n - y - - smtpd - -o content_filter=spamassassin -submission inet n - y - - smtpd - -o syslog_name=postfix/submission - -o smtpd_tls_security_level=encrypt - -o smtpd_sasl_auth_enable=yes - -o smtpd_tls_auth_only=yes -smtps inet n - y - - smtpd - -o syslog_name=postfix/smtps - -o smtpd_tls_wrappermode=yes - -o smtpd_sasl_auth_enable=yes -spamassassin unix - n n - - pipe - user=spamd argv=/usr/bin/vendor_perl/spamc -f -e /usr/sbin/sendmail -oi -f \${sender} \${recipient} -</code></pre> -<p>Now, I ran into some problems with postfix, one being <a href="https://www.faqforge.com/linux/fix-for-opensuse-error-postfixmaster-fatal-0-0-0-0smtps-servname-not-supported-for-ai_socktype/">smtps: Servname not supported for ai_socktype</a>, to fix it, as <em>Till</em> posted in that site, edit <code>/etc/services</code> and add:</p> -<pre><code class="language-apache">smtps 465/tcp -smtps 465/udp -</code></pre> -<p>Before starting the <code>postfix</code> service, you need to run <code>newaliases</code> first, but you can do a bit of configuration beforehand editing the file <code>/etc/postfix/aliases</code>. I only change the <code>root: you</code> line (where <code>you</code> is the account that will be receiving “root” mail). After you’re done, run:</p> -<pre><code class="language-sh">postalias /etc/postfix/aliases -newaliases -</code></pre> -<p>At this point you’re done configuring <code>postfix</code> and you can already start/enable the <code>postfix</code> service:</p> -<pre><code class="language-sh">systemctl start postfix.service -systemctl enable postfix.service -</code></pre> -<h2 id="dovecot">Dovecot<a class="headerlink" href="#dovecot" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Dovecot">Dovecot</a> is an IMAP and POP3 server, which is what lets an email application retrieve the mail.</p> -<p>Install the <code>dovecot</code> and <code>pigeonhole</code> (sieve for <code>dovecot</code>) packages:</p> -<pre><code class="language-sh">pacman -S dovecot pigeonhole -</code></pre> -<p>On arch, by default, there is no <code>/etc/dovecot</code> directory with default configurations set in place, but the package does provide the example configuration files. Create the <code>dovecot</code> directory under <code>/etc</code> and, optionally, copy the <code>dovecot.conf</code> file and <code>conf.d</code> directory under the just created <code>dovecot</code> directory:</p> -<pre><code class="language-sh">mkdir /etc/dovecot -cp /usr/share/doc/dovecot/example-config/dovecot.conf /etc/dovecot/dovecot.conf -cp -r /usr/share/doc/dovecot/example-config/conf.d /etc/dovecot -</code></pre> -<p>As Luke stated, <code>dovecot</code> comes with a lot of “modules” (under <code>/etc/dovecot/conf.d/</code> if you copied that folder) for all sorts of configurations that you can include, but I do as he does and just edit/create the whole <code>dovecot.conf</code> file; although, I would like to check each of the separate configuration files <code>dovecot</code> provides I think the options Luke provides are more than good enough.</p> -<p>I’m working with an empty <code>dovecot.conf</code> file. Add the following lines for SSL and login configuration (also replace <code>{yourcertdir}</code> with the same certificate directory described in the Postfix section above, note that the <code><</code> is required):</p> -<pre><code class="language-apache">ssl = required -ssl_cert = <{yourcertdir}/fullchain.pem -ssl_key = <{yourcertdir}/privkey.pem -ssl_min_protocol = TLSv1.2 -ssl_cipher_list = ALL:!RSA:!CAMELLIA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!SHA1:!SHA256:!SHA384:!LOW@STRENGTH -ssl_prefer_server_ciphers = yes -ssl_dh = </etc/dovecot/dh.pem - -auth_mechanisms = plain login -auth_username_format = %n -protocols = $protocols imap -</code></pre> -<p>You may notice we specify a file we don’t have under <code>/etc/dovecot</code>: <code>dh.pem</code>. We need to create it with <code>openssl</code> (you should already have it installed if you’ve been following this entry and the one for <code>nginx</code>). Just run (might take a few minutes):</p> -<pre><code class="language-sh">openssl dhparam -out /etc/dovecot/dh.pem 4096 -</code></pre> -<p>After that, the next lines define what a “valid user is” (really just sets the database for users and passwords to be the local users with their password):</p> -<pre><code class="language-apache">userdb { - driver = passwd -} - -passdb { - driver = pam -} -</code></pre> -<p>Next, comes the mail directory structure (has to match the one described in the Postfix section). Here, the <code>LAYOUT</code> option is important so the boxes are <code>.Sent</code> instead of <code>Sent</code>. Add the next lines (plus any you like):</p> -<pre><code class="language-apache">mail_location = maildir:~/Mail:INBOX=~/Mail/Inbox:LAYOUT=fs -namespace inbox { - inbox = yes - - mailbox Drafts { - special_use = \Drafts - auto = subscribe - } - - mailbox Junk { - special_use = \Junk - auto = subscribe - autoexpunge = 30d - } - - mailbox Sent { - special_use = \Sent - auto = subscribe - } - - mailbox Trash { - special_use = \Trash - } - - mailbox Archive { - special_use = \Archive - } -} -</code></pre> -<p>Also include this so Postfix can use Dovecot’s authentication system:</p> -<pre><code class="language-apache">service auth { - unix_listener /var/spool/postfix/private/auth { - mode = 0660 - user = postfix - group = postfix - } -} -</code></pre> -<p>Lastly (for Dovecot at least), the plugin configuration for <code>sieve</code> (<code>pigeonhole</code>):</p> -<pre><code class="language-apache">protocol lda { - mail_plugins = $mail_plugins sieve -} - -protocol lmtp { - mail_plugins = $mail_plugins sieve -} - -plugin { - sieve = ~/.dovecot.sieve - sieve_default = /var/lib/dovecot/sieve/default.sieve - sieve_dir = ~/.sieve - sieve_global_dir = /var/lib/dovecot/sieve/ -</code></pre> -<p>Where <code>/var/lib/dovecot/sieve/default.sieve</code> doesn’t exist yet. Create the folders:</p> -<pre><code class="language-sh">mkdir -p /var/lib/dovecot/sieve -</code></pre> -<p>And create the file <code>default.sieve</code> inside that just created folder with the content:</p> -<pre><code class="language-nginx">require ["fileinto", "mailbox"]; -if header :contains "X-Spam-Flag" "YES" { - fileinto "Junk"; -} -</code></pre> -<p>Now, if you don’t have a <code>vmail</code> (virtual mail) user, create one and change the ownership of the <code>/var/lib/dovecot</code> directory to this user:</p> -<pre><code class="language-sh">grep -q "^vmail:" /etc/passwd || useradd -m vmail -s /usr/bin/nologin -chown -R vmail:vmail /var/lib/dovecot -</code></pre> -<p>Note that I also changed the shell for <code>vmail</code> to be <code>/usr/bin/nologin</code>. After that, to compile the configuration file run:</p> -<pre><code class="language-sh">sievec /var/lib/dovecot/sieve/default.sieve -</code></pre> -<p>A <code>default.svbin</code> file will be created next to <code>default.sieve</code>.</p> -<p>Next, add the following lines to <code>/etc/pam.d/dovecot</code> if not already present (shouldn’t be there if you’ve been following these notes):</p> -<pre><code class="language-txt">auth required pam_unix.so nullok -account required pam_unix.so -</code></pre> -<p>That’s it for Dovecot, at this point you can start/enable the <code>dovecot</code> service:</p> -<pre><code class="language-sh">systemctl start dovecot.service -systemctl enable dovecot.service -</code></pre> -<h2 id="opendkim">OpenDKIM<a class="headerlink" href="#opendkim" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/OpenDKIM">OpenDKIM</a> is needed so services like G**gle don’t throw the mail to the trash. DKIM stands for “DomainKeys Identified Mail”.</p> -<p>Install the <code>opendkim</code> package:</p> -<pre><code class="language-sh">pacman -S opendkim -</code></pre> -<p>Generate the keys for your domain:</p> -<pre><code class="language-sh">opendkim-genkey -D /etc/opendkim -d {yourdomain} -s {yoursubdomain} -r -b 2048 -</code></pre> -<p>Where you need to change <code>{yourdomain}</code> and <code>{yoursubdomain}</code> (doesn’t really need to be the sub-domain, could be anything that describes your key) accordingly, for me it’s <code>luevano.xyz</code> and <code>mail</code>, respectively. After that, we need to create some files inside the <code>/etc/opendkim</code> directory. First, create the file <code>KeyTable</code> with the content:</p> -<pre><code class="language-txt">{yoursubdomain}._domainkey.{yourdomain} {yourdomain}:{yoursubdomain}:/etc/opendkim/{yoursubdomain}.private -</code></pre> -<p>So, for me it would be:</p> -<pre><code class="language-txt">mail._domainkey.luevano.xyz luevano.xyz:mail:/etc/opendkim/mail.private -</code></pre> -<p>Next, create the file <code>SigningTable</code> with the content:</p> -<pre><code class="language-txt">*@{yourdomain} {yoursubdomain}._domainkey.{yourdomain} -</code></pre> -<p>Again, for me it would be:</p> -<pre><code class="language-txt">*@luevano.xyz mail._domainkey.luevano.xyz -</code></pre> -<p>And, lastly create the file <code>TrustedHosts</code> with the content:</p> -<pre><code class="language-txt">127.0.0.1 -::1 -10.1.0.0/16 -1.2.3.4/24 -localhost -{yourserverip} -... -</code></pre> -<p>And more, make sure to include your server IP and something like <code>subdomain.domainname</code>.</p> -<p>Next, edit <code>/etc/opendkim/opendkim.conf</code> to reflect the changes (or rather, addition) of these files, as well as some other configuration. You can look up the example configuration file located at <code>/usr/share/doc/opendkim/opendkim.conf.sample</code>, but I’m creating a blank one with the contents:</p> -<pre><code class="language-apache">Domain {yourdomain} -Selector {yoursubdomain} - -Syslog Yes -UserID opendkim - -KeyFile /etc/opendkim/{yoursubdomain}.private -Socket inet:8891@localhost -</code></pre> -<p>Now, change the permissions for all the files inside <code>/etc/opendkim</code>:</p> -<pre><code class="language-sh">chown -R root:opendkim /etc/opendkim -chmod g+r /etc/postfix/dkim/* -</code></pre> -<p>I’m using <code>root:opendkim</code> so <code>opendkim</code> doesn’t complain about the <code>{yoursubdomani}.private</code> being insecure (you can change that by using the option <code>RequireSafeKeys False</code> in the <code>opendkim.conf</code> file, as stated <a href="http://lists.opendkim.org/archive/opendkim/users/2014/12/3331.html">here</a>).</p> -<p>That’s it for the general configuration, but you could go more in depth and be more secure with some extra configuration.</p> -<p>Now, just start/enable the <code>opendkim</code> service:</p> -<pre><code class="language-sh">systemctl start opendkim.service -systemctl enable opendkim.service -</code></pre> -<h3 id="opendkim-dns-txt-records">OpenDKIM DNS TXT records<a class="headerlink" href="#opendkim-dns-txt-records" title="Permanent link">¶</a></h3> -<p>Add the following <strong>TXT</strong> records on your domain registrar (these examples are for Epik):</p> -<ol> -<li><em>DKIM</em> entry: look up your <code>{yoursubdomain}.txt</code> file, it should look something like:</li> -</ol> -<pre><code class="language-txt">{yoursubdomain}._domainkey IN TXT ( "v=DKIM1; k=rsa; s=email; " - "p=..." - "..." ) ; ----- DKIM key mail for {yourdomain} -</code></pre> -<p>In the <strong>TXT</strong> record you will place <code>{yoursubdomain}._domainkey</code> as the “Host” and <code>"v=DKIM1; k=rsa; s=email; " "p=..." "..."</code> in the “TXT Value” (replace the dots with the actual value you see in your file).</p> -<ol start="2"> -<li> -<p><em>DMARC</em> entry: just <code>_dmarc.{yourdomain}</code> as the “Host” and <code>"v=DMARC1; p=reject; rua=mailto:dmarc@{yourdomain}; fo=1"</code> as the “TXT Value”.</p> -</li> -<li> -<p><em>SPF</em> entry: just <code>@</code> as the “Host” and <code>"v=spf1 mx a:{yoursubdomain}.{yourdomain} - all"</code> as the “TXT Value”.</p> -</li> -</ol> -<p>And at this point you could test your mail for spoofing and more.</p> -<h2 id="spamassassin">SpamAssassin<a class="headerlink" href="#spamassassin" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/SpamAssassin">SpamAssassin</a> is just <em>a mail filter to identify spam</em>.</p> -<p>Install the <code>spamassassin</code> package (which will install a bunch of ugly <code>perl</code> packages…):</p> -<pre><code class="language-sh">pacman -S spamassassin -</code></pre> -<p>For some reason, the permissions on all <code>spamassassin</code> stuff are all over the place. First, change owner of the executables, and directories:</p> -<pre><code class="language-sh">chown spamd:spamd /usr/bin/vendor_perl/sa-* -chown spamd:spamd /usr/bin/vendor_perl/spam* -chwown -R spamd:spamd /etc/mail/spamassassin -</code></pre> -<p>Then, you can edit <code>local.cf</code> (located in <code>/etc/mail/spamassassin</code>) to fit your needs (I only uncommented the <code>rewrite_header Subject ...</code> line). And then you can run the following command to update the patterns and compile them:</p> -<pre><code class="language-sh">sudo -u spamd sa-update -sudo -u spamd sa-compile -</code></pre> -<p>And since this should be run periodically, create the service <code>spamassassin-update.service</code> under <code>/etc/systemd/system</code> with the following content:</p> -<pre><code class="language-ini">[Unit] -Description=SpamAssassin housekeeping -After=network.target - -[Service] -User=spamd -Group=spamd -Type=oneshot - -ExecStart=/usr/bin/vendor_perl/sa-update --allowplugins -SuccessExitStatus=1 -ExecStart=/usr/bin/vendor_perl/sa-compile -ExecStart=/usr/bin/systemctl -q --no-block try-restart spamassassin.service -</code></pre> -<p>And you could also execute <code>sa-learn</code> to train <code>spamassassin</code>‘s bayes filter, but this works for me. Then create the timer <code>spamassassin-update.timer</code> under the same directory, with the content:</p> -<pre><code class="language-ini">[Unit] -Description=SpamAssassin housekeeping - -[Timer] -OnCalendar=daily -Persistent=true - -[Install] -WantedBy=timers.target -</code></pre> -<p>You can now start/enable the <code>spamassassin-update</code> timer:</p> -<pre><code class="language-sh">systemctl start spamassassin-update.timer -systemctl enable spamassassin-update.timer -</code></pre> -<p>Next, you may want to edit the <code>spamassassin</code> service before starting and enabling it, because by default, it could <a href="https://rimuhosting.com/howto/memory.jsp">spawn a lot of “childs”</a> eating a lot of resources and you really only need one child. Append <code>--max-children=1</code> to the line <code>ExecStart=...</code> in <code>/usr/bin/systemd/system/spamassassin.service</code>:</p> -<pre><code class="language-ini">... -ExecStart=/usr/bin/vendor_perl/spamd -x -u spamd -g spamd --listen=/run/spamd/spamd.sock --listen=localhost --max-children=1 -... -</code></pre> -<p>Finally, start and enable the <code>spamassassin</code> service:</p> -<pre><code class="language-sh">systemctl start spamassassin.service -systemctl enable spamassassin.service -</code></pre> -<h2 id="wrapping-up">Wrapping up<a class="headerlink" href="#wrapping-up" title="Permanent link">¶</a></h2> -<p>We should have a working mail server by now. Before continuing check your journal logs (<code>journalctl -xe --unit={unit}</code>, where <code>{unit}</code> could be <code>spamassassin.service</code> for example) to see if there was any error whatsoever and try to debug it, it should be a typo somewhere because all the settings and steps detailed here just worked; I literally just finished doing everything on a new server as of the writing of this text, <mark>it just werks on my machine</mark>.</p> -<p>Now, to actually use the mail service: first of all, you need a <em>normal</em> account (don’t use root) that belongs to the <code>mail</code> group (<code>gpasswd -a user group</code> to add a user <code>user</code> to group <code>group</code>) and that has a password.</p> -<p>Next, to actually login into a mail app/program, you will use the following settings, at least for <code>thunderdbird</code>(I tested in windows default mail app and you don’t need a lot of settings):</p> -<ul> -<li>* server: subdomain.domain (mail.luevano.xyz in my case)</li> -<li><strong>SMTP</strong> port: 587</li> -<li><strong>SMTPS</strong> port: 465 (I use this one)</li> -<li><strong>IMAP</strong> port: 143</li> -<li><strong>IMAPS</strong> port: 993 (again, I use this one)</li> -<li>Connection/security: SSL/TLS</li> -<li>Authentication method: Normal password</li> -<li>Username: just your <code>user</code>, not the whole email (<code>david</code> in my case)</li> -<li>Password: your <code>user</code> password (as in the password you use to login to the server with that user)</li> -</ul> -<p>All that’s left to do is test your mail server for spoofing, and to see if everything is setup correctly. Go to <a href="https://www.appmaildev.com/en/dkim">DKIM Test</a> and follow the instructions (basically click next, and send an email with whatever content to the email that they provide). After you send the email, you should see something like:</p> -<figure id="__yafg-figure-10"> -<img alt="DKIM Test successful" src="https://static.luevano.xyz/images/b/mail/dkim_test_successful.png" title="DKIM Test successful"> -<figcaption>DKIM Test successful</figcaption> -</figure>]]></content:encoded> - </item> - <item> - <title>Set up a website with Nginx and Certbot</title> - <link>https://blog.luevano.xyz/a/website_with_nginx.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/website_with_nginx.html</guid> - <pubDate>Fri, 19 Mar 2021 02:58:15 GMT</pubDate> - <category>Code</category> - <category>English</category> - <category>Server</category> - <category>Tools</category> - <category>Tutorial</category> - <description>How to set up a website using Nginx for web server and Certbot for SSL certificates, on Arch. This is a base for future blog posts about similar topics.</description> - <content:encoded><![CDATA[<p>These are general notes on how to setup a Nginx web server plus Certbot for SSL certificates, initially learned from <a href="https://www.youtube.com/watch?v=OWAqilIVNgE">Luke’s video</a> and after some use and research I added more stuff to the mix. And, actually at the time of writing this entry, I’m configuring the web server again on a new VPS instance, so this is going to be fresh.</p> -<p>As a side note, <mark>i use arch btw</mark> so everything here es aimed at an Arch Linux distro, and I’m doing everything on a VPS. Also note that most if not all commands here are executed with root privileges.</p> -<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2> -<div class="toc"> -<ul> -<li><a href="#table-of-contents">Table of contents</a></li> -<li><a href="#prerequisites">Prerequisites</a></li> -<li><a href="#nginx">Nginx</a></li> -<li><a href="#certbot">Certbot</a></li> -</ul> -</div> -<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2> -<p>You will need two things:</p> -<ul> -<li>A domain name (duh!). I got mine on <a href="https://www.epik.com/?affid=da5ne9ru4">Epik</a> (affiliate link, btw).<ul> -<li>With the corresponding <strong>A</strong> and <strong>AAA</strong> records pointing to the VPS’ IPs. I have three records for each type: empty string, “www” and “*” for a wildcard, that way “domain.name”, “www.domain.name”, “anythingelse.domain.name” point to the same VPS (meaning that you can have several VPS for different sub-domains). These depend on the VPS provider.</li> -</ul> -</li> -<li>A VPS or somewhere else to host it. I’m using <a href="https://www.vultr.com/?ref=8732849">Vultr</a> (also an affiliate link, btw).<ul> -<li>With <code>ssh</code> already configured both on the local machine and on the remote machine.</li> -<li>Firewall already configured to allow ports <code>80</code> (HTTP) and <code>443</code> (HTTPS). I use <code>ufw</code> so it’s just a matter of doing <code>ufw allow 80,443/tcp</code> (for example) as root and you’re golden.</li> -<li><code>cron</code> installed if you follow along (you could use <code>systemd</code> timers, or some other method you prefer to automate running commands every certain time).</li> -</ul> -</li> -</ul> -<h2 id="nginx">Nginx<a class="headerlink" href="#nginx" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Nginx">Nginx</a> is a web (HTTP) server and reverse proxy server.</p> -<p>You have two options: <code>nginx</code> and <code>nginx-mainline</code>. I prefer <code>nginx-mainline</code> because it’s the “up to date” package even though <code>nginx</code> is labeled to be the “stable” version. Install the package and enable/start the service:</p> -<pre><code class="language-sh">pacman -S nginx-mainline -systemctl enable nginx.service -systemctl start nginx.service -</code></pre> -<p>And that’s it, at this point you can already look at the default initial page of Nginx if you enter the IP of your server in a web browser. You should see something like this:</p> -<figure id="__yafg-figure-3"> -<img alt="Nginx welcome page" src="https://static.luevano.xyz/images/b/nginx/nginx_welcome_page.png" title="Nginx welcome page"> -<figcaption>Nginx welcome page</figcaption> -</figure> -<p>As stated in the welcome page, configuration is needed, head to the directory of Nginx:</p> -<pre><code class="language-sh">cd /etc/nginx -</code></pre> -<p>Here you have several files, the important one is <code>nginx.conf</code>, which as its name implies, contains general configuration of the web server. If you peek into the file, you will see that it contains around 120 lines, most of which are commented out and contains the welcome page server block. While you can configure a website in this file, it’s common practice to do it on a separate file (so you can scale really easily if needed for mor websites or sub-domains).</p> -<p>Inside the <code>nginx.conf</code> file, delete the <code>server</code> blocks and add the lines <code>include sites-enabled/*;</code> (to look into individual server configuration files) and <code>types_hash_max_size 4096;</code> (to get rid of an ugly warning that will keep appearing) somewhere inside the <code>http</code> block. The final <code>nginx.conf</code> file would look something like (ignoring the comments just for clarity, but you can keep them as side notes):</p> -<pre><code class="language-nginx">worker_processes 1; - -events { - worker_connections 1024; -} - -http { - include sites-enabled/*; - include mime.types; - default_type application/octet-stream; - - sendfile on; - - keepalive_timeout 65; - - types_hash_max_size 4096; -} -</code></pre> -<p>Next, inside the directory <code>/etc/nginx/</code> create the <code>sites-available</code> and <code>sites-enabled</code> directories, and go into the <code>sites-available</code> one:</p> -<pre><code class="language-sh">mkdir sites-available -mkdir sites-enabled -cd sites-available -</code></pre> -<p>Here, create a new <code>.conf</code> file for your website and add the following lines (this is just the sample content more or less):</p> -<pre><code class="language-nginx">server { - listen 80; - listen [::]:80; - - root /path/to/root/directory; - server_name domain.name another.domain.name; - index index.html anotherindex.otherextension; - - location /{ - try_files $uri $uri/ =404; - } -} -</code></pre> -<p>That could serve as a template if you intend to add more domains.</p> -<p>Note some things:</p> -<ul> -<li><code>listen</code>: we’re telling Nginx which port to listen to (IPv4 and IPv6, respectively).</li> -<li><code>root</code>: the root directory of where the website files (<code>.html</code>, <code>.css</code>, <code>.js</code>, etc. files) are located. I followed Luke’s directory path <code>/var/www/some_folder</code>.</li> -<li><code>server_name</code>: the actual domain to “listen” to (for my website it is: <code>server_name luevano.xyz www.luevano.xyz;</code> and for this blog is: <code>server_name blog.luevano.xyz www.blog.luevano.xyz;</code>).</li> -<li><code>index</code>: what file to serve as the index (could be any <code>.html</code>, <code>.htm</code>, <code>.php</code>, etc. file) when just entering the website.</li> -<li><code>location</code>: what goes after <code>domain.name</code>, used in case of different configurations depending on the URL paths (deny access on <code>/private</code>, make a proxy on <code>/proxy</code>, etc).<ul> -<li><code>try_files</code>: tells what files to look for.</li> -</ul> -</li> -</ul> -<p>Then, make a symbolic link from this configuration file to the <code>sites-enabled</code> directory:</p> -<pre><code class="language-sh">ln -s /etc/nginx/sites-available/your_config_file.conf /etc/nginx/sites-enabled -</code></pre> -<p>This is so the <code>nginx.conf</code> file can look up the newly created server configuration. With this method of having each server configuration file separate you can easily “deactivate” any website by just deleting the symbolic link in <code>sites-enabled</code> and you’re good, or just add new configuration files and keep everything nice and tidy.</p> -<p>All you have to do now is restart (or enable and start if you haven’t already) the Nginx service (and optionally test the configuration):</p> -<pre><code class="language-sh">nginx -t -systemctl restart nginx -</code></pre> -<p>If everything goes correctly, you can now go to your website by typing <code>domain.name</code> on a web browser. But you will see a “404 Not Found” page like the following (maybe with different Nginx version):</p> -<figure id="__yafg-figure-4"> -<img alt="Nginx 404 Not Found page" src="https://static.luevano.xyz/images/b/nginx/nginx_404_page.png" title="Nginx 404 Not Found page"> -<figcaption>Nginx 404 Not Found page</figcaption> -</figure> -<p>That’s no problem, because it means that the web server it’s actually working. Just add an <code>index.html</code> file with something simple to see it in action (in the <code>/var/www/some_folder</code> that you decided upon). If you keep seeing the 404 page make sure your <code>root</code> line is correct and that the directory/index file exists.</p> -<p>I like to remove the <code>.html</code> and trailing <code>/</code> on the URLs of my website, for that you need to add the following <code>rewrite</code> lines and modify the <code>try_files</code> line (for more: <a href="https://www.seancdavis.com/blog/remove-html-extension-and-trailing-slash-in-nginx-config/">Sean C. Davis: Remove HTML Extension And Trailing Slash In Nginx Config</a>):</p> -<pre><code class="language-nginx">server { - ... - rewrite ^(/.*)\.html(\?.*)?$ $1$2 permanent; - rewrite ^/(.*)/$ /$1 permanent; - ... - try_files $uri/index.html $uri.html $uri/ $uri =404; - ... -</code></pre> -<h2 id="certbot">Certbot<a class="headerlink" href="#certbot" title="Permanent link">¶</a></h2> -<p><a href="https://wiki.archlinux.org/title/Certbot">Certbot</a> is what provides the SSL certificates via <a href="https://letsencrypt.org/">Let’s Encrypt</a>.</p> -<p>The only “bad” (bloated) thing about Certbot, is that it uses <code>python</code>, but for me it doesn’t matter too much. You may want to look up another alternative if you prefer. Install the packages <code>certbot</code> and <code>certbot-nginx</code>:</p> -<pre><code class="language-sh">pacman -S certbot certbot-nginx -</code></pre> -<p>After that, all you have to do now is run <code>certbot</code> and follow the instructions given by the tool:</p> -<pre><code class="language-sh">certbot --nginx -</code></pre> -<p>It will ask you for some information, for you to accept some agreements and the names to activate HTTPS for. Also, you will want to “say yes” to the redirection from HTTP to HTTPS. And that’s it, you can now go to your website and see that you have HTTPS active.</p> -<p>Now, the certificate given by <code>certbot</code> expires every 3 months or something like that, so you want to renew this certificate every once in a while. I did this before using <code>cron</code> or manually creating a <code>systemd</code> timer and service, but now it’s just a matter of enabling the <code>certbot-renew.timer</code>:</p> -<pre><code class="language-sh">systemctl start certbot-renew.timer -</code></pre> -<p>The <code>deploy-hook</code> is not needed anymore, only for plugins. For more, visit the <a href="https://wiki.archlinux.org/title/Certbot#Automatic_renewal">Arch Linux Wiki</a>.</p>]]></content:encoded> - </item> - <item> - <title>Así es raza, el blog ya tiene timestamps</title> - <link>https://blog.luevano.xyz/a/el_blog_ya_tiene_timestamps.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/el_blog_ya_tiene_timestamps.html</guid> - <pubDate>Tue, 16 Mar 2021 02:46:24 GMT</pubDate> - <category>Short</category> - <category>Spanish</category> - <category>Tools</category> - <category>Update</category> - <description>Actualización en el estado del blog y el sistema usado para crearlo.</description> - <content:encoded><![CDATA[<p>Pues eso, esta entrada es sólo para tirar update sobre mi <a href="https://blog.luevano.xyz/a/first_blog_post.html">primer post</a>. Ya modifiqué el <code>ssg</code> lo suficiente como para que maneje los <em>timestamps</em>, y ya estoy más familiarizado con este script entonces ya lo podré extender más, pero por ahora las entradas ya tienen su fecha de creación (y modificación en dado caso) al final y en el índice ya están organizados por fecha, que por ahora está algo simple pero está sencillo de extender.</p> -<p>Ya lo único que queda es cambiar un poco el formato del blog (y de la página en general), porque en un momento de desesperación puse todo el texto en justificado y pues no se ve chido siempre, entonces queda corregir eso. <em>Y aunque me tomó más tiempo del que quisiera, así nomás quedó, diría un cierto personaje.</em></p> -<p><del>El <code>ssg</code> modificado está en mis <a href="https://git.luevano.xyz/.dots">dotfiles</a> (o directamente <a href="https://git.luevano.xyz/.dots/tree/.local/bin/ssg">aquí</a>).</del> -<ins>Como al final ya no usé el <code>ssg</code> modificado, este pex ya no existe.</ins></p> -<p>Por último, también quité las extensiones <code>.html</code> de las URLs, porque se ve bien pitero, pero igual los links con <code>.html</code> al final redirigen a su link sin <code>.html</code>, así que no hay rollo alguno.</p> -<p><strong>Actualización</strong>: Ahora estoy usando mi propia solución en vez de <code>ssg</code>, que la llamé <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, de la cual empiezo a hablar <a href="https://blog.luevano.xyz/a/new_blogging_system.html">acá</a>.</p>]]></content:encoded> - </item> - <item> - <title>This is the first blog post, just for testing purposes</title> - <link>https://blog.luevano.xyz/a/first_blog_post.html</link> - <guid isPermaLink="true">https://blog.luevano.xyz/a/first_blog_post.html</guid> - <pubDate>Sat, 27 Feb 2021 13:08:33 GMT</pubDate> - <category>English</category> - <category>Short</category> - <category>Tools</category> - <category>Update</category> - <description>Just my first blog post where I state what tools I'm using to build this blog.</description> - <content:encoded><![CDATA[<p>I’m making this post just to figure out how <a href="https://www.romanzolotarev.com/ssg.html"><code>ssg5</code></a> and <a href="https://kristaps.bsd.lv/lowdown/"><code>lowdown</code></a> are supposed to work, and eventually <a href="https://www.romanzolotarev.com/rssg.html"><code>rssg</code></a>.</p> -<p>At the moment I’m not satisfied because there’s no automatic date insertion into the 1) html file, 2) the blog post itself and 3) the listing system in the <a href="https://blog.luevano.xyz/">blog homepage</a> which also has a problem with the ordering of the entries. And all of this just because I didn’t want to use Luke’s <a href="https://github.com/LukeSmithxyz/lb">lb</a> solution as I don’t really like that much how he handles the scripts (<em>but they just work</em>).</p> -<p>Hopefully, for tomorrow all of this will be sorted out and I’ll have a working blog system.</p> -<p><strong>Update</strong>: I’m now using my own solution which I called <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, of which I talk about <a href="https://blog.luevano.xyz/a/new_blogging_system.html">here</a>.</p>]]></content:encoded> - </item> - </channel> -</rss>
\ No newline at end of file |