1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
3547
3548
3549
3550
3551
3552
3553
3554
3555
3556
3557
3558
3559
3560
3561
3562
3563
3564
3565
3566
3567
3568
3569
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
3591
3592
3593
3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
3615
3616
3617
3618
3619
3620
3621
3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
3716
3717
3718
3719
3720
3721
3722
3723
3724
3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736
3737
3738
3739
3740
3741
3742
3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758
3759
3760
3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789
3790
3791
3792
3793
3794
3795
3796
3797
3798
3799
3800
3801
3802
3803
3804
3805
3806
3807
3808
3809
3810
3811
3812
3813
3814
3815
3816
3817
3818
3819
3820
3821
3822
3823
3824
3825
3826
3827
3828
3829
3830
3831
3832
3833
3834
3835
3836
3837
3838
3839
3840
3841
3842
3843
3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
3859
3860
3861
3862
3863
3864
3865
3866
3867
3868
3869
3870
3871
3872
3873
3874
3875
3876
3877
3878
3879
3880
3881
3882
3883
3884
3885
3886
3887
3888
3889
3890
3891
3892
3893
3894
3895
3896
3897
3898
3899
3900
3901
3902
3903
3904
3905
3906
3907
3908
3909
3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
3920
3921
3922
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959
3960
3961
3962
3963
3964
3965
3966
3967
3968
3969
3970
3971
3972
3973
3974
3975
3976
3977
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992
3993
3994
3995
3996
3997
3998
3999
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
4027
4028
4029
4030
4031
4032
4033
4034
4035
4036
4037
4038
4039
4040
4041
4042
4043
4044
4045
4046
4047
4048
4049
4050
4051
4052
4053
4054
4055
4056
4057
4058
4059
4060
4061
4062
4063
4064
4065
4066
4067
4068
4069
4070
4071
4072
4073
4074
4075
4076
4077
4078
4079
4080
4081
4082
4083
4084
4085
4086
4087
4088
4089
4090
4091
4092
4093
4094
4095
4096
4097
4098
4099
4100
4101
4102
4103
4104
4105
4106
4107
4108
4109
4110
4111
4112
4113
4114
4115
4116
4117
4118
4119
4120
4121
4122
4123
4124
4125
4126
4127
4128
4129
4130
4131
4132
4133
4134
4135
4136
4137
4138
4139
4140
4141
4142
4143
4144
4145
4146
4147
4148
4149
4150
4151
4152
4153
4154
4155
4156
4157
4158
4159
4160
4161
4162
4163
4164
4165
4166
4167
4168
4169
4170
4171
4172
4173
4174
4175
4176
4177
4178
4179
4180
4181
4182
4183
4184
4185
4186
4187
4188
4189
4190
4191
4192
4193
4194
4195
4196
4197
4198
4199
4200
4201
4202
4203
4204
4205
4206
4207
4208
4209
4210
4211
4212
4213
4214
4215
4216
4217
4218
4219
4220
4221
4222
4223
4224
4225
4226
4227
4228
4229
4230
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
4246
4247
4248
4249
4250
4251
4252
4253
4254
4255
4256
4257
4258
4259
4260
4261
4262
4263
4264
4265
4266
4267
4268
4269
4270
4271
4272
4273
4274
4275
4276
4277
4278
4279
4280
4281
4282
4283
4284
4285
4286
4287
4288
4289
4290
4291
4292
4293
4294
4295
4296
4297
4298
4299
4300
4301
4302
4303
4304
4305
4306
4307
4308
4309
4310
4311
4312
4313
4314
4315
4316
4317
4318
4319
4320
4321
4322
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
4336
4337
4338
4339
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
4367
4368
4369
4370
4371
4372
4373
4374
4375
4376
4377
4378
4379
4380
4381
4382
4383
4384
4385
4386
4387
4388
4389
4390
4391
4392
4393
4394
4395
4396
4397
4398
4399
4400
4401
4402
4403
4404
4405
4406
4407
4408
4409
4410
4411
4412
4413
4414
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
4447
4448
4449
4450
4451
4452
4453
4454
4455
4456
4457
4458
4459
4460
4461
4462
4463
4464
4465
4466
4467
4468
4469
4470
4471
4472
4473
4474
4475
4476
4477
4478
4479
4480
4481
4482
4483
4484
4485
4486
4487
4488
4489
4490
4491
4492
4493
4494
4495
4496
4497
4498
4499
4500
4501
4502
4503
4504
4505
4506
4507
4508
4509
4510
4511
4512
4513
4514
4515
4516
4517
4518
4519
4520
4521
4522
4523
4524
4525
4526
4527
4528
4529
4530
4531
4532
4533
4534
4535
4536
4537
4538
4539
4540
4541
4542
4543
4544
4545
4546
4547
4548
4549
4550
4551
4552
4553
4554
4555
4556
4557
4558
4559
4560
4561
4562
4563
4564
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
4580
4581
4582
4583
4584
4585
4586
4587
4588
4589
4590
4591
4592
4593
4594
4595
4596
4597
4598
4599
4600
4601
4602
4603
4604
4605
4606
4607
4608
4609
4610
4611
4612
4613
4614
4615
4616
4617
4618
4619
4620
4621
4622
4623
4624
4625
4626
4627
4628
4629
4630
4631
4632
4633
4634
4635
4636
4637
4638
4639
4640
4641
4642
4643
4644
4645
4646
4647
4648
4649
4650
4651
4652
4653
4654
4655
4656
4657
4658
4659
4660
4661
4662
4663
4664
4665
4666
4667
4668
4669
4670
4671
4672
4673
4674
4675
4676
4677
4678
4679
4680
4681
4682
4683
4684
4685
4686
4687
4688
4689
4690
4691
4692
4693
4694
4695
|
<?xml version="1.0" encoding="UTF-8" ?>
<rss version="2.0"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<title>Luévano's Blog</title>
<link>https://blog.luevano.xyz</link>
<atom:link href="https://blog.luevano.xyz/rss.xml" rel="self" type="application/rss+xml"/>
<description>My personal blog where I post about my thoughts, some how-to's, or general ranting.</description>
<language>en-us</language>
<category></category>
<copyright>Copyright 2023 David Luévano Alvarado</copyright>
<managingEditor>david@luevano.xyz (David Luévano Alvarado)</managingEditor>
<webMaster>david@luevano.xyz (David Luévano Alvarado)</webMaster>
<pubDate></pubDate>
<lastBuildDate></lastBuildDate>
<generator>pyssg v0.9.0</generator>
<docs>https://validator.w3.org/feed/docs/rss2.html</docs>
<ttl>30</ttl>
<image>
<url>https://static.luevano.xyz/images/b/default.png</url>
<title>Luévano's Blog</title>
<link>https://blog.luevano.xyz</link>
</image>
<item>
<title>Set up a pastebin alternative with PrivateBin and YOURLS</title>
<link>https://blog.luevano.xyz/a/pastebin_alt_with_privatebin.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/pastebin_alt_with_privatebin.html</guid>
<pubDate>Sun, 20 Aug 2023 09:46:33 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a pastebin alternative with PrivateBin and YOURLS as shortener, on Arch.</description>
<content:encoded><![CDATA[<p>I learned about PrivateBin a few weeks back and ever since I’ve been looking into installing it, along with a URL shortener (a service I wanted to self host since forever). It took me a while as I ran into some problems while experimenting and documenting all the necessary bits in here.</p>
<p>My setup is exposed to the public, and as always is heavily based on previous entries as described in <a href="#prerequisites">Prerequisites</a>. Descriptions on setting up MariaDB (preferred MySQL replacement for Arch) and PHP are written in this entry as this is the first time I’ve needed them.</p>
<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#mariadb">MariaDB</a><ul>
<li><a href="#create-usersdatabases">Create users/databases</a></li>
</ul>
</li>
<li><a href="#php">PHP</a><ul>
<li><a href="#configuration">Configuration</a></li>
<li><a href="#nginx">Nginx</a></li>
</ul>
</li>
<li><a href="#yourls">YOURLS</a><ul>
<li><a href="#configuration_1">Configuration</a></li>
<li><a href="#nginx_1">Nginx</a><ul>
<li><a href="#ssl-certificate">SSL certificate</a></li>
</ul>
</li>
<li><a href="#usage">Usage</a></li>
</ul>
</li>
<li><a href="#privatebin">PrivateBin</a><ul>
<li><a href="#configuration_2">Configuration</a><ul>
<li><a href="#yourls-integration">YOURLS integration</a></li>
</ul>
</li>
<li><a href="#nginx_2">Nginx</a><ul>
<li><a href="#ssl-certificate_1">SSL certificate</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>If you want to expose to a (sub)domain, then similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> for the reverse proxy plus certificates):</p>
<ul>
<li><code>nginx</code> for the reverse proxy.</li>
<li><code>certbot</code> for the SSL certificates.</li>
<li><code>yay</code> to install AUR packages.<ul>
<li>I briefly mention how to install and use it on <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html#yay">Manga server with Komga: yay</a>.</li>
</ul>
</li>
<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>privatebin</code> and <code>yourls</code> (or whatever you want to call them).</li>
</ul>
<h2 id="mariadb">MariaDB<a class="headerlink" href="#mariadb" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/MariaDB">MariaDB</a> is a drop-in replacement of <a href="https://wiki.archlinux.org/title/MySQL">MySQL</a>.</p>
<p>Install the <code>mariadb</code> package:</p>
<pre><code class="language-sh">pacman -S mariadb
</code></pre>
<p>Before starting/enabling the systemd service run:</p>
<pre><code class="language-sh">mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
</code></pre>
<p><code>start</code>/<code>enable</code> the <code>mariadb.service</code>:</p>
<pre><code class="language-sh">systemctl start mariadb.service
systemctl enable mariadb.service
</code></pre>
<p>Run and follow the secure installation script before proceding any further:</p>
<pre><code class="language-sh">mariadb-secure-installation
</code></pre>
<p>Change the binding address so the service listens on <code>localhost</code> only by modifying <code>/etc/my.cnf.d/server.cnf</code>:</p>
<pre><code class="language-ini">[mariadb]
bind-address = localhost
</code></pre>
<h3 id="create-usersdatabases">Create users/databases<a class="headerlink" href="#create-usersdatabases" title="Permanent link">¶</a></h3>
<p>To use <code>mariadb</code> simply run the command and it will try to login with the corresponding linux user running it. The general login command is:</p>
<pre><code class="language-sh">mariadb -u <username> -p <database_name>
</code></pre>
<p>The <code>database_name</code> is optional. It will prompt a password input field.</p>
<p>Using <code>mariadb</code> as root, create users with their respective database if needed with the following queries:</p>
<pre><code class="language-sql">MariaDB> CREATE USER '<username>'@'localhost' IDENTIFIED BY '<password>';
MariaDB> CREATE DATABASE <database_name>;
MariaDB> GRANT ALL PRIVILEGES ON <database_name>.* TO '<username>'@'localhost';
MariaDB> quit
</code></pre>
<p>The <code>database_name</code> will depend on how YOURLS and PrivateBin are configured, that is if the services use a separate database and/or table prefixes are used.</p>
<h2 id="php">PHP<a class="headerlink" href="#php" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/PHP">PHP</a> is a general-purpose scripting language that is usually used for web development, which was supposed to be ass for a long time but it seems to be a misconseption from the <em>old times</em>.</p>
<p>Install the <code>php</code>, <code>php-fpm</code>, <code>php-gd</code> packages:</p>
<pre><code class="language-sh">pacman -S php php-fpm php-gd
</code></pre>
<p><code>start</code>/<code>enable</code> the <code>php-fpm.service</code>:</p>
<pre><code class="language-sh">systemctl start php-fpm.service
systemctl enable php-fpm.service
</code></pre>
<h3 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h3>
<p>Only showing changes needed, main config file is located at <code>/etc/php/php.ini</code>, or drop-in files can be placed at <code>/etc/php/conf.d/</code> instead.</p>
<p>Set timezone (<a href="https://www.php.net/manual/en/timezones.php">list of timezones</a>):</p>
<pre><code class="language-ini">date.timezone = Europe/Berlin
</code></pre>
<p>Enable the <code>gd</code> and <code>mysql</code> extensions:</p>
<pre><code class="language-ini">extension=gd
extension=pdo_mysql
extension=mysqli
</code></pre>
<h3 id="nginx">Nginx<a class="headerlink" href="#nginx" title="Permanent link">¶</a></h3>
<p>Create a PHP specific config that can be reusable at <code>/etc/nginx/php_fastcgi.conf</code>:</p>
<pre><code class="language-nginx">location ~ \.php$ {
# required for yourls
add_header Access-Control-Allow-Origin $http_origin;
# 404
try_files $fastcgi_script_name =404;
# default fastcgi_params
include fastcgi_params;
# fastcgi settings
fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
fastcgi_index index.php;
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
# fastcgi params
fastcgi_param DOCUMENT_ROOT $realpath_root;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
#fastcgi_param PHP_ADMIN_VALUE "open_basedir=$base/:/usr/lib/php/:/tmp/";
}
</code></pre>
<p>This then can be imported by any <code>server</code> directive that needs it.</p>
<h2 id="yourls">YOURLS<a class="headerlink" href="#yourls" title="Permanent link">¶</a></h2>
<p><a href="https://yourls.org/">YOURLS</a> is a self-hosted URL shortener that is supported by PrivateBin.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S yourls
</code></pre>
<p>Create a new user and database as described in <a href="#create-usersdatabases">MariaDB: Create users/databases</a>.</p>
<h3 id="configuration_1">Configuration<a class="headerlink" href="#configuration_1" title="Permanent link">¶</a></h3>
<p>The default configuration file is self explanatory, it is located at <code>/etc/webapps/yourls/config.php</code>. Make sure to correctly set the user/database YOURLS will use and either create a cookie or get one from <a href="http://yourls.org/cookie">URL provided</a>.</p>
<p>It is important to change the <code>$yours_user_passwords</code> variable, YOURLS will hash the passwords on login so it is not stored in plaintext. Password hashing can be disabled with:</p>
<pre><code class="language-php">define( 'YOURLS_NO_HASH_PASSWORD', true );
</code></pre>
<p>I also changed the “shortening method” to <code>62</code> to include more characters:</p>
<pre><code class="language-php">define( 'YOURLS_URL_CONVERT', 62 );
</code></pre>
<p>The <code>$yourls_reserved_URL</code> variable will need more blacklisted words depending on the use-case. Make sure the <code>YOURLS_PRIVATE</code> variable is set to <code>true</code> (default) if the service will be exposed to the public.</p>
<h3 id="nginx_1">Nginx<a class="headerlink" href="#nginx_1" title="Permanent link">¶</a></h3>
<p>Create a <code>yourls.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p>
<pre><code class="language-nginx">server {
listen 80;
root /usr/share/webapps/yourls/;
server_name short.example.com;
index index.php;
location / {
try_files $uri $uri/ /yourls-loader.php$is_args$args;
}
include /etc/nginx/php_fastcgi.conf;
}
</code></pre>
<p>Make sure the following header is included in the <code>php</code>‘s <code>nginx</code> location block described in <a href="#nginx">YOURLS: Nginx</a>:</p>
<pre><code class="language-nginx">add_header Access-Control-Allow-Origin $http_origin;
</code></pre>
<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4>
<p>Create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>Restart the <code>nginx</code> service for changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="usage">Usage<a class="headerlink" href="#usage" title="Permanent link">¶</a></h3>
<p>The admin area is located at <code>https://short.example.com/admin/</code>, login with any of the configured users set with the <code>$yours_user_passwords</code> in the config. Activate plugins by going to the “Manage Plugins” page (located at the top left) and clicking in the respective “Activate” button by hovering the “Actin” column, as shown below:</p>
<figure id="__yafg-figure-1">
<img alt="YOURLS: Activate plugin" src="https://static.luevano.xyz/images/b/yourls/yourls_activate_plugin.png" title="YOURLS: Activate plugin">
<figcaption>YOURLS: Activate plugin</figcaption>
</figure>
<p>I personally activated the “Random ShortURLs” and “Allow Hyphens in Short URLs”. Once the “Random ShortURLs” plugin is activated it can be configured by going to the “Random ShortURLs Settings” page (located at the top left, right below “Manage Plugins”), only config available is “Random Keyword Length”.</p>
<p>The main admin area can be used to manually shorten any link provided, by using the automatic shortening or by providing a custom short URL.</p>
<p>Finally, the “Tools” page (located at the top left) conains the <code>signature</code> token, used for <a href="https://yourls.org/docs/guide/advanced/passwordless-api">YOURLS: Passwordless API</a> as well as useful bookmarklets for URL shortening while browsing.</p>
<h2 id="privatebin">PrivateBin<a class="headerlink" href="#privatebin" title="Permanent link">¶</a></h2>
<p><a href="https://privatebin.info/">PrivateBin</a> is a minimalist self-hosted alternative to <a href="https://pastebin.com/">pastebin</a>.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S privatebin
</code></pre>
<p>Create a new user and database as described in <a href="#create-usersdatabases">MariaDB: Create users/databases</a>.</p>
<h3 id="configuration_2">Configuration<a class="headerlink" href="#configuration_2" title="Permanent link">¶</a></h3>
<p>This heavily depends on personal preference, all defaults are fine. Make a copy of the sample config template:</p>
<pre><code class="language-sh">cp /etc/webapps/privatebin/conf.sample.php /etc/webapps/privatebin/conf.php
</code></pre>
<p>The most important changes needed are <code>basepath</code> according to the <code>privatebin</code> URL and the <code>[model]</code> and <code>[model_options]</code> to use MySQL instead of plain filesystem files:</p>
<pre><code class="language-php">[model]
; example of DB configuration for MySQL
class = Database
[model_options]
dsn = "mysql:host=localhost;dbname=privatebin;charset=UTF8"
tbl = "privatebin_" ; table prefix
usr = "privatebin"
pwd = "<password>"
opt[12] = true ; PDO::ATTR_PERSISTENT
</code></pre>
<p>Any other <code>[model]</code> or <code>[model_options]</code> needs to be commented out (for example, the default filesystem setting).</p>
<h4 id="yourls-integration">YOURLS integration<a class="headerlink" href="#yourls-integration" title="Permanent link">¶</a></h4>
<p>I recommend creating a separate user for <code>privatebin</code> in <code>yourls</code> by modifying the <code>$yours_user_passwords</code> variable in <code>yourls</code> config file. Then login with this user and get the <code>signature</code> from the “Tools” section in the admin page, for more: <a href="https://yourls.org/docs/guide/advanced/passwordless-api">YOURLS: Passwordless API</a>.</p>
<p>For a “private” <code>yourls</code> installation (that needs username/pasword), set <code>urlshortener</code>:</p>
<pre><code class="language-php">urlshortener = "https://short.example.com/yourls-api.php?signature=xxxxxxxxxx&action=shorturl&format=json&url="
</code></pre>
<p><mark>Note that this will expose the <code>signature</code> in the HTTP requests and anybody with the signature can use it to shorten external URLs.</mark></p>
<h3 id="nginx_2">Nginx<a class="headerlink" href="#nginx_2" title="Permanent link">¶</a></h3>
<p>To deny access to some bots/crawlers, PrivateBin provides a sample <code>.htaccess</code>, which is used in Apache. We need an Nginx version, which I found <a href="https://gist.github.com/benediktg/948a70136e2104c8601da7d355061323">here</a>.</p>
<p>Add the following at the beginning of the <code>http</code> block of the <code>/etc/nginx/nginx.conf</code> file:</p>
<pre><code class="language-nginx">http {
map $http_user_agent $pastebin_badagent {
~*bot 1;
~*spider 1;
~*crawl 1;
~https?:// 1;
WhatsApp 1;
SkypeUriPreview 1;
facebookexternalhit 1;
}
#...
}
</code></pre>
<p>Create a <code>privatebin.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p>
<pre><code class="language-nginx">server {
listen 80;
root //usr/share/webapps/privatebin/;
server_name bin.example.com;
index index.php;
if ($pastebin_badagent) {
return 403;
}
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
include /etc/nginx/php_fastcgi.conf;
}
</code></pre>
<h4 id="ssl-certificate_1">SSL certificate<a class="headerlink" href="#ssl-certificate_1" title="Permanent link">¶</a></h4>
<p>Create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>Restart the <code>nginx</code> service for changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>]]></content:encoded>
</item>
<item>
<title>Set up a media server with Jellyfin, Sonarr and Radarr</title>
<link>https://blog.luevano.xyz/a/jellyfin_server_with_sonarr_radarr.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/jellyfin_server_with_sonarr_radarr.html</guid>
<pubDate>Mon, 24 Jul 2023 04:30:14 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a media server with Jellyfin, Sonarr and Radarr, on Arch. With Bazarr, too.</description>
<content:encoded><![CDATA[<p>Second part of my self hosted media server. This is a direct continuation of <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html">Set up qBitTorrent with Jackett for use with Starr apps</a>, which will be mentioned as “first part” going forward. Sonarr, Radarr, Bazarr (Starr apps) and Jellyfin setups will be described in this part. Same introduction applies to this entry, regarding the use of documentation and configuration.</p>
<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p>
<p><mark>Kindly note that I do not condone the use of BitTorrent for illegal activities. I take no responsibility for what you do when setting up anything shown here. It is for you to check your local laws before using automated downloaders such as Sonarr and Radarr.</mark></p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#radarr">Radarr</a><ul>
<li><a href="#reverse-proxy">Reverse proxy</a></li>
<li><a href="#start-using-radarr">Start using Radarr</a><ul>
<li><a href="#configuration">Configuration</a><ul>
<li><a href="#media-management">Media Management</a></li>
<li><a href="#quality">Quality</a></li>
<li><a href="#custom-formats">Custom Formats</a></li>
<li><a href="#profiles">Profiles</a></li>
<li><a href="#download-clients">Download clients</a></li>
<li><a href="#indexers">Indexers</a></li>
</ul>
</li>
<li><a href="#download-content">Download content</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#sonarr">Sonarr</a><ul>
<li><a href="#reverse-proxy_1">Reverse proxy</a></li>
<li><a href="#start-using-sonarr">Start using Sonarr</a><ul>
<li><a href="#configuration_1">Configuration</a><ul>
<li><a href="#media-management_1">Media Management</a></li>
<li><a href="#quality_1">Quality</a></li>
<li><a href="#profiles_1">Profiles</a></li>
<li><a href="#download-clients_1">Download clients</a></li>
<li><a href="#indexers_1">Indexers</a></li>
</ul>
</li>
<li><a href="#download-content_1">Download content</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#jellyfin">Jellyfin</a><ul>
<li><a href="#reverse-proxy_2">Reverse proxy</a><ul>
<li><a href="#ssl-certificate">SSL certificate</a></li>
</ul>
</li>
<li><a href="#start-using-jellyfin">Start using Jellyfin</a><ul>
<li><a href="#plugins">Plugins</a></li>
<li><a href="#transcoding">Transcoding</a><ul>
<li><a href="#nvidia-drivers">NVIDIA drivers</a></li>
<li><a href="#enable-hardware-acceleration">Enable hardware acceleration</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
<li><a href="#bazarr">Bazarr</a><ul>
<li><a href="#reverse-proxy_3">Reverse proxy</a></li>
<li><a href="#start-using-bazarr">Start using Bazarr</a><ul>
<li><a href="#configuration_2">Configuration</a><ul>
<li><a href="#providers">Providers</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Same prerequisites as with the <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#prerequisites">First part: Prerequisites</a> plus:</p>
<ul>
<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>jellyfin</code>. Only if you want to expose Jellyfin to a subdomain.</li>
</ul>
<p>The <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">First part: Directory structure</a> is the same here. The <code>servarr</code> user and group should be available, too.</p>
<p><mark>It is assumed that the first part was followed.</mark></p>
<h2 id="radarr">Radarr<a class="headerlink" href="#radarr" title="Permanent link">¶</a></h2>
<p><a href="https://radarr.video/">Radarr</a> is a movie collection manager that can be used to download movies via torrents. This is actually a fork of Sonarr, so they’re pretty similar, I just wanted to set up movies first.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S radarr
</code></pre>
<p><mark>Add the <code>radarr</code> user to the <code>servarr</code> group:</mark></p>
<pre><code class="language-sh">gpasswd -a radarr servarr
</code></pre>
<p>The default port that Radarr uses is <code>7878</code> for http (the one you need for the reverse proxy).</p>
<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3>
<p>Add the following <code>location</code> blocks into the <code>isos.conf</code> with whatever subdirectory name you want, I’ll leave it as <code>radarr</code>:</p>
<pre><code class="language-nginx">location /radarr/ {
proxy_pass http://127.0.0.1:7878/radarr/; # change port if needed
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_redirect off;
}
# Allow the API External Access via NGINX
location /radarr/api {
auth_basic off;
proxy_pass http://127.0.0.1:7878/radarr/api; # change port if needed
}
</code></pre>
<p>This is taken from <a href="https://wiki.servarr.com/radarr/installation#nginx">Radarr Nginx reverse proxy configuration</a>. Restart the <code>nginx</code> service for the changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-radarr">Start using Radarr<a class="headerlink" href="#start-using-radarr" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>radarr.service</code>:</p>
<pre><code class="language-sh">systemctl enable radarr.service
systemctl start radarr.service
</code></pre>
<p>This will start the service and create the default configs under <code>/var/lib/radarr</code>. You need to change the <code>URLBase</code> as the reverse proxy is under a subdirectory (<code>/radarr</code>). Edit <code>/var/lib/radarr/config.xml</code>:</p>
<pre><code class="language-xml">...
<UrlBase>/radarr</UrlBase>
...
</code></pre>
<p>Then restart the <code>radarr</code> service:</p>
<pre><code class="language-sh">systemctl restart radarr.service
</code></pre>
<p>Now <code>https://isos.yourdomain.com/radarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials.</p>
<p>Note that if you want to have an anime movies library, it is recommended to run a second instance of Radarr for this as shown in <a href="https://wiki.servarr.com/radarr/installation#linux-multiple-instances">Radarr: Linux multiple instances</a> and follow <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles-anime/">TRaSH: How to setup quality profiles anime</a> if an anime instance is what you want.</p>
<h4 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h4>
<p>Will be following the official <a href="https://wiki.servarr.com/radarr/quick-start-guide">Radarr: Quick start guide</a> as well as the recommendations by <a href="https://trash-guides.info/Radarr/">TRaSH: Radarr</a>.</p>
<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p>
<h5 id="media-management">Media Management<a class="headerlink" href="#media-management" title="Permanent link">¶</a></h5>
<ul>
<li><strong>File Management</strong>:<ul>
<li><em>Propers and Repacks</em>: set it to “Do Not Prefer” and instead you’ll use the <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#repackproper">Repack/Proper</a> <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats">custom format by TRaSH</a>.</li>
</ul>
</li>
</ul>
<h5 id="quality">Quality<a class="headerlink" href="#quality" title="Permanent link">¶</a></h5>
<p>This is personal preference and it dictates your preferred file sizes. You can follow <a href="https://trash-guides.info/Radarr/Radarr-Quality-Settings-File-Size/">TRaSH: Quality settings</a> to maximize the quality of the downloaded content and restrict low quality stuff.</p>
<p>Personally, I think TRaSH’s quality settings are a bit elitist and first world-y. I’m fine with whatever and the tracker I’m using has the quality I want anyways. I did, however, set it to a minimum of <code>0</code> and maximum of <code>400</code> for the qualities shown in TRaSH’s guide. Configuring anything below <code>720p</code> shouldn’t be necessary anyways.</p>
<h5 id="custom-formats">Custom Formats<a class="headerlink" href="#custom-formats" title="Permanent link">¶</a></h5>
<p>Again, this is also completely a personal preference selection and depends on the quality and filters you want. My custom format selections are mostly based on <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles/#hd-bluray-webA">TRaSH: HD Bluray + WEB quality profile</a>.</p>
<p>The only <em>Unwanted</em> format that I’m not going to use is the Low Quality (<a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#lq">LQ</a>) as it blocks one of the sources I’m using to download a bunch of movies. The reasoning behind the LQ custom format is that these release groups don’t care much about quality (they keep low file sizes) and name tagging, which I understand but I’m fine with this as I can upgrade movies individually whenever I want (I want a big catalog of content that I can quickly watch).</p>
<h5 id="profiles">Profiles<a class="headerlink" href="#profiles" title="Permanent link">¶</a></h5>
<p>As mentioned in <a href="#custom-formats">Custom Formats</a> and <a href="#quality">Quality</a> this is completly a personal preference. I’m going to go for “Low Quality” downloads by still following some of the conventions from TRaSH. I’m using the <a href="https://trash-guides.info/Radarr/radarr-setup-quality-profiles/#hd-bluray-webA">TRaSH: HD Bluray + WEB quality profile</a> with the exclusion of the <a href="https://trash-guides.info/Radarr/Radarr-collection-of-custom-formats/#lq">LQ</a> profile.</p>
<p>I set the name to “HD Bluray + WEB”. I’m also not upgrading the torrents for now. Language set to “Original”.</p>
<h5 id="download-clients">Download clients<a class="headerlink" href="#download-clients" title="Permanent link">¶</a></h5>
<p>Pretty straight forward, just click on the giant “+” button and click on the qBitTorrent option. Then configure:</p>
<ul>
<li>Name: can be anything, just an identifier.</li>
<li>Enable: enable it.</li>
<li>Host: use <code>127.0.0.1</code>. For some reason I can’t make it work with the reverse proxied qBitTorrent.</li>
<li>Port: the port number you chose, <code>30000</code> in my case.</li>
<li>Url Base: leave blank as even though you have it exposed under <code>/qbt</code>, the service itself is not.</li>
<li>Username: the Web UI username, <code>admin</code> by default.</li>
<li>Password: the Web UI username, <code>adminadmin</code> by default (you should’ve changed it if you have the service exposed).</li>
<li>Category: <code>movies</code>.</li>
</ul>
<p>Everything else can be left as default, but maybe change <em>Completed Download Handling</em> if you’d like. Same goes for the general <em>Failed Download Handling</em> download clients’ option.</p>
<h5 id="indexers">Indexers<a class="headerlink" href="#indexers" title="Permanent link">¶</a></h5>
<p>Also easy to set up, also just click on the giant “+” button and click on the <em>custom</em> Torznab option (you can also use the <em>preset -> Jackett</em> Torznab option). Then configure:</p>
<ul>
<li>Name: can be anything, just an identifier. I like to do “Jackett - INDEXER”, where “INDEXER” is just an identifier.</li>
<li>URL: <code>http://127.0.0.1:9117/jack/api/v2.0/indexers/YOURINDEXER/results/torznab/</code>, where <code>YOURINDEXER</code> is specific to each indexer (<code>yts</code>, <code>nyaasi</code>, etc.). Can be directly copied from the indexer’s “Copy Torznab Feed” button on the Jackett Web UI.</li>
<li>API Path: <code>/api</code>, leave as is.</li>
<li>API Key: this can be found at the top right corner in Jackett’s Web UI.</li>
<li>Categories: which categories to use when searching, these are generic categories until you test/add the indexer. After you add the indexer you can come back and select your prefered categories (like just toggling the movies categories).</li>
<li>Tags: I like to add a tag for the indexer name like <code>yts</code> or <code>nyaa</code>. This is useful to control which indexers to use when adding new movies.</li>
</ul>
<p>Everything else on default. <em>Download Client</em> can also be set, which can be useful to keep different categories per indexer or something similar. <em>Seed Ratio</em> and <em>Seed Time</em> can also be set and are used to manage when to stop the torrent, this can also be set globally on the qBitTorrent Web UI, this is a personal setting.</p>
<h4 id="download-content">Download content<a class="headerlink" href="#download-content" title="Permanent link">¶</a></h4>
<p>You can now start to download content by going to <em>Movies -> Add New</em>. Basically just follow the <a href="https://wiki.servarr.com/radarr/quick-start-guide#how-to-add-a-movie">Radarr: How to add a movie</a> guide. The screenshots from the guide are a bit outdated but it contains everything you need to know.</p>
<p>I personally use:</p>
<ul>
<li>Monitor: Movie Only.</li>
<li>Minimum Availability: Released.</li>
<li>Quiality Profile: “HD Bluray + WEB”, the one configured in this entry.</li>
<li>Tags: the indexer name I want to use to download the movie, usually just <code>yts</code> for me (remember this is a “LQ” release group, so if you have that custom format disable it) as mentioned in <a href="#indexers">Indexers</a>. If you don’t specify a tag it will only use indexers that don’t have a tag set.</li>
<li>Start search for missing movie: toggled on. Immediatly start searching for the movie and start the download.</li>
</ul>
<p>Once you click on “Add Movie” it will add it to the <em>Movies</em> section and start searching and selecting the best torrent it finds, according to the “filters” (quality settings, profile and indexer(s)).</p>
<p>When it selects a torrent it sends it to qBitTorrent and you can even go ahead and monitor it over there. Else you can also monitor at <em>Activity -> Queue</em>.</p>
<p>After the movie is downloaded and processed by Radarr, it will create the appropriate hardlinks to the <code>media/movies</code> directory, as set in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">First part: Directory structure</a>.</p>
<p>Optionally, you can add subtitles using <a href="#bazarr">Bazarr</a>.</p>
<h2 id="sonarr">Sonarr<a class="headerlink" href="#sonarr" title="Permanent link">¶</a></h2>
<p><a href="https://sonarr.tv/">Sonarr</a> is a TV series collection manager that can be used to download series via torrents. Most of the install process, configuration and whatnot is going to be basically the same as with Radarr.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S sonarr
</code></pre>
<p><mark>Add the <code>sonarr</code> user to the <code>servarr</code> group:</mark></p>
<pre><code class="language-sh">gpasswd -a sonarr servarr
</code></pre>
<p>The default port that Radarr uses is <code>8989</code> for http (the one you need for the reverse proxy).</p>
<h3 id="reverse-proxy_1">Reverse proxy<a class="headerlink" href="#reverse-proxy_1" title="Permanent link">¶</a></h3>
<p>Basically the same as with <a href="#reverse-proxy">Radarr: Reverse proxy</a>, <mark>except that the <code>proxy_set_header</code> changes from <code>$proxy_host</code> to <code>$host</code>.</mark></p>
<p>Add the following <code>location</code> blocks into the <code>isos.conf</code>, I’ll leave it as <code>sonarr</code>:</p>
<pre><code class="language-nginx">location /sonarr/ {
proxy_pass http://127.0.0.1:8989/sonarr/; # change port if needed
proxy_http_version 1.1;
proxy_set_header Host $proxy_host; # this differs from the radarr reverse proxy
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_redirect off;
}
# Allow the API External Access via NGINX
location /sonarr/api {
auth_basic off;
proxy_pass http://127.0.0.1:8989/sonarr/api; # change port if needed
}
</code></pre>
<p>This is taken from <a href="https://wiki.servarr.com/sonarr/installation#nginx">Sonarr: Nginx reverse proxy configuration</a>. Restart the <code>nginx</code> service for the changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-sonarr">Start using Sonarr<a class="headerlink" href="#start-using-sonarr" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>sonarr.service</code>:</p>
<pre><code class="language-sh">systemctl enable sonarr.service
systemctl start sonarr.service
</code></pre>
<p>This will start the service and create the default configs under <code>/var/lib/sonarr</code>. You need to change the <code>URLBase</code> as the reverse proxy is under a subdirectory (<code>/sonarr</code>). Edit <code>/var/lib/sonarr/config.xml</code>:</p>
<pre><code class="language-xml">...
<UrlBase>/sonarr</UrlBase>
...
</code></pre>
<p>Then restart the <code>sonarr</code> service:</p>
<pre><code class="language-sh">systemctl restart sonarr.service
</code></pre>
<p>Now <code>https://isos.yourdomain.com/sonarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials.</p>
<p>Similar to <a href="#radarr">Radarr</a> if you want to have an anime library, it is recommended to run a second instance of Sonarr for this as shown in <a href="https://wiki.servarr.com/sonarr/installation#linux-multiple-instances">Sonarr: Linux multiple instances</a> and follow <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx-Anime/">TRaSH: Release profile regex (anime)</a> and the <a href="https://trash-guides.info/Sonarr/Sonarr-recommended-naming-scheme/#anime-episode-format">TRaSH: Anime recommended naming scheme</a> if an anime instance is what you want.</p>
<h4 id="configuration_1">Configuration<a class="headerlink" href="#configuration_1" title="Permanent link">¶</a></h4>
<p>Will be following the official <a href="https://wiki.servarr.com/sonarr/quick-start-guide">Sonarr: Quick start guide</a> as well as the recommendations by <a href="https://trash-guides.info/Sonarr/">TRaSH: Sonarr</a>.</p>
<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p>
<h5 id="media-management_1">Media Management<a class="headerlink" href="#media-management_1" title="Permanent link">¶</a></h5>
<ul>
<li><strong>File Management</strong>:<ul>
<li><em>Propers and Repacks</em>: set it to “Do Not Prefer” and instead you’ll use the <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#propers-and-repacks">Propers and Repacks</a> release profile and fill with <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#p2p-groups-repackproper">P2P Groups + Repack/Proper</a>.</li>
</ul>
</li>
</ul>
<h5 id="quality_1">Quality<a class="headerlink" href="#quality_1" title="Permanent link">¶</a></h5>
<p>Similar to <a href="#quality">Radarr: Quality</a> this is personal preference and it dictates your preferred file sizes. You can follow <a href="https://trash-guides.info/Sonarr/Sonarr-Quality-Settings-File-Size/">TRaSH: Quality settings</a> to maximize the quality of the downloaded content and restrict low quality stuff.</p>
<p>Will basically do the same as in <a href="#quality">Radarr: Quality</a>: set minimum of <code>0</code> and maximum of <code>400</code> for everything <code>720p</code> and above.</p>
<h5 id="profiles_1">Profiles<a class="headerlink" href="#profiles_1" title="Permanent link">¶</a></h5>
<p>This is a bit different than with <a href="#radarr">Radarr</a>, the way it is configured is by setting “Release profiles”. I took the profiles from <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/">TRaSH: WEB-DL Release profile regex</a>. The only possible change I’ll do is disable the Low Quality Groups and/or the “Golden rule” filter (for <code>x265</code> encoded video).</p>
<p>For me it ended up looking like this:</p>
<figure id="__yafg-figure-5">
<img alt="Sonarr: Release profiles" src="https://static.luevano.xyz/images/b/sonarr/sonarr_release_profiles.png" title="Sonarr: Release profiles">
<figcaption>Sonarr: Release profiles</figcaption>
</figure>
<p>But yours can differ as its mostly personal preference. For the “Quality profile” I’ll be using the default “HD-1080p” most of the time, but I also created a “HD + WEB (720/1080)” which works best for some.</p>
<h5 id="download-clients_1">Download clients<a class="headerlink" href="#download-clients_1" title="Permanent link">¶</a></h5>
<p>Exactly the same as with <a href="#download-clients">Radarr: Download clients</a> only change is the category from <code>movies</code> to <code>tv</code> (or whatever you want), click on the giant “+” button and click on the qBitTorrent option. Then configure:</p>
<ul>
<li>Name: can be anything, just an identifier.</li>
<li>Enable: enable it.</li>
<li>Host: use <code>127.0.0.1</code>.</li>
<li>Port: the port number you chose, <code>30000</code> in my case.</li>
<li>Url Base: leave blank as even though you have it exposed under <code>/qbt</code>, the service itself is not.</li>
<li>Username: the Web UI username, <code>admin</code> by default.</li>
<li>Password: the Web UI username, <code>adminadmin</code> by default (you should’ve changed it if you have the service exposed).</li>
<li>Category: <code>tv</code>.</li>
</ul>
<p>Everything else can be left as default, but maybe change <em>Completed Download Handling</em> if you’d like. Same goes for the general <em>Failed Download Handling</em> download clients’ option.</p>
<h5 id="indexers_1">Indexers<a class="headerlink" href="#indexers_1" title="Permanent link">¶</a></h5>
<p>Also exactly the same as with <a href="#indexers">Radarr: Indexers</a>, click on the giant “+” button and click on the <em>custom</em> Torznab option (this doesn’t have the Jackett preset). Then configure:</p>
<ul>
<li>Name: can be anything, just an identifier. I like to do “Jackett - INDEXER”, where “INDEXER” is just an identifier.</li>
<li>URL: <code>http://127.0.0.1:9117/jack/api/v2.0/indexers/YOURINDEXER/results/torznab/</code>, where <code>YOURINDEXER</code> is specific to each indexer (<code>eztv</code>, <code>nyaasi</code>, etc.). Can be directly copied from the indexer’s “Copy Torznab Feed” button on the Jackett Web UI.</li>
<li>API Path: <code>/api</code>, leave as is.</li>
<li>API Key: this can be found at the top right corner in Jackett’s Web UI.</li>
<li>Categories: which categories to use when searching, these are generic categories until you test/add the indexer. After you add the indexer you can come back and select your prefered categories (like just toggling the TV categories).</li>
<li>Tags: I like to add a tag for the indexer name like <code>eztv</code> or <code>nyaa</code>. This is useful to control which indexers to use when adding new series.</li>
</ul>
<p>Everything else on default. <em>Download Client</em> can also be set, which can be useful to keep different categories per indexer or something similar. <em>Seed Ratio</em> and <em>Seed Time</em> can also be set and are used to manage when to stop the torrent, this can also be set globally on the qBitTorrent Web UI, this is a personal setting.</p>
<h4 id="download-content_1">Download content<a class="headerlink" href="#download-content_1" title="Permanent link">¶</a></h4>
<p>Almost the same as with <a href="#download-content">Radarr: Download content</a>, but I’ve been personally selecting the torrents I want to download for each season/episode so far, as the indexers I’m using are all over the place and I like consistencies. Will update if I find a (near) 100% automation process, but I’m fine with this anyways as I always monitor that everything is going fine.</p>
<p>Add by going to <em>Series -> Add New</em>. Basically just follow the <a href="https://wiki.servarr.com/sonarr/library#add-new">Sonarr: Library add new</a> guide. Adding series needs a bit more options that movies in Radarr, but it’s straight forward.</p>
<p>I personally use:</p>
<ul>
<li>Monitor: All Episodes.</li>
<li>Quiality Profile: “HD + WEB (720/1080)”. This depends on what I want for that how, lately I’ve been experimenting with this one.</li>
<li>Series Type: Standard. For now I’m just downloading shows, but it has an Anime option.</li>
<li>Tags: the “indexer_name” I want to use to download the movie, I’ve been using all indexers so I just use all tags as I’m experimenting and trying multiple options.</li>
<li>Season Folder: enabled. I like as much organization as possible.</li>
<li>Start search for missing episodes: disabled. Depends on you, due to my indexers, I prefer to check manually the season packs, for example.</li>
<li>Start search for cutoff unmet episodes: disabled. Honestly don’t really know what this is.</li>
</ul>
<p>Once you click on “Add X” it will add it to the <em>Series</em> section and will start as monitored. So far I haven’t noticed that it immediately starts downloading (because of the “Start search for missing episodes” setting) but I always click on unmonitor the series, so I can manually check (again, due to the low quality of my indexers).</p>
<p>When it automatically starts to download an episode/season it will send it to qBitTorrent and you can monitor it over there. Else you can also monitor at <em>Activity -> Queue</em>. Same thing goes if you download manually each episode/season via the interactive search.</p>
<p>To interactively search episodes/seasons go to <em>Series</em> and then click on any series, then click either on the interactive search button for the episode or the season, it is an icon of a person as shown below:</p>
<figure id="__yafg-figure-6">
<img alt="Sonarr: Interactive search button" src="https://static.luevano.xyz/images/b/sonarr/sonarr_interactive_search_button.png" title="Sonarr: Interactive search button">
<figcaption>Sonarr: Interactive search button</figcaption>
</figure>
<p>Then it will bring a window with the search results, where it shows the indexer it got the result from, the size of the torrent, peers, language, quality, the score it received from the configured release profiles an alert in case that the torrent is “bad” and the download button to manually download the torrent you want. An example shown below:</p>
<figure id="__yafg-figure-7">
<img alt="Sonarr: Interactive search results" src="https://static.luevano.xyz/images/b/sonarr/sonarr_interactive_search_results.png" title="Sonarr: Interactive search results">
<figcaption>Sonarr: Interactive search results</figcaption>
</figure>
<p>After the movie is downloaded and processed by Sonarr, it will create the appropriate hardlinks to the <code>media/tv</code> directory, as set in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">Directory structure</a>.</p>
<p>Optionally, you can add subtitles using <a href="#bazarr">Bazarr</a>.</p>
<h2 id="jellyfin">Jellyfin<a class="headerlink" href="#jellyfin" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Jellyfin">Jellyfin</a> is a media server “manager”, usually used to manage and organize video content (movies, TV series, etc.) which could be compared with <a href="https://wiki.archlinux.org/title/plex">Plex</a> or <a href="https://wiki.archlinux.org/title/Emby">Emby</a> for example (take them as possible alternatives).</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S jellyfin-bin
</code></pre>
<p>I’m installing the pre-built binary instead of building it as I was getting a lot of errors and the server was even crashing. You can try installing <code>jellyfin</code> instead.</p>
<p><mark>Add the <code>jellyfin</code> user to the <code>servarr</code> group:</mark></p>
<pre><code class="language-sh">gpasswd -a jellyfin servarr
</code></pre>
<p>You can already <code>start</code>/<code>enable</code> the <code>jellyfin.service</code> which will start at <code>http://127.0.0.1:8096/</code> by default where you need to complete the initial set up. But let’s create the reverse proxy first then start everything and finish the set up.</p>
<h3 id="reverse-proxy_2">Reverse proxy<a class="headerlink" href="#reverse-proxy_2" title="Permanent link">¶</a></h3>
<p>I’m going to have my <code>jellyfin</code> instance under a subdomain with an <code>nginx</code> reverse proxy as shown in the <a href="https://wiki.archlinux.org/title/Jellyfin#Nginx_reverse_proxy">Arch wiki</a>. For that, create a <code>jellyfin.conf</code> at the usual <code>sites-<available/enabled></code> path for <code>nginx</code>:</p>
<pre><code class="language-nginx">server {
listen 80;
server_name jellyfin.yourdomain.com; # change accordingly to your wanted subdomain and domain name
set $jellyfin 127.0.0.1; # jellyfin is running at localhost (127.0.0.1)
# Security / XSS Mitigation Headers
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";
# Content Security Policy
# See: https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
# Enforces https content and restricts JS/CSS to origin
# External Javascript (such as cast_sender.js for Chromecast) must be whitelisted.
add_header Content-Security-Policy "default-src https: data: blob: http://image.tmdb.org; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' https://www.gstatic.com/cv/js/sender/v1/cast_sender.js https://www.youtube.com blob:; worker-src 'self' blob:; connect-src 'self'; object-src 'none'; frame-ancestors 'self'";
location = / {
return 302 https://$host/web/;
}
location / {
# Proxy main Jellyfin traffic
proxy_pass http://$jellyfin:8096;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
# Disable buffering when the nginx proxy gets very resource heavy upon streaming
proxy_buffering off;
}
# location block for /web - This is purely for aesthetics so /web/#!/ works instead of having to go to /web/index.html/#!/
location = /web/ {
# Proxy main Jellyfin traffic
proxy_pass http://$jellyfin:8096/web/index.html;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
}
location /socket {
# Proxy Jellyfin Websockets traffic
proxy_pass http://$jellyfin:8096;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
}
}
</code></pre>
<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4>
<p>Create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>Similarly to the <code>isos</code> subdomain, that will autodetect the new subdomain and extend the existing certificate(s). Restart the <code>nginx</code> service for changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-jellyfin">Start using Jellyfin<a class="headerlink" href="#start-using-jellyfin" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>jellyfin.service</code> if you haven’t already:</p>
<pre><code class="language-sh">systemctl enable jellyfin.service
systemctl start jellyfin.service
</code></pre>
<p>Then navigate to <code>https://jellyfin.yourdomain.com</code> and either continue with the set up wizard if you didn’t already or continue with the next steps to configure your libraries.</p>
<p>The initial setup wizard makes you create an user (will be the admin for now) and at least one library, though these can be done later. For more check <a href="https://jellyfin.org/docs/general/quick-start/">Jellyfin: Quick start</a>.</p>
<p>Remember to use the configured directory as mentioned in <a href="https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html#directory-structure">Directory structure</a>. Any other configuration (like adding users or libraries) can be done at the dashboard: click on the 3 horizontal lines on the top left of the Web UI then navigate to <em>Administration -> Dashboard</em>. I didn’t configure much other than adding a couple of users for me and friends, I wouldn’t recommend using the admin account to watch (personal preference).</p>
<p>Once there is at least one library it will show at <em>Home</em> along with the latest movies (if any) similar to the following (don’t judge, these are just the latest I added due to friend’s requests):</p>
<figure id="__yafg-figure-8">
<img alt="Jellyfin: Home libraries" src="https://static.luevano.xyz/images/b/jellyfin/jellyfin_home_libraries.png" title="Jellyfin: Home libraries">
<figcaption>Jellyfin: Home libraries</figcaption>
</figure>
<p>And inside the “Movies” library you can see the whole catalog where you can filter or just scroll as well as seeing <em>Suggestions</em> (I think this starts getting populated after a while) and <em>Genres</em>:</p>
<figure id="__yafg-figure-9">
<img alt="Jellyfin: Library catalog options" src="https://static.luevano.xyz/images/b/jellyfin/jellyfin_library_catalog_options.png" title="Jellyfin: Library catalog options">
<figcaption>Jellyfin: Library catalog options</figcaption>
</figure>
<h4 id="plugins">Plugins<a class="headerlink" href="#plugins" title="Permanent link">¶</a></h4>
<p>You can also install/activate <a href="https://jellyfin.org/docs/general/server/plugins/">plugins</a> to get extra features. Most of the plugins you might want to use are already available in the official repositories and can be found in the “Catalog”. There are a lot of plugins that are focused around anime and TV metadata, as well as an Open Subtitles plugin to automatically download missing subtitles (though this is managed with <a href="#bazarr">Bazarr</a>).</p>
<p>To activate plugins click on the 3 horizontal lines on the top left of the Web UI then navigate to <em>Administration -> Dashboard -> Advanced -> Plugins</em> and click on the <em>Catalog</em> tab (top of the Web UI). Here you can select the plugins you want to install. By default only the official ones are shown, for more you can add more <a href="https://jellyfin.org/docs/general/server/plugins/#repositories">repositories</a>.</p>
<p>The only plugin I’m using is the “Playback Reporting”, to get a summary of what is being used in the instance. But I might experiment with some anime-focused plugins when the time comes.</p>
<h4 id="transcoding">Transcoding<a class="headerlink" href="#transcoding" title="Permanent link">¶</a></h4>
<p>Although not recommended and explicitly set to not download any <code>x265</code>/<code>HEVC</code> content (by using the <a href="https://trash-guides.info/Sonarr/Sonarr-Release-Profile-RegEx/#golden-rule">Golden rule</a>) there might be cases where the only option you have is to download such content. If that is the case and you happen to have a way to do <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/">Hardware Acceleration</a>, for example by having an NVIDIA graphics card (in my case) then you should enable it to avoid using lots of resources on your system.</p>
<p>Using hardware acceleration will leverage your GPU to do the transcoding and save resources on your CPU. I tried streaming <code>x265</code> content and it basically used 70-80% on all cores of my CPU, while on the other hand using my GPU it used the normal amount on the CPU (70-80% on a single core).</p>
<p>This will be the steps to install on an <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/nvidia/">NVIDIA</a> graphics card, specifically a GTX 1660 Ti. But more info and guides can be found at <a href="https://jellyfin.org/docs/general/administration/hardware-acceleration/">Jellyfin: Hardware Acceleration</a> for other acceleration methods.</p>
<h5 id="nvidia-drivers">NVIDIA drivers<a class="headerlink" href="#nvidia-drivers" title="Permanent link">¶</a></h5>
<p>Ensure you have the NVIDIA drivers and utils installed. I’ve you’ve done this in the past then you can skip this part, else you might be using the default <code>nouveau</code> drivers. Follow the next steps to set up the NVIDIA drivers, which basically is a summary of <a href="https://wiki.archlinux.org/title/NVIDIA#Installation">NVIDIA: Installation</a> for my setup, so <mark>double check the wiki in case you have an older NVIDIA graphics card</mark>.</p>
<p>Install the <code>nvidia</code> and <code>nvidia-utils</code> packages:</p>
<pre><code class="language-sh">pacman -S nvidia nvidia-utils
</code></pre>
<p>Modify <code>/etc/mkinitcpio.conf</code> to remove <code>kms</code> from the <code>HOOKS</code> array. It should look like this (commented line is how it was for me before the change):</p>
<pre><code class="language-sh">...
# HOOKS=(base udev autodetect modconf kms keyboard keymap consolefont block filesystems fsck)
HOOKS=(base udev autodetect modconf keyboard keymap consolefont block filesystems fsck)
...
</code></pre>
<p><a href="https://wiki.archlinux.org/title/Mkinitcpio#Image_creation_and_activation">Regenerate the initramfs</a> by executing:</p>
<pre><code class="language-sh">mkinitcpio -P
</code></pre>
<p>Finally, reboot the system. After the reboot you should be able to check your GPU info and processes being run with the GPU by executing <code>nvidia-smi</code>.</p>
<h5 id="enable-hardware-acceleration">Enable hardware acceleration<a class="headerlink" href="#enable-hardware-acceleration" title="Permanent link">¶</a></h5>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S jellyfin-ffmpeg6-bin
</code></pre>
<p>This provides the <code>jellyfin-ffmpeg</code> executable, which is necessary for Jellyfin to do hardware acceleration, it needs to be this specific one.</p>
<p>Then in the Jellyfin go to the transcoding settings by clicking on the 3 horizontal lines on the top left of the Web UI and navigating to <em>Administration -> Dashboard -> Playback -> Transcoding</em> and:</p>
<ul>
<li>Change the <em>Hardware acceleration</em> from “None” to “Nvidia NVENC”.</li>
<li>Some other options will pop up, just make sure you enable “HEVC” (which is <code>x265</code>) in the list of <em>Enable hardware encoding for:</em>.</li>
<li>Scroll down and specify the <code>ffmpeg</code> path, which is <code>/usr/lib/jellyfin-ffmpeg/ffmpeg</code>.</li>
</ul>
<p>Don’t forget to click “Save” at the bottom of the Web UI, it will ask if you want to enable hardware acceleration.</p>
<h2 id="bazarr">Bazarr<a class="headerlink" href="#bazarr" title="Permanent link">¶</a></h2>
<p><a href="https://www.bazarr.media/">Bazarr</a> is a companion for Sonarr and Radarr that manages and downloads subtitles.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S bazarr
</code></pre>
<p><mark>Add the <code>bazarr</code> user to the <code>servarr</code> group:</mark></p>
<pre><code class="language-sh">gpasswd -a bazarr servarr
</code></pre>
<p>The default port that Bazarr uses is <code>6767</code> for http (the one you need for the reverse proxy), and it has pre-configured the default ports for Radarr and Sonarr.</p>
<h3 id="reverse-proxy_3">Reverse proxy<a class="headerlink" href="#reverse-proxy_3" title="Permanent link">¶</a></h3>
<p>Basically the same as with <a href="#reverse-proxy">Radarr: Reverse proxy</a> and <a href="#reverse-proxy-1">Sonarr: Reverse proxy</a>.</p>
<p>Add the following setting in the <code>server</code> block of the <code>isos.conf</code>:</p>
<pre><code class="language-nginx">server {
# server_name and other directives
...
# Increase http2 max sizes
large_client_header_buffers 4 16k;
# some other blocks like location blocks
...
}
</code></pre>
<p>Then add the following <code>location</code> blocks in the <code>isos.conf</code>, where I’ll keep it as <code>/bazarr/</code>:</p>
<pre><code class="language-nginx">location /bazarr/ {
proxy_pass http://127.0.0.1:6767/bazarr/; # change port if needed
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_redirect off;
}
# Allow the Bazarr API through if you enable Auth on the block above
location /bazarr/api {
auth_request off;
proxy_pass http://127.0.0.1:6767/bazarr/api;
}
</code></pre>
<p>This is taken from <a href="https://wiki.bazarr.media/Additional-Configuration/Reverse-Proxy-Help/">Bazarr: Reverse proxy help</a>. Restart the <code>nginx</code> service for the changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-bazarr">Start using Bazarr<a class="headerlink" href="#start-using-bazarr" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>bazarr.service</code> if you haven’t already:</p>
<pre><code class="language-sh">systemctl start bazarr.service
systemctl enable bazarr.service
</code></pre>
<p>This will start the service and create the default configs under <code>/var/lib/bazarr</code>. You need to change the <code>base_url</code> for the necessary services as they’re running under a reverse proxy and under subdirectories. Edit <code>/var/lib/bazarr/config/config.ini</code>:</p>
<pre><code class="language-ini">[general]
port = 6767
base_url = /bazarr
[sonarr]
port = 8989
base_url = /sonarr
[radarr]
port = 7878
base_url = /radarr
</code></pre>
<p>Then restart the <code>bazarr</code> service:</p>
<pre><code class="language-sh">systemctl restart bazarr.service
</code></pre>
<p>Now <code>https://isos.yourdomain.com/bazarr</code> is accessible. <mark>Secure the instance right away</mark> by adding authentication under <em>Settings -> General -> Security</em>. I added the “Forms” option, just fill in the username and password then click on save changes on the top left of the page. You can restart the service again and check that it asks for login credentials. I also disabled <em>Settings -> General -> Updates -> Automatic</em>.</p>
<h4 id="configuration_2">Configuration<a class="headerlink" href="#configuration_2" title="Permanent link">¶</a></h4>
<p>Will be following the official <a href="https://wiki.bazarr.media/Getting-Started/setup-guide/">Bazarr: Setup guide</a> as well as the recommendations by <a href="https://trash-guides.info/Bazarr/">TRaSH: Bazarr</a>.</p>
<p>Anything that is not mentioned in either guide or that is specific to how I’m setting up stuff will be stated below.</p>
<h5 id="providers">Providers<a class="headerlink" href="#providers" title="Permanent link">¶</a></h5>
<p>This doesn’t require much thinking and its up to personal preference, but I’ll list the ones I added:</p>
<ul>
<li><a href="https://www.opensubtitles.com/">OpenSubtitles.com</a>: requires an account (the <code>.org</code> option is deprecated).<ul>
<li>For a free account it only lets you download around 20 subtitles per day, and they contain ads. You could pay for a VIP account ($3 per month) and that will give you 1000 subtitles per day and no ads. But if you’re fine with 20 ads per day you can try to get rid of the ads by running an automated script. Such option can be found at <a href="https://github.com/brianspilner01/media-server-scripts/blob/master/sub-clean.sh">brianspilner01/media-server-scripts: sub-clean.sh</a>.</li>
</ul>
</li>
<li>YIFY Subtitles</li>
<li>Subscenter</li>
<li>Supersubtitles</li>
<li>TVSubtitles</li>
<li>Subtitulamos.tv: Spanish subtitles provider.</li>
<li>Argenteam: LATAM Spanish subtitles provider.</li>
<li>Subdivx: LATAM Spanish / Spanish subtitles provider.</li>
</ul>
<p>I’ve tested this setup for the following languages (with all default settings as stated in the guides):</p>
<ul>
<li>English</li>
<li>Spanish</li>
</ul>
<p>I tried with “Latin American Spanish” but they’re hard to find, those two work pretty good.</p>
<p>None of these require an <a href="https://anti-captcha.com/">Anti-Captcha</a> account (which is a paid service), but I created one anyways in case I need it. Though you need to add credits to it (pretty cheap though) if you ever use it.</p>]]></content:encoded>
</item>
<item>
<title>Set up qBitTorrent with Jackett for use with Starr apps</title>
<link>https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/torrenting_with_qbittorrent.html</guid>
<pubDate>Mon, 24 Jul 2023 02:06:24 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a torrenting solution with qBitTorrent in preparation for a media server with Jellyfin and Starr apps, on Arch. With Jackett and flaresolverr, too.</description>
<content:encoded><![CDATA[<p>Riding on my excitement of having a good internet connection and having setup my <em>home server</em> now it’s time to self host a media server for movies, series and anime. I’ll setup qBitTorrent as the downloader, Jackett for the trackers, the <em>Starr apps</em> for the automatic downloading and Jellyfin as the media server manager/media viewer. This was going to be a single entry but it ended up being a really long one so I’m splitting it, this being the first part.</p>
<p>I’ll be exposing my stuff on a subdomain only so I can access it while out of home and for SSL certificates (not required), but shouldn’t be necessary and instead you can use a VPN (<a href="https://blog.luevano.xyz/a/vpn_server_with_openvpn.html">how to set up</a>). For your reference, whenever I say “Starr apps” (*arr apps) I mean the family of apps such as Sonarr, Radarr, Bazarr, Readarr, Lidarr, etc..</p>
<p>Most of my config is based on the <a href="https://trash-guides.info/">TRaSH-Guides</a> (will be mentioned as “TRaSH” going forward). Specially get familiar with the <a href="https://trash-guides.info/Hardlinks/How-to-setup-for/Native/">TRaSH: Native folder structure</a> and with the <a href="https://trash-guides.info/Hardlinks/Hardlinks-and-Instant-Moves/">TRaSH: Hardlinks and instant moves</a>. Will also use the default configurations based on the respective documentation for each Starr app and service, except when stated otherwise.</p>
<p>Everything here is performed in <mark>arch btw</mark> and all commands should be run as root unless stated otherwise.</p>
<p><mark>Kindly note that I do not condone the use of torrenting for illegal activities. I take no responsibility for what you do when setting up anything shown here. It is for you to check your local laws before using automated downloaders such as Sonarr and Radarr.</mark></p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a><ul>
<li><a href="#directory-structure">Directory structure</a></li>
</ul>
</li>
<li><a href="#jackett">Jackett</a><ul>
<li><a href="#reverse-proxy">Reverse proxy</a><ul>
<li><a href="#ssl-certificate">SSL certificate</a></li>
</ul>
</li>
<li><a href="#start-using-jackett">Start using Jackett</a><ul>
<li><a href="#indexers">Indexers</a></li>
</ul>
</li>
<li><a href="#flaresolverr">FlareSolverr</a></li>
</ul>
</li>
<li><a href="#qbittorrent">qBitTorrent</a><ul>
<li><a href="#reverse-proxy_1">Reverse proxy</a></li>
<li><a href="#start-using-qbittorrent">Start using qBitTorrent</a><ul>
<li><a href="#configuration">Configuration</a></li>
<li><a href="#trackers">Trackers</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>The specific programs are mostly recommendations, if you’re familiar with something else or want to change things around, feel free to do so but everything will be written with them in mind.</p>
<p>If you want to expose to a (sub)domain, then similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> for the reverse proxy plus certificates):</p>
<ul>
<li><code>nginx</code> for the reverse proxy.</li>
<li><code>certbot</code> for the SSL certificates.</li>
<li><code>ufw</code> for the firewall.</li>
<li><code>yay</code> to install AUR packages.<ul>
<li>I mentioned how to install and use it on my previous entry: <a href="https://blog.luevano.xyz/a/manga_server_with_komga.html#yay">Manga server with Komga: yay</a>.</li>
</ul>
</li>
<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>isos</code> (or whatever you want to call it).<ul>
<li>For automation software (qBitTorrent, Jackett, Starr apps, etc.). One subdomain per service can be used instead.</li>
</ul>
</li>
</ul>
<p><mark>Note: I’m using the explicit <code>127.0.0.1</code> ip instead of <code>localhost</code> in the reverse proxies/services config as <code>localhost</code> resolves to <code>ipv6</code> sometimes which is not configured on my server correctly.</mark> If you have it configured you can use <code>localhost</code> without any issue.</p>
<h3 id="directory-structure">Directory structure<a class="headerlink" href="#directory-structure" title="Permanent link">¶</a></h3>
<p>Basically following <a href="https://trash-guides.info/Hardlinks/How-to-setup-for/Native/">TRaSH: Native folder structure</a> except for the directory permissions part, I’ll do the same as with my <a href="https://blog.luevano.xyz/a/manga_server_with_komga#set-default-directory-permissions.html">Komga setup guide</a> to stablish default group permissions.</p>
<p>The desired behaviour is: set <code>servarr</code> as group ownership, set write access to group and whenever a new directory/file is created, inherit these permission settings. <code>servarr</code> is going to be a service user and I’ll use the root of a mounted drive at <code>/mnt/a</code>.</p>
<ol>
<li>Create a service user called <code>servarr</code> (it could just be a group, too):</li>
</ol>
<pre><code class="language-sh">useradd -r -s /usr/bin/nologin -M -c "Servarr applications" servarr
</code></pre>
<ol start="2">
<li>Create the <code>torrents</code> directory and set default permissions:</li>
</ol>
<pre><code class="language-sh">cd /mnt/a # change this according to your setup
mkdir torrents
chown servarr:servarr torrents
chmod g+w torrents
chmod g+s torrents
setfacl -d -m g::rwx torrents
setfacl -d -m o::rx torrents
</code></pre>
<ol start="3">
<li>Check that the permissions are set correctly (<code>getfacl torrents</code>)</li>
</ol>
<pre><code># file: torrents/
# owner: servarr
# group: servarr
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x
</code></pre>
<ol start="4">
<li>Create the subdirectories you want with any user (I’ll be using <code>servarr</code> personally):</li>
</ol>
<pre><code class="language-sh">mkdir torrents/{tv,movies,anime}
chown -R servarr: torrents
</code></pre>
<ol start="5">
<li>Finally repeat steps 2 - 4 for the <code>media</code> directory.</li>
</ol>
<p>The final directory structure should be the following:</p>
<pre><code>root_dir
├── torrents
│ ├── movies
│ ├── music
│ └── tv
└── media
├── movies
├── music
└── tv
</code></pre>
<p>Where <code>root_dir</code> is <code>/mnt/a</code> in my case. This is going to be the reference for the following applications set up.</p>
<p>Later, add the necessary users to the <code>servarr</code> group if they need write access, by executing:</p>
<pre><code class="language-sh">gpasswd -a <USER> servarr
</code></pre>
<h2 id="jackett">Jackett<a class="headerlink" href="#jackett" title="Permanent link">¶</a></h2>
<p><a href="https://github.com/Jackett/Jackett">Jackett</a> is a “proxy server” (or “middle-ware”) that translates queries from apps (such as the Starr apps in this case) into tracker-specific http queries. Note that there is an alternative called <a href="https://github.com/Prowlarr/Prowlarr">Prowlarr</a> that is better integrated with most if not all Starr apps, requiring less maintenance; I’ll still be sticking with Jackett, though.</p>
<p>Install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S jackett
</code></pre>
<p>I’ll be using the default <code>9117</code> port, but change accordingly if you decide on another one.</p>
<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3>
<p>I’m going to have most of the services under the same subdomain, with different subdirectories. Create the config file <code>isos.conf</code> at the usual <code>sites-available/enabled</code> path for <code>nginx</code>:</p>
<pre><code class="language-nginx">server {
listen 80;
server_name isos.yourdomain.com;
location /jack { # you can change this to jackett or anything you'd like, but it has to match the jackett config on the next steps
proxy_pass http://127.0.0.1:9117; # change the port according to what you want
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_redirect off;
}
}
</code></pre>
<p>This is the basic reverse proxy config as shown in <a href="https://github.com/Jackett/Jackett#running-jackett-behind-a-reverse-proxy">Jackett: Running Jackett behind a reverse proxy</a>. The rest of the services will be added under different <code>location</code> block on their respective steps.</p>
<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4>
<p>Create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>That will automatically detect the new subdomain config and create/extend your existing certificate(s). Restart the <code>nginx</code> service for changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-jackett">Start using Jackett<a class="headerlink" href="#start-using-jackett" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>jackett.service</code>:</p>
<pre><code class="language-sh">systemctl enable jackett.service
systemctl start jackett.service
</code></pre>
<p>It will autocreate the default configuration under <code>/var/lib/jackett/ServerConfig.json</code>, which you need to edit at least to change the <code>BasePathOverride</code> to match what you used in the <code>nginx</code> config:</p>
<pre><code class="language-json">{
"Port": 9117,
"SomeOtherConfigs": "some_other_values",
"BasePathOverride": "/jack",
"MoreConfigs": "more_values",
}
</code></pre>
<p>Also modify the <code>Port</code> if you changed it. Restart the <code>jackett</code> service:</p>
<pre><code class="language-sh">systemctl restart jackett.service
</code></pre>
<p>It should now be available at <code>https://isos.yourdomain.com/jack</code>. <mark>Add an admin password right away</mark> by scroll down and until the first config setting; don’t forget to click on “Set Password”. Change any other config you want from the Web UI, too (you’ll need to click on the blue “Apply server settings” button).</p>
<p>Note that you need to set the “Base URL override” to <code>http://127.0.0.1:9117</code> (or whatever port you used) so that the “Copy Torznab Feed” button works for each indexer.</p>
<h4 id="indexers">Indexers<a class="headerlink" href="#indexers" title="Permanent link">¶</a></h4>
<p>For Jackett, an indexer is just a configured tracker for some of the commonly known torrent sites. Jackett comes with a lot of pre-configured public and private indexers which usually have multiple URLs (mirrors) per indexer, useful when the main torrent site is down. Some indexers come with extra features/configuration depending on what the site specializes on.</p>
<p>To add an indexer click on the “+ Add Indexer” at the top of the Web UI and look for indexers you want, then click on the “+” icon on the far-most right for each indexer or select the ones you want (clicking on the checkbox on the far-most left of the indexer) and scroll all the way to the bottom to click on “Add Selected”. They then will show as a list with some available actions such as “Copy RSS Feed”, “Copy Torznab Feed”, “Copy Potato Feed”, a button to search, configure, delete and test the indexer, as shown below:</p>
<figure id="__yafg-figure-2">
<img alt="Jacket: configured indexers" src="https://static.luevano.xyz/images/b/jack/jack_configured_indexers.png" title="Jackett: configured indexers">
<figcaption>Jackett: configured indexers</figcaption>
</figure>
<p>You can manually test the indexers by doing a basic search to see if they return anything, either by searching on each individual indexer by clicking on the magnifying glass icon on the right of the indexer or clicking on “Manual Search” button which is next to the “+ Add Indexer” button at the top right.</p>
<p>Explore each indexer’s configuration in case there is stuff you might want to change.</p>
<h3 id="flaresolverr">FlareSolverr<a class="headerlink" href="#flaresolverr" title="Permanent link">¶</a></h3>
<p><a href="https://github.com/FlareSolverr/FlareSolverr">FlareSolverr</a> is used to bypass <em>certain</em> protection that some torrent sites have. This is not 100% necessary and only needed for some trackers sometimes, it also doesn’t work 100%.</p>
<p>You could install from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S flaresolverr-bin
</code></pre>
<p>At the time of writing, the <code>flaresolverr</code> package didn’t work for me because of <code>python-selenium</code>. <code>flaresolverr-bin</code> was updated around the time I was writing this, so that is what I’m using and what’s working fine so far, it contains almost everything needed (it has self contained libraries) except for <code>xfvb</code>.</p>
<p>Install dependencies via <code>pacman</code>:</p>
<pre><code class="language-sh">pacman -S xorg-server-xvfb
</code></pre>
<p>You can now <code>start</code>/<code>enable</code> the <code>flaresolverr.service</code>:</p>
<pre><code class="language-sh">systemctl enable flaresolverr.service
systemctl start flaresolverr.service
</code></pre>
<p>Verify that the service started correctly by checking the logs:</p>
<pre><code class="language-sh">journalctl -fxeu flaresolverr
</code></pre>
<p>It should show “Test successful” and “Serving on http://0.0.0.0:8191” (which is the default). Jackett now needs to be configured by adding <code>http://127.0.0.1:8191</code> almost at the end in the “FlareSolverr API URL” field, then click on the blue “Apply server settings” button at the beginning of the config section. This doesn’t need to be exposed or anything, it’s just an internal API that Jackett (or anything you want) will use.</p>
<h2 id="qbittorrent">qBitTorrent<a class="headerlink" href="#qbittorrent" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/QBittorrent">qBitTorrent</a> is a fast, stable and light BitTorrent client that comes with many features and in my opinion it’s a really user friendly client and my personal choice for years now. But you can choose whatever client you want, there are more lightweight alternatives such as <a href="https://wiki.archlinux.org/title/transmission">Transmission</a>.</p>
<p>Install the <code>qbittorrent-nox</code> package (“nox” as in “no X server”):</p>
<pre><code class="language-sh">pacman -S qbittorrent-nox
</code></pre>
<p>By default the package doesn’t create any (service) user, but it is recommended to have one so you can run the service under it. Create the user, I’ll call it <code>qbittorrent</code> and leave it with the default homedir (<code>/home</code>):</p>
<pre><code class="language-sh">useradd -r -m qbittorrent
</code></pre>
<p><mark>Add the <code>qbittorrent</code> user to the <code>servarr</code> group:</mark></p>
<pre><code class="language-sh">gpasswd -a qbittorrent servarr
</code></pre>
<p>Decide a port number you’re going to run the service on for the next steps, the default is <code>8080</code> but I’ll use <code>30000</code>; it doesn’t matter much, as long as it matches for all the config. This is the <code>qbittorrent</code> service port, it is used to connect to the instance itself through the Web UI or via API, <mark>you also need to open a port for listening to peer connections.</mark> Choose any port you want, for example <code>50000</code> and open it with your firewall, <code>ufw</code> in my case:</p>
<pre><code class="language-sh">ufw allow 50000/tcp comment "qBitTorrent - Listening port"
</code></pre>
<h3 id="reverse-proxy_1">Reverse proxy<a class="headerlink" href="#reverse-proxy_1" title="Permanent link">¶</a></h3>
<p>Add the following <code>location</code> block into the <code>isos.conf</code> with whatever subdirectory name you want, I’ll call it <code>qbt</code>:</p>
<pre><code class="language-nginx">location /qbt/ {
proxy_pass http://localhost:30000/; # change port to whatever number you want
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_cookie_path / "/; Secure";
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
</code></pre>
<p>This is taken from <a href="https://github.com/qbittorrent/qBittorrent/wiki/NGINX-Reverse-Proxy-for-Web-UI">qBitTorrent: Nginx reverse proxy for Web UI</a>. Restart the <code>nginx</code> service for the changes to take effect:</p>
<pre><code class="language-sh">systemctl restart nginx.service
</code></pre>
<h3 id="start-using-qbittorrent">Start using qBitTorrent<a class="headerlink" href="#start-using-qbittorrent" title="Permanent link">¶</a></h3>
<p>You can now <code>start</code>/<code>enable</code> the <code>qbittorrent-nox@.service</code> using the service account created (<code>qbittorrent</code>):</p>
<pre><code class="language-sh">systemctl enable `qbittorrent-nox@qbittorrent.service
systemctl start `qbittorrent-nox@qbittorrent.service
</code></pre>
<p>This will start <code>qbittorrent</code> using default config. You need to change the port (in my case to <code>30000</code>) and set <code>qbittorrent</code> to restart on exit (the Web UI has a close button). I guess this can be done before enabling/starting the service, but either way let’s create a “drop-in” file by “editing” the service:</p>
<pre><code class="language-sh">systemctl edit `qbittorrent-nox@qbittorrent.service
</code></pre>
<p>Which will bring up a file editing mode containing the service unit and a space where you can add/override anything, add:</p>
<pre><code class="language-ini">[Service]
Environment="QBT_WEBUI_PORT=30000" # or whatever port number you want
Restart=on-success
RestartSec=5s
</code></pre>
<p>When exiting from the file (if you wrote anything) it will create the override config. Restart the service for changes to take effect (you might be asked to reload the systemd daemon):</p>
<pre><code class="language-sh">systemctl restart `qbittorrent-nox@qbittorrent.service
</code></pre>
<p>You can now head to <code>https://isos.yourdomain.com/qbt/</code> and login with user <code>admin</code> and password <code>adminadmin</code> (by default). <mark>Change the default password right away</mark> by going to <em>Tools -> Options -> Web UI -> Authentication</em>. The Web UI is basically the same as the normal desktop UI so if you’ve used it it will feel familiar. From here you can threat it as a normal torrent client and even start using for other stuff other than the specified here.</p>
<h4 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h4>
<p>It should be usable already but you can go further and fine tune it, specially to some kind of “convention” as shown in <a href="https://trash-guides.info/Downloaders/qBittorrent/Basic-Setup/">TRaSH: qBitTorrent basic setup</a> and subsequent <code>qbittorrent</code> guides.</p>
<p>I use all the suggested settings by <em>TRaSH</em>, where the only “changes” are for personal paths, ports, and in general connection settings that depend on my setup. The only super important setting I noticed that can affect our setup (meaning what is described in this entry) is the <em>Web UI -> Authentication</em> for the “Bypass authentication for clients on localhost”. This will be an issue because the reverse proxy is accessing <code>qbittorrent</code> via <code>localhost</code>, so this will make the service open to the world, experiment at your own risk.</p>
<p>Finally, add categories by following <a href="https://trash-guides.info/Downloaders/qBittorrent/How-to-add-categories/">TRaSH: qBitTorrent how to add categories</a>, basically right clicking on <em>Categories -> All (x)</em> (located at the left of the Web UI) and then on “Add category”; I use the same “Category” and “Save Path” (<code>tv</code> and <code>tv</code>, for example), where the “Save Path” will be a subdirectory of the configured global directory for torrents (<a href="https://trash-guides.info/Downloaders/qBittorrent/How-to-add-categories/#paths-and-categories-breakdown">TRaSH: qBitTorent paths and categories breakdown</a>). I added 3: <code>tv</code>, <code>movies</code> and <code>anime</code>.</p>
<h4 id="trackers">Trackers<a class="headerlink" href="#trackers" title="Permanent link">¶</a></h4>
<p>Often some of the trackers that come with torrents are dead or just don’t work. You have the option to add extra trackers to torrents either by:</p>
<ul>
<li>Automatically add a predefined list on new torrents: configure at <em>Tools -> Options -> BitTorrent</em>, enable the last option “Automatically add these trackers to new downloads” then add the list of trackers. This should only be done on public torrents as private ones might ban you or something.</li>
<li>Manually add a list of trackers to individual torrents: configure by selecting a torrent, clicking on <em>Trackers</em> on the bottom of the Web UI, right clicking on an empty space and selecting “Add trackers…” then add the list of trackers.</li>
</ul>
<p>On both options, the list of trackers need to have at least one new line in between each new tracker. You can find trackers from the following sources:</p>
<ul>
<li><a href="https://newtrackon.com/list">List of stable trackers</a></li>
<li><a href="https://github.com/ngosang/trackerslist">ngosang/trackerslist</a><ul>
<li>It also mentions <a href="https://github.com/ngosang/trackerslist#third-party-tools">Third party tools</a> to automate this process.</li>
</ul>
</li>
</ul>
<p>Both sources maintain an updated list of trackers. You also might need to enable an advanced option so all the new trackers are contacted (<a href="https://github.com/qbittorrent/qBittorrent/issues/7882">Only first tracker contacted</a>): configure at <em>Tools -> Options -> Advanced -> libtorrent Section</em> and enable both “Always announce to all tiers” and “Always announce to all trackers in a tier”.</p>]]></content:encoded>
</item>
<item>
<title>Configure system logs on Arch to avoid filled up disk</title>
<link>https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/arch_logs_flooding_disk.html</guid>
<pubDate>Thu, 15 Jun 2023 10:22:20 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Short</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to configure the system logs, mostly journald, from filling up the disk, on Arch.</description>
<content:encoded><![CDATA[<p>It’s been a while since I’ve been running a minimal server on a VPS, and it is a pretty humble VPS with just 32 GB of storage which works for me as I’m only hosting a handful of services. At some point I started noticing that the disk keept filling up on each time I checked.</p>
<p>Turns out that out of the box, Arch has a default config for <code>systemd</code>‘s <code>journald</code> that keeps a persistent <code>journal</code> log, but doesn’t have a limit on how much logging is kept. This means that depending on how many services, and how aggresive they log, it can be filled up pretty quickly. For me I had around 15 GB of logs, from the normal <code>journal</code> directory, <code>nginx</code> directory and my now unused <code>prosody</code> instance.</p>
<p>For <code>prosody</code> it was just a matter of deleting the directory as I’m not using it anymore, which freed around 4 GB of disk space.
For <code>journal</code> I did a combination of configuring <code>SystemMaxUse</code> and creating a <em>Namespace</em> for all “email” related services as mentioned in the <a href="https://wiki.archlinux.org/title/Systemd/Journal#Per_unit_size_limit_by_a_journal_namespace">Arch wiki: systemd/Journal</a>; basically just configuring <code>/etc/systemd/journald.conf</code> (and <code>/etc/systemd/journald@email.conf</code> with the comment change) with:</p>
<pre><code class="language-ini">[Journal]
Storage=persistent
SystemMaxUse=100MB # 50MB for the "email" Namespace
</code></pre>
<p>And then for each service that I want to use this “email” <em>Namespace</em> I add:</p>
<pre><code class="language-ini">[Service]
LogNamespace=email
</code></pre>
<p>Which can be changed manually or by executing <code>systemctl edit service_name.service</code> and it will create an override file which will be read on top of the normal service configuration. Once configured restart by running <code>systemctl daemon-reload</code> and <code>systemctl restart service_name.service</code> (probably also restart <code>systemd-journald</code>).</p>
<p>I also disabled the logging for <code>ufw</code> by running <code>ufw logging off</code> as it logs everything to the <code>kernel</code> “unit”, and I didn’t find a way to pipe its logs to a separate directory. It really isn’t that useful as most of the logs are the normal <code>[UFW BLOCK]</code> log, which is normal. If I need debugging then I’ll just enable that again. Note that you can change the logging level, if you still want some kind of logging.</p>
<p>Finally to clean up the <code>nginx</code> logs, you need to install <code>logrotate</code> (<code>pacman -S logrotate</code>) as that is what is used to clean up the <code>nginx</code> log directory. <code>nginx</code> already “installs” a config file for <code>logrotate</code> which is located at <code>/etc/logrotate.d/</code>, I just added a few lines:</p>
<pre><code>/var/log/nginx/*log {
rotate 7
weekly
dateext
dateformat -%Y-%m-%d
missingok
notifempty
create 640 http log
sharedscripts
compress
postrotate
test ! -r /run/nginx.pid || kill -USR1 `cat /run/nginx.pid`
endscript
}
</code></pre>
<p>Once you’re ok with your config, it’s just a matter of running <code>logrotate -v -f /etc/logrotate.d/nginx</code> which forces the run of the rule for <code>nginx</code>. After this, <code>logrotate</code> will be run daily if you <code>enable</code> the <code>logrotate</code> timer: <code>systemctl enable logrotate.timer</code>.</p>]]></content:encoded>
</item>
<item>
<title>Set up a manga server with Komga and mangal</title>
<link>https://blog.luevano.xyz/a/manga_server_with_komga.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/manga_server_with_komga.html</guid>
<pubDate>Sat, 10 Jun 2023 19:36:07 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a manga server with Komga as media server and mangal for downloading manga, on Arch. Tachiyomi integration is available thanks to Komga.</description>
<content:encoded><![CDATA[<p>I’ve been wanting to set up a manga media server to hoard some mangas/comics and access them via Tachiyomi, but I didn’t have enough space in my vultr VPS. Now that I have symmetric fiber optic at home and my spare PC to use as a server I decided to go ahead and create one. As always, <mark>i use arch btw</mark> so these instructions are specifically for it, I’m not sure how easier/harder it is for other distros, I’m just too comfortable with arch honestly.</p>
<p>I’m going to run it as an exposed service using a subdomain of my own, so the steps are taking that into account, if you want to run it locally (or on a LAN/VPN) then it is going to be easier/with less steps (you’re on your own). Also, as you might notice I don’t like to use D*ck*r images or anything (ew).</p>
<p><mark>At the time of editing this entry (06-28-2023) Komga has already upgraded to <code>v.1.0.0</code> and it introduces some breaking changes if you already had your instance set up. Read more <a href="https://komga.org/installation/upgrade.html#prepare-for-v1-0-0">here</a>.</mark> The only change I did here was changing the port to the new default.</p>
<p>As always, all commands are run as root unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#yay">yay</a><ul>
<li><a href="#install">Install</a></li>
<li><a href="#usage">Usage</a></li>
</ul>
</li>
<li><a href="#mangal">mangal</a><ul>
<li><a href="#install-from-source">Install from source</a></li>
<li><a href="#configuration">Configuration</a></li>
<li><a href="#usage_1">Usage</a><ul>
<li><a href="#headless-browser">Headless browser</a></li>
<li><a href="#tui">TUI</a></li>
<li><a href="#inline">Inline</a></li>
<li><a href="#automation">Automation</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#komga">Komga</a><ul>
<li><a href="#reverse-proxy">Reverse proxy</a><ul>
<li><a href="#ssl-certificate">SSL certificate</a></li>
</ul>
</li>
<li><a href="#start-using-komga">Start using Komga</a></li>
<li><a href="#library-creation">Library creation</a><ul>
<li><a href="#set-default-directory-permissions">Set default directory permissions</a></li>
<li><a href="#populate-manga-library">Populate manga library</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#alternative-downloaders">Alternative downloaders</a></li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Similar to my early <a href="https://blog.luevano.xyz/tag/@tutorial.html">tutorial</a> entries, if you want it as a subdomain:</p>
<ul>
<li>An <strong>A</strong> (and/or <strong>AAAA</strong>) or a <strong>CNAME</strong> for <code>komga</code> (or whatever you want).</li>
<li>An SSL certificate, if you’re following the other entries (specially the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> entry), add a <code>komga.conf</code> and run <code>certbot --nginx</code> (or similar) to extend/create the certificate. More details below: <a href="#reverse-proxy">Reverse proxy</a> and <a href="#ssl-certificate">SSL certificate</a>.</li>
</ul>
<h2 id="yay">yay<a class="headerlink" href="#yay" title="Permanent link">¶</a></h2>
<p>This is the first time I mention the <strong>AUR</strong> (and <code>yay</code>) in my entries, so I might as well just write a bit about it.</p>
<p>The <a href="https://aur.archlinux.org/">AUR</a> is the <strong>A</strong>rch Linux <strong>U</strong>ser <strong>R</strong>epository and it’s basically like an extension of the official one which is supported by the community, the only thing is that it requires a different package manager. The one I use (and I think everyone does, too) is <code>yay</code>, which as far as I know is like a wrapper of <code>pacman</code>.</p>
<h3 id="install">Install<a class="headerlink" href="#install" title="Permanent link">¶</a></h3>
<p>To install and use <code>yay</code> we need a normal account with sudo access, <mark>all the commands related to <code>yay</code> are run as normal user and then it asks for sudo password</mark>. <a href="https://github.com/Jguer/yay#installation">Installation</a> its straight forward: clone <code>yay</code> repo and install. Only dependencies are <code>git</code> and <code>base-devel</code>:</p>
<p>Install dependencies:</p>
<pre><code class="language-sh">sudo pacman -S git base-devel
</code></pre>
<p>Clone <code>yay</code> and install it (I also like to delete the cloned git repo):</p>
<pre><code class="language-sh">git clone git@github.com:Jguer/yay.git
cd yay
makepkg -si
cd ..
sudo rm -r yay
</code></pre>
<h3 id="usage">Usage<a class="headerlink" href="#usage" title="Permanent link">¶</a></h3>
<p><code>yay</code> is used basically the same as <code>pacman</code> with the difference that it is run as normal user (then later requiring sudo password) and that it asks extra input when installing something, such as if we want to build the package from source or if we want to show package diffs.</p>
<p>To install a package (for example Komga in this blog entry), run:</p>
<pre><code class="language-sh">yay -S komga
</code></pre>
<h2 id="mangal">mangal<a class="headerlink" href="#mangal" title="Permanent link">¶</a></h2>
<p><a href="https://github.com/metafates/mangal">mangal</a> is a CLI/TUI manga downloader with anilist integration and custom Lua scrapers.</p>
<p>You could install it from the AUR with <code>yay</code>:</p>
<pre><code class="language-sh">yay -S mangal-bin
</code></pre>
<p>But I’ll use my <a href="https://github.com/luevano/mangal">fork</a> as it contains some fixes and extra stuff.</p>
<h3 id="install-from-source">Install from source<a class="headerlink" href="#install-from-source" title="Permanent link">¶</a></h3>
<p>As I mentioned in my past <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a> I had to <a href="https://github.com/luevano/mangal">fork</a> <code>mangal</code> and related repositories to fix/change a few things. Currently the major fix I did in <code>mangal</code> is for the built in <a href="https://mangadex.org/">MangaDex</a> scraper which had really annoying bug in the chunking of the manga chapter listing.</p>
<p>So instad of installing with <code>yay</code> we’ll build it from source. We need to have <code>go</code> installed:</p>
<pre><code class="language-sh">pacman -S go
</code></pre>
<p>Then clone my fork of <code>mangal</code> and <code>install</code> it:</p>
<pre><code class="language-sh">git clone https://github.com/luevano/mangal.git # not sure if you can use SSH to clone
cd mangal
make install # or just `make build` and then move the binary to somewhere in your $PATH
</code></pre>
<p>This will use <code>go install</code> so it will install to a path specified by the <code>go</code> environment variables, for more run <code>go help install</code>. It was installed to <code>$HOME/.local/bin/go/mangal</code> for me because my env vars, then just make sure this is included in your <code>PATH</code>.</p>
<p>Check it was correctly installed by running <code>mangal version</code>, which should print something like:</p>
<pre><code>▇▇▇ mangal
Version ...
Git Commit ...
Build Date ...
Built By ...
Platform ...
</code></pre>
<h3 id="configuration">Configuration<a class="headerlink" href="#configuration" title="Permanent link">¶</a></h3>
<p>I’m going to do everything with a normal user (<code>manga-dl</code>) which I created just to download manga. So all of the commands will be run without sudo/root privileges.</p>
<p>Change some of the configuration options:</p>
<pre><code class="language-sh">mangal config set -k downloader.path -v "/mnt/d/mangal" # downloads to current dir by default
mangal config set -k formats.use -v "cbz" # downloads as pdf by default
mangal config set -k installer.user -v "luevano" # points to my scrapers repository which contains a few extra scrapers and fixes, defaults to metafates' one; this is important if you're using my fork, don't use otherwise as it uses extra stuff I added
mangal config set -k logs.write -v true # I like to get logs for what happens
</code></pre>
<p><strong>Note</strong>: For testing purposes (if you want to explore <code>mangal</code>) set <code>downloader.path</code> once you’re ready to start to populate the Komga library directory (at <a href="#populate-manga-library">Komga: populate manga library</a>).</p>
<p>For more configs and to read what they’re for:</p>
<pre><code class="language-sh">mangal config info
</code></pre>
<p>Also install the custom Lua scrapers by running:</p>
<pre><code class="language-sh">mangal sources install
</code></pre>
<p>And install whatever you want, it picks up the sources/scrapers from the configured repository (<code>installer.<key></code> config), if you followed, it will show my scrapers.</p>
<h3 id="usage_1">Usage<a class="headerlink" href="#usage_1" title="Permanent link">¶</a></h3>
<p>Two main ways of using <code>mangal</code>: </p>
<ul>
<li><strong>TUI</strong>: for initial browsing/downloading and testing things out. If the manga finished publishing, this should be enough.</li>
<li><strong>inline</strong>: for automation on manga that is still publishing and I need to check/download every once in a while.</li>
</ul>
<h4 id="headless-browser">Headless browser<a class="headerlink" href="#headless-browser" title="Permanent link">¶</a></h4>
<p>Before continuing, I gotta say I went through some bullshit while trying to use the custom Lua scrapers that use the <em>headless</em> browser (actually just a wrapper of <a href="https://github.com/go-rod/rod">go-rod/rod</a>, and honestly it is not really a “headless” browser, <code>mangal</code> “documentation” is just wrong). For more on my rant check out my last <a href="https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html">entry</a>.</p>
<p>There is no concrete documentation on the “headless” browser, only that it is automatically set up and ready to use… but it doesn’t install any library/dependency needed. I discovered the following libraries that were missing on my Arch minimal install:</p>
<ul>
<li>library -> arch package containing it</li>
<li>libnss3.so -> nss</li>
<li>libatk-1.0.so.0 -> at-spi2-core</li>
<li>libcups.so.2 -> libcups</li>
<li>libdrm.so.2 -> libdrm</li>
<li>libXcomposite.so.1 -> libxcomposite</li>
<li>libXdamage.so.1 -> libxdamage</li>
<li>libXrandr.so.2 -> libxrandr</li>
<li>libgbm.so.1 -> mesa</li>
<li>libxkbcommon.so.0 -> libxkbcommon</li>
<li>libpango-1.0.so.0 -> pango</li>
<li>libasound.so.2 -> alsa-lib</li>
</ul>
<p>To install them::</p>
<pre><code class="language-sh">pacman -S nss at-spi2-core libcups libdrm libxcomposite libxdamage libxrandr mesa libxkbcommon pango alsa-lib
</code></pre>
<p>I can’t guarantee that those are all the packages needed, those are the ones I happen to discover (had to <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the lua libs and add some logging because the error message was too fucking generic).</p>
<p>These dependencies are probably met by installing either <code>chromedriver</code> or <code>google-chrome</code> from the AUR (for what I could see on the package dependencies).</p>
<h4 id="tui">TUI<a class="headerlink" href="#tui" title="Permanent link">¶</a></h4>
<p>Use the TUI by running</p>
<pre><code class="language-sh">mangal
</code></pre>
<p>Download manga using the TUI by selecting the source/scrapper, search the manga/comic you want and then you can select each chapter to download (use <code>tab</code> to select all). This is what I use when downloading manga that already finished publishing, or when I’m just searching and testing out how it downloads the manga (directory name, and manga information).</p>
<p>Note that some scrapters will contain duplicated chapters, as they have multiple uploaded chapters from the community, usually for different <em>scanlation groups</em>. This happens a lot with <a href="https://mangadex.org/">MangaDex</a>.</p>
<h4 id="inline">Inline<a class="headerlink" href="#inline" title="Permanent link">¶</a></h4>
<p>The inline mode is a single terminal command meant to be used to automate stuff or for more advanced options. You can peek a bit into the “<a href="https://github.com/metafates/mangal/wiki/Inline-mode#command-examples">documentation</a>” which honestly it’s ass because it doesn’t explain much. The minimal command for inline according to the <code>mangal help</code> is:</p>
<pre><code class="language-sh">mangal inline --manga <option> --query <manga-title>
</code></pre>
<p>But this will not produce anything because it also needs <code>--source</code> (or set the default using the config key <code>downloader.default_sources</code>) and either <code>--json</code> which basically just does the search and returns the result in <code>json</code> format or <code>--download</code> to actually download whatever is found; I recommend to do <code>--json</code> first to check that the correct manga will be downloaded then do <code>--download</code>.</p>
<p>Something not mentioned anywhere is the <code>--manga</code> flag options (found it at the source code), it has 3 available options:</p>
<ul>
<li><code>first</code>: first manga entry found for the search.</li>
<li><code>last</code>: last manga entry found for the search.</li>
<li><code>exact</code>: exact manga title match. This is the one I use.</li>
</ul>
<p>Similar to <code>--chapters</code>, there are a few options not explained (that I found at the source code, too). I usually just use <code>all</code> but other options:</p>
<ul>
<li><code>all</code>: all chapters found in the chapter list.</li>
<li><code>first</code>: first chapter found in the chapter list.</li>
<li><code>last</code>: last chapter found in the chapter list</li>
<li><code>[from]-[to]</code>: selector for the chapters found in the chapter list, index starts at 0.<ul>
<li>If the selectors (<code>from</code> or <code>to</code>) exceed the amount of chapters in the chapterlist it just adjusts to he maximum available.</li>
<li>I had to fix this at the source code because if you wanted <code>to</code> to be the last chapter, it did <code>to + 1</code> and it failed due to index out of range.</li>
</ul>
</li>
<li><code>@[sub]@</code>: not sure how this works exactly, my understanding is that it’s for “named” chapters.</li>
</ul>
<p>That said, I’ll do an example by using <a href="https://mangapill.com">Mangapill</a> as source, and will search for <a href="https://mangapill.com/manga/2285/kimetsu-no-yaiba">Demon Slayer: Kimetsu no Yaiba</a>:</p>
<ol>
<li>Search first and make sure my command will pull the manga I want:</li>
</ol>
<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --json | jq # I use jq to pretty format the output
</code></pre>
<ol start="2">
<li>I make sure the json output contains the correct manga information: name, url, etc..</li>
</ol>
<ul>
<li>You can also include the flag <code>--include-anilist-manga</code> to include anilist information (if any) so you can check that the correct anilist id is attached. If the correct one is not attached (and it exists) then you can bind the <code>--query</code> (search term) to a specific anilist id by running:</li>
</ul>
<pre><code class="language-sh">mangal inline anilist set --name "Kimetsu no Yaiba" --id 101922
</code></pre>
<ol start="3">
<li>If I’m okay with the outputs, then I change <code>--json</code> for <code>--download</code> to actually download:</li>
</ol>
<pre><code class="language-sh">mangal inline --source "Mangapill" --manga "exact" --query "Kimetsu no Yaiba" --download
</code></pre>
<ol start="4">
<li>Check if the manga is downloaded correctly. I do this by going to my download directory and checking the directory name (I’m picky with this stuff), that all chapters where downloaded, that it includes a correct <code>series.json</code> file and it contains a <code>cover.<img-ext></code>; this usually means it correctly pulled information from anilist and that it will contain metadata Komga will be able to use.</li>
</ol>
<h4 id="automation">Automation<a class="headerlink" href="#automation" title="Permanent link">¶</a></h4>
<p>The straight forward approach for automation is just to bundle a bunch of <code>mangal inline</code> commands in a shell script and schedule it’s execution either via <a href="https://wiki.archlinux.org/title/cron">cron</a> or <a href="https://wiki.archlinux.org/title/systemd/Timers">systemd/Timers</a>. But, as always, I overcomplicated/overengineered my approach, which is the following:</p>
<ol>
<li>Group manga names per source.</li>
<li>Configure anything that should always be set before executing <code>mangal</code>, this includes anilist bindings.</li>
<li>Have a way to track the changes/updates on each run.</li>
<li>Use that tracker to know where to start downloading chapters from.<ul>
<li>This is optional, as you can just do <code>--chapters "all"</code> and it will work but I do it mostly to keep the logs/output cleaner/shorter.</li>
</ul>
</li>
<li>Download/update each manga using <code>mangal inline</code>.</li>
<li>Wrap everything in a <code>systemd</code> <code>service</code> and <code>timer</code>.</li>
</ol>
<p>Manga list example:</p>
<pre><code class="language-sh">mangapill="Berserk|Chainsaw Man|Dandadan|Jujutsu Kaisen|etc..."
</code></pre>
<p>Function that handles the download per manga in the list:</p>
<pre><code class="language-sh">mangal_src_dl () {
source_name=$1
manga_list=$(echo "$2" | tr '|' '\n')
while IFS= read -r line; do
# By default download all chapters
chapters="all"
last_chapter_n=$(grep -e "$line" "$TRACKER_FILE" | cut -d'|' -f2 | grep -v -e '^$' | tail -n 1)
if [ -n "${last_chapter_n}" ]; then
chapters="$last_chapter_n-9999"
echo "Downloading [${last_chapter_n}-] chapters for $line from $source_name..."
else
echo "Downloading all chapters for $line from $source_name..."
fi
dl_output=$(mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "$chapters" -d)
if [ $? -ne 0 ]; then
echo "Failed to download chapters for $line."
continue
fi
line_count=$(echo "$dl_output" | grep -v -e '^$' | wc -l)
if [ $line_count -gt 0 ]; then
echo "Downloaded $line_count chapters for $line:"
echo "$dl_output"
new_last_chapter_n=$(echo "$dl_output" | tail -n 1 | cut -d'[' -f2 | cut -d']' -f1)
# manga_name|last_chapter_number|downloaded_chapters_on_this_update|manga_source
echo "$line|$new_last_chapter_n|$line_count|$source_name" >> $TRACKER_FILE
else
echo "No new chapters for $line."
fi
done <<< "$manga_list"
}
</code></pre>
<p>Where <code>$TRACKER_FILE</code> is just a variable holding a path to some file where you can store the tracking and <code>$DOWNLOAD_FORMAT</code> the format for the mangas, for me it’s <code>cbz</code>. Then the usage would be something like <code>mangal_src_dl "Mangapill" "$mangapill"</code>, meaning that it is a function call per source.</p>
<p>A simpler function without “tracking” would be:</p>
<pre><code class="language-sh">mangal_src_dl () {
source_name=$1
manga_list=$(echo "$2" | tr '|' '\n')
while IFS= read -r line; do
echo "Downloading all chapters for $line from $source_name..."
mangal inline -S "$source_name" -q "$line" -m "exact" -F "$DOWNLOAD_FORMAT" -c "all" -d
if [ $? -ne 0 ]; then
echo "Failed to download chapters for $line."
continue
fi
echo "Finished downloading chapters for $line."
done <<< "$manga_list"
}
</code></pre>
<p>The tracker file would have a format like follows:</p>
<pre><code># Updated: 06/10/23 10:53:15 AM CST
Berserk|0392|392|Mangapill
Dandadan|0110|110|Mangapill
...
</code></pre>
<p>And note that if you already had manga downloaded and you run the script for the first time, then it will show as if it downloaded everything from the first chapter, but that’s just how <code>mangal</code> works, it will actually just discover downloaded chapters and only download anything missing.</p>
<p>Any configuration the downloader/updater might need needs to be done before the <code>mangal_src_dl</code> calls. I like to configure mangal for download path, format, etc.. I found that it is needed to clear the <code>mangal</code> and <code>rod</code> browser cache (headless browser used in some custom sources) from personal experience and from others: <a href="https://github.com/metafates/mangal/issues/170">mangal#170</a> and <a href="https://github.com/oae/kaizoku/issues/89">kaizoku#89</a>.</p>
<p>Also you should set any anilist binding necessary for the downloading (as the cache was cleared). An example of an anilist binding I had to do is for Mushoku Tensei, as it has both a <a href="https://anilist.co/manga/85470/Mushoku-Tensei-Jobless-Reincarnation/">light novel</a> and <a href="https://anilist.co/manga/85564/Mushoku-Tensei-Jobless-Reincarnation/">manga</a> version, which for me it’s the following binding:</p>
<pre><code class="language-sh">mangal inline anilist set --name "Mushoku Tensei - Isekai Ittara Honki Dasu" --id 85564
</code></pre>
<p>Finally is just a matter of using your prefered way of scheduling, I’ll use <code>systemd/Timers</code> but anything is fine. You could make the downloader script more sophisticated and only running every week on which each manga gets (usually) released but that’s too much work; I’ll just run it once daily probably.</p>
<p>A feature I want to add and probably will is sending notifications (probably through email) on a summary for manga downloaded or failed to download so I’m on top of the updates. For now this is good enough and it’s been working so far.</p>
<h2 id="komga">Komga<a class="headerlink" href="#komga" title="Permanent link">¶</a></h2>
<p><a href="https://komga.org/">Komga</a> is a comics/mangas media server.</p>
<p>Install from the AUR:</p>
<pre><code class="language-sh">yay -S komga
</code></pre>
<p>This <code>komga</code> package creates a <code>komga</code> (service) user and group which is tied to the also included <code>komga.service</code>.</p>
<p>Configure it by editing <code>/etc/komga.conf</code>:</p>
<pre><code class="language-sh">SERVER_PORT=25600
SERVER_SERVLET_CONTEXT_PATH=/ # this depends a lot of how it's going to be served (domain, subdomain, ip, etc)
KOMGA_LIBRARIES_SCAN_CRON="0 0 * * * ?"
KOMGA_LIBRARIES_SCAN_STARTUP=false
KOMGA_LIBRARIES_SCAN_DIRECTORY_EXCLUSIONS='#recycle,@eaDir,@Recycle'
KOMGA_FILESYSTEM_SCANNER_FORCE_DIRECTORY_MODIFIED_TIME=false
KOMGA_REMEMBERME_KEY=USE-WHATEVER-YOU-WANT-HERE
KOMGA_REMEMBERME_VALIDITY=2419200
KOMGA_DATABASE_BACKUP_ENABLED=true
KOMGA_DATABASE_BACKUP_STARTUP=true
KOMGA_DATABASE_BACKUP_SCHEDULE="0 0 */8 * * ?"
</code></pre>
<p>My changes (shown above):</p>
<ul>
<li><code>cron</code> schedules.<ul>
<li>It’s not actually <code>cron</code> but rather a <code>cron</code>-like syntax used by <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronSequenceGenerator.html">Spring</a> as stated in the <a href="https://komga.org/installation/configuration.html#optional-configuration">Komga config</a>.</li>
</ul>
</li>
<li>Added the remember me key.</li>
<li>For more check out <a href="https://komga.org/installation/configuration.html">Komga: Configuration options</a>.</li>
</ul>
<p>If you’re going to run it locally (or LAN/VPN) you can start the <code>komga.service</code> and access it via IP at <code>http://<your-server-ip>:<port>(/base_url)</code> as stated at <a href="https://komga.org/installation/webui.html">Komga: Accessing the web interface</a>, then you can continue with the <a href="#mangal">mangal</a> section, else continue with the next steps for the reverse proxy and certificate.</p>
<h3 id="reverse-proxy">Reverse proxy<a class="headerlink" href="#reverse-proxy" title="Permanent link">¶</a></h3>
<p>Create the reverse proxy configuration (this is for <code>nginx</code>). In my case I’ll use a subdomain, so this is a new config called <code>komga.conf</code> at the usual <code>sites-available/enabled</code> path:</p>
<pre><code class="language-nginx">server {
listen 80;
server_name komga.yourdomain.com; # change accordingly to your wanted subdomain and domain name
location / {
proxy_pass http://localhost:25600; # change port if needed
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 600s;
proxy_send_timeout 600s;
}
}
</code></pre>
<p>If it’s going to be used as a subdir on another domain then just change the <code>location</code> with <code>/subdir</code> instead of <code>/</code>; be careful with the <code>proxy_pass</code> directive, it has to match what you configured at <code>/etc/komga.conf</code> for the <code>SERVER_SERVLET_CONTEXT_PATH</code> regardless of the <code>/subdir</code> you selected at <code>location</code>.</p>
<h4 id="ssl-certificate">SSL certificate<a class="headerlink" href="#ssl-certificate" title="Permanent link">¶</a></h4>
<p>If using a subdir then the same certificate for the subdomain/domain should work fine and no extra stuff is needed, else if following along me then we can create/extend the certificate by running:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>That will automatically detect the new subdomain config and create/extend your existing certificate(s). In my case I manage each certificate’s subdomain:</p>
<pre><code class="language-sh">certbot --nginx -d domainname.com -d subdomain.domainname.com -d komga.domainname.com
</code></pre>
<h3 id="start-using-komga">Start using Komga<a class="headerlink" href="#start-using-komga" title="Permanent link">¶</a></h3>
<p>We can now <code>start</code>/<code>enable</code> the <code>komga.service</code>:</p>
<pre><code class="language-sh">systemctl enable komga.service
systemctl start komga.service
</code></pre>
<p>And access the web interface at <code>https://komga.domainname.com</code> which should show the login page for Komga. The first time it will ask to create an account as shown in <a href="https://komga.org/installation/webui.html#create-user-account">Komga: Create user account</a>, this will be an admin account. Fill in the email and password (can be changed later). The email doesn’t have to be an actual email, for now it’s just for management purposes.</p>
<p>Next thing would be to add any extra account (for read-only/download manga permissions), add/import libraries, etc.. For now I’ll leave it here until we start downloading manga on the next steps.</p>
<h3 id="library-creation">Library creation<a class="headerlink" href="#library-creation" title="Permanent link">¶</a></h3>
<p>Creating a library is as simple as creating a directory somewhere and point to it in Komga. The following examples are for my use case, change accordingly. I’ll be using <code>/mnt/d/mangal</code> for my library (as stated in the <a href="#configuration">mangal: configuration</a> section):</p>
<pre><code class="language-sh">mkdir /mnt/d/mangal
</code></pre>
<p>Where I chose the name <code>mangal</code> as its the name of the downloader/scrapper, it could be anything, this is just how I like to organize stuff.</p>
<p>For the most part, the permissions don’t matter much (as long as it’s readable by the <code>komga</code> user) unless you want to delete some manga, then <code>komga</code> user also needs write permissions.</p>
<p>Then just create the library in Komga web interface (the <code>+</code> sign next to <em>Libraries</em>), choose a name <em>“Mangal”</em> and point to the root folder <code>/mnt/d/mangal</code>, then just click <em>Next</em>, <em>Next</em> and <em>Add</em> for the defaults (that’s how I’ve been using it so far). This is well explained at <a href="https://komga.org/guides/libraries.html">Komga: Libraries</a>.</p>
<p>The real important part (for me) is the permissions of the <code>/mnt/d/mangal</code> directory, as I want to have write access for <code>komga</code> so I can manage from the web interface itself. It looks like it’s just a matter of giving ownership to the <code>komga</code> user either for owner or for group (or to all for that matter), but since I’m going to use a separate user to download manga then I need to choose carefully.</p>
<h4 id="set-default-directory-permissions">Set default directory permissions<a class="headerlink" href="#set-default-directory-permissions" title="Permanent link">¶</a></h4>
<p>The desired behaviour is: set <code>komga</code> as group ownership, set write access to group and whenever a new directory/file is created, inherit these permission settings. I found out via <a href="https://unix.stackexchange.com/a/1315">this</a> stack exchange answer how to do it. So, for me:</p>
<pre><code class="language-sh">chown manga-dl:komga /mnt/d/mangal # required for group ownership for komga
chmod g+s /mnt/d/mangal # required for group permission inheritance
setfacl -d -m g::rwx /mnt/d/mangal # default permissions for group
setfacl -d -m o::rx /mnt/d/mangal # default permissions for other (as normal, I think this command can be excluded)
</code></pre>
<p>Where <code>manga-dl</code> is the user I created to download manga with. Optionally add <code>-R</code> flag to those 4 commands in case it already has subdirectories/files (this might mess file permissions, but it’s not an issue as far as I konw).</p>
<p>Checking that the permissions are set correctly (<code>getfacl /mnt/d/mangal</code>):</p>
<pre><code>getfacl: Removing leading '/' from absolute path names
# file: mnt/d/mangal
# owner: manga-dl
# group: komga
# flags: -s-
user::rwx
group::rwx
other::r-x
default:user::rwx
default:group::rwx
default:other::r-x
</code></pre>
<p>You can then check by creating a new subdirectory (in <code>/mnt/d/mangal</code>) and it should have the same group permissions.</p>
<h4 id="populate-manga-library">Populate manga library<a class="headerlink" href="#populate-manga-library" title="Permanent link">¶</a></h4>
<p>You can now start downloading manga using <code>mangal</code> either manually or by running the <code>cron</code>/<code>systemd/Timers</code> and it will be detected by Komga automatically when it scans the library (once every hour according to my config). You can manually scan the library, though, by clicking on the 3 dots to the right of the library name (in Komga) and click on “Scan library files”.</p>
<p>Then you can check that the metadata is correct (once the manga is fully indexed and metadata finished building), such as title, summary, chapter count, language, tags, genre, etc., which honestly it never works fine as <code>mangal</code> creates the <code>series.json</code> with the <code>comicId</code> field with an upper case <code>I</code> and Komga expects it to be a lower case <code>i</code> (<code>comicid</code>) so it falls back to using the info from the first chapter. I’ll probably will fix this on <code>mangal</code> side, and see how it goes.</p>
<p>So, what I do is manually edit the metadata for the manga, by changing whatever it’s wrong or add what’s missing (I like adding anilist and MyAnimeList links) and then leave it as is. This is up to you.</p>
<h2 id="alternative-downloaders">Alternative downloaders<a class="headerlink" href="#alternative-downloaders" title="Permanent link">¶</a></h2>
<p>Just for the record, here is a list of downloaders/scrapers I considered before starting to use <code>mangal</code>:</p>
<ul>
<li><a href="https://github.com/oae/kaizoku">kaizoku</a>: NodeJS web server that uses <code>mangal</code> for its “backend” and honestly since I liked <code>mangal</code> so much I should use it, the only reason I don’t is because I’m a bitch and I don’t want to use a D*ck*r image and NodeJS (ew) (in general is pretty bloated in my opinion). If I get tired of my solution with pure <code>mangal</code> I might as well just migrate to it as It’s a more automatic solution.</li>
<li><a href="https://github.com/manga-py/manga-py">manga-py</a>: Python CLI application that’s a really good option as far as I’ve explored it, I’m just not using it yet as <code>mangal</code> has been really smooth and has everything I need, but will definitely explore it in the future if I need to. The cool thing out of the box is the amount of sources it can scrape from (somethign lacking from <code>mangal</code>).</li>
<li><a href="https://github.com/mylar3/mylar3">mylar3</a>: Python web server that should be the easier way to download manga with once correctly set up, but I guess I’m too dumb and don’t know how to configure it. Looks like you need to have access to specific private torrent trackers or whatever the other ways to download are, I just couldn’t figure out how to set it up and for public torrent stuff everything will be all over the place, so this was no option for me at the end.</li>
</ul>
<p>Others:</p>
<ul>
<li><a href="https://hakuneko.download/">HakuNeku</a>: It looks pretty easy to use and future rich, only thing is that it’s not designed for headless servers, just a normal app. So this is also not an option for me. You could use it on your computer and <code>rsync</code> to your server or use some other means to upload to your server (a <em>nono</em> for me).</li>
<li><a href="https://github.com/riderkick/FMD">FMD</a>: No fucking idea on how to use it and it’s not been updated since 2019, just listing it here as an option if it interests you.</li>
</ul>]]></content:encoded>
</item>
<item>
<title>Updated the how-to entries titles</title>
<link>https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/updating_creating_entries_titles_to_setup.html</guid>
<pubDate>Sat, 03 Jun 2023 03:46:44 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Update</category>
<description>Just a small update on the title for some old entries.</description>
<content:encoded><![CDATA[<p>One of the main reasons I started “blogging” was basically just to document how I set up stuff up so I can reference them later in the future if I ever needed to replicate the steps or just to show somebody, and these entries had helped to do so multiple times. I’ll keep creating these entries but after a while the <em>Creating a</em> title started to feel weird, because we’re not <em>creating</em> anything really, it is just a set up/configuration/how-to/etc. So I think that using <em>Set up a</em> for the titles is better and makes more sense; probably using <em>How to set up a</em> is better for the SEO bullshit.</p>
<p>Anyways, so I’ll start using <em>Set up a</em> instead of <em>Creating a</em> and will retroactively change the titles for these entries (by this entry the change should be applied already). This might impact some RSS feeds as they keep up a cache of the feed and might duplicate the entries, heads up if for some reason somebody is using it.</p>]]></content:encoded>
</item>
<item>
<title>I had to learn Go and Lua the hard way</title>
<link>https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/learned_go_and_lua_hard_way.html</guid>
<pubDate>Sat, 03 Jun 2023 03:32:17 GMT</pubDate>
<category>English</category>
<category>Rant</category>
<category>Short</category>
<category>Tools</category>
<description>Thanks to the issues of a program (mangal) I'm starting to use for my manga media server, I had to learn Go and Lua the hard way so that I can fix it and use it.</description>
<content:encoded><![CDATA[<p><mark>TL;DR</mark>: I learned Go and Lua the hard way by forking (for fixing):</p>
<ul>
<li><a href="https://github.com/luevano/mangal">mangal</a>: main manga scraper written in Go.</li>
<li><a href="https://github.com/luevano/mangal-lua-libs">mangal-lua-libs</a>: <a href="https://github.com/yuin/gopher-lua">gopher-lua</a> libraries for mangal.</li>
<li><a href="https://github.com/luevano/mangal-scrapers">mangal-scrapers</a>: custom mangal scrapers written in Lua.</li>
</ul>
<p>In the last couple of days I’ve been setting up a <a href="https://komga.org/">Komga</a> server for manga downloaded using <a href="https://github.com/metafates/mangal">metafates/mangal</a> (upcoming set up entry about it) and everything was fine so far until I tried to download One Piece from <a href="https://mangadex.org/">MangaDex</a> of which <code>mangal</code> has a built-in scraper. Long story short the issue was that MangaDex’s API only allows requesting manga chapters on chunks of 500 and the way that was being handled was completely wrong, specifics can be found on my <a href="https://github.com/luevano/mangal/commit/395bce96e439ee828d0180328a5cf9204bfd818a">commit</a> (and the subsequent minor fix <a href="https://github.com/luevano/mangal/commit/6bf709fe9b333ec9d4375ed80f9b055d07a40c1c">commit</a>).</p>
<p>I tried to do a PR, but the project hasn’t been active since Feb 2023 (same reason I didn’t even try to do PRs on the other repos) so I closed it and will start working on my own <a href="https://github.com/luevano/mangal">fork</a>, probaly just merging everything <a href="https://github.com/Belphemur">Belphemur</a>‘s <a href="https://github.com/Belphemur/mangal">fork</a> has to offer, as he’s been working on <code>mangal</code> actively. I could probably just fork from him and/or just submit PR requests to him, but I think I saw some changes I didn’t really like, will have to look more into it.</p>
<p>Also, while trying to use some of the custom scrapers I ran into issues with the headless chrome explorer implementation where it didn’t close on each manga chapter download, causig my CPU and Mem usage to get maxed out and losing control of the system, so I had to also <a href="https://github.com/luevano/mangal-lua-libs">fork</a> the <a href="https://github.com/metafates/mangal-lua-libs">metafates/mangal-lua-libs</a> and “fixed” (I say fixed because that wasn’t the issue at the end, it was how the custom scrapers where using it, shitty documentation) the issue by adding the <code>browser.Close()</code> function to the <code>headless</code> Lua API (<a href="https://github.com/luevano/mangal-lua-libs/commit/97fba97ab23efe88278dfacbeed2dd83c5472de0">commit</a>) and merged some commits from the original <a href="https://github.com/vadv/gopher-lua-libs">vadv/gopher-lua-libs</a> just to include any features added to the Lua libs needed.</p>
<p>Finally I <a href="https://github.com/luevano/mangal-scrapers">forked</a> the <a href="https://github.com/metafates/mangal-scrapers">metafates/mangal-scrapers</a> (which I actually forked <a href="https://github.com/NotPhantomX">NotPhantomX</a>‘s <a href="https://github.com/NotPhantomX/mangal-scrapers">fork</a> as they had included more scrapers from some pull requests) to be able to have updated custom Lua scrapers (in which I also fixed the <code>headless</code> bullshit) and use them on my <code>mangal</code>.</p>
<p>So, I went into the rabbit hole of manga scrapping because I wanted to set up my Komga server, and more importantly I had to quickly learn Go and Lua (Lua was easier) and I have to say that Go is super convoluted on the module management, all research I did lead me to totally different responses, but it is just because of different Go versions and the year of the responses.</p>]]></content:encoded>
</item>
<item>
<title>Al fin tengo fibra ópticona</title>
<link>https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/al_fin_tengo_fibra_opticona.html</guid>
<pubDate>Tue, 09 May 2023 08:59:00 GMT</pubDate>
<category>Rant</category>
<category>Short</category>
<category>Spanish</category>
<category>Update</category>
<description>Por fin pude contratar fibra óptica simétrica y ya no sufro con el cobre de cierta compañía horrible.</description>
<content:encoded><![CDATA[<p>Quienes me conocen sabrán que llevo como 2 años intentando contratar internet de fibra óptica (específicamente el de T*lm*x). El problema es que nunca había <em>nodos/terminales</em> disponibles o, la verdad, que los técnicos ni querían hacer su jale porque están acostumbrados a que les debes soltar una feria para que te la instalen.</p>
<p>Pues bueno, el punto es que me tocó estar aguantando la compañía horrible de *zz*, que sólo tiene cobre; el servicio es malo y a cada rato le suben de precio. Por esto último volví a checar precios de otras compañías para comparar y resulta que me estaban cobrando como $100 - $150 pesos extra con el mismo paquete que ya tenía/tengo. Hasta ahí estaba encabronado, y no ayudó nada que intenté hablar con los muy incompetentes de <em>soporte</em> y no pudieron digamos “resolverme”, porque ¿cómo es posible que siendo cliente de como 5 años ni si quiera pueden avisarme que ya tienen mejores paquetes (que la neta es el mismo paquete pero más barato)?</p>
<p>Intenté pedirles que me cambien al paquete actual (mismo todo, única diferencia el precio), pero resulta que me meterían a plazo forzoso. Obviamente esto me prendió un cuete en la cola y por eso chequé con T*lm*x, que a mi sorpresa salía que sí había fibra óptica disponible en mi cantón. Inicié el proceso de portabilidad y me dijeron que en como dos semanas me la instalaban, pero resulta que el basado del técnico me marcó al día siguiente para decirme que <mark>YA ESTABA AFUERA DE MI CASA</mark> para instalarlo. Gané.</p>
<p>Resulta que ahora sí hay <em>nodos/terminales</em>, de hecho instalaron 3 nuevos y están completamente vacíos, me tocó muy buena suerte y el muy basado del técnico se lo aventó en medio segundo sin ningún pedo, no me pidió nada más que detalles de dónde quería el módem. No tenía efectivo si no le soltaba un varo, se portó muy chingón.</p>]]></content:encoded>
</item>
<item>
<title>Updated pyssg to include pymdvar and the website</title>
<link>https://blog.luevano.xyz/a/updated_pyssg_pymdvar_and_website.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/updated_pyssg_pymdvar_and_website.html</guid>
<pubDate>Sat, 06 May 2023 12:39:14 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Tools</category>
<category>Update</category>
<description>Worked on another update of pyssg which now includes my extension pymdvar and updated the website overall.</description>
<content:encoded><![CDATA[<p>Again, I’ve updated <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> to add a bit of unit-testing as well as to include my extension <a href="https://github.com/luevano/pymdvar"><code>pymdvar</code></a> which is used to convert <code>${some_variables}</code> into their respective <code>values</code> based on a config file and/or environment variables. With this I also updated a bit of the CSS of the site as well as basically all the entries and base templates, a much needed update (for me, because externally doesn’t look like much). Along with this I also added a “return to top” button, once you scroll enough on the site, a new button appears on the bottom right to get back to the top, also added table of contents to entries taht could use them (as well as a bit of CSS to them).</p>
<p>This update took a long time because I had a fundamental issue with how I was managing the “static” website, where I host all assets such as CSS, JS, images, etc.. Because I was using the <code><base></code> HTML tag. The issue is that this tag affects everything and there is no “opt-out” on some body tags, meaning that I would have to write the whole URL for all static assets. So I tried looking into changing how the image extension for <a href="https://python-markdown.github.io/"><code>python-markdown</code></a> works, so that it includes this “base” URL I needed. But it was too much hassle, so I ended up developing my own extension mentioned earlier. Just as a side note, I noticed that my extension doesn’t cover all my needs, so probably it wont cover yours, if you end up using it just test it out a bit yourself and then go ahead, PRs are welcomed.</p>
<p>One thing led to another so I ended up changing a lot of stuff, and with changes comes tireness and eded up leaving the project for a while (again). This also led to not wanting to write or add anything else to the site until I sorted things out. But I’m again reviving it I guess, and up to the next cycle.</p>
<p>The next things I’ll be doing are continuing with my <a href="https://blog.luevano.xyz/tag/@gamedev">@gamedev</a> journey and probably upload some drawings if I feel like doing some.</p>]]></content:encoded>
</item>
<item>
<title>Rewrote pyssg again</title>
<link>https://blog.luevano.xyz/a/rewrote_pyssg_again.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/rewrote_pyssg_again.html</guid>
<pubDate>Tue, 20 Dec 2022 04:31:05 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Tools</category>
<category>Update</category>
<description>Rewrote pyssg to make it more flexible and to work with YAML configuration files.</description>
<content:encoded><![CDATA[<p>I’ve been wanting to change the way <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> reads config files and generates <code>HTML</code> files so that it is more flexible and I don’t need to have 2 separate build commands and configs (for <a href="https://blog.luevano.xyz">blog</a> and <a href="https://art.luevano.xyz">art</a>), and also to handle other types of “sites”; because <code>pyssg</code> was built with blogging in mind, so it was a bit limited to how it could be used. So I had to kind of <em>rewrite</em> <code>pyssg</code>, and with the latest version I can now generate the whole site and use the same templates for everything, quite neat for my use case.</p>
<p>Anyways, so I bought a new domain for all <code>pyssg</code> related stuff, mostly because I wanted somewhere to test live builds while developing, it is of course <a href="https://pyssg.xyz">pyssg.xyz</a>; as of now it is the same template, CSS and scripts that I use here, probably will change in the future. I’ll be testing new features and anything <code>pyssg</code> related stuff.</p>
<p>I should start pointing all links to <code>pyssg</code> to the actual site instead of the github repository (or my <a href="https://git.luevano.xyz">git</a> repository), but I haven’t decided how to handle everything.</p>]]></content:encoded>
</item>
<item>
<title>Creating my Go Godot Jam 3 entry using Godot 3.5 devlog 1</title>
<link>https://blog.luevano.xyz/g/gogodot_jam3_devlog_1.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/g/gogodot_jam3_devlog_1.html</guid>
<pubDate>Fri, 10 Jun 2022 09:17:05 GMT</pubDate>
<category>English</category>
<category>Gamedev</category>
<category>Gamejam</category>
<category>Gdscript</category>
<category>Godot</category>
<description>Details on the implementation for the game I created for the Go Godot Jam 3, which theme is Evolution.</description>
<content:encoded><![CDATA[<p>The jam’s theme is Evolution and all the details are listed <a href="https://itch.io/jam/go-godot-jam-3">here</a>. <del>This time I’m logging as I go, so there might be some changes to the script or scenes along the way.</del> <ins>I couldn’t actually do this, as I was running out of time</ins>. Note that I’m not going to go into much details, the obvious will be ommitted.</p>
<p>I wanted to do a <em>Snake</em> clone, and I’m using this jam as an excuse to do it and add something to it. The features include:</p>
<ul>
<li>Snakes will pass their stats in some form to the next snakes.</li>
<li>Non-grid snake movement. I just hate the grid constraint, so I wanted to make it move in any direction.</li>
<li>Depending on the food you eat, you’ll gain new mutations/abilities <del>and the more you eat the more that mutation develops</del> <ins>didn’t have time to add this feature, sad</ins>.</li>
<li>Procedural map creation.</li>
</ul>
<p>I created this game using <em>Godot 3.5-rc3</em>. You can find the source code in my GitHub <a href="https://github.com/luevano/gogodot_jam3">here</a> which at the time of writing this it doesn’t contain any exported files, for that you can go ahead and play it in your browser at <a href="https://lorentzeus.itch.io/snake-tronic">itch.io</a>, which you can find below:</p>
<p style="text-align:center"><iframe src="https://itch.io/embed/1562701?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/snake-tronic">Snake-tronic by Lorentzeus</a></iframe></p>
<p>You can also find the jam entry <a href="https://itch.io/jam/go-godot-jam-3/rate/1562701">here</a>.</p>
<p>Similarly with the my FlappyBird clone, I plan to update this to a better state.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#initial-setup">Initial setup</a></li>
<li><a href="#assets">Assets</a></li>
<li><a href="#the-snake">The snake</a><ul>
<li><a href="#basic-movement">Basic movement</a></li>
<li><a href="#setting-up-path-following">Setting up path following</a></li>
<li><a href="#define-body-parts-for-the-snake">Define body parts for the snake</a></li>
<li><a href="#adding-body-parts">Adding body parts</a></li>
<li><a href="#fix-on-body-segments-following-head">Fix on body segments following head</a></li>
</ul>
</li>
<li><a href="#the-food">The food</a></li>
<li><a href="#za-warudo-the-world">Za warudo! (The world)</a><ul>
<li><a href="#food-placement">Food placement</a></li>
</ul>
</li>
<li><a href="#stats-clas-and-loadingsaving-data">Stats clas and loading/saving data</a><ul>
<li><a href="#stats-class">Stats class</a></li>
<li><a href="#loadsave-data">Load/save data</a></li>
</ul>
</li>
<li><a href="#scoring">Scoring</a></li>
<li><a href="#snake-redesigned-with-the-state-machine-pattern">Snake redesigned with the state machine pattern</a></li>
<li><a href="#other-minor-stuff">Other minor stuff</a></li>
<li><a href="#final-notes">Final notes</a></li>
</ul>
</div>
<h2 id="initial-setup">Initial setup<a class="headerlink" href="#initial-setup" title="Permanent link">¶</a></h2>
<p>Again, similar to the <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html">FlappyBird</a> clone I created, I’m using the directory structure I wrote about on <a href="https://blog.luevano.xyz/g/godot_project_structure.html">Godot project structure</a> with slight modifications to test things out. Also using similar <em>Project settings</em> as those from the <em>FlappyBird</em> clone like the pixel art texture imports, keybindings, layers, etc..</p>
<p>I’ve also setup <a href="https://github.com/bram-dingelstad/godot-gifmaker">GifMaker</a>, with slight modifications as the <em>AssetLib</em> doesn’t install it correctly and contains unnecessry stuff: moved necessary files to the <code>res://addons</code> directory, deleted test scenes and files in general, and copied the license to the <code>res://docs</code> directory. Setting this up was a bit annoying because the tutorial it’s bad (with all due respect). I might do a separate entry just to explain how to set it up, because I couldn’t find it anywhere other than by inspecting some of the code/scenes. <ins>I ended up leaving this disabled in the game as it hit the performance by a lot, but it’s an option I’ll end up researching more</ins>.</p>
<p>This time I’m also going to be using an <a href="https://www.gdquest.com/docs/guidelines/best-practices/godot-gdscript/event-bus/">Event bus</a> singleton (which I’m going to just call <em>Event</em>) as managing signals was pretty annoying on my last project; as well as a <em>Global</em> singleton for essential stuff so I don’t have to do as many cross references between nodes/scenes.</p>
<h2 id="assets">Assets<a class="headerlink" href="#assets" title="Permanent link">¶</a></h2>
<p>This time I’ll be creating my own assets in <a href="https://www.aseprite.org/">Aseprite</a>, wont be that good, but enough to prototype and get things going.</p>
<p>Other than that I used few key sprites from <a href="https://vryell.itch.io/">vryell</a>: <a href="https://vryell.itch.io/controller-keyboard-icons">Controller & Keyboard Icons</a> and a font from <a href="https://datagoblin.itch.io/">datagoblin</a>: <a href="https://datagoblin.itch.io/monogram">Monogram</a>.</p>
<h2 id="the-snake">The snake<a class="headerlink" href="#the-snake" title="Permanent link">¶</a></h2>
<p>This is the most challenging part in my opinion as making all the body parts follow the head in a user defined path it’s kinda hard. I tried with like 4-5 options and the one I’m detailing here is the only one that worked as I wanted for me. This time the directory structure I’m using is the following:</p>
<figure id="__yafg-figure-39">
<img alt="FileSystem - Snake dir structure" src="https://static.luevano.xyz/images/g/gogodot_jam3/file_system_snake_dir_structure.png" title="FileSystem - Snake dir structure">
<figcaption>FileSystem - Snake dir structure</figcaption>
</figure>
<h3 id="basic-movement">Basic movement<a class="headerlink" href="#basic-movement" title="Permanent link">¶</a></h3>
<p>The most basic thing is to move the head, this is what we have control of. Create a scene called <code>Head.tscn</code> and setup the basic <em>KinematicBody2D</em> with it’s own <em>Sprite</em> and <em>CollisionShape2D</em> (I used a small circle for the tip of the head), and set the <em>Collision Layer/Mask</em> accordingly, for now just <code>layer = bit 1</code>. And all we need to do, is keep moving the snake forwards and be able to rotate left or right. Created a new script called <code>head.gd</code> attached to the root (<em>KinematicBody2D</em>) and added:</p>
<pre><code class="language-gdscript">extends KinematicBody2D
enum {
LEFT=-1,
RIGHT=1
}
var velocity: Vector2 = Vector2.ZERO
var _direction: Vector2 = Vector2.UP
func _physics_process(delta: float) -> void:
if Input.is_action_pressed("move_left"):
_rotate_to(LEFT)
if Input.is_action_pressed("move_right"):
_rotate_to(RIGHT)
velocity = _direction * Global.SNAKE_SPEED
velocity = move_and_slide(velocity)
_handle_time_elapsed(delta)
func _rotate_to(direction: int) -> void:
rotate(deg2rad(direction * Global.SNAKE_ROT_SPEED * get_physics_process_delta_time()))
_direction = _direction.rotated(deg2rad(direction * Global.SNAKE_ROT_SPEED * get_physics_process_delta_time()))
</code></pre>
<p>After tunning all the necessary parameters you should get something like this:</p>
<figure id="__yafg-figure-40">
<img alt="Snake - Basic movement (left and right controls)" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement.gif" title="Snake - Basic movement (left and right controls)">
<figcaption>Snake - Basic movement (left and right controls)</figcaption>
</figure>
<h3 id="setting-up-path-following">Setting up path following<a class="headerlink" href="#setting-up-path-following" title="Permanent link">¶</a></h3>
<p>To move other snake parts by following the snake head the only solution I found was to use the <em>Path2D</em> and <em>PathFollow2D</em> nodes. <em>Path2D</em> basically just handles the curve/path that <em>PathFollow2D</em> will use to move its child node; and I say “child node” in singular… as <em>PathFollow2D</em> can only handle one damn child, all the other ones will have weird transformations and/or rotations. So, the next thing to do is to setup a way to compute (and draw so we can validate) the snake’s path/curve.</p>
<p>Added the signal <code>snake_path_new_point(coordinates)</code> to the <em>Event</em> singleton and then add the following to <code>head.gd</code>:</p>
<pre><code class="language-gdscript">var _time_elapsed: float = 0.0
# using a timer is not recommended for < 0.01
func _handle_time_elapsed(delta: float) -> void:
if _time_elapsed >= Global.SNAKE_POSITION_UPDATE_INTERVAL:
Event.emit_signal("snake_path_new_point", global_position)
_time_elapsed = 0.0
_time_elapsed += delta
</code></pre>
<p>This will be pinging the current snake head position every <code>0.01</code> seconds (defined in <em>Global</em>). Now create a new scene called <code>Snake.tscn</code> which will contain a <em>Node2D</em>, a <em>Path2D</em> and an instance of <em>Head</em> as its childs. Create a new script called <code>snake.gd</code> attached to the root (<em>Node2D</em>) with the following content:</p>
<pre><code class="language-gdscript">class_name Snake
extends Node2D
onready var path: Path2D = $Path
func _ready():
Event.connect("snake_path_new_point", self, "_on_Head_snake_path_new_point")
func _draw() -> void:
if path.curve.get_baked_points().size() >= 2:
draw_polyline(path.curve.get_baked_points(), Color.aquamarine, 1, true)
func _on_Head_snake_path_new_point(coordinates: Vector2) -> void:
path.curve.add_point(coordinates)
# update call is to draw curve as there are new points to the path's curve
update()
</code></pre>
<p>With this, we’re now populating the <em>Path2D</em> curve points with the position of the snake head. You should be able to see it because of the <code>_draw</code> call. If you run it you should see something like this:</p>
<figure id="__yafg-figure-41">
<img alt="Snake - Basic movement with path" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_with_path.gif" title="Snake - Basic movement with path">
<figcaption>Snake - Basic movement with path</figcaption>
</figure>
<h3 id="define-body-parts-for-the-snake">Define body parts for the snake<a class="headerlink" href="#define-body-parts-for-the-snake" title="Permanent link">¶</a></h3>
<p>At this point the only thing to do is to add the corresponding next body parts and tail of the snake. To do so, we need a <em>PathFollow2D</em> to use the live-generating <em>Path2D</em>, the only caveat is that we need one of these per body part/tail (this took me hours to figure out, <em>thanks documentation</em>).</p>
<p>Create a new scene called <code>Body.tscn</code> with a <em>PathFollow2D</em> as its root and an <em>Area2D</em> as its child, then just add the necessary <em>Sprite</em> and <em>CollisionShap2D</em> for the <em>Area2D</em>, I’m using <code>layer = bit 2</code> for its collision. Create a new script called <code>generic_segment.gd</code> with the following code:</p>
<pre><code class="language-gdscript">extends PathFollow2D
export(String, "body", "tail") var TYPE: String = "body"
func _physics_process(delta: float) -> void:
offset += Global.SNAKE_SPEED * delta
</code></pre>
<p>And this can be attached to the <em>Body</em>‘s root node (<em>PathFollow2D</em>), no extra setup needed. Repeat the same steps for creating the <code>Tail.tscn</code> scene and when attaching the <code>generic_segment.gd</code> script just configure the <code>Type</code> parameter to <code>tail</code> in the GUI (by selecting the node with the script attached and editing in the <em>Inspector</em>).</p>
<h3 id="adding-body-parts">Adding body parts<a class="headerlink" href="#adding-body-parts" title="Permanent link">¶</a></h3>
<p>Now it’s just a matter of handling when to add new body parts in the <code>snake.gd</code> script. For now I’ve only setup for adding body parts to fulfill the initial length of the snake (this doesn’t include the head or tail). The extra code needed is the following:</p>
<pre><code class="language-gdscript">export(PackedScene) var BODY_SEGMENT_NP: PackedScene
export(PackedScene) var TAIL_SEGMENT_NP: PackedScene
var current_body_segments: int = 0
var max_body_segments: int = 1
func _add_initial_segment(type: PackedScene) -> void:
if path.curve.get_baked_length() >= (current_body_segments + 1.0) * Global.SNAKE_SEGMENT_SIZE:
var _temp_body_segment: PathFollow2D = type.instance()
path.add_child(_temp_body_segment)
current_body_segments += 1
func _on_Head_snake_path_new_point(coordinates: Vector2) -> void:
path.curve.add_point(coordinates)
# update call is to draw curve as there are new points to the path's curve
update()
# add the following lines
if current_body_segments < max_body_segments:
_add_initial_segment(BODY_SEGMENT_NP)
elif current_body_segments == max_body_segments:
_add_initial_segment(TAIL_SEGMENT_NP)
</code></pre>
<p>Select the <em>Snake</em> node and add the <em>Body</em> and <em>Tail</em> scene to the parameters, respectively. Then when running you should see something like this:</p>
<figure id="__yafg-figure-42">
<img alt="Snake - Basic movement with all body parts" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_added_body_parts.gif" title="Snake - Basic movement with all body parts">
<figcaption>Snake - Basic movement with all body parts</figcaption>
</figure>
<p>Now, we need to handle adding body parts after the snake is complete and already moved for a bit, this will require a queue so we can add part by part in the case that we eat multiple pieces of food in a short period of time. For this we need to add some signals: <code>snake_adding_new_segment(type)</code>, <code>snake_added_new_segment(type)</code>, <code>snake_added_initial_segments</code> and use them when makes sense. Now we need to add the following:</p>
<pre><code class="language-gdscript">var body_segment_stack: Array
var tail_segment: PathFollow2D
# didn't konw how to name this, basically holds the current path lenght
# whenever the add body segment, and we use this stack to add body parts
var body_segment_queue: Array
</code></pre>
<p>As well as updating <code>_add_initial_segment</code> with the following so it adds the new segment on the specific variable:</p>
<pre><code class="language-gdscript">if _temp_body_segment.TYPE == "body":
body_segment_stack.append(_temp_body_segment)
else:
tail_segment = _temp_body_segment
</code></pre>
<p>Now that it’s just a matter of creating the segment queue whenever a new segment is needed, as well as adding each segment in a loop whenever we have items in the queue and it’s a good distance to place the segment on. These two things can be achieved with the following code:</p>
<pre><code class="language-gdscript"># this will be called in _physics_process
func _add_new_segment() -> void:
var _path_length_threshold: float = body_segment_queue[0] + Global.SNAKE_SEGMENT_SIZE
if path.curve.get_baked_length() >= _path_length_threshold:
var _removed_from_queue: float = body_segment_queue.pop_front()
var _temp_body_segment: PathFollow2D = BODY_SEGMENT_NP.instance()
var _new_body_offset: float = body_segment_stack.back().offset - Global.SNAKE_SEGMENT_SIZE
_temp_body_segment.offset = _new_body_offset
body_segment_stack.append(_temp_body_segment)
path.add_child(_temp_body_segment)
tail_segment.offset = body_segment_stack.back().offset - Global.SNAKE_SEGMENT_SIZE
current_body_segments += 1
func _add_segment_to_queue() -> void:
# need to have the queues in a fixed separation, else if the eating functionality
# gets spammed, all next bodyparts will be spawned almost at the same spot
if body_segment_queue.size() == 0:
body_segment_queue.append(path.curve.get_baked_length())
else:
body_segment_queue.append(body_segment_queue.back() + Global.SNAKE_SEGMENT_SIZE)
</code></pre>
<p>With everything implemented and connected accordingly then we can add segments on demand (for testing I’m adding with a key press), it should look like this:</p>
<figure id="__yafg-figure-43">
<img alt="Snake - Basic movement with dynamic addition of new segments" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_basic_movement_with_dynamic_segments.gif" title="Snake - Basic movement with dynamic addition of new segments">
<figcaption>Snake - Basic movement with dynamic addition of new segments</figcaption>
</figure>
<p>For now, this should be enough, I’ll add more stuff as needed as I go. Last thing is that after finished testing that the movement felt ok, I just added a way to stop the snake whenever it collides with itself by using the following code (and the signal <code>snake_segment_body_entered(body)</code>) in a <code>main.gd</code> script that is the entry point for the game:</p>
<pre><code class="language-gdscript">func _snake_disabled(on_off: bool) -> void:
_snake.propagate_call("set_process", [on_off])
_snake.propagate_call("set_process_internal", [on_off])
_snake.propagate_call("set_physics_process", [on_off])
_snake.propagate_call("set_physics_process_internal", [on_off])
_snake.propagate_call("set_process_input", [on_off])
</code></pre>
<p>Which will stop the snake node and all children.</p>
<h3 id="fix-on-body-segments-following-head">Fix on body segments following head<a class="headerlink" href="#fix-on-body-segments-following-head" title="Permanent link">¶</a></h3>
<p>After a while of testing and developing, I noticed that sometimes the head “detaches” from the body when a lot of rotations happen (moving the snake left or right), because of how imprecise the <em>Curve2D</em> is. To do this I just send a signal (<code>snake_rotated</code>) whenever the snake rotates and make a small correction (in <code>generic_segment.gd</code>):</p>
<pre><code class="language-gdscript">func _on_snake_rotated() -> void:
offset -= 0.75 * Global.SNAKE_SPEED * pow(get_physics_process_delta_time(), 2)
</code></pre>
<p>This is completely random, I tweaked it manually after a lot of iterations.</p>
<h2 id="the-food">The food<a class="headerlink" href="#the-food" title="Permanent link">¶</a></h2>
<p>For now I just decided to setup a simple system to see everything works fine. The idea is to make some kind of generic food node/scene and a “food manager” to spawn them, for now in totally random locations. For this I added the following signals: <code>food_placing_new_food(type)</code>, <code>food_placed_new_food(type)</code> and <code>food_eaten(type)</code>.</p>
<p>First thing is creating the <code>Food.tscn</code> which is just an <em>Area2D</em> with its necessary children with an attached script called <code>food.gd</code>. The script is really simple:</p>
<pre><code class="language-gdscript">class_name Food # needed to access Type enum outside of the script, this registers this script as a node
extends Area2D
enum Type {
APPLE
}
var _type_texture: Dictionary = {
Type.APPLE: preload("res://entities/food/sprites/apple.png")
}
export(Type) var TYPE
onready var _sprite: Sprite = $Sprite
func _ready():
connect("body_entered", self, "_on_body_entered")
_sprite.texture = _type_texture[TYPE]
func _on_body_entered(body: Node) -> void:
Event.emit_signal("food_eaten", TYPE)
queue_free()
</code></pre>
<p>Then this <code>food_eaten</code> signal is received in <code>snake.gd</code> to add a new segment to the queue.</p>
<p>Finally, for the food manager I just created a <code>FoodManager.tscn</code> with a <em>Node2D</em> with an attached script called <code>food_manager.gd</code>. To get a random position:</p>
<pre><code class="language-gdscript">func _get_random_pos() -> Vector2:
var screen_size: Vector2 = get_viewport().get_visible_rect().size
var temp_x: float = randf() * screen_size.x - screen_size.x / 2.0
var temp_y: float = randf() * screen_size.y - screen_size.y / 2.0
return Vector2(temp_x, temp_y)
</code></pre>
<p>Which gets the job done, but later I’ll have to add a way to check that the position is valid. And to actually place the food:</p>
<pre><code class="language-gdscript">func _place_new_food() -> void:
var food: Area2D = FOOD.instance()
var position: Vector2 = _get_random_pos()
food.global_position = position
add_child(food)
</code></pre>
<p>And this is used in <code>_process</code> to place new food whenever needed. For now I added a condition to add food until 10 pieces are in place, and keep adding whenever the food is is lower than 10. After setting everything up, this is the result:</p>
<figure id="__yafg-figure-44">
<img alt="Snake - Food basic interaction" src="https://static.luevano.xyz/images/g/gogodot_jam3/snake_food_basic_interaction.gif" title="Snake - Food basic interaction">
<figcaption>Snake - Food basic interaction</figcaption>
</figure>
<h2 id="za-warudo-the-world">Za warudo! (The world)<a class="headerlink" href="#za-warudo-the-world" title="Permanent link">¶</a></h2>
<p>It just happend that I saw a video to create random maps by using a method called <a href="https://www.mit.edu/~kardar/teaching/projects/chemotaxis(AndreaSchmidt)/random.htm">random walks</a>, this video was made by <a href="https://www.youtube.com/c/NADLABS">NAD LABS</a>: <a href="https://www.youtube.com/watch?v=ppP2Doq3p7s">Nuclear Throne Like Map Generation In Godot</a>. It’s a pretty simple but powerful script, he provided the source code from which I based my random walker, just tweaked a few things and added others. Some of the maps than can be generated with this method (already aded some random sprites):</p>
<figure id="__yafg-figure-45">
<img alt="World map generator - Random map 1" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_1.png" title="World map generator - Random map 1">
<figcaption>World map generator - Random map 1</figcaption>
</figure>
<figure id="__yafg-figure-46">
<img alt="World map generator - Random map 2" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_2.png" title="World map generator - Random map 2">
<figcaption>World map generator - Random map 2</figcaption>
</figure>
<figure id="__yafg-figure-47">
<img alt="World map generator - Random map 3" src="https://static.luevano.xyz/images/g/gogodot_jam3/world_generator_3.png" title="World map generator - Random map 3">
<figcaption>World map generator - Random map 3</figcaption>
</figure>
<p>It started with just black and white tiles, but I ended up adding some sprites as it was really harsh to the eyes. My implementation is basically the same as <em>NAD LABS</em>‘ with few changes, most importantly: I separated the generation in 2 diferent tilemaps (floor and wall) to have better control as well as wrapped everything in a single scene with a “main” script with the following important functions:</p>
<pre><code class="language-gdscript">func get_valid_map_coords() -> Array:
var safe_area: Array = walker_head.get_cells_around()
var cells_used: Array = ground_tilemap.get_used_cells()
for location in safe_area:
cells_used.erase(location)
return cells_used
func get_centered_world_position(location: Vector2) -> Vector2:
return ground_tilemap.map_to_world(location) + Vector2.ONE * Global.TILE_SIZE / 2.0
</code></pre>
<p>Where <code>get_cells_around</code> is just a function that gets the safe cells around the origin. And this <code>get_valid_map_coords</code> just returns used cells minus the safe cells, to place food. <code>get_centered_world_position</code> is so we can center the food in the tiles.</p>
<p>Some signals I used for the world gen: <code>world_gen_walker_started(id)</code>, <code>world_gen_walker_finished(id)</code>, <code>world_gen_walker_died(id)</code> and <code>world_gen_spawn_walker_unit(location)</code>.</p>
<h3 id="food-placement">Food placement<a class="headerlink" href="#food-placement" title="Permanent link">¶</a></h3>
<p>The last food algorithm doesn’t check anything related to the world, and thus the food could spawn in the walls and outside the map.</p>
<p>First thing is I generalized the food into a single script and added basic food and special food which inherit from base food. The most important stuff for the base food is to be able to set all necessary properties at first:</p>
<pre><code class="language-gdscript">func update_texture() -> void:
_sprite.texture = texture[properties["type"]]
func set_properties(pos: Vector2, loc: Vector2, special: bool, type: int, points: int=1, special_points: int=1, ttl: float = -1.0) -> void:
properties["global_position"] = pos
global_position = pos
properties["location"] = loc
properties["special"] = special
properties["type"] = type
properties["points"] = points
properties["special_points"] = special_points
properties["ttl"] = ttl
if properties["ttl"] != -1.0:
timer.wait_time = properties["ttl"]
timer.start()
</code></pre>
<p>Where the <code>update_texture</code> needs to be a separate function, because we need to create the food first, set properties, add as a child and then update the sprite; we also need to keep track of the global position, location (in tilemap coordinates) and identifiers for the type of food.</p>
<p>Then basic/special food just extend base food, define a <code>Type</code> enum and preloads the necessary textures, for example:</p>
<pre><code class="language-gdscript">enum Type {
APPLE,
BANANA,
RAT
}
func _ready():
texture[Type.APPLE] = preload("res://entities/food/sprites/apple.png")
texture[Type.BANANA] = preload("res://entities/food/sprites/banana.png")
texture[Type.RAT] = preload("res://entities/food/sprites/rat.png")
</code></pre>
<p>Now, some of the most important change to <code>food_manager.gd</code> is to get an actual random valid position:</p>
<pre><code class="language-gdscript">func _get_random_pos() -> Array:
var found_valid_loc: bool = false
var index: int
var location: Vector2
while not found_valid_loc:
index = randi() % possible_food_locations.size()
location = possible_food_locations[index]
if current_basic_food.find(location) == -1 and current_special_food.find(location) == -1:
found_valid_loc = true
return [world_generator.get_centered_world_position(location), location]
</code></pre>
<p>Other than that, there are some differences between placing normal and special food (specially the signal they send, and if an extra “special points” property is set). Some of the signals that I used that might be important: <code>food_placing_new_food(type)</code>, <code>food_placed_new_food(type, location)</code> and <code>food_eaten(type, location)</code>.</p>
<h2 id="stats-clas-and-loadingsaving-data">Stats clas and loading/saving data<a class="headerlink" href="#stats-clas-and-loadingsaving-data" title="Permanent link">¶</a></h2>
<p>I got the idea of saving the current stats (points, max body segments, etc.) in a separate <code>Stats</code> class for easier load/save data. This option I went with didn’t work as I would liked it to work, as it was a pain in the ass to setup and each time a new property is added you have to manually setup the load/save helper functions… so not the best option. This option I used was json but saving a Node directly could work better or using resources (saving <code>tres</code> files).</p>
<h3 id="stats-class">Stats class<a class="headerlink" href="#stats-class" title="Permanent link">¶</a></h3>
<p>The <code>Stats</code> “class” is just a script that extends from <em>Node</em> called <code>stats.gd</code>. It needs to define the <code>class_name</code> as <code>Stats</code>. The main content:</p>
<pre><code class="language-gdscript"># main
var points: int = 0
var segments: int = 0
# track of trait points
var dash_points: int = 0
var slow_points: int = 0
var jump_points: int = 0
# times trait achieved
var dash_segments: int = 0
var slow_segments: int = 0
var jump_segments: int = 0
# trait properties
var dash_percentage: float = 0.0
var slow_percentage: float = 0.0
var jump_lenght: float = 0.0
# trait active
var trait_dash: bool = false
var trait_slow: bool = false
var trait_jump: bool = false
</code></pre>
<p>And with the ugliest functions:</p>
<pre><code class="language-gdscript">func get_stats() -> Dictionary:
return {
"points": points,
"segments": segments,
"dash_points": dash_points,
"dash_segments": dash_segments,
"dash_percentage": dash_percentage,
"slow_points": slow_points,
"slow_segments": slow_segments,
"slow_percentage": slow_percentage,
"jump_points": jump_points,
"jump_segments": jump_segments,
"jump_lenght": jump_lenght,
"trait_dash": trait_dash,
"trait_slow": trait_slow,
"trait_jump": trait_jump
}
func set_stats(stats: Dictionary) -> void:
points = stats["points"]
segments = stats["segments"]
dash_points = stats["dash_points"]
slow_points = stats["slow_points"]
jump_points = stats["jump_points"]
dash_segments = stats["dash_segments"]
slow_segments = stats["slow_segments"]
jump_segments = stats["jump_segments"]
dash_percentage = stats["dash_percentage"]
slow_percentage = stats["slow_percentage"]
jump_lenght = stats["jump_lenght"]
trait_dash = stats["trait_dash"]
trait_slow = stats["trait_slow"]
trait_jump = stats["trait_jump"]
</code></pre>
<p>And this is not scalable at all, but I had to do this at the end of the jam so no way of optimizing and/or doing it correctly, sadly.</p>
<h3 id="loadsave-data">Load/save data<a class="headerlink" href="#loadsave-data" title="Permanent link">¶</a></h3>
<p>The load/save function is pretty standard. It’s a singleton/autoload called <code>SavedData</code> with a script that extends from <em>Node</em> called <code>save_data.gd</code>:</p>
<pre><code class="language-gdscript">const DATA_PATH: String = "user://data.save"
var _stats: Stats
func _ready() -> void:
_load_data()
# called when setting "stats" and thus saving
func save_data(stats: Stats) -> void:
_stats = stats
var file: File = File.new()
file.open(DATA_PATH, File.WRITE)
file.store_line(to_json(_stats.get_stats()))
file.close()
func get_stats() -> Stats:
return _stats
func _load_data() -> void:
# create an empty file if not present to avoid error while loading settings
_handle_new_file()
var file = File.new()
file.open(DATA_PATH, File.READ)
_stats = Stats.new()
_stats.set_stats(parse_json(file.get_line()))
file.close()
func _handle_new_file() -> void:
var file: File = File.new()
if not file.file_exists(DATA_PATH):
file.open(DATA_PATH, File.WRITE)
_stats = Stats.new()
file.store_line(to_json(_stats.get_stats()))
file.close()
</code></pre>
<p>It uses json as the file format, but I might end up changing this in the future to something else more reliable and easier to use (<code>Stats</code> class related issues).</p>
<h2 id="scoring">Scoring<a class="headerlink" href="#scoring" title="Permanent link">¶</a></h2>
<p>For this I created a scoring mechanisms and just called it <code>ScoreManager</code> (<code>score_manager.gd</code>) which just basically listens to <code>food_eaten</code> signal and adds points accordingly to the current <em>Stats</em> object loaded. The main function is:</p>
<pre><code class="language-gdscript">func _on_food_eaten(properties: Dictionary) -> void:
var is_special: bool = properties["special"]
var type: int = properties["type"]
var points: int = properties["points"]
var special_points: int = properties["special_points"]
var location: Vector2 = properties["global_position"]
var amount_to_grow: int
var special_amount_to_grow: int
amount_to_grow = _process_points(points)
_spawn_added_score_text(points, location)
_spawn_added_segment_text(amount_to_grow)
if is_special:
special_amount_to_grow = _process_special_points(special_points, type)
# _spawn_added_score_text(points, location)
_spawn_added_special_segment_text(special_amount_to_grow, type)
_check_if_unlocked(type)
</code></pre>
<p>Where the most important function is:</p>
<pre><code class="language-gdscript">func _process_points(points: int) -> int:
var score_to_grow: int = (stats.segments + 1) * Global.POINTS_TO_GROW - stats.points
var amount_to_grow: int = 0
var growth_progress: int
stats.points += points
if points >= score_to_grow:
amount_to_grow += 1
points -= score_to_grow
# maybe be careful with this
amount_to_grow += points / Global.POINTS_TO_GROW
stats.segments += amount_to_grow
Event.emit_signal("snake_add_new_segment", amount_to_grow)
growth_progress = Global.POINTS_TO_GROW - ((stats.segments + 1) * Global.POINTS_TO_GROW - stats.points)
Event.emit_signal("snake_growth_progress", growth_progress)
return amount_to_grow
</code></pre>
<p>Which will add the necessary points to <code>Stats.points</code> and return the amount of new snake segments to grow. After this <code>_spawn_added_score_segment</code> and <code>_spawn_added_segment_text</code> just spawn a <em>Label</em> with the info on the points/segments gained; this is custom UI I created, nothing fancy.</p>
<p>Last thing is taht in <code>_process_points</code> there is a check at the end, where if the food eaten is “special” then a custom variation of the last 3 functions are executed. These are really similar, just specific to each kind of food.</p>
<p>This <code>ScoreManager</code> also handles the calculation for the <code>game_over</code> signal, to calculte progress, set necessary <code>Stats</code> values and save the data:</p>
<pre><code class="language-gdscript">func _on_game_over() -> void:
var max_stats: Stats = _get_max_stats()
SaveData.save_data(max_stats)
Event.emit_signal("display_stats", initial_stats, stats, mutation_stats)
func _get_max_stats() -> Stats:
var old_stats_dict: Dictionary = initial_stats.get_stats()
var new_stats_dict: Dictionary = stats.get_stats()
var max_stats: Stats = Stats.new()
var max_stats_dict: Dictionary = max_stats.get_stats()
var bool_stats: Array = [
"trait_dash",
"trait_slow",
"trait_jump"
]
for i in old_stats_dict:
if bool_stats.has(i):
max_stats_dict[i] = old_stats_dict[i] or new_stats_dict[i]
else:
max_stats_dict[i] = max(old_stats_dict[i], new_stats_dict[i])
max_stats.set_stats(max_stats_dict)
return max_stats
</code></pre>
<p>Then this sends a signal <code>display_stats</code> to activate UI elements that shows the progression.</p>
<p>Naturally, the saved <code>Stats</code> are loaded whenever needed. For example, for the <code>Snake</code>, we load the stats and setup any value needed from there (like a flag to know if any ability is enabled), and since we’re saving the new <code>Stats</code> at the end, then on restart we load the updated one.</p>
<h2 id="snake-redesigned-with-the-state-machine-pattern">Snake redesigned with the state machine pattern<a class="headerlink" href="#snake-redesigned-with-the-state-machine-pattern" title="Permanent link">¶</a></h2>
<p>I redesigned the snake code (the head, actually) to use the state machine pattern by following <a href="https://gdscript.com/solutions/godot-state-machine/">this guide</a> which is definitely a great guide, straight to the point and easy to implement.</p>
<p>Other than what is shown in the guide, I implemented some important functions in the <code>state_machine.gd</code> script itself, to be used by each of the states as needed:</p>
<pre><code class="language-gdscript">func rotate_on_input() -> void:
if Input.is_action_pressed("move_left"):
player.rotate_to(player.LEFT)
if Input.is_action_pressed("move_right"):
player.rotate_to(player.RIGHT)
func slow_down_on_collisions(speed_backup: float):
if player.get_last_slide_collision():
Global.SNAKE_SPEED = player.velocity.length()
else:
Global.SNAKE_SPEED = speed_backup
func handle_slow_speeds() -> void:
if Global.SNAKE_SPEED <= Global.SNAKE_SPEED_BACKUP / 4.0:
Global.SNAKE_SPEED = Global.SNAKE_SPEED_BACKUP
Event.emit_signal("game_over")
</code></pre>
<p>And then in the <code>StateMachine</code>‘s <code>_process</code>:</p>
<pre><code class="language-gdscript">func _physics_process(delta: float) -> void:
# state specific code, move_and_slide is called here
if state.has_method("physics_process"):
state.physics_process(delta)
handle_slow_speeds()
player.handle_time_elapsed(delta)
</code></pre>
<p>And now it’s just a matter of implementing the necessary states. I used 4: <code>normal_stage.gd</code>, <code>slow_state.gd</code>, <code>dash_state.gd</code> and <code>jump_state.gd</code>.</p>
<p>The <code>normal_state.gd</code> contains what the original <code>head.gd</code> code contained:</p>
<pre><code class="language-gdscript">func physics_process(delta: float) -> void:
fsm.rotate_on_input()
fsm.player.velocity = fsm.player.direction * Global.SNAKE_SPEED
fsm.player.velocity = fsm.player.move_and_slide(fsm.player.velocity)
fsm.slow_down_on_collisions(Global.SNAKE_SPEED_BACKUP)
func input(event: InputEvent) -> void:
if fsm.player.can_dash and event.is_action_pressed("dash"):
exit("DashState")
if fsm.player.can_slow and event.is_action_pressed("slow"):
exit("SlowState")
if fsm.player.can_jump and event.is_action_pressed("jump"):
exit("JumpState")
</code></pre>
<p>Here, the <code>exit</code> method is basically to change to the next state. And lastly, I’m only gonna show the <code>dash_state.gd</code> as the other ones are pretty similar:</p>
<pre><code class="language-gdscript">func enter():
if fsm.DEBUG:
print("Got inside %s." % name)
Event.emit_signal("snake_started_dash")
Global.SNAKE_SPEED = Global.SNAKE_DASH_SPEED
yield(get_tree().create_timer(Global.SNAKE_DASH_TIME), "timeout")
exit()
func exit():
Event.emit_signal("snake_finished_dash")
Global.SNAKE_SPEED = Global.SNAKE_SPEED_BACKUP
fsm.back()
func physics_process(delta: float) -> void:
fsm.rotate_on_input()
fsm.player.velocity = fsm.player.direction * Global.SNAKE_SPEED
fsm.player.velocity = fsm.player.move_and_slide(fsm.player.velocity)
fsm.slow_down_on_collisions(Global.SNAKE_DASH_SPEED)
</code></pre>
<p>Where the important parts happen in the <code>enter</code> and <code>exit</code> functions. We need to change the <code>Global.SNAKE_SPEED</code> with the <code>Global.SNAKE_DASH_SPEED</code> on <code>start</code>and start the timer for how long should the dash last. And on the <code>exit</code> we reset the <code>Global.SNAKE_SPEED</code> back to normal. There is probably a better way of updating the <code>Global.SNAKE_SPEED</code> but this works just fine.</p>
<p>For the other ones is the same. Only difference with the <code>jump_state.gd</code> is that the collision from head to body is disabled, and no rotation is allowed (by not calling the <code>rotate_on_input</code> function).</p>
<h2 id="other-minor-stuff">Other minor stuff<a class="headerlink" href="#other-minor-stuff" title="Permanent link">¶</a></h2>
<p>Not as important but worth mentioning:</p>
<ul>
<li>Added restartability function.</li>
<li>Added signals for game control: <code>game_over</code> and <code>game_start</code>, but ended not using them.</li>
<li>Fixed issue where the <em>Curve2D</em> stayed the same even when restarting by just setting an empty curve on starting the node.</li>
<li>Added a debug mode for drawing of the <em>Curve2D</em> instead of always drawing.</li>
<li>Tweaked the tracking of the snake size.</li>
<li>Tweaked the food system to contain more attributes and use a base food node.</li>
<li>Added a HUD with mini snake sprites.</li>
<li>Added a HUD for growth progress on snake body segments and abilities.</li>
<li>Refactored the nodes to make it work with <code>change_scene_to</code>, and added a main menu.</li>
<li>Added GUI for dead screen, showing the progress.</li>
</ul>
<h2 id="final-notes">Final notes<a class="headerlink" href="#final-notes" title="Permanent link">¶</a></h2>
<p>I actually didn’t finish this game (as how I visualized it), but I got it in a <em>semi-playable</em> state which is good. My big learning during this jam is the time management that it requires to plan and design a game. I lost a lot of time trying to implement some mechanics because I was facing many issues, because of my lack of practice (which was expected) as well as trying to blog and create the necessary sprites myself. Next time I should just get an asset pack and do something with it, as well as keeping the scope of my game shorter.</p>
<p>For exporting and everything else, I went with what I did for my <a href="https://blog.luevano.xyz/g/flappybird_godot_devlog_1#final-notes-and-exporting">FlappyBird Godot clone: final notes and exporting</a></p>]]></content:encoded>
</item>
<item>
<title>Creating a FlappyBird clone in Godot 3.5 devlog 1</title>
<link>https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/g/flappybird_godot_devlog_1.html</guid>
<pubDate>Sun, 29 May 2022 03:38:43 GMT</pubDate>
<category>English</category>
<category>Gamedev</category>
<category>Gdscript</category>
<category>Godot</category>
<description>Since I'm starting to get more into gamedev stuff, I'll start blogging about it just to keep consistent. This shows as "devlog 1" just in case I want to include more parts for extra stuff.</description>
<content:encoded><![CDATA[<p>I just have a bit of experience with <em>Godot</em> and with gamedev in general, so I started with this game as it is pretty straight forward. On a high level the main characteristics of the game are:</p>
<ul>
<li>Literally just one sprite going up and down.</li>
<li>Constant horizontal move of the world/player.</li>
<li>If you go through the gap in the pipes you score a point.</li>
<li>If you touch the pipes, the ground or go past the “ceiling” you lose.</li>
</ul>
<p>The game was originally developed with <em>Godot 4.0 alpha 8</em>, but it didn’t support HTML5 (webassembly) export… so I backported to <em>Godot 3.5 rc1</em>.</p>
<p>Not going to specify all the details, only the needed parts and what could be confusing, as the source code is available and can be inspected; also this assumes minimal knowledge of <em>Godot</em> in general. Usually when I mention that a set/change of something it usually it’s a property and it can be found under the <em>Inspector</em> on the relevant node, unless stated otherwise; also, all scripts attached have the same name as the scenes, but in <em>snake_case</em> (scenes/nodes in <em>PascalCase</em>).</p>
<p>One thing to note, is that I started writing this when I finished the game, so it’s hard to go part by part, and it will be hard to test individual parts when going through this as everything is depending on each other. For the next devlog, I’ll do it as I go and it will include all the changes to the nodes/scripts as I was finding them, probably better idea and easier to follow.</p>
<p>The source code can be found at <a href="https://github.com/luevano/flappybird_godot/tree/godot-3.5">luevano/flappybird_godot#godot-3.5</a>, it also contains the exported versions for HTML5, Windows and Linux (be aware that the sound might be too high and I’m too lazy to make it configurable, it was the last thing I added), or you could also go to the <a href="https://lorentzeus.itch.io/flappybirdgodot">itch.io</a> page I setup where it’s playable in the browser:</p>
<p style="text-align:center"><iframe src="https://itch.io/embed/1551015?dark=true" width="208" height="167" frameborder="0"><a href="https://lorentzeus.itch.io/flappybirdgodot">FlappyBirdGodot by Lorentzeus</a></iframe></p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#initial-setup">Initial setup</a><ul>
<li><a href="#directory-structure">Directory structure</a></li>
<li><a href="#config">Config</a><ul>
<li><a href="#default-import-settings">Default import settings</a></li>
<li><a href="#general-settings">General settings</a></li>
<li><a href="#keybindings">Keybindings</a></li>
<li><a href="#layers">Layers</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#assets">Assets</a><ul>
<li><a href="#importing">Importing</a></li>
</ul>
</li>
<li><a href="#scenes">Scenes</a><ul>
<li><a href="#tilemaps">TileMaps</a><ul>
<li><a href="#default-ground-tiles">Default ground tiles</a></li>
</ul>
</li>
<li><a href="#player">Player</a></li>
<li><a href="#other">Other</a></li>
<li><a href="#game">Game</a></li>
<li><a href="#ui">UI</a><ul>
<li><a href="#fonts">Fonts</a></li>
<li><a href="#scene-setup">Scene setup</a></li>
</ul>
</li>
<li><a href="#main">Main</a></li>
</ul>
</li>
<li><a href="#scripting">Scripting</a><ul>
<li><a href="#player_1">Player</a></li>
<li><a href="#worlddetector">WorldDetector</a></li>
<li><a href="#worldtiles">WorldTiles</a><ul>
<li><a href="#groundtilemap">GroundTileMap</a></li>
<li><a href="#pipetilemap">PipeTileMap</a></li>
</ul>
</li>
<li><a href="#saved-data">Saved data</a></li>
<li><a href="#game_1">Game</a></li>
<li><a href="#ui_1">UI</a></li>
<li><a href="#main_1">Main</a></li>
</ul>
</li>
<li><a href="#final-notes-and-exporting">Final notes and exporting</a><ul>
<li><a href="#preparing-the-files">Preparing the files</a></li>
<li><a href="#exporting">Exporting</a></li>
</ul>
</li>
</ul>
</div>
<h2 id="initial-setup">Initial setup<a class="headerlink" href="#initial-setup" title="Permanent link">¶</a></h2>
<h3 id="directory-structure">Directory structure<a class="headerlink" href="#directory-structure" title="Permanent link">¶</a></h3>
<p>I’m basically going with what I wrote on <a href="https://blog.luevano.xyz/g/godot_project_structure.html">Godot project structure</a> recently, and probably with minor changes depending on the situation.</p>
<h3 id="config">Config<a class="headerlink" href="#config" title="Permanent link">¶</a></h3>
<h4 id="default-import-settings">Default import settings<a class="headerlink" href="#default-import-settings" title="Permanent link">¶</a></h4>
<p>Since this is just pixel art, the importing settings for textures needs to be adjusted so the sprites don’t look blurry. Go to <em>Project -> Project settings… -> Import defaults</em> and on the drop down select <code>Texture</code>, untick everything and make sure <em>Compress/Mode</em> is set to <code>Lossless</code>.</p>
<figure id="__yafg-figure-11">
<img alt="Project settings - Import defaults - Texture settings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_import_texture.png" title="Project settings - Import defaults - Texture settings">
<figcaption>Project settings - Import defaults - Texture settings</figcaption>
</figure>
<h4 id="general-settings">General settings<a class="headerlink" href="#general-settings" title="Permanent link">¶</a></h4>
<p>It’s also a good idea to setup some config variables project-wide. To do so, go to <em>Project -> Project settings… -> General</em>, select <em>Application/config</em> and add a new property (there is a text box at the top of the project settings window) for game scale: <code>application/config/game_scale</code> for the type use <code>float</code> and then click on add; configure the new property to <code>3.0</code>; On the same window, also add <code>application/config/version</code> as a <code>string</code>, and make it <code>1.0.0</code> (or whatever number you want).</p>
<figure id="__yafg-figure-12">
<img alt="Project settings - General - Game scale and version properties" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_config_properties.png" title="Project settings - General - Game scale and version properties">
<figcaption>Project settings - General - Game scale and version properties</figcaption>
</figure>
<p>For my personal preferences, also disable some of the <em>GDScript</em> debug warnings that are annoying, this is done at <em>Project -> Project settings… -> General</em>, select <em>Debug/GDScript</em> and toggle off <code>Unused arguments</code>, <code>Unused signal</code> and <code>Return value discarded</code>, and any other that might come up too often and don’t want to see.</p>
<figure id="__yafg-figure-13">
<img alt="Project settings - General - GDScript debug warnings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_debug_gdscript.png" title="Project settings - General - GDScript debug warnings">
<figcaption>Project settings - General - GDScript debug warnings</figcaption>
</figure>
<p>Finally, set the initial window size in <em>Project -> Project settings… -> General</em>, select <em>Display/Window</em> and set <em>Size/Width</em> and <em>Size/Height</em> to <code>600</code> and <code>800</code>, respectively. As well as the <em>Stretch/Mode</em> to <code>viewport</code> , and <em>Stretch/Aspect</em> to <code>keep</code>:</p>
<figure id="__yafg-figure-14">
<img alt="Project settings - General - Initial window size" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_window_settings.png" title="Project settings - General - Initial window size">
<figcaption>Project settings - General - Initial window size</figcaption>
</figure>
<h4 id="keybindings">Keybindings<a class="headerlink" href="#keybindings" title="Permanent link">¶</a></h4>
<p>I only used 3 actions (keybindings): jump, restart and toggle_debug (optional). To add custom keybindings (so that the <code>Input.something()</code> API can be used), go to <em>Project -> Project settings… -> Input Map</em> and on the text box write <code>jump</code> and click add, then it will be added to the list and it’s just a matter of clicking the <code>+</code> sign to add a <em>Physical key</em>, press any key you want to be used to jump and click ok. Do the same for the rest of the actions.</p>
<figure id="__yafg-figure-15">
<img alt="Project settings - Input Map - Adding necessary keybindings" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_input_map.png" title="Project settings - Input Map - Adding necessary keybindings">
<figcaption>Project settings - Input Map - Adding necessary keybindings</figcaption>
</figure>
<h4 id="layers">Layers<a class="headerlink" href="#layers" title="Permanent link">¶</a></h4>
<p>Finally, rename the physics layers so we don’t lose track of which layer is which. Go to <em>Project -> Layer Names -> 2d Physics</em> and change the first 5 layer names to (in order): <code>player</code>, <code>ground</code>, <code>pipe</code>, <code>ceiling</code> and <code>score</code>.</p>
<figure id="__yafg-figure-16">
<img alt="Project settings - Layer Names - 2D Physics" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_layer_names_2d_physics.png" title="Project settings - Layer Names - 2D Physics">
<figcaption>Project settings - Layer Names - 2D Physics</figcaption>
</figure>
<h2 id="assets">Assets<a class="headerlink" href="#assets" title="Permanent link">¶</a></h2>
<p>For the assets I found out about a pack that contains just what I need: <a href="https://megacrash.itch.io/flappy-bird-assets">flappy-bird-assets</a> by <a href="https://megacrash.itch.io/">MegaCrash</a>; I just did some minor modifications on the naming of the files. For the font I used <a href="https://poppyworks.itch.io/silver">Silver</a>, and for the sound the resources from <a href="https://github.com/meeq/FlappyBird-N64">FlappyBird-N64</a> (which seems to be taken from <a href="https://www.101soundboards.com/boards/10178-flappy-bird-sounds">101soundboards.com</a> which the orignal copyright holder is <a href="https://dotgears.com/">.Gears</a> anyways).</p>
<h3 id="importing">Importing<a class="headerlink" href="#importing" title="Permanent link">¶</a></h3>
<p>Create the necessary directories to hold the respective assets and it’s just a matter of dragging and dropping, I used directories: <code>res://entities/actors/player/sprites/</code>, <code>res://fonts/</code>, <code>res://levels/world/background/sprites/</code>, <code>res://levels/world/ground/sprites/</code>, <code>res://levels/world/pipe/sprites/</code>, <code>res://sfx/</code>. For the player sprites, the
<em>FileSystem</em> window looks like this (<code>entities/actor</code> directories are really not necessary):</p>
<figure id="__yafg-figure-17">
<img alt="FileSystem - Player sprite imports" src="https://static.luevano.xyz/images/g/flappybird_godot/player_sprite_imports.png" title="FileSystem - Player sprite imports">
<figcaption>FileSystem - Player sprite imports</figcaption>
</figure>
<p>It should look similar for other directories, except maybe for the file extensions. For example, for the sfx:</p>
<figure id="__yafg-figure-18">
<img alt="FileSystem - SFX imports" src="https://static.luevano.xyz/images/g/flappybird_godot/sfx_imports.png" title="FileSystem - SFX imports">
<figcaption>FileSystem - SFX imports</figcaption>
</figure>
<h2 id="scenes">Scenes<a class="headerlink" href="#scenes" title="Permanent link">¶</a></h2>
<p>Now it’s time to actually create the game, by creating the basic scenes that will make up the game. The hardest part and the most confusing is going to be the <em>TileMaps</em>, so that goes first.</p>
<h3 id="tilemaps">TileMaps<a class="headerlink" href="#tilemaps" title="Permanent link">¶</a></h3>
<p>I’m using a scene called <code>WorldTiles</code> with a <em>Node2D</em> node as root called the same. With 2 different <em>TileMap</em> nodes as children named <code>GroundTileMap</code> and <code>PipeTileMap</code> (these are their own scene); yes 2 different <em>TileMaps</em> because we need 2 different physics colliders (in <em>Godot 4.0</em> you can have a single <em>TileMap</em> with different physics colliders in it). Each node has its own script. It should look something like this:</p>
<figure id="__yafg-figure-19">
<img alt="Scene - WorldTiles (TileMaps)" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_world_tiles.png" title="Scene - WorldTiles (TileMaps)">
<figcaption>Scene - WorldTiles (TileMaps)</figcaption>
</figure>
<p>I used the following directory structure:</p>
<figure id="__yafg-figure-20">
<img alt="Scene - WorldTiles - Directory structure" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_world_tiles_directory_structure.png" title="Scene - WorldTiles - Directory structure">
<figcaption>Scene - WorldTiles - Directory structure</figcaption>
</figure>
<p>To configure the <code>GroundTileMap</code>, select the node and click on <code>(empty)</code> on the <em>TileMap/Tile set</em> property and then click on <code>New TileSet</code>, then click where the <code>(empty)</code> used to be, a new window should open on the bottom:</p>
<figure id="__yafg-figure-21">
<img alt="TileSet - Configuration window" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_config_window.png" title="TileSet - Configuration window">
<figcaption>TileSet - Configuration window</figcaption>
</figure>
<p>Click on the plus on the bottom left and you can now select the specific tile set to use. Now click on the yellow <code>+ New Single Tile</code>, activate the grid and select any of the tiles. Should look like this:</p>
<figure id="__yafg-figure-22">
<img alt="TileSet - New single tile" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_new_single_tile.png" title="TileSet - New single tile">
<figcaption>TileSet - New single tile</figcaption>
</figure>
<p>We need to do this because for some reason we can’t change the snap options before selecting a tile. After selecting a random tile, set up the <em>Snap Options/Step</em> (in the <em>Inspector</em>) and set it to <code>16x16</code> (or if using a different tile set, to it’s tile size):</p>
<figure id="__yafg-figure-23">
<img alt="TileSet - Tile - Step snap options" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_tile_step_snap_options.png" title="TileSet - Tile - Step snap options">
<figcaption>TileSet - Tile - Step snap options</figcaption>
</figure>
<p>Now you can select the actual single tile. Once selected click on <code>Collision</code>, use the rectangle tool and draw the rectangle corresponding to that tile’s collision:</p>
<figure id="__yafg-figure-24">
<img alt="TileSet - Tile - Selection and collision" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_tile_selection_collision.png" title="TileSet - Tile - Selection and collision">
<figcaption>TileSet - Tile - Selection and collision</figcaption>
</figure>
<p>Do the same for the other 3 tiles. If you select the <em>TileMap</em> itself again, it should look like this on the right (on default layout it’s on the left of the <em>Inspector</em>):</p>
<figure id="__yafg-figure-25">
<img alt="TileSet - Available tiles" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_available_tiles.png" title="TileSet - Available tiles">
<figcaption>TileSet - Available tiles</figcaption>
</figure>
<p>The ordering is important only for the “underground tile”, which is the filler ground, it should be at the end (index 3); if this is not the case, repeat the process (it’s possible to rearrange them but it’s hard to explain as it’s pretty weird).</p>
<p>At this point the tilemap doesn’t have any physics and the cell size is wrong. Select the <code>GroundTileMap</code>, set the <em>TileMap/Cell/Size</em> to <code>16x16</code>, the <em>TileMap/Collision/Layer</em> set to <code>bit 2</code> only (ground layer) and disable any <em>TileMap/Collision/Mask</em> bits. Should look something like this:</p>
<figure id="__yafg-figure-26">
<img alt="TileMap - Cell size and collision configuration" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_map_cell_collision_configuration.png" title="TileMap - Cell size and collision configuration">
<figcaption>TileMap - Cell size and collision configuration</figcaption>
</figure>
<p>Now it’s just a matter of repeating the same for the pipes (<code>PipeTileMap</code>), only difference is that when selecting the tiles you need to select 2 tiles, as the pipe is 2 tiles wide, or just set the <em>Snap Options/Step</em> to <code>32x16</code>, for example, just keep the cell size to <code>16x16</code>.</p>
<h4 id="default-ground-tiles">Default ground tiles<a class="headerlink" href="#default-ground-tiles" title="Permanent link">¶</a></h4>
<p>I added few default ground tiles to the scene, just for testing purposes but I left them there. These could be place programatically, but I was too lazy to change things. On the <code>WorldTiles</code> scene, while selecting the <code>GroundTileMap</code>, you can select the tiles you want to paint with, and left click in the grid to paint with the selected tile. Need to place tiles from <code>(-8, 7)</code> to <code>(10, 7)</code> as well as the tile below with the filler ground (the tile position/coordinates show at the bottom left, refer to the image below):</p>
<figure id="__yafg-figure-27">
<img alt="Scene - WorldTiles - Default ground tiles" src="https://static.luevano.xyz/images/g/flappybird_godot/world_tiles_default_tiles.png" title="Scene - WorldTiles - Default ground tiles">
<figcaption>Scene - WorldTiles - Default ground tiles</figcaption>
</figure>
<h3 id="player">Player<a class="headerlink" href="#player" title="Permanent link">¶</a></h3>
<p>On a new scene called <code>Player</code> with a <em>KinematicBody2D</em> node named <code>Player</code> as the root of the scene, then for the children: <em>AnimatedSprite</em> as <code>Sprite</code>, <em>CollisionShape2D</em> as <code>Collision</code> (with a circle shape) and 3 <em>AudioStreamPlayers</em> for <code>JumpSound</code>, <code>DeadSound</code> and <code>HitSound</code>. Not sure if it’s a good practice to have the audio here, since I did that at the end, pretty lazy. Then, attach a script to the <code>Player</code> node and then it should look like this:</p>
<figure id="__yafg-figure-28">
<img alt="Scene - Player - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_node_setup.png" title="Scene - Player - Node setup">
<figcaption>Scene - Player - Node setup</figcaption>
</figure>
<p>Select the <code>Player</code> node and set the <em>CollisionShape2D/Collision/Layer</em> to <code>1</code> and the <em>CollisionObject2D/Collision/Mask</em> to <code>2</code> and <code>3</code> (ground and pipe).</p>
<p>For the <code>Sprite</code> node, when selecting it click on the <code>(empty)</code> for the <em>AnimatedSprite/Frames</em> property and click <code>New SpriteFrames</code>, click again where the <code>(empty)</code> used to be and ane window should open on the bottom:</p>
<figure id="__yafg-figure-29">
<img alt="Scene - Player - SpriteFrames window" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_spriteframes_window.png" title="Scene - Player - SpriteFrames window">
<figcaption>Scene - Player - SpriteFrames window</figcaption>
</figure>
<p>Right off the bat, set the <code>Speed</code> to <code>10 FPS</code> (bottom left) and rename <code>default</code> to <code>bird_1</code>. With the <code>bird_1</code> selected, click on the <code>Add frames from a Sprite Sheet</code>, which is the second button under <code>Animation Frames:</code> which looks has an icon of a small grid (next to the folder icon), a new window will popup where you need to select the respective sprite sheet to use and configure it for importing. On the <code>Select Frames</code> window, change the <code>Vertical</code> to <code>1</code>, and then select all 4 frames (<em>Ctrl + Scroll</em> wheel to zoom in):</p>
<figure id="__yafg-figure-30">
<img alt="Scene - Player - Sprite sheet importer" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_sprite_sheet_importer.png" title="Scene - Player - Sprite sheet importer">
<figcaption>Scene - Player - Sprite sheet importer</figcaption>
</figure>
<p>After that, the <em>SpriteFrames</em> window should look like this:</p>
<figure id="__yafg-figure-31">
<img alt="Scene - Player - SpriteFrames window with sprite sheet configured" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_player_spriteframes_window_with_sprite_sheet.png" title="Scene - Player - SpriteFrames window with sprite sheet configured">
<figcaption>Scene - Player - SpriteFrames window with sprite sheet configured</figcaption>
</figure>
<p>Finally, make sure the <code>Sprite</code> node has the <em>AnimatedSprite/Animation</em> is set to <code>bird_1</code> and that the <code>Collision</code> node is configured correctly for its size and position (I just have it as a radius of <code>7</code>). As well as dropping the SFX files into the corresponding <em>AudioStreamPlayer</em> (into the <em>AudioStreamPlayer/Stream</em> property).</p>
<h3 id="other">Other<a class="headerlink" href="#other" title="Permanent link">¶</a></h3>
<p>These are really simple scenes that don’t require much setup:</p>
<ul>
<li><code>CeilingDetector</code>: just an <em>Area2D</em> node with a <em>CollisionShape2D</em> in the form of a rectangle (<em>CollisionShape2D/Shape/extents</em> to <code>(120, 10)</code>), stretched horizontally so it fits the whole screen. <em>CollisionObject2D/Collision/Layer</em> set to <code>bit 4</code> (ceiling) and <em>CollisionObject2D/Collision/Mask</em> set to <code>bit 1</code> (player).</li>
<li><code>ScoreDetector</code>: similar to the <code>CeilingDetector</code>, but vertical (<em>CollisionShape2D/Shape/extents</em> to <code>(2.5, 128)</code>) and <em>CollisionObject2D/Collision/Layer</em> set to <code>bit 1</code> (player).</li>
<li><code>WorldDetector</code>: <em>Node2D</em> with a script attached, and 3 <em>RayCast2D</em> as children:<ul>
<li><code>NewTile</code>: <em>Raycast2D/Enabled</em> to true (checked), <em>Raycast2D/Cast To</em> <code>(0, 400)</code>, <em>Raycast2D/Collision Mask</em> to <code>bit 2</code> (ground) and <em>Node2D/Transform/Position</em> to <code>(152, -200)</code></li>
<li><code>OldTile</code>: same as “NewTile”, except for the <em>Node2D/Transform/Position</em>, set it to <code>(-152, -200)</code>.</li>
<li><code>OldPipe</code>: same as “OldTile”, except for the <em>Raycast2D/Collision Mask</em>, set it to <code>bit 3</code> (pipe).</li>
</ul>
</li>
</ul>
<h3 id="game">Game<a class="headerlink" href="#game" title="Permanent link">¶</a></h3>
<p>This is the actual <code>Game</code> scene that holds all the playable stuff, here we will drop in all the previous scenes; the root node is a <em>Node2D</em> and also has an attached script. Also need to add 2 additional <em>AudioStreamPlayers</em> for the “start” and “score” sounds, as well as a <em>Sprite</em> for the background (<em>Sprite/Offset/Offset</em> set to <code>(0, 10)</code>) and a <em>Camera2D</em> (<em>Camera2D/Current</em> set to true (checked)). It should look something like this:</p>
<figure id="__yafg-figure-32">
<img alt="Scene - Game - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_game_node_setup.png" title="Scene - Game - Node setup">
<figcaption>Scene - Game - Node setup</figcaption>
</figure>
<p>The scene viewport should look something like the following:</p>
<figure id="__yafg-figure-33">
<img alt="Scene - Game - Viewport" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_game_viewport.png" title="Scene - Game - Viewport">
<figcaption>Scene - Game - Viewport</figcaption>
</figure>
<h3 id="ui">UI<a class="headerlink" href="#ui" title="Permanent link">¶</a></h3>
<h4 id="fonts">Fonts<a class="headerlink" href="#fonts" title="Permanent link">¶</a></h4>
<p>We need some font <code>Resources</code> to style the <em>Label</em> fonts. Under the <em>FileSystem</em> window, right click on the fonts directory (create one if needed) and click on <code>New Resource...</code> and select <em>DynamicFontData</em>, save it in the “fonts” directory as <code>SilverDynamicFontData.tres</code> (<code>Silver</code> as it is the font I’m using) then double click the just created resource and set the <em>DynamicFontData/Font Path</em> to the actual <code>Silver.ttf</code> font (or whatever you want).</p>
<p>Then create a new resource and this time select <em>DynamicFont</em>, name it <code>SilverDynamicFont.tres</code>, then double click to edit and add the <code>SilverDynamicFontData.tres</code> to the <em>DynamicFont/Font/Font Data</em> property (and I personally toggled off the <em>DynamicFont/Font/Antialiased</em> property), now just set the <em>DynamicFont/Settings/(Size, Outline Size, Outline Color)</em> to <code>32</code>, <code>1</code> and <code>black</code>, respectively (or any other values you want). It should look something like this:</p>
<figure id="__yafg-figure-34">
<img alt="Resource - DynamicFont - Default font" src="https://static.luevano.xyz/images/g/flappybird_godot/resource_dynamic_font.png" title="Resource - DynamicFont - Default font">
<figcaption>Resource - DynamicFont - Default font</figcaption>
</figure>
<p>Do the same for another <em>DynamicFont</em> which will be used for the score label, named <code>SilverScoreDynamicFont.tres</code>. Only changes are <em>Dynamic/Settings/(Size, Outline Size)</em> which are set to <code>128</code> and <code>2</code>, respectively. The final files for the fonts should look something like this:</p>
<figure id="__yafg-figure-35">
<img alt="Resource - Dynamicfont - Directory structure" src="https://static.luevano.xyz/images/g/flappybird_godot/resource_dynamic_font_directory_structure.png" title="Resource - Dynamicfont - Directory structure">
<figcaption>Resource - Dynamicfont - Directory structure</figcaption>
</figure>
<h4 id="scene-setup">Scene setup<a class="headerlink" href="#scene-setup" title="Permanent link">¶</a></h4>
<p>This has a bunch of nested nodes, so I’ll try to be concise here. The root node is a <em>CanvasLayer</em> named <code>UI</code> with its own script attached, and for the children:</p>
<ul>
<li><code>MarginContainer</code>: <em>MarginContainer</em> with <em>Control/Margin/(Left, Top)</em> set to <code>10</code> and <em>Control/Margin/(Right, Bottom)</em> set to <code>-10</code>.<ul>
<li><code>InfoContainer</code>: <em>VBoxContainer</em> with <em>Control/Theme Overrides/Constants/Separation</em> set to <code>250</code>.<ul>
<li><code>ScoreContainer</code>: <em>VBoxContainer</em>.<ul>
<li><code>Score</code>: <em>Label</em> with <em>Label/Align</em> set to <code>Center</code>, <em>Control/Theme Overrides/Fonts/Font</em> to the <code>SilverScoreDynamicFont.tres</code>, if needed adjust the <em>DynamicFont</em> settings.</li>
<li><code>HighScore</code>: same as <code>Score</code>, escept for the <em>Control/Theme Overrides/Fonts/Font</em> which is set to <code>SilverDynamicFont.tres</code>.</li>
</ul>
</li>
<li><code>StartGame</code>: Same as <code>HighScore</code>.</li>
</ul>
</li>
<li><code>DebugContainer</code>: <em>VBoxContainer</em>.<ul>
<li><code>FPS</code>: <em>Label</em>.</li>
</ul>
</li>
<li><code>VersionContainer</code>: <em>VBoxContainer</em> with <em>BoxContainer/Alignment</em> set to <code>Begin</code>.<ul>
<li><code>Version</code>: <em>Label</em> with <em>Label/Align</em> set to <code>Right</code>.</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>The scene ends up looking like this:</p>
<figure id="__yafg-figure-36">
<img alt="Scene - UI - Node setup" src="https://static.luevano.xyz/images/g/flappybird_godot/scene_ui.png" title="Scene - UI - Node setup">
<figcaption>Scene - UI - Node setup</figcaption>
</figure>
<h3 id="main">Main<a class="headerlink" href="#main" title="Permanent link">¶</a></h3>
<p>This is the final scene where we connect the <code>Game</code> and the <code>UI</code>. It’s made of a <em>Node2D</em> with it’s own script attached and an instance of <code>Game</code> and <code>UI</code> as it’s children.</p>
<p>This is a good time to set the default scene when we run the game by going to <em>Project -> Project settings… -> General</em> and in <em>Application/Run</em> set the <em>Main Scene</em> to the <code>Main.tscn</code> scene.</p>
<h2 id="scripting">Scripting<a class="headerlink" href="#scripting" title="Permanent link">¶</a></h2>
<p>I’m going to keep this scripting part to the most basic code blocks, as it’s too much code, for a complete view you can head to the <a href="https://github.com/luevano/flappybird_godot/tree/godot-3.5">source code</a>.</p>
<p>As of now, the game itself doesn’t do anything if we hit play. The first thing to do so we have something going on is to do the minimal player scripting.</p>
<h3 id="player_1">Player<a class="headerlink" href="#player_1" title="Permanent link">¶</a></h3>
<p>The most basic code needed so the bird goes up and down is to just detect <code>jump</code> key presses and add a negative jump velocity so it goes up (<code>y</code> coordinate is reversed in godot…), we also check the velocity sign of the <code>y</code> coordinate to decide if the animation is playing or not.</p>
<pre><code class="language-gdscript">class_name Player
extends KinematicBody2D
export(float, 1.0, 1000.0, 1.0) var JUMP_VELOCITY: float = 380.0
onready var sprite: AnimatedSprite = $Sprite
var gravity: float = 10 * ProjectSettings.get_setting("physics/2d/default_gravity")
var velocity: Vector2 = Vector2.ZERO
func _physics_process(delta: float) -> void:
velocity.y += gravity * delta
if Input.is_action_just_pressed("jump"):
velocity.y = -JUMP_VELOCITY
if velocity.y < 0.0:
sprite.play()
else:
sprite.stop()
velocity = move_and_slide(velocity)
</code></pre>
<p>You can play it now and you should be able to jump up and down, and the bird should stop on the ground (although you can keep jumping). One thing to notice is that when doing <code>sprite.stop()</code> it stays on the last frame, we can fix that using the code below (and then change <code>sprite.stop()</code> for <code>_stop_sprite()</code>):</p>
<pre><code class="language-gdscript">func _stop_sprite() -> void:
if sprite.playing:
sprite.stop()
if sprite.frame != 0:
sprite.frame = 0
</code></pre>
<p>Where we just check that the last frame has to be the frame 0.</p>
<p>Now just a matter of adding other needed code for moving horizontally, add sound by getting a reference to the <em>AudioStreamPlayers</em> and doing <code>sound.play()</code> when needed, as well as handling death scenarios by adding a <code>signal died</code> at the beginning of the script and handle any type of death scenario using the below function:</p>
<pre><code class="language-gdscript">func _emit_player_died() -> void:
# bit 2 corresponds to pipe (starts from 0)
set_collision_mask_bit(2, false)
dead = true
SPEED = 0.0
emit_signal("died")
# play the sounds after, because yield will take a bit of time,
# this way the camera stops when the player "dies"
velocity.y = -DEATH_JUMP_VELOCITY
velocity = move_and_slide(velocity)
hit_sound.play()
yield(hit_sound, "finished")
dead_sound.play()
</code></pre>
<p>Finally need to add the actual checks for when the player dies (like collision with ground or pipe) as well as a function that listens to a signal for when the player goes to the ceiling.</p>
<h3 id="worlddetector">WorldDetector<a class="headerlink" href="#worlddetector" title="Permanent link">¶</a></h3>
<p>The code is pretty simple, we just need a way of detecting if we ran out of ground and send a signal, as well as sending as signal when we start detecting ground/pipes behind us (to remove it) because the world is being generated as we move. The most basic functions needed are:</p>
<pre><code class="language-gdscript">func _was_colliding(detector: RayCast2D, flag: bool, signal_name: String) -> bool:
if detector.is_colliding():
return true
if flag:
emit_signal(signal_name)
return false
return true
func _now_colliding(detector: RayCast2D, flag: bool, signal_name: String) -> bool:
if detector.is_colliding():
if not flag:
emit_signal(signal_name)
return true
return false
</code></pre>
<p>We need to keep track of 3 “flags”: <code>ground_was_colliding</code>, <code>ground_now_colliding</code> and <code>pipe_now_colliding</code> (and their respective signals), which are going to be used to do the checks inside <code>_physics_process</code>. For example for checking for new ground: <code>ground_now_colliding = _now_colliding(old_ground, ground_now_colliding, "ground_started_colliding")</code>.</p>
<h3 id="worldtiles">WorldTiles<a class="headerlink" href="#worldtiles" title="Permanent link">¶</a></h3>
<p>This script is what handles the <code>GroundTileMap</code> as well as the <code>PipeTileMap</code> and just basically functions as a “Signal bus” connecting a bunch of signals from the <code>WorldDetector</code> with the <em>TileMaps</em> and just tracking how many pipes have been placed:</p>
<pre><code class="language-gdscript">export(int, 2, 20, 2) var PIPE_SEP: int = 6
var tiles_since_last_pipe: int = PIPE_SEP - 1
func _on_WorldDetector_ground_stopped_colliding() -> void:
emit_signal("place_ground")
tiles_since_last_pipe += 1
if tiles_since_last_pipe == PIPE_SEP:
emit_signal("place_pipe")
tiles_since_last_pipe = 0
func _on_WorldDetector_ground_started_colliding() -> void:
emit_signal("remove_ground")
func _on_WorldDetector_pipe_started_colliding() -> void:
emit_signal("remove_pipe")
</code></pre>
<h4 id="groundtilemap">GroundTileMap<a class="headerlink" href="#groundtilemap" title="Permanent link">¶</a></h4>
<p>This is the node that actually places the ground tiles upong receiving a signal. In general, what you want is to keep track of the newest tile that you need to place (empty spot) as well as the last tile that is in the tilemap (technically the first one if you count from left to right). I was experimenting with <code>enum</code>s so I used them to define the possible <code>Ground</code> tiles:</p>
<pre><code class="language-gdscript">enum Ground {
TILE_1,
TILE_2,
TILE_3,
TILE_DOWN_1,
}
</code></pre>
<p>This way you can just select the tile by doing <code>Ground.TILE_1</code>, which will correspond to the <code>int</code> value of <code>0</code>. So most of the code is just:</p>
<pre><code class="language-gdscript"># old_tile is the actual first tile, whereas the new_tile_position
# is the the next empty tile; these also correspond to the top tile
const _ground_level: int = 7
const _initial_old_tile_x: int = -8
const _initial_new_tile_x: int = 11
var old_tile_position: Vector2 = Vector2(_initial_old_tile_x, _ground_level)
var new_tile_position: Vector2 = Vector2(_initial_new_tile_x, _ground_level)
func _place_new_ground() -> void:
set_cellv(new_tile_position, _get_random_ground())
set_cellv(new_tile_position + Vector2.DOWN, Ground.TILE_DOWN_1)
new_tile_position += Vector2.RIGHT
func _remove_first_ground() -> void:
set_cellv(old_tile_position, -1)
set_cellv(old_tile_position + Vector2.DOWN, -1)
old_tile_position += Vector2.RIGHT
</code></pre>
<p>Where you might notice that the <code>_initial_new_tile_x</code> is <code>11</code>, instead of <code>10</code>, refer to <a href="#default-ground-tiles">Default ground tiles</a> where we placed tiles from <code>-8</code> to <code>10</code>, so the next empty one is <code>11</code>. These <code>_place_new_ground</code> and <code>_remove_first_ground</code> functions are called upon receiving the signal.</p>
<h4 id="pipetilemap">PipeTileMap<a class="headerlink" href="#pipetilemap" title="Permanent link">¶</a></h4>
<p>This is really similar to the <code>GroundTileMap</code> code, instead of defining an <code>enum</code> for the ground tiles, we define it for the pipe patterns (because each pipe is composed of multiple pipe tiles). If your pipe tile set looks like this (notice the index):</p>
<figure id="__yafg-figure-37">
<img alt="PipeTileMap - Tile set indexes" src="https://static.luevano.xyz/images/g/flappybird_godot/tile_set_pipes_indexes.png" title="PipeTileMap - Tile set indexes">
<figcaption>PipeTileMap - Tile set indexes</figcaption>
</figure>
<p>Then you can use the following “pipe patterns”:</p>
<pre><code class="language-gdscript">var pipe: Dictionary = {
PipePattern.PIPE_1: [0, 1, 2, 2, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2],
PipePattern.PIPE_2: [0, 1, 2, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2],
PipePattern.PIPE_3: [0, 1, 2, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2],
PipePattern.PIPE_4: [0, 1, 2, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2],
PipePattern.PIPE_5: [0, 1, 2, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2, 2],
PipePattern.PIPE_6: [0, 1, 2, 3, 4, -1, -1, -1, 0, 1, 2, 2, 2, 2, 2, 2]
}
</code></pre>
<p>Now, the pipe system requires a bit more of tracking as we need to instantiate a <code>ScoreDetector</code> here, too. I ended up keeping track of the placed pipes/detectors by using a “pipe stack” (and “detector stack”) which is just an array of placed objects from which I pop the first when deleting them:</p>
<pre><code class="language-gdscript">onready var _pipe_sep: int = get_parent().PIPE_SEP
const _pipe_size: int = 16
const _ground_level: int = 7
const _pipe_level_y: int = _ground_level - 1
const _initial_new_pipe_x: int = 11
var new_pipe_starting_position: Vector2 = Vector2(_initial_new_pipe_x, _pipe_level_y)
var pipe_stack: Array
# don't specify type for game, as it results in cyclic dependency,
# as stated here: https://godotengine.org/qa/39973/cyclic-dependency-error-between-actor-and-actor-controller
onready var game = get_parent().get_parent()
var detector_scene: PackedScene = preload("res://levels/detectors/score_detector/ScoreDetector.tscn")
var detector_offset: Vector2 = Vector2(16.0, -(_pipe_size / 2.0) * 16.0)
var detector_stack: Array
</code></pre>
<p>The <code>detector_offset</code> is just me being picky. For placing a new pipe, we get the starting position (bottom pipe tile) and build upwards, then instantiate a new <code>ScoreDetector</code> (<code>detector_scene</code>) and set it’s position to the pipe starting position plus the offset, so it’s centered in the pipe, then just need to connect the <code>body_entered</code> signal from the detector with the game, so we keep track of the scoring. Finally just add the placed pipe and detector to their corresponding stacks:</p>
<pre><code class="language-gdscript">func _place_new_pipe() -> void:
var current_pipe: Vector2 = new_pipe_starting_position
for tile in pipe[_get_random_pipe()]:
set_cellv(current_pipe, tile)
current_pipe += Vector2.UP
var detector: Area2D = detector_scene.instance()
detector.position = map_to_world(new_pipe_starting_position) + detector_offset
detector.connect("body_entered", game, "_on_ScoreDetector_body_entered")
detector_stack.append(detector)
add_child(detector)
pipe_stack.append(new_pipe_starting_position)
new_pipe_starting_position += _pipe_sep * Vector2.RIGHT
</code></pre>
<p>For removing pipes, it’s really similar but instead of getting the position from the next tile, we pop the first element from the (pipe/detector) stack and work with that. To remove the cells we just set the index to <code>-1</code>:</p>
<pre><code class="language-gdscript">func _remove_old_pipe() -> void:
var current_pipe: Vector2 = pipe_stack.pop_front()
var c: int = 0
while c < _pipe_size:
set_cellv(current_pipe, -1)
current_pipe += Vector2.UP
c += 1
var detector: Area2D = detector_stack.pop_front()
remove_child(detector)
detector.queue_free()
</code></pre>
<p>These functions are called when receiving the signal to place/remove pipes.</p>
<h3 id="saved-data">Saved data<a class="headerlink" href="#saved-data" title="Permanent link">¶</a></h3>
<p>Before proceeding, we require a way to save/load data (for the high scores). We’re going to use the <em>ConfigFile</em> node that uses a custom version of the <code>ini</code> file format. Need to define where to save the data:</p>
<pre><code class="language-gdscript">const DATA_PATH: String = "user://data.cfg"
const SCORE_SECTION: String = "score"
var _data: ConfigFile
</code></pre>
<p>Note that <code>user://</code> is a OS specific path in which the data can be stored on a per user basis, for more: <a href="https://docs.godotengine.org/en/stable/tutorials/io/data_paths.html">File paths</a>. Then, a way to load the save file:</p>
<pre><code class="language-gdscript">func _load_data() -> void:
# create an empty file if not present to avoid error while loading settings
var file: File = File.new()
if not file.file_exists(DATA_PATH):
file.open(DATA_PATH, file.WRITE)
file.close()
_data = ConfigFile.new()
var err: int = _data.load(DATA_PATH)
if err != OK:
print("[ERROR] Cannot load data.")
</code></pre>
<p>A way to save the data:</p>
<pre><code class="language-gdscript">func save_data() -> void:
var err: int = _data.save(DATA_PATH)
if err != OK:
print("[ERROR] Cannot save data.")
</code></pre>
<p>And of course, a way to get and set the high score:</p>
<pre><code class="language-gdscript">func set_new_high_score(high_score: int) -> void:
_data.set_value(SCORE_SECTION, "high_score", high_score)
func get_high_score() -> int:
return _data.get_value(SCORE_SECTION, "high_score")
</code></pre>
<p>Then, whenever this script is loaded we load the data and if it’s a new file, then add the default high score of 0:</p>
<pre><code class="language-gdscript">func _ready() -> void:
_load_data()
if not _data.has_section(SCORE_SECTION):
set_new_high_score(0)
save_data()
</code></pre>
<p>Now, this script in particular will need to be a <a href="https://docs.godotengine.org/en/stable/tutorials/scripting/singletons_autoload.html">Singleton (AutoLoad)</a>, which means that there will be only one instance and will be available across all scripts. To do so, go to <em>Project -> Project settings… -> AutoLoad</em> and select this script in the <code>Path:</code> and add a <code>Node Name:</code> (I used <code>SavedData</code>, if you use something else, be careful while following this devlog) which will be the name we’ll use to access the singleton. Toggle on <code>Enable</code> if needed, it should look like this:</p>
<figure id="__yafg-figure-38">
<img alt="Project settings - AutoLoad - SavedData singleton" src="https://static.luevano.xyz/images/g/flappybird_godot/project_settings_autoload_saved_data.png" title="Project settings - AutoLoad - SavedData singleton">
<figcaption>Project settings - AutoLoad - SavedData singleton</figcaption>
</figure>
<h3 id="game_1">Game<a class="headerlink" href="#game_1" title="Permanent link">¶</a></h3>
<p>The game script it’s also like a “Signal bus” in the sense that it connects all its childs’ signals together, and also has the job of starting/stopping the <code>_process</code> and <code>_physics_process</code> methods from the childs as needed. First, we need to define the signals and and references to all child nodes:</p>
<pre><code class="language-gdscript">signal game_started
signal game_over
signal new_score(score, high_score)
onready var player: Player = $Player
onready var background: Sprite= $Background
onready var world_tiles: WorldTiles = $WorldTiles
onready var ceiling_detector: Area2D = $CeilingDetector
onready var world_detector: Node2D = $WorldDetector
onready var camera: Camera2D = $Camera
onready var start_sound: AudioStreamPlayer = $StartSound
onready var score_sound: AudioStreamPlayer = $ScoreSound
</code></pre>
<p>It’s important to get the actual “player speed”, as we’re using a scale to make the game look bigger (remember, pixel art), to do so we need a reference to the <code>game_scale</code> we setup at the beginning and compute the <code>player_speed</code>:</p>
<pre><code class="language-gdscript">var _game_scale: float = ProjectSettings.get_setting("application/config/game_scale")
var player_speed: float
func _ready() -> void:
scale = Vector2(_game_scale, _game_scale)
# so we move at the actual speed of the player
player_speed = player.SPEED / _game_scale
</code></pre>
<p>This <code>player_speed</code> will be needed as we need to move all the nodes (<code>Background</code>, <code>Camera</code>, etc.) in the <code>x</code> axis as the player is moving. This is done in the <code>_physics_process</code>:</p>
<pre><code class="language-gdscript">func _physics_process(delta: float) -> void:
ceiling_detector.move_local_x(player_speed * delta)
world_detector.move_local_x(player_speed * delta)
background.move_local_x(player_speed * delta)
camera.move_local_x(player_speed * delta)
</code></pre>
<p>We also need a way to start and stop the processing of all the nodes:</p>
<pre><code class="language-gdscript">func _set_processing_to(on_off: bool, include_player: bool = true) -> void:
set_process(on_off)
set_physics_process(on_off)
if include_player:
player.set_process(on_off)
player.set_physics_process(on_off)
world_tiles.set_process(on_off)
world_tiles.set_physics_process(on_off)
ceiling_detector.set_process(on_off)
ceiling_detector.set_physics_process(on_off)
</code></pre>
<p>Where the <code>player</code> is a special case, as when the player dies, it should still move (only down), else it would just freeze in place. In <code>_ready</code> we connect all the necessary signals as well as initially set the processing to <code>false</code> using the last function. To start/restart the game we need to keep a flag called <code>is_game_running</code> initially set to <code>false</code> and then handle the (re)startability in <code>_input</code>:</p>
<pre><code class="language-gdscript">func _input(event: InputEvent) -> void:
if not is_game_running and event.is_action_pressed("jump"):
_set_processing_to(true)
is_game_running = true
emit_signal("game_started")
start_sound.play()
if event.is_action_pressed("restart"):
get_tree().reload_current_scene()
</code></pre>
<p>Then we handle two specific signals:</p>
<pre><code class="language-gdscript">func _on_Player_died() -> void:
_set_processing_to(false, false)
emit_signal("game_over")
func _on_ScoreDetector_body_entered(body: Node2D) -> void:
score += 1
if score > high_score:
high_score = score
SavedData.set_new_high_score(high_score)
SavedData.save_data()
emit_signal("new_score", score, high_score)
score_sound.play()
</code></pre>
<p>When the <code>player</code> dies, we set all processing to <code>false</code>, except for the player itself (so it can drop all the way to the ground). Also, when receiving a “scoring” signal, we manage the current score, as well as saving the new high score when applicable, note that we need to read the <code>high_score</code> at the beginning by calling <code>SavedData.get_high_score()</code>. This signal we emit will be received by the <code>UI</code> so it updates accordingly.</p>
<h3 id="ui_1">UI<a class="headerlink" href="#ui_1" title="Permanent link">¶</a></h3>
<p>First thing is to get a reference to all the child <em>Labels</em>, an initial reference to the high score as well as the version defined in the project settings:</p>
<pre><code class="language-gdscript">onready var fps_label: Label = $MarginContainer/DebugContainer/FPS
onready var version_label: Label = $MarginContainer/VersionContainer/Version
onready var score_label: Label = $MarginContainer/InfoContainer/ScoreContainer/Score
onready var high_score_label: Label = $MarginContainer/InfoContainer/ScoreContainer/HighScore
onready var start_game_label: Label = $MarginContainer/InfoContainer/StartGame
onready var _initial_high_score: int = SavedData.get_high_score()
var _version: String = ProjectSettings.get_setting("application/config/version")
</code></pre>
<p>Then set the initial <em>Label</em> values as well as making the <code>fps_label</code> invisible:</p>
<pre><code class="language-gdscript">func _ready() -> void:
fps_label.visible = false
version_label.set_text("v%s" % _version)
high_score_label.set_text("High score: %s" % _initial_high_score)
</code></pre>
<p>Now we need to handle the <code>fps_label</code> update and toggle:</p>
<pre><code class="language-gdscript">func _input(event: InputEvent) -> void:
if event.is_action_pressed("toggle_debug"):
fps_label.visible = !fps_label.visible
func _process(delta: float) -> void:
if fps_label.visible:
fps_label.set_text("FPS: %d" % Performance.get_monitor(Performance.TIME_FPS))
</code></pre>
<p>Finally the signal receiver handlers which are straight forward:</p>
<pre><code class="language-gdscript">func _on_Game_game_started() -> void:
start_game_label.visible = false
high_score_label.visible = false
func _on_Game_game_over() -> void:
start_game_label.set_text("Press R to restart")
start_game_label.visible = true
high_score_label.visible = true
func _on_Game_new_score(score: int, high_score: int) -> void:
score_label.set_text(String(score))
high_score_label.set_text("High score: %s" % high_score)
</code></pre>
<h3 id="main_1">Main<a class="headerlink" href="#main_1" title="Permanent link">¶</a></h3>
<p>This is the shortest script, it just connects the signals between the <code>Game</code> and the <code>UI</code>:</p>
<pre><code class="language-gdscript">onready var game: Game = $Game
onready var ui: UI = $UI
var _game_over: bool = false
func _ready() -> void:
game.connect("game_started", ui, "_on_Game_game_started")
game.connect("game_over", ui, "_on_Game_game_over")
game.connect("new_score", ui, "_on_Game_new_score")
</code></pre>
<h2 id="final-notes-and-exporting">Final notes and exporting<a class="headerlink" href="#final-notes-and-exporting" title="Permanent link">¶</a></h2>
<p>At this point the game should be fully playable (if any detail missing feel free to look into the source code linked at the beginning). Only thing missing is an icon for the game; I did one pretty quicly with the assets I had.</p>
<h3 id="preparing-the-files">Preparing the files<a class="headerlink" href="#preparing-the-files" title="Permanent link">¶</a></h3>
<p>If you followed the directory structure I used, then only thing needed is to transform the icon to a native Windows <code>ico</code> format (if exporting to Windows, else ignore this part). For this you need <a href="https://imagemagick.org/index.php">ImageMagick</a> or some other program that can transform <code>png</code> (or whatever file format you used for the icon) to <code>ico</code>. I used [Chocolatey][https://chocolatey.org/] to install <code>imagemagick</code>, then to convert the icon itself used: <code>magick convert icon.png -define icon:auto-resize=256,128,64,48,32,16 icon.ico</code> as detailed in <em>Godot</em>‘s <a href="https://docs.godotengine.org/en/stable/tutorials/export/changing_application_icon_for_windows.html">Changing application icon for Windows</a>.</p>
<h3 id="exporting">Exporting<a class="headerlink" href="#exporting" title="Permanent link">¶</a></h3>
<p>You need to download the templates for exporting as detailed in <em>Godot</em>‘s <a href="https://docs.godotengine.org/en/stable/tutorials/export/exporting_projects.html">Exporting projects</a>. Basically you go to <em>Editor -> Manage Export Templates…</em> and download the latest one specific to your <em>Godot</em> version by clicking on <code>Download and Install</code>.</p>
<p>If exporting for Windows then you also need to download <code>rcedit</code> from <a href="https://github.com/electron/rcedit/releases/latest">here</a>. Just place it wherever you want (I put it next to the <em>Godot</em> executable).</p>
<p>Then go to <em>Project -> Export…</em> and the Window should be empty, add a new template by clicking on <code>Add...</code> at the top and then select the template you want. I used HTML5, Windows Desktop and Linux/X11. Really the only thing you need to set is the “Export Path” for each template, which is te location of where the executable will be written to, and in the case of the <em>Windows Desktop</em> template you could also setup stuff like <code>Company Name</code>, <code>Product Name</code>, <code>File/Product Version</code>, etc..</p>
<p>Once the templates are setup, select any and click on <code>Export Project</code> at the bottom, and make sure to untoggle <code>Export With Debug</code> in the window that pops up, this checkbox should be at the bottom of the new window.</p>]]></content:encoded>
</item>
<item>
<title>General Godot project structure</title>
<link>https://blog.luevano.xyz/g/godot_project_structure.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/g/godot_project_structure.html</guid>
<pubDate>Sun, 22 May 2022 01:16:10 GMT</pubDate>
<category>English</category>
<category>Gamedev</category>
<category>Godot</category>
<category>Short</category>
<description>Details on the project structure I'm using for Godot, based on preference and some research I did.</description>
<content:encoded><![CDATA[<p>One of my first issues when starting a project is how to structure everything. So I had to spend some time researching best practices and go with what I like the most and after trying some of them I wanted to write down somewhere what I’m sticking with.</p>
<p>The first place to look for is, of course, the official <em>Godot</em> documentation on <a href="https://docs.godotengine.org/en/stable/tutorials/best_practices/project_organization.html">Project organization</a>; along with project structure discussion, also comes with best practices for code style and what-not. I don’t like this project/directory structure that much, just because it tells you to bundle everything under the same directory but it’s a really good starting point, for example it tells you to use:</p>
<ul>
<li>/models/town/house/<ul>
<li>house.dae</li>
<li>window.png</li>
<li>door.png</li>
</ul>
</li>
</ul>
<p>Where I would prefer to have more modularity, for example:</p>
<ul>
<li>/levels/structures/town/house (or /levels/town/structures/house)<ul>
<li>window/<ul>
<li>window.x</li>
<li>window.y</li>
<li>window.z</li>
</ul>
</li>
<li>door/<ul>
<li>…</li>
</ul>
</li>
<li>house.x</li>
<li>house.y</li>
<li>house.z</li>
</ul>
</li>
</ul>
<p>It might look like it’s more work, but I prefer it like this. I wish <a href="https://www.braindead.bzh/entry/creating-a-game-with-godot-engine-ep-2-project-organization">this site</a> was still available, as I got most of my ideas from there and was a pretty good resource, but apparently the owner is not maintaining his site anymore; but there is <a href="https://www.reddit.com/r/godot/comments/7786ee/comment/dojuzuf/?utm_source=share&utm_medium=web2x&context=3">this excelent comment on reddit</a> which shows a project/directory structure more in line with what I’m currently using (and similr to the site that is down that I liked). I ended up with:</p>
<ul>
<li>/.git</li>
<li>/assets (raw assets/editable assets/asset packs)</li>
<li>/releases (executables ready to publish)</li>
<li>/src (the actual godot project)<ul>
<li>.godot/</li>
<li>actors/ (or entities)<ul>
<li>player/<ul>
<li>sprites/</li>
<li>player.x</li>
<li>…</li>
</ul>
</li>
<li>enemy/ (this could be a dir with subdirectories for each type of enemy for example…)<ul>
<li>sprites/</li>
<li>enemy.x</li>
<li>…</li>
</ul>
</li>
<li>actor.x</li>
<li>…</li>
</ul>
</li>
<li>levels/ (or scenes)<ul>
<li>common/<ul>
<li>sprites/</li>
<li>…</li>
</ul>
</li>
<li>main/<ul>
<li>…</li>
</ul>
</li>
<li>overworld/<ul>
<li>…</li>
</ul>
</li>
<li>dugeon/<ul>
<li>…</li>
</ul>
</li>
<li>Game.tscn (I’m considering the “Game” as a level/scene)</li>
<li>game.gd</li>
</ul>
</li>
<li>objects/<ul>
<li>box/<ul>
<li>…</li>
</ul>
</li>
<li>…</li>
</ul>
</li>
<li>screens/<ul>
<li>main_menu/<ul>
<li>…</li>
</ul>
</li>
<li>…</li>
</ul>
</li>
<li>globals/ (singletons/autoloads)</li>
<li>ui/<ul>
<li>menus/<ul>
<li>…</li>
</ul>
</li>
<li>…</li>
</ul>
</li>
<li>sfx/<ul>
<li>…</li>
</ul>
</li>
<li>vfx/<ul>
<li>…</li>
</ul>
</li>
<li>etc/<ul>
<li>…</li>
</ul>
</li>
<li>Main.tscn (the entry point of the game)</li>
<li>main.gd</li>
<li>icon.png (could also be on a separate “icons” directory)</li>
<li>project.godot</li>
<li>…</li>
</ul>
</li>
<li>\<any other repository related files></li>
</ul>
<p>And so on, I hope the idea is clear. I’ll probably change my mind on the long run, but for now this has been working fine.</p>]]></content:encoded>
</item>
<item>
<title>Will start blogging about gamedev</title>
<link>https://blog.luevano.xyz/g/starting_gamedev_blogging.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/g/starting_gamedev_blogging.html</guid>
<pubDate>Tue, 17 May 2022 05:19:54 GMT</pubDate>
<category>English</category>
<category>Gamedev</category>
<category>Godot</category>
<category>Short</category>
<category>Update</category>
<description>Since I'm starting to get more into gamedev stuff, I'll start blogging about it just to keep consistent.</description>
<content:encoded><![CDATA[<p>I’ve been wanting to get into gamedev for a while now, but it’s always a pain to stay consistent. I just recently started to get into it again, and this time I’m trying to actually do stuff.</p>
<p>So, the plan is to blog about my progress and clone some simple games just to get started. I’m thinking on sticking with <a href="https://godotengine.org/">Godot</a> just because I like that it’s open source, it’s getting better and better overtime (big rewrite happening right now) and I already like how the engine works. <del>Specifically I’ll start using <em>Godot 4</em> even though it’s not done yet, to get used to the new features, specifically pumped for <a href="https://godotengine.org/article/gdscript-progress-report-feature-complete-40">GDScript 2.0</a>.</del> <ins>Actually… (for the small clones/ripoffs) I’ll need to use <em>Godot 3.X</em> (probably 3.5), as <em>Godot 4</em> doesn’t have support to export to webassembly (HTML5) yet, and I want that to publish to <a href="https://itch.io/">itch.io</a> and my website. I’ll continue to use <em>Godot 4</em> for bigger projects, as they will take longer and I hope that by the time I need to publish, there’s no issues to export.</ins></p>
<p>For a moment I almost started a new subdomain just for gamedev stuff, but decided to just use a different directory for subtleness; this directory and use of tags should be enough. I’ll be posting the entry about the first rip-off I’m <em>developing</em> (FlappyBird L O L) shortly.</p>
<p><strong>Update</strong>: <a href="https://godotengine.org/article/godot-4-0-sets-sail/">Godot 4</a> already released and it now has HTML5 support, so back to the original plan.</p>]]></content:encoded>
</item>
<item>
<title>My setup for a password manager and MFA authenticator</title>
<link>https://blog.luevano.xyz/a/password_manager_authenticator_setup.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/password_manager_authenticator_setup.html</guid>
<pubDate>Sun, 15 May 2022 22:40:34 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Tools</category>
<description>A short description on my personal setup regarding a password manager and alternatives to G\*\*gl\* authenticator.</description>
<content:encoded><![CDATA[<p><strong>Disclaimer</strong>: I won’t go into many technical details here of how to install/configure/use the software, this is just supposed to be a short description on my setup.</p>
<p>It’s been a while since I started using a password manager at all, and I’m happy that I started with <a href="https://keepassxc.org/">KeePassXC</a> (open source, multiplatform password manager that it’s completely offline) as a direct recommendation from <a href="https://www.lmcj.xyz/"><mark>EL ELE EME</mark></a>; before this I was using the same password for everything (like a lot of people), which is a well know privacy issue as noted in detail by <a href="https://askleo.com/different-passwords-for-everything/">Leo</a> (I don’t personally recommed LastPass as Leo does). Note that you will still need a <em>master password</em> to lock/unlock your password database (you can additionally use a hardware key and a key file).</p>
<p>Anyways, setting up <em>keepass</em> is pretty simple, as there is a client for almost any device; note that <em>keepass</em> is basically just the format and the base for all of the clients, as its common with pretty much any open source software. In my case I’m using <a href="https://keepassxc.org/">KeePassXC</a> in my computer and <a href="https://www.keepassdx.com/">KeePassDX</a> in my phone (Android). The only concern is keeping everything in sync because <em>keepass</em> doesn’t have any automatic method of synchronizing between devices because of security reasons (as far as I know), meaning that you have to manage that yourself.</p>
<p>Usually you can use something like G**gl* drive, dropbox, mega, nextcloud, or any other cloud solution that you like to sync your <em>keepass</em> database between devices; I personally prefer to use <a href="https://syncthing.net/">Syncthing</a> as it’s open source, it’s really easy to setup and has worked wonders for me since I started using it, also it keeps versions of your files that can serve as backups in any scenario where the database gets corrupted or something.</p>
<p>Finally, when I went through the issue with the micro SD and the <em>adoptable storage</em> bullshit (you can find the rant <a href="https://blog.luevano.xyz/a/devs_android_me_trozaron.html">here</a>, in spanish) I had to also migrate from <em>G**gl* authenticator</em> (<em>gauth</em>) to something else for the simple reason that <em>gauth</em> doesn’t even let you do backups, nor it’s synched with your account… nothing, it is just standalone and if you ever lose your phone you’re fucked; so I decided to go with <a href="https://getaegis.app/">Aegis authenticator</a>, as it is open source, you have control over all your secret keys, you can do backups directly to the filesystem, you can secure your database with an extra password, etc., etc.. In general <em>aegis</em> is the superior MFA authenticator (at least compared with <em>gauth</em>) and everything that’s compatible with <em>gauth</em> is compatible with <em>aegis</em> as the format is a standard (as a matter of fact, <em>keepass</em> also has this MFA feature which is called TOPT and is also compatible, but I prefer to have things separate). I also use <em>syncthing</em> to keep a backup of my <em>aegis</em> database.</p>
<p><strong>TL;DR</strong>:</p>
<ul>
<li><a href="https://syncthing.net/">Syncthing</a> to sync files between devices (for the password databases).</li>
<li><a href="https://keepassxc.org/">KeePassXC</a> for the password manager in my computer.</li>
<li><a href="https://www.keepassdx.com/">KeePassDX</a> for the password manager in my phone.</li>
<li><a href="https://getaegis.app/">Aegis authenticator</a> for the universal MFA authenticator.</li>
</ul>]]></content:encoded>
</item>
<item>
<title>Los devs de Android/MIUI me trozaron</title>
<link>https://blog.luevano.xyz/a/devs_android_me_trozaron.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/devs_android_me_trozaron.html</guid>
<pubDate>Sun, 15 May 2022 09:51:04 GMT</pubDate>
<category>Rant</category>
<category>Spanish</category>
<category>Update</category>
<description>Perdí un día completo resolviendo un problema muy estúpido, por culpa de los devs de Android/MIUI.</description>
<content:encoded><![CDATA[<p>Llevo dos semanas posponiendo esta entrada porque andaba bien enojado (todavía, pero ya se anda pasando) y me daba <em>zzz</em>. Pero bueno, antes que nada este pex ocupa un poco de contexto sobre dos cositas:</p>
<ul>
<li><a href="https://tachiyomi.org/">Tachiyomi</a>: Una aplicación de android que uso para descargar y leer manga. Lo importante aquí es que por default se guardan los mangas con cada página siendo una sola imagen, por lo que al mover el manga de un lado a otro tarda mucho tiempo.</li>
<li><a href="https://source.android.com/devices/storage/adoptable">Adoptable storage</a>: Un <em>feature</em> de android que básicamente te deja usar una micro SD (mSD) externa como si fuera interna, encriptando y dejando la mSD inutilizable en cualquier otro dispositivo. La memoria interna se <em>pierde</em> o algo por el estilo (bajo mi experiencia), por lo que parece es bastante útil cuando la capacidad de la memoria interna es baja.</li>
</ul>
<p>Ahora sí vamonos por partes, primero que nada lo que sucedió fue que ordené una mSD con más capacidad que la que ya tenía (64 GB -> 512 GB, poggies), porque últimamente he estado bajando y leyendo mucho manga entonces me estaba quedando sin espacio. Ésta llegó el día de mi cumpleaños lo cuál estuvo chingón, me puse a hacer backup de la mSD que ya tenía y preparando todo, muy bonito, muy bonito.</p>
<p>Empecé a tener problemas, porque al estar moviendo tanto archivo pequeño (porque recordemos que el <em>tachiyomi</em> trata a cada página como una sola imagen), la conexión entre el celular y mi computadora se estaba corte y corte por alguna razón; en general muchos pedos. Por lo que mejor le saqué la nueva mSD y la metí directo a mi computadora por medio de un adaptador para batallar menos y que fuera más rápido.</p>
<p>Hacer este pedo de mover archivos directamente en la mSD puede llevar a corromper la memoria, no se los detalles pero pasa (o quizá estoy meco e hice algo mal). Por lo que al terminar de mover todo a la nueva mSD y ponerla en el celular, éste se emputó que porque no la detectaba y que quería tirar un formateo a la mSD. A este punto no me importaba mucho, sólo era questión de volvera mover archivos y ser más cuidadoso; “<em>no issues from my end</em>” diría en mis <em>standups</em>.</p>
<p>Todo valió <strong>vergota</strong> porque en cierto punto al elegir sí formatear la mSD mi celular me daba la opción de “<em>usar la micro SD para el celular</em>” o “<em>usar la micro SD como memoria portátil</em>” (o algo entre esas líneas), y yo, estúpidamente, elegí la primera, porque me daba sentido: “no, pues simón, voy a usar esta memoria para este celular”.</p>
<p>Pues mamé, resulta que esa primera opción lo que realmente quería decir es que se iba a usar la micro SD como interna usando el pex este de <em>adoptable storage</em>. Entonces básicamente <em>perdí</em> mi capacidad de memoria interna (128 GB aprox.), y toda la mSD nueva se usó como memoria interna. Todo se juntó, si intentaba sacar la mSD todo se iba a la mierda y no podía usar muchas aplicaciones. “<em>No hay pedo</em>”, pensé, “<em>nada más es cuestión de desactivar esta mamada de adoptable storage</em>”.</p>
<p>Ni madres dijeron los devs de Android, este pedo nada más es un <em>one-way</em>: puedes activar <em>adoptable storage</em> pero para desactivarlo <strong>ocupas, a huevo, formatear tu celular a estado de fábrica</strong>. Chingué a mi madre, comí mierda, perdí.</p>
<p>Pues eso fue lo que hice, ni modo. Hice backup de todo lo que se me ocurrió (también me di cuenta que G**gl* authenticator es cagada ya que no te deja hacer backup, entre otras cosas, mejor usen <a href="https://getaegis.app/">Aegis authenticator</a>), desactivé todo lo que se tenía que desactivar y tocó hacer <em>factory reset</em>, ni modo. Pero como siempre las cosas salen mal y tocó comer mierda del banco porque me bloquearon la tarjeta, perdí credenciales necesarias para el trabajo (se resolvió rápido), etc., etc.. Ya no importa, ya casi todo está resuelto, sólo queda ir al banco a resolver lo de la tarjeta bloqueada (esto es para otro <em>rant</em>, pinches apps de bancos piteras, ocupan hacer una sola cosa y la hacen mal).</p>
<p>Al final del día, la causa del problema fueron los malditos mangas (por andar queriendo <em>backupearlos</em>), que terminé bajando de nuevo manualmente y resultó mejor porque aparentemente <em>tachiyomi</em> agregó la opción de “<em>zippear</em>” los mangas en formato <a href="https://docs.fileformat.com/ebook/cbz/">CBZ</a>, por lo que ya son más fácil de mover de un lado para otro, el fono no se queda pendejo, etc., etc..</p>
<p>Por último, quiero decir que los devs de Android son unos pendejos por no hacer reversible la opción de <em>adoptable storage</em>, y los de MIUI son todavía más por no dar detalles de lo que significan sus opciones de formateo, especialmente si una opción es tan chingadora que para revertirla necesitas formatear a estado de fábrica tu celular; más que nada es culpa de los de MIUI, todavía que ponen un chingo de A(i)DS en todas sus apps, no pueden poner una buena descripción en sus opciones. <strong>REEEE</strong>.</p>]]></content:encoded>
</item>
<item>
<title>Volviendo a usar la página</title>
<link>https://blog.luevano.xyz/a/volviendo_a_usar_la_pagina.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/volviendo_a_usar_la_pagina.html</guid>
<pubDate>Thu, 28 Apr 2022 03:21:02 GMT</pubDate>
<category>Short</category>
<category>Spanish</category>
<category>Update</category>
<description>Actualización en el estado de la página, después de mucho tiempo de ausencia.</description>
<content:encoded><![CDATA[<p>Después de mucho tiempo de estar luchando con querer volver a usar este pex (maldita <em>d</em> word y demás), ya me volví a acomodar el setup para agregar nuevas entradas.</p>
<p>Entre las cosas que tuve que hacer fue actualizar el <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> porque no lo podía usar de una como estaba; y de pasado le agregue una que otra feature nueva. Luego quiero agregarle más funcionalidad para poder <em>buildear</em> la página completa; por ahora se hace en segmentos: todo lo de <a href="https://luevano.xyz">luevano.xyz</a> está hecho manual, mientras que <a href="https://blog.luevano.xyz">blog</a> y <a href="https://art.luevano.xyz">art</a> usan <a href="https://github.com/luevano/pyssg">pyssg</a>.</p>
<p>Otra cosa es que quizá me devuelva a editar alguans entradas nada más para homogeneizar las entradas específicas a <em>Create a…</em> (tiene más sentido que sean <em>Setup x…</em> o algo similar).</p>
<p>En otras noticias, estoy muy agusto en el jale que tengo actualmente aunque lleve alrededor de 3 semanas de un infierno en el jale. Debo pensar en si debo omitir cosas personales o del trabajo aquí, ya que quién sabe quién se pueda llegar a topar con esto <em>*thinking emoji*</em>.</p>]]></content:encoded>
</item>
<item>
<title>Set up a VPN server with OpenVPN</title>
<link>https://blog.luevano.xyz/a/vpn_server_with_openvpn.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/vpn_server_with_openvpn.html</guid>
<pubDate>Sun, 01 Aug 2021 09:27:02 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a VPN server using OpenVPN on a server running Nginx, on Arch. Only for IPv4.</description>
<content:encoded><![CDATA[<p>I’ve been wanting to do this entry, but had no time to do it since I also have to set up the VPN service as well to make sure what I’m writing makes sense, today is the day.</p>
<p>Like with any other of my entries I based my setup on the <a href="https://wiki.archlinux.org/title/OpenVPN">Arch Wiki</a>, <a href="https://github.com/Nyr/openvpn-install">this install script</a> and <a href="https://github.com/graysky2/ovpngen">this profile generator script</a>.</p>
<p>This will be installed and working alongside the other stuff I’ve wrote about on other posts (see the <a href="https://blog.luevano.xyz/tag/@server.html">server</a> tag). All commands here are executes as root unless specified otherwise. Also, this is intended only for IPv4 (it’s not that hard to include IPv6, but meh). As always, all commands are executed as root unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#create-pki-from-scratch">Create PKI from scratch</a></li>
<li><a href="#openvpn">OpenVPN</a><ul>
<li><a href="#enable-forwarding">Enable forwarding</a></li>
<li><a href="#create-client-configurations">Create client configurations</a></li>
</ul>
</li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Pretty simple:</p>
<ul>
<li>Working server with root access, and with <code>ufw</code> as the firewall.</li>
<li>Open port <code>1194</code> (default), or as a fallback on <code>443</code> (click <a href="https://openvpn.net/vpn-server-resources/advanced-option-settings-on-the-command-line/">here</a> for more). I will do mine on port <code>1194</code> but it’s just a matter of changing 2 lines of configuration and one <code>ufw</code> rule.</li>
</ul>
<h2 id="create-pki-from-scratch">Create PKI from scratch<a class="headerlink" href="#create-pki-from-scratch" title="Permanent link">¶</a></h2>
<p>PKI stands for <em>Public Key Infrastructure</em> and basically it’s required for certificates, private keys and more. This is supposed to work between two servers and one client: a server in charge of creating, signing and verifying the certificates, a server with the OpenVPN service running and the client making the request.</p>
<p>In a nutshel, this is supposed to work something like: 1) a client wants to use the VPN service, so it creates a requests and sends it to the signing server, 2) this server checks the requests and signs the request, returning the certificates to both the VPN service and the client and 3) the client can now connect to the VPN service using the signed certificate which the OpenVPN server knows about.</p>
<p>That’s how the it should be st up… but, to be honest, all of this is a hassle and (in my case) I want something simple to use and manage. So I’m gonna do all on one server and then just give away the configuration file for the clients, effectively generating files that anyone can run and will work, meaning that you need to be careful who you give this files (it also comes with a revoking mechanism, so no worries).</p>
<p>This is done with <a href="https://wiki.archlinux.org/title/Easy-RSA">Easy-RSA</a>.</p>
<p>Install the <code>easy-rsa</code> package:</p>
<pre><code class="language-sh">pacman -S easy-rsa
</code></pre>
<p>Initialize the PKI and generate the CA keypair:</p>
<pre><code class="language-sh">cd /etc/easy-rsa
easyrsa init-pki
easyrsa build-ca nopass
</code></pre>
<p>Create the server certificate and private key (while in the same directory):</p>
<pre><code class="language-sh">EASYRSA_CERT_EXPIRE=3650 easyrsa build-server-full server nopass
</code></pre>
<p>Where <code>server</code> is just a name to identify your server certificate keypair, I just use <code>server</code> but could be anything (like <code>luevano.xyz</code> in my case).</p>
<p>Create the client revocation list AKA CRL (will be used later, but might as well have it now):</p>
<pre><code class="language-sh">EASYRSA_CRL_DAYS=3650 easyrsa gen-crl
</code></pre>
<p>After this we should have 6 new files:</p>
<pre><code>/etc/easy-rsa/pki/ca.crt
/etc/easy-rsa/pki/private/ca.key
/etc/easy-rsa/pki/issued/server.crt
/etc/easy-rsa/pki/reqs/server.req
/etc/easy-rsa/pki/private/server.key
/etc/easy-rsa/pki/crl.pem
</code></pre>
<p>It is recommended to copy some of these files over to the <code>openvpn</code> directory, but I prefer to keep them here and just change some of the permissions:</p>
<pre><code class="language-sh">chmod o+rx pki
chmod o+rx pki/ca.crt
chmod o+rx pki/issued
chmod o+rx pki/issued/server.crt
chmod o+rx pki/private
chmod o+rx pki/private/server.key
chown nobody:nobody pki/crl.pem
chmod o+r pki/crl.pem
</code></pre>
<p>Finally, go to the <code>openvpn</code> directory and create the required files there:</p>
<pre><code class="language-sh">cd /etc/openvpn/server
openssl dhparam -out dh.pem 2048
openvpn --genkey secret ta.key
</code></pre>
<h2 id="openvpn">OpenVPN<a class="headerlink" href="#openvpn" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/OpenVPN">OpenVPN</a> is a robust and highly flexible VPN daemon, that’s pretty complete feature-wise.</p>
<p>Install the <code>openvpn</code> package:</p>
<pre><code class="language-sh">pacman -S openvpn
</code></pre>
<p>Now, most of the stuff is going to be handled by (each, if you have more than one) server configuration. This might be the hardest thing to configure, but I’ve used a basic configuration file that worked a lot to me, which is a compilation of stuff that I found on the internet while configuring the file a while back.</p>
<pre><code># Server ip addres (ipv4).
local 1.2.3.4 # your server public ip
# Port.
port 1194 # Might want to change it to 443
# TCP or UDP.
;proto tcp
proto udp # If ip changes to 443, you should change this to tcp, too
# "dev tun" will create a routed IP tunnel,
# "dev tap" will create an ethernet tunnel.
;dev tap
dev tun
# Server specific certificates and more.
ca /etc/easy-rsa/pki/ca.crt
cert /etc/easy-rsa/pki/issued/server.crt
key /etc/easy-rsa/pki/private/server.key # This file should be kept secret.
dh /etc/openvpn/server/dh.pem
auth SHA512
tls-crypt /etc/openvpn/server/ta.key 0 # This file is secret.
crl-verify /etc/easy-rsa/pki/crl.pem
# Network topology.
topology subnet
# Configure server mode and supply a VPN subnet
# for OpenVPN to draw client addresses from.
server 10.8.0.0 255.255.255.0
# Maintain a record of client <-> virtual IP address
# associations in this file.
ifconfig-pool-persist ipp.txt
# Push routes to the client to allow it
# to reach other private subnets behind
# the server.
;push "route 192.168.10.0 255.255.255.0"
;push "route 192.168.20.0 255.255.255.0"
# If enabled, this directive will configure
# all clients to redirect their default
# network gateway through the VPN, causing
# all IP traffic such as web browsing and
# and DNS lookups to go through the VPN
push "redirect-gateway def1 bypass-dhcp"
# Certain Windows-specific network settings
# can be pushed to clients, such as DNS
# or WINS server addresses.
# Google DNS.
;push "dhcp-option DNS 8.8.8.8"
;push "dhcp-option DNS 8.8.4.4"
# The keepalive directive causes ping-like
# messages to be sent back and forth over
# the link so that each side knows when
# the other side has gone down.
keepalive 10 120
# The maximum number of concurrently connected
# clients we want to allow.
max-clients 5
# It's a good idea to reduce the OpenVPN
# daemon's privileges after initialization.
user nobody
group nobody
# The persist options will try to avoid
# accessing certain resources on restart
# that may no longer be accessible because
# of the privilege downgrade.
persist-key
persist-tun
# Output a short status file showing
# current connections, truncated
# and rewritten every minute.
status openvpn-status.log
# Set the appropriate level of log
# file verbosity.
#
# 0 is silent, except for fatal errors
# 4 is reasonable for general usage
# 5 and 6 can help to debug connection problems
# 9 is extremely verbose
verb 3
# Notify the client that when the server restarts so it
# can automatically reconnect.
# Only usable with udp.
explicit-exit-notify 1
</code></pre>
<p><code>#</code> and <code>;</code> are comments. Read each and every line, you might want to change some stuff (like the logging), specially the first line which is your server public IP.</p>
<h4 id="enable-forwarding">Enable forwarding<a class="headerlink" href="#enable-forwarding" title="Permanent link">¶</a></h4>
<p>Now, we need to enable <em>packet forwarding</em> (so we can access the web while connected to the VPN), which can be enabled on the interface level or globally (you can check the different options with <code>sysctl -a | grep forward</code>). I’ll do it globally, run:</p>
<pre><code class="language-sh">sysctl net.ipv4.ip_forward=1
</code></pre>
<p>And create/edit the file <code>/etc/sysctl.d/30-ipforward.conf</code>:</p>
<pre><code>net.ipv4.ip_forward=1
</code></pre>
<p>Now we need to configure <code>ufw</code> to forward traffic through the VPN. Append the following to <code>/etc/default/ufw</code> (or edit the existing line):</p>
<pre><code>...
DEFAULT_FORWARD_POLICY="ACCEPT"
...
</code></pre>
<p>And change the <code>/etc/ufw/before.rules</code>, appending the following lines after the header <strong>but before the *filter line</strong>:</p>
<pre><code>...
# NAT (Network Address Translation) table rules
*nat
:POSTROUTING ACCEPT [0:0]
# Allow traffic from clients to the interface
-A POSTROUTING -s 10.8.0.0/24 -o interface -j MASQUERADE
# do not delete the "COMMIT" line or the NAT table rules above will not be processed
COMMIT
# Don't delete these required lines, otherwise there will be errors
*filter
...
</code></pre>
<p>Where <code>interface</code> must be changed depending on your system (in my case it’s <code>ens3</code>, another common one is <code>eth0</code>); I always check this by running <code>ip addr</code> which gives you a list of interfaces (the one containing your server public IP is the one you want, or whatever interface your server uses to connect to the internet):</p>
<pre><code>...
2: ens3: <SOMETHING,SOMETHING> bla bla
link/ether bla:bla
altname enp0s3
inet my.public.ip.addr bla bla
...
</code></pre>
<p>And also make sure the <code>10.8.0.0/24</code> matches the subnet mask specified in the <code>server.conf</code> file (in this example it matches). You should check this very carefully, because I just spent a good 2 hours debugging why my configuration wasn’t working, and this was te reason (I could connect to the VPN, but had no external connection to the web).</p>
<p>Finally, allow the OpenVPN port you specified (in this example its <code>1194/udp</code>) and reload <code>ufw</code>:</p>
<pre><code class="language-sh">ufw allow 1194/udp comment "OpenVPN"
ufw reload
</code></pre>
<p>At this point, the server-side configuration is done and you can start and enable the service:</p>
<pre><code class="language-sh">systemctl start openvpn-server@server.service
systemctl enable openvpn-server@server.service
</code></pre>
<p>Where the <code>server</code> after <code>@</code> is the name of your configuration, <code>server.conf</code> without the <code>.conf</code> in my case.</p>
<h3 id="create-client-configurations">Create client configurations<a class="headerlink" href="#create-client-configurations" title="Permanent link">¶</a></h3>
<p>You might notice that I didn’t specify how to actually connect the VPN. For that we need a configuration file similar to the <code>server.conf</code> file that we created.</p>
<p>The real way of doing this would be to run similar steps as the ones with <code>easy-rsa</code> locally, send them to the server, sign them, and retrieve them. Fuck all that, we’ll just create all configuration files on the server as I was mentioning earlier.</p>
<p>Also, the client configuration file has to match the server one (to some degree), to make this easier you can create a <code>client-common</code> file in <code>/etc/openvpn/server</code> with the following content:</p>
<pre><code>client
dev tun
remote 1.2.3.4 1194 udp # change this to match your ip and port
resolv-retry infinite
nobind
persist-key
persist-tun
remote-cert-tls server
auth SHA512
verb 3
</code></pre>
<p>Where you should make any changes necessary, depending on your configuration.</p>
<p>Now, we need a way to create and revoke new configuration files. For this I created a script, heavily based on one of the links I mentioned at the beginning. You can place these scripts anywhere you like, and you should take a look before running them because you’ll be running them with elevated privileges (sudo).</p>
<p>In a nutshell, what it does is: generate a new client certificate keypair, update the CRL and create a new <code>.ovpn</code> configuration file that consists on the <code>client-common</code> data and all of the required certificates; or, revoke an existing client and refresh the CRL. The file is placed under <code>~/ovpn</code>.</p>
<p>Create a new file with the following content (name it whatever you like) and don’t forget to make it executable (<code>chmod +x vpn_script</code>):</p>
<pre><code class="language-sh">#!/bin/sh
# Client ovpn configuration creation and revoking.
MODE=$1
if [ ! "$MODE" = "new" -a ! "$MODE" = "rev" ]; then
echo "$1 is not a valid mode, using default 'new'"
MODE=new
fi
CLIENT=${2:-guest}
if [ -z $2 ];then
echo "there was no client name passed as second argument, using 'guest' as default"
fi
# Expiration config.
EASYRSA_CERT_EXPIRE=3650
EASYRSA_CRL_DAYS=3650
# Current PWD.
CPWD=$PWD
cd /etc/easy-rsa/
if [ "$MODE" = "rev" ]; then
easyrsa --batch revoke $CLIENT
echo "$CLIENT revoked."
elif [ "$MODE" = "new" ]; then
easyrsa build-client-full $CLIENT nopass
# This is what actually generates the config file.
{
cat /etc/openvpn/server/client-common
echo "<ca>"
cat /etc/easy-rsa/pki/ca.crt
echo "</ca>"
echo "<cert>"
sed -ne '/BEGIN CERTIFICATE/,$ p' /etc/easy-rsa/pki/issued/$CLIENT.crt
echo "</cert>"
echo "<key>"
cat /etc/easy-rsa/pki/private/$CLIENT.key
echo "</key>"
echo "<tls-crypt>"
sed -ne '/BEGIN OpenVPN Static key/,$ p' /etc/openvpn/server/ta.key
echo "</tls-crypt>"
} > "$(eval echo ~${SUDO_USER:-$USER}/ovpn/$CLIENT.ovpn)"
eval echo "~${SUDO_USER:-$USER}/ovpn/$CLIENT.ovpn file generated."
fi
# Finish up, re-generates the crl
easyrsa gen-crl
chown nobody:nobody pki/crl.pem
chmod o+r pki/crl.pem
cd $CPWD
</code></pre>
<p>And the way to use is to run <code>bash vpn_script <mode> <client_name></code> where <code>mode</code> is <code>new</code> or <code>rev</code> (revoke) as sudo (when revoking, it doesn’t actually delete the <code>.ovpn</code> file in <code>~/ovpn</code>). Again, this is a little script that I put together, so you should check it out, it may need tweaks (specially depending on your directory structure for <code>easy-rsa</code>).</p>
<p>Now, just get the <code>.ovpn</code> file generated, import it to OpenVPN in your client of preference and you should have a working VPN service.</p>]]></content:encoded>
</item>
<item>
<title>Hoy me tocó desarrollo de personaje</title>
<link>https://blog.luevano.xyz/a/hoy_toco_desarrollo_personaje.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/hoy_toco_desarrollo_personaje.html</guid>
<pubDate>Wed, 28 Jul 2021 06:10:55 GMT</pubDate>
<category>Rant</category>
<category>Spanish</category>
<category>Update</category>
<description>Una breve historia sobre cómo estuvo mi día, porque me tocó desarrollo de personaje y lo quiero sacar del coraje que traigo.</description>
<content:encoded><![CDATA[<p>Sabía que hoy no iba a ser un día tan bueno, pero no sabía que iba a estar tan horrible; me tocó desarrollo de personaje y saqué el <em>bad ending</em>.</p>
<p>Básicamente tenía que cumplir dos misiones hoy: ir al banco a un trámite y vacunarme contra el Covid-19. Muy sencillas tareas.</p>
<p>Primero que nada me levanté de una pesadilla horrible en la que se puede decir que se me subió el muerto al querer despertar, esperé a que fuera casi la hora de salida de mi horario de trabajo, me bañé y fui directo al banco primero. Todo bien hasta aquí.</p>
<p>En el camino al banco, durante la plática con el conductor del Uber salió el tema del horario del banco. Yo muy tranquilo dije “pues voy algo tarde, pero sí alcanzo, cierran a las 5, ¿no?” a lo que me respondió el conductor “nel jefe, a las 4, y se van media hora antes”; quedé. Chequé y efectivamente cerraban a las 4. Entonces le dije que le iba a cambiar la ruta directo a donde me iba a vacunar, pero ya era muy tarde y quedaba para la dirección opuesta.”Ni pedo, ahí déjame y pido otro viaje, no te apures”, le dije y como siempre pues me deseó que se compusiera mi día; <strong>afortunadamente</strong> el banco sí estaba abierto para lo que tenía que hacer, así que fue un buen giro. Me puse muy feliz y asumí que sería un buen día, como me lo dijo mi conductor; <strong>literalmente NO SABÍA</strong>.</p>
<p>Salí feliz de poder haber completado esa misión y poder irme a vacunar. Pedí otro Uber a donde tenía que ir y todo bien. Me tocó caminar mucho porque la entrada estaba en punta de la chingada de donde me dejó el conductor, pero no había rollo, era lo de menos. Me desanimé cuando vi que había una cantidad estúpida de gente, era una fila que abarcaba todo el estacionamiento y daba demasiadas vueltas; “ni pedo”, dije, “si mucho me estaré aquí una hora, hora y media”… otra vez, <strong>literalmente NO SABÍA</strong>.</p>
<p>Pasó media hora y había avanzado lo que parecía ser un cuarto de la fila, entonces todo iba bien. Pues nel, había avanzado el equivalente a un octavo de la fila, este pedo no iba a salir en una hora-hora y media. Para acabarla de chingar era todo bajo el tan amado sol de Chiwawa. “No hay pedo, me entretengo tirando chal con alguien en el wasap”, pues no, aparentemente no cargué el celular y ya tenía 15-20% de batería… volví a quedar.</p>
<p>Se me acabó la pila, ya había pasado una hora y parecía que la fila era infinita, simplemente avanzábamos demasiado lento, a pesar de que los que venían atrás de mí repetían una y otra vez “mira, avanza bien rápido, ya mero llegamos”, ilusos. Duré aproximadamente 3 horas formado, aguantando conversaciones estúpidas a mi alrededor, gente quejándose por estar parada (yo también me estaba quejando pero dentro de mi cabeza), y por alguna razón iban familias completas de las cuales al final del día sólo uno o dos integrantes de la familia entraban a vacunarse.</p>
<p>En fin que se acabó la tortura y ya tocaba irse al cantón, todo bien. “No hay pedo, no me tocó irme en Uber, aquí agarro un camíon” pensé. Pero no, ningún camión pasó durante la hora que estuve esperando y de los 5 taxis que intenté parar <strong>NINGUNO</strong> se detuvo. Decidí irme caminado, ya qué más daba, en ese punto ya nada más era hacer corajes <em>dioquis</em>.</p>
<p>En el camino vi un Oxxo y decidí desviarme para comprar algo de tomar porque andaba bien deshidratado. En el mismo segundo que volteé para ir hacia el Oxxo pasó un camión volando y lo único que pensaba era que el conductor me decía “Jeje ni pedo:)”. Exploté, me acabé, simplemente perdí, saqué el <em>bad ending</em>.</p>
<p>Ya estaba harto y hasta iba a comprar un cargador para ya irme rápido, estaba cansado del día, simplemente ahí terminó la quest, había sacado el peor final. Lo bueno es que se me ocurrió pedirle al cajero un cargador y que me <em>tirara paro</em>. Todo bien, pedí mi Uber y llegué a mi casa sano y a salvo, pero con la peor rabia que me había dado en mucho tiempo. Simplemente ¿mi culo? explotado. Este día me tocó un desarrollo de personaje muy cabrón, se mamó el D*****o.</p>
<p>Lo único rescatable fue que había una (más bien como 5) chica muy guapa en la fila, lástima que los <em>stats</em> de mi personaje me tienen bloqueadas las conversaciones con desconocidos.</p>
<p>Y pues ya, este pex ya me sirvió para desahogarme, una disculpa por la redacción tan <em>pitera</em>. Sobres.</p>]]></content:encoded>
</item>
<item>
<title>Tenía este pex algo descuidado</title>
<link>https://blog.luevano.xyz/a/tenia_esto_descuidado.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/tenia_esto_descuidado.html</guid>
<pubDate>Sun, 18 Jul 2021 07:51:50 GMT</pubDate>
<category>Short</category>
<category>Spanish</category>
<category>Update</category>
<description>Nada más un update en el estado del blog y lo que he andado haciendo.</description>
<content:encoded><![CDATA[<p>Así es, tenía un poco descuidado este pex, siendo la razón principal que andaba ocupado con cosas de <em>la vida profesional</em>, ayay. Pero ya que ando un poco más despejado y menos estresado voy a seguir usando el blog y a ver qué más hago.</p>
<p>Tengo unas entradas pendientes que quiero hacer del estilo de “tutorial” o “how-to”, pero me lo he estado debatiendo, porque Luke ya empezó a hacerlo más de verdad en <a href="https://landchad.net/">landchad.net</a>, lo cual recomiendo bastante pues igual yo empecé a hacer esto por él (y por <a href="https://lmcj.xyz/"><mark>EL ELE EME</mark></a>); aunque la verdad pues es muy específico a como él hace las cosas y quizá sí puede haber diferencias, pero ya veré en estos días. La próxima que quiero hacer es sobre el VPN, porque no lo he <em>setupeado</em> desde que reinicié El Página Web y La Servidor, entonces acomodaré el VPN de nuevo y de pasada tiro entrada de eso.</p>
<p>También dejé un dibujo pendiente, que la neta lo dejé por 2 cosas: está bien cabrón (porque también lo quiero colorear) y porque estaba ocupado; de lo cuál ya sólo queda el <em>está bien cabrón</em> pero no he tenido el valor de retomarlo. Lo triste es que ya pasó el tiempo del hype y ya no tengo mucha motivación para terminarlo más que el hecho de que cuando lo termine empezaré a usar Clip Studio Paint en vez de Krita, porque compré una licencia ahora que estuvo en 50% de descuento.</p>
<p>Algo bueno es que me he estado sintiendo muy bien conmigo mismo últimamente, aunque casi no hable de eso. Sí hay una razón en específico, pero es una razón algo tonta. Espero así siga.</p>
<p><del>Ah, y también quería acomodarme una sección de comentarios, pero como siempre, todas las opciones están bien <em>bloated</em>, entonces pues me voy a hacer una en corto seguramente en Python para <em>el back</em>, MySQL para la base de datos y Javascript para la conexión acá en <em>el front</em>, algo tranqui.</del> <ins>Nel, siempre no ocupo esto, pa’ qué.</ins></p>
<p>Sobres pues.</p>]]></content:encoded>
</item>
<item>
<title>Set up an XMPP server with Prosody compatible with Conversations and Movim</title>
<link>https://blog.luevano.xyz/a/xmpp_server_with_prosody.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/xmpp_server_with_prosody.html</guid>
<pubDate>Wed, 09 Jun 2021 05:24:30 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up an XMPP server using Prosody on a server running Nginx, on Arch. This server will be compatible with at least Conversations and Movim.</description>
<content:encoded><![CDATA[<p><strong>Update</strong>: I no longer host this XMPP server as it consumed a lot of resources and I wasn’t using it that much. I’ll probably re-create it in the future, though.</p>
<p>Recently I set up an <a href="https://xmpp.org/">XMPP</a> server (and a Matrix one, too) for my personal use and for friends if they want one; made one for <a href="https://lmcj.xyz"><mark>EL ELE EME</mark></a> for example. So, here are the notes on how I set up the server that is compatible with the <a href="https://conversations.im/">Conversations</a> app and the <a href="https://movim.eu/">Movim</a> social network. You can see my addresses at <a href="https://luevano.xyz/contact.html">contact</a> and the XMPP compliance/score of the server.</p>
<p>One of the best resources I found that helped me a lot was <a href="https://community.hetzner.com/tutorials/prosody-debian9">Installing and Configuring Prosody XMPP Server on Debian 9</a>, the <a href="https://wiki.archlinux.org/title/Prosody">Arch Wiki</a> and the <a href="https://prosody.im/">oficial documentation</a>.</p>
<p>As with my other entries, this is under a server running Arch Linux, with the Nginx web server and Certbot certificates. And all commands here are executed as root, unless specified otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#prosody">Prosody</a></li>
<li><a href="#nginx-configuration-file">Nginx configuration file</a></li>
<li><a href="#coturn">Coturn</a></li>
<li><a href="#wrapping-up">Wrapping up</a></li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Same as with my other entries (<a href="https://luevano.xyz/a/website_with_nginx.html">website</a>, <a href="https://blog.luevano.xyz/a/mail_server_with_postfix.html">mail</a> and <a href="https://blog.luevano.xyz/a/git_server_with_cgit.html">git</a>) plus:</p>
<ul>
<li><strong>A</strong> and (optionally) <strong>AAA</strong> DNS records for:<ul>
<li><code>xmpp</code>: the actual XMPP server and the file upload service.</li>
<li><code>muc</code> (or <code>conference</code>): for multi-user chats.</li>
<li><code>pubsub</code>: the publish-subscribe service.</li>
<li><code>proxy</code>: a proxy in case one of the users needs it.</li>
<li><code>vjud</code>: user directory.</li>
</ul>
</li>
<li>(Optionally, but recommended) the following <strong>SRV</strong> DNS records; make sure it is pointing to an <strong>A</strong> or <strong>AAA</strong> record (matching the records from the last point, for example):<ul>
<li><code>_xmpp-client._tcp.{your.domain}.</code> for port <code>5222</code> pointing to <code>xmpp.{your.domain}.</code></li>
<li><code>_xmpp-server._tcp.{your.domain}.</code> for port <code>5269</code> pointing to <code>xmpp.{your.domain}.</code></li>
<li><code>_xmpp-server._tcp.muc.{your.domain}.</code> for port <code>5269</code> pointing to <code>xmpp.{your.domain}.</code></li>
</ul>
</li>
<li>SSL certificates for the previous subdomains; similar that with my other entries just create the appropriate <code>prosody.conf</code> (where <code>server_name</code> will be all the subdomains defined above) file and run <code>certbot --nginx</code>. You can find the example configuration file almost at the end of this entry.</li>
<li>Email addresses for <code>admin</code>, <code>abuse</code>, <code>contact</code>, <code>security</code>, etc. Or use your own email for all of them, doesn’t really matter much as long as you define them in the configuration and are valid, I have aliases so those emails are forwarded to me.</li>
<li>Allow ports <code>5000</code>, <code>5222</code>, <code>5269</code>, <code>5280</code> and <code>5281</code> for <a href="https://prosody.im/doc/ports">Prosody</a> and, <code>3478</code> and <code>5349</code> for <a href="https://webrtc.org/getting-started/turn-server">Turnserver</a> which are the defaults for <code>coturn</code>.</li>
</ul>
<h2 id="prosody">Prosody<a class="headerlink" href="#prosody" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Prosody">Prosody</a> is an implementation of the XMPP protocol that is flexible and extensible.</p>
<p>Install the <code>prosody</code> package (with optional dependencies) and the <code>mercurial</code> package:</p>
<pre><code class="language-sh">pacman -S prosody, mercurial, lua52-sec, lua52-dbi, lua52-zlib
</code></pre>
<p>We need mercurial to be able to download and update the extra modules needed to make the server compliant with <code>conversations.im</code> and <code>mov.im</code>. Go to <code>/var/lib/prosody</code>, clone the latest Prosody modules repository and prepare the directories:</p>
<pre><code class="language-sh">cd /var/lib/prosody
hg clone https://hg.prosody.im/prosody-modules modules-available
mkdir modules-enabled
</code></pre>
<p>You can see that I follow a similar approach that I used with Nginx and the server configuration, where I have all the modules available in a directory, and make a symlink to another to keep track of what is being used. You can update the repository by running <code>hg pull --update</code> while inside the <code>modules-available</code> directory (similar to Git).</p>
<p>Make symbolic links to the following modules:</p>
<pre><code>ln -s /var/lib/prosody/modules-available/{module_name} /var/lib/prosody/modules-enabled/
...
</code></pre>
<ul>
<li>Modules (<code>{module_name}</code>):<ul>
<li><code>mod_bookmarks</code></li>
<li><code>mod_cache_c2s_caps</code></li>
<li><code>mod_checkcerts</code></li>
<li><code>mod_cloud_notify</code></li>
<li><code>mod_csi_battery_saver</code></li>
<li><code>mod_default_bookmarks</code></li>
<li><code>mod_external_services</code></li>
<li><code>mod_http_avatar</code></li>
<li><code>mod_http_pep_avatar</code></li>
<li><code>mod_http_upload</code></li>
<li><code>mod_http_upload_external</code></li>
<li><code>mod_idlecompat</code></li>
<li><code>mod_muc_limits</code></li>
<li><code>mod_muc_mam_hints</code></li>
<li><code>mod_muc_mention_notifications</code></li>
<li><code>mod_presence_cache</code></li>
<li><code>mod_pubsub_feeds</code></li>
<li><code>mod_pubsub_text_interface</code></li>
<li><code>mod_smacks</code></li>
<li><code>mod_strict_https</code></li>
<li><code>mod_vcard_muc</code></li>
<li><code>mod_vjud</code></li>
<li><code>mod_watchuntrusted</code></li>
</ul>
</li>
</ul>
<p>And add other modules if needed, but these work for the apps that I mentioned. You should also change the permissions for these files:</p>
<pre><code class="language-sh">chown -R prosody:prosody /var/lib/prosody
</code></pre>
<p>Now, configure the server by editing the <code>/etc/prosody/prosody.cfg.lua</code> file. It’s a bit tricky to configure, so here is my configuration file (lines starting with <code>--</code> are comments). Make sure to change according to your domain, and maybe preferences. Read each line and each comment to know what’s going on, It’s easier to explain it with comments in the file itself than strip it in a lot of pieces.</p>
<p>And also, note that the configuration file has a “global” section and a per “virtual server”/”component” section, basically everything above all the VirtualServer/Component sections are global, and bellow each VirtualServer/Component, corresponds to that section.</p>
<pre><code>-- important for systemd
daemonize = true
pidfile = "/run/prosody/prosody.pid"
-- or your account, not that this is an xmpp jid, not email
admins = { "admin@your.domain" }
contact_info = {
abuse = { "mailto:abuse@your.domain", "xmpp:abuse@your.domain" };
admin = { "mailto:admin@your.domain", "xmpp:admin@your.domain" };
admin = { "mailto:feedback@your.domain", "xmpp:feedback@your.domain" };
security = { "mailto:security@your.domain" };
support = { "mailto:support@your.domain", "xmpp:support@muc.your.domain" };
}
-- so prosody look up the plugins we added
plugin_paths = { "/var/lib/prosody/modules-enabled" }
modules_enabled = {
-- Generally required
"roster"; -- Allow users to have a roster. Recommended ;)
"saslauth"; -- Authentication for clients and servers. Recommended if you want to log in.
"tls"; -- Add support for secure TLS on c2s/s2s connections
"dialback"; -- s2s dialback support
"disco"; -- Service discovery
-- Not essential, but recommended
"carbons"; -- Keep multiple clients in sync
"pep"; -- Enables users to publish their avatar, mood, activity, playing music and more
"private"; -- Private XML storage (for room bookmarks, etc.)
"blocklist"; -- Allow users to block communications with other users
"vcard4"; -- User profiles (stored in PEP)
"vcard_legacy"; -- Conversion between legacy vCard and PEP Avatar, vcard
"limits"; -- Enable bandwidth limiting for XMPP connections
-- Nice to have
"version"; -- Replies to server version requests
"uptime"; -- Report how long server has been running
"time"; -- Let others know the time here on this server
"ping"; -- Replies to XMPP pings with pongs
"register"; -- Allow users to register on this server using a client and change passwords
"mam"; -- Store messages in an archive and allow users to access it
"csi_simple"; -- Simple Mobile optimizations
-- Admin interfaces
"admin_adhoc"; -- Allows administration via an XMPP client that supports ad-hoc commands
--"admin_telnet"; -- Opens telnet console interface on localhost port 5582
-- HTTP modules
"http"; -- Explicitly enable http server.
"bosh"; -- Enable BOSH clients, aka "Jabber over HTTP"
"websocket"; -- XMPP over WebSockets
"http_files"; -- Serve static files from a directory over HTTP
-- Other specific functionality
"groups"; -- Shared roster support
"server_contact_info"; -- Publish contact information for this service
"announce"; -- Send announcement to all online users
"welcome"; -- Welcome users who register accounts
"watchregistrations"; -- Alert admins of registrations
"motd"; -- Send a message to users when they log in
--"legacyauth"; -- Legacy authentication. Only used by some old clients and bots.
--"s2s_bidi"; -- not yet implemented, have to wait for v0.12
"bookmarks";
"checkcerts";
"cloud_notify";
"csi_battery_saver";
"default_bookmarks";
"http_avatar";
"idlecompat";
"presence_cache";
"smacks";
"strict_https";
--"pep_vcard_avatar"; -- not compatible with this version of pep, wait for v0.12
"watchuntrusted";
"webpresence";
"external_services";
}
-- only if you want to disable some modules
modules_disabled = {
-- "offline"; -- Store offline messages
-- "c2s"; -- Handle client connections
-- "s2s"; -- Handle server-to-server connections
-- "posix"; -- POSIX functionality, sends server to background, enables syslog, etc.
}
external_services = {
{
type = "stun",
transport = "udp",
host = "proxy.your.domain",
port = 3478
}, {
type = "turn",
transport = "udp",
host = "proxy.your.domain",
port = 3478,
-- you could decide this now or come back later when you install coturn
secret = "YOUR SUPER SECRET TURN PASSWORD"
}
}
--- general global configuration
http_ports = { 5280 }
http_interfaces = { "*", "::" }
https_ports = { 5281 }
https_interfaces = { "*", "::" }
proxy65_ports = { 5000 }
proxy65_interfaces = { "*", "::" }
http_default_host = "xmpp.your.domain"
http_external_url = "https://xmpp.your.domain/"
-- or if you want to have it somewhere else, change this
https_certificate = "/etc/prosody/certs/xmpp.your.domain.crt"
hsts_header = "max-age=31556952"
cross_domain_bosh = true
--consider_bosh_secure = true
cross_domain_websocket = true
--consider_websocket_secure = true
trusted_proxies = { "127.0.0.1", "::1", "192.169.1.1" }
pep_max_items = 10000
-- this is disabled by default, and I keep it like this, depends on you
--allow_registration = true
-- you might want this options as they are
c2s_require_encryption = true
s2s_require_encryption = true
s2s_secure_auth = false
--s2s_insecure_domains = { "insecure.example" }
--s2s_secure_domains = { "jabber.org" }
-- where the certificates are stored (/etc/prosody/certs by default)
certificates = "certs"
checkcerts_notify = 7 -- ( in days )
-- rate limits on connections to the server, these are my personal settings, because by default they were limited to something like 30kb/s
limits = {
c2s = {
rate = "2000kb/s";
};
s2sin = {
rate = "5000kb/s";
};
s2sout = {
rate = "5000kb/s";
};
}
-- again, this could be yourself, it is a jid
unlimited_jids = { "admin@your.domain" }
authentication = "internal_hashed"
-- if you don't want to use sql, change it to internal and comment the second line
-- since this is optional, i won't describe how to setup mysql or setup the user/database, that would be out of the scope for this entry
storage = "sql"
sql = { driver = "MySQL", database = "prosody", username = "prosody", password = "PROSODY USER SECRET PASSWORD", host = "localhost" }
archive_expires_after = "4w" -- configure message archive
max_archive_query_results = 20;
mam_smart_enable = true
default_archive_policy = "roster" -- archive only messages from users who are in your roster
-- normally you would like at least one log file of certain level, but I keep all of them, the default is only the info = "*syslog" one
log = {
info = "*syslog";
warn = "prosody.warn";
error = "prosody.err";
debug = "prosody.debug";
-- "*console"; -- Needs daemonize=false
}
-- cloud_notify
push_notification_with_body = false -- Whether or not to send the message body to remote pubsub node
push_notification_with_sender = false -- Whether or not to send the message sender to remote pubsub node
push_max_errors = 5 -- persistent push errors are tolerated before notifications for the identifier in question are disabled
push_max_devices = 5 -- number of allowed devices per user
-- by default every user on this server will join these muc rooms
default_bookmarks = {
{ jid = "room@muc.your.domain", name = "The Room" };
{ jid = "support@muc.your.domain", name = "Support Room" };
}
-- could be your jid
untrusted_fail_watchers = { "admin@your.domain" }
untrusted_fail_notification = "Establishing a secure connection from $from_host to $to_host failed. Certificate hash: $sha1. $errors"
----------- Virtual hosts -----------
VirtualHost "your.domain"
name = "Prosody"
http_host = "xmpp.your.domain"
disco_items = {
{ "your.domain", "Prosody" };
{ "muc.your.domain", "MUC Service" };
{ "pubsub.your.domain", "Pubsub Service" };
{ "proxy.your.domain", "SOCKS5 Bytestreams Service" };
{ "vjud.your.domain", "User Directory" };
}
-- Multi-user chat
Component "muc.your.domain" "muc"
name = "MUC Service"
modules_enabled = {
--"bob"; -- not compatible with this version of Prosody
"muc_limits";
"muc_mam"; -- message archive in muc, again, a placeholder
"muc_mam_hints";
"muc_mention_notifications";
"vcard_muc";
}
restrict_room_creation = false
muc_log_by_default = true
muc_log_presences = false
log_all_rooms = false
muc_log_expires_after = "1w"
muc_log_cleanup_interval = 4 * 60 * 60
-- Upload
Component "xmpp.your.domain" "http_upload"
name = "Upload Service"
http_host= "xmpp.your.domain"
-- you might want to change this, these are numbers in bytes, so 10MB and 100MB respectively
http_upload_file_size_limit = 1024*1024*10
http_upload_quota = 1024*1024*100
-- Pubsub
Component "pubsub.your.domain" "pubsub"
name = "Pubsub Service"
pubsub_max_items = 10000
modules_enabled = {
"pubsub_feeds";
"pubsub_text_interface";
}
-- personally i don't have any feeds configured
feeds = {
-- The part before = is used as PubSub node
--planet_jabber = "http://planet.jabber.org/atom.xml";
--prosody_blog = "http://blog.prosody.im/feed/atom.xml";
}
-- Proxy
Component "proxy.your.domain" "proxy65"
name = "SOCKS5 Bytestreams Service"
proxy65_address = "proxy.your.domain"
-- Vjud, user directory
Component "vjud.your.domain" "vjud"
name = "User Directory"
vjud_mode = "opt-in"
</code></pre>
<p>You <mark>HAVE</mark> to read all of the configuration file, because there are a lot of things that you need to change to make it work with your server/domain. Test the configuration file with:</p>
<pre><code class="language-sh">luac5.2 -p /etc/prosody/prosody.cfg.lua
</code></pre>
<p>Notice that by default <code>prosody</code> will look up certificates that look like <code>sub.your.domain</code>, but if you get the certificates like I do, you’ll have a single certificate for all subdomains, and by default it is in <code>/etc/letsencrypt/live</code>, which has some strict permissions. So, to import it you can run:</p>
<pre><code class="language-sh">prosodyctl --root cert import /etc/letsencrypt/live
</code></pre>
<p>Ignore the complaining about not finding the subdomain certificates and note that you will have to run that command on each certificate renewal, to automate this, add the <code>--deploy-hook</code> flag to your automated Certbot renewal system; for me it’s a <code>systemd</code> timer with the following <code>certbot.service</code>:</p>
<pre><code class="language-ini">[Unit]
Description=Let's Encrypt renewal
[Service]
Type=oneshot
ExecStart=/usr/bin/certbot renew --quiet --agree-tos --deploy-hook "systemctl reload nginx.service && prosodyctl --root cert import /etc/letsencrypt/live"
</code></pre>
<p>And if you don’t have it already, the <code>certbot.timer</code>:</p>
<pre><code class="language-ini">[Unit]
Description=Twice daily renewal of Let's Encrypt's certificates
[Timer]
OnCalendar=0/12:00:00
RandomizedDelaySec=1h
Persistent=true
[Install]
WantedBy=timers.target
</code></pre>
<p>Also, go to the <code>certs</code> directory and make the appropriate symbolic links:</p>
<pre><code class="language-sh">cd /etc/prosody/certs
ln -s your.domain.crt SUBDOMAIN.your.domain.crt
ln -s your.domain.key SUBDOMAIN.your.domain.key
...
</code></pre>
<p>That’s basically all the configuration that needs Prosody itself, but we still have to configure Nginx and Coturn before starting/enabling the <code>prosody</code> service.</p>
<h2 id="nginx-configuration-file">Nginx configuration file<a class="headerlink" href="#nginx-configuration-file" title="Permanent link">¶</a></h2>
<p>Since this is not an ordinary configuration file I’m going to describe this, too. Your <code>prosody.conf</code> file should have the following location blocks under the main server block (the one that listens to HTTPS):</p>
<pre><code class="language-nginx"># HTTPS server block
server {
root /var/www/prosody/;
server_name xmpp.luevano.xyz muc.luevano.xyz pubsub.luevano.xyz vjud.luevano.xyz proxy.luevano.xyz;
index index.html;
# for extra https discovery (XEP-0256)
location /.well-known/acme-challenge {
allow all;
}
# bosh specific
location /http-bind {
proxy_pass https://localhost:5281/http-bind;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
tcp_nodelay on;
}
# websocket specific
location /xmpp-websocket {
proxy_pass https://localhost:5281/xmpp-websocket;
proxy_http_version 1.1;
proxy_set_header Connection "Upgrade";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900s;
}
# general proxy
location / {
proxy_pass https://localhost:5281;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
}
...
# Certbot stuff
}
# HTTP server block (the one that certbot creates)
server {
...
}
</code></pre>
<p>Also, you need to add the following to your actual <code>your.domain</code> (this cannot be a subdomain) configuration file:</p>
<pre><code class="language-nginx">server {
...
location /.well-known/host-meta {
default_type 'application/xrd+xml';
add_header Access-Control-Allow-Origin '*' always;
}
location /.well-known/host-meta.json {
default_type 'application/jrd+json';
add_header Access-Control-Allow-Origin '*' always;
}
...
}
</code></pre>
<p>And you will need the following <code>host-meta</code> and <code>host-meta.json</code> files inside the <code>.well-known/acme-challenge</code> directory for <code>your.domain</code> (following my nomenclature: <code>/var/www/yourdomaindir/.well-known/acme-challenge/</code>).</p>
<p>For <code>host-meta</code> file:</p>
<pre><code class="language-xml"><?xml version='1.0' encoding='utf-8'?>
<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>
<Link rel="urn:xmpp:alt-connections:xbosh"
href="https://xmpp.your.domain:5281/http-bind" />
<Link rel="urn:xmpp:alt-connections:websocket"
href="wss://xmpp.your.domain:5281/xmpp-websocket" />
</XRD>
</code></pre>
<p>And <code>host-meta.json</code> file:</p>
<pre><code class="language-json">{
"links": [
{
"rel": "urn:xmpp:alt-connections:xbosh",
"href": "https://xmpp.your.domain:5281/http-bind"
},
{
"rel": "urn:xmpp:alt-connections:websocket",
"href": "wss://xmpp.your.domain:5281/xmpp-websocket"
}
]
}
</code></pre>
<p>Remember to have your <code>prosody.conf</code> file symlinked (or discoverable by Nginx) to the <code>sites-enabled</code> directory. You can now test and restart your <code>nginx</code> service (and test the configuration, optionally):</p>
<pre><code class="language-sh">nginx -t
systemctl restart nginx.service
</code></pre>
<h2 id="coturn">Coturn<a class="headerlink" href="#coturn" title="Permanent link">¶</a></h2>
<p><a href="https://github.com/coturn/coturn">Coturn</a> is the implementation of TURN and STUN server, which in general is for (at least in the XMPP world) voice support and external service discovery.</p>
<p>Install the <code>coturn</code> package:</p>
<pre><code class="language-sh">pacman -S coturn
</code></pre>
<p>You can modify the configuration file (located at <code>/etc/turnserver/turnserver.conf</code>) as desired, but at least you need to make the following changes (uncomment or edit):</p>
<pre><code class="language-ini">use-auth-secret
realm=proxy.your.domain
static-auth-secret=YOUR SUPER SECRET TURN PASSWORD
</code></pre>
<p>I’m sure there is more configuration to be made, like using SQL to store data and whatnot, but for now this is enough for me. Note that you may not have some functionality that’s needed to create dynamic users to use the TURN server, and to be honest I haven’t tested this since I don’t use this feature in my XMPP clients, but if it doesn’t work, or you know of an error or missing configuration don’t hesitate to <a href="https://luevano.xyz/contact.html">contact me</a>.</p>
<p>Start/enable the <code>turnserver</code> service:</p>
<pre><code class="language-sh">systemctl start turnserver.service
systemctl enable turnserver.service
</code></pre>
<p>You can test if your TURN server works at <a href="https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/">Trickle ICE</a>. You may need to add a user in the <code>turnserver.conf</code> to test this.</p>
<h2 id="wrapping-up">Wrapping up<a class="headerlink" href="#wrapping-up" title="Permanent link">¶</a></h2>
<p>At this point you should have a working XMPP server, start/enable the <code>prosody</code> service now:</p>
<pre><code class="language-sh">systemctl start prosody.service
systemctl enable prosody.service
</code></pre>
<p>And you can add your first user with the <code>prosodyctl</code> command (it will prompt you to add a password):</p>
<pre><code class="language-sh">prosodyctl adduser user@your.domain
</code></pre>
<p>You may want to add a <code>compliance</code> user, so you can check if your server is set up correctly. To do so, go to <a href="https://compliance.conversations.im/add/">XMPP Compliance Tester</a> and enter the <code>compliance</code> user credentials. It should have similar compliance score to mine:</p>
<p><a href='https://compliance.conversations.im/server/luevano.xyz'><img src='https://compliance.conversations.im/badge/luevano.xyz'></a></p>
<p>Additionally, you can test the security of your server in <a href="https://xmpp.net/index.php">IM Observatory</a>, here you only need to specify your <code>domain.name</code> (not <code>xmpp.domain.name</code>, if you set up the <strong>SRV</strong> DNS records correctly). Again, it should have a similar score to mine:</p>
<p><a href='https://xmpp.net/result.php?domain=luevano.xyz&type=client'><img src='https://xmpp.net/badge.php?domain=luevano.xyz' alt='xmpp.net score' /></a></p>
<p>You can now log in into your XMPP client of choice, if it asks for the server it should be <code>xmpp.your.domain</code> (or <code>your.domain</code> for some clients) and your login credentials <code>you@your.domain</code> and the password you chose (which you can change in most clients).</p>
<p>That’s it, send me a message at <a href="xmpp:david@luevano.xyz">david@luevano.xyz</a> if you were able to set up the server successfully.</p>]]></content:encoded>
</item>
<item>
<title>Al fin ya me acomodé la página pa' los dibujos</title>
<link>https://blog.luevano.xyz/a/acomodada_la_pagina_de_arte.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/acomodada_la_pagina_de_arte.html</guid>
<pubDate>Sun, 06 Jun 2021 19:06:09 GMT</pubDate>
<category>Short</category>
<category>Spanish</category>
<category>Update</category>
<description>Actualización en el estado de la página, en este caso sobre la existencia de una nueva página para los dibujos y arte en general.</description>
<content:encoded><![CDATA[<p>Así es, ya quedó acomodado el sub-dominio <code>art.luevano.xyz</code> pos pal <a href="https://art.luevano.xyz">arte</a> veda. Entonces pues ando feliz por eso.</p>
<p>Este pedo fue gracias a que me reescribí la forma en la que <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> maneja los templates, ahora uso el sistema de <a href="https://jinja.palletsprojects.com/en/3.1.x/"><code>jinja</code></a> en vez del cochinero que hacía antes.</p>
<p>Y pues nada más eso, aquí está el <a href="https://art.luevano.xyz/a/elephant_octopus.html">primer post</a> y por supuesto acá está el link del RSS <a href="https://art.luevano.xyz/rss.xml">https://art.luevano.xyz/rss.xml</a>.</p>]]></content:encoded>
</item>
<item>
<title>Así nomás está quedando el página</title>
<link>https://blog.luevano.xyz/a/asi_nomas_esta_quedando.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/asi_nomas_esta_quedando.html</guid>
<pubDate>Fri, 04 Jun 2021 08:24:03 GMT</pubDate>
<category>Short</category>
<category>Spanish</category>
<category>Update</category>
<description>Actualización en el estado de la página, el servidor de XMPP y Matrix que me acomodé y próximas cosas que quiero hacer.</description>
<content:encoded><![CDATA[<p>Estuve acomodando un poco más el <em>sItIo</em>, al fin agregué la “sección” de <a href="https://luevano.xyz/contact.html">contact</a> y de <a href="https://luevano.xyz/donate.html">donate</a> por si hay algún loco que quiere tirar varo.</p>
<p>También me puse a acomodar un servidor de <a href="https://xmpp.org/">XMPP</a> el cual, en pocas palabras, es un protocolo de mensajería instantánea (y más) descentralizado, por lo cual cada quien puede hacer una cuenta en el servidor que quiera y conectarse con cuentas creadas en otro servidor… exacto, como con los correos electrónicos. Y esto está perro porque si tú tienes tu propio server, así como con uno de correo electrónico, puedes controlar qué características tiene, quiénes pueden hacer cuenta, si hay <em>end-to-end encryption</em> (o mínimo <em>end-to-server</em>), entre un montón de otras cosas.</p>
<p>Ahorita este server es SUMISO (<em>compliant</em> en español, jeje) para jalar con la app <a href="https://conversations.im/">conversations</a> y con la red social <a href="https://movim.eu/">movim</a>, pero realmente funcionaría con casi cualquier cliente de XMPP, amenos que ese cliente implemente algo que no tiene mi server. Y también acomodé un server de <a href="https://matrix.org/">Matrix</a> que es muy similar pero es bajo otro protocolo y se siente más como un discord/slack (al menos en el <a href="https://element.io/">element</a>), muy chingón también.</p>
<p>Si bien aún quedan cosas por hacer sobre estos dos servers que me acomodé (además de hacerles unas entradas para documentar cómo lo hice), quiero moverme a otra cosa que sería acomodar una sección de dibujos, lo cual en teoría es bien sencillo, pero como quiero poder automatizar la publicación de estos, quiero modificar un poco el <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> para que jale chido para este pex.</p>
<p>Ya por último también quiero moverle un poco al CSS, porque lo dejé en un estado muy culerón y quiero meterle/ajustar unas cosas para que quede más limpio y medianamente bonito… <em>dentro de lo que cabe porque evidentemente me vale verga si se ve como una página del 2000</em>.</p>
<p><strong>Actualización</strong>: Ya tumbé el servidor de XMPP porque consumía bastantes recursos y no lo usaba tanto, si en un futuro consigo un mejor servidor podría volver a hostearlo.</p>]]></content:encoded>
</item>
<item>
<title>I'm using a new blogging system</title>
<link>https://blog.luevano.xyz/a/new_blogging_system.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/new_blogging_system.html</guid>
<pubDate>Fri, 28 May 2021 03:21:39 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Tools</category>
<category>Update</category>
<description>I created a new blogging system called pyssg, which is based on what I was using but, to be honest, better.</description>
<content:encoded><![CDATA[<p>So, I was tired of working with <code>ssg</code> (and then <code>sbg</code> which was a modified version of <code>ssg</code> that I “wrote”), for one general reason: not being able to extend it as I would like; and not just dumb little stuff, I wanted to be able to have more control, to add tags (which another tool that I found does: <code>blogit</code>), and even more in a future.</p>
<p>The solution? Write a new program “from scratch” in <em>pYtHoN</em>. Yes it is bloated, yes it is in its early stages, but it works just as I want it to work, and I’m pretty happy so far with the results and have with even more ideas in mind to “optimize” and generally clean my wOrKfLoW to post new blog entries. I even thought of using it for posting into a “feed” like gallery for drawings or pictures in general.</p>
<p>I called it <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, because it sounds nice and it wasn’t taken in the PyPi. It is just a terminal program that reads either a configuration file or the options passed as flags when calling the program.</p>
<p>It still uses Markdown files because I find them very easy to work with. And instead of just having a “header” and a “footer” applied to each parsed entry, you will have templates (generated with the program) for each piece that I thought made sense (idea taken from <code>blogit</code>): the common header and footer, the common header and footer for each entry and, header, footer and list elements for articles and tags. When parsing the Markdown file these templates are applied and stitched together to make a single HTML file. Also generates an RSS feed and the <code>sitemap.xml</code> file, which is nice.</p>
<p>It might sound convoluted, but it works pretty well, with of course room to improve; I’m open to suggestions, issue reporting or direct contributions <a href="https://github.com/luevano/pyssg">here</a>. For now, it is only tested on Linux (and don’t think on making it work on windows, but feel free to do PR for the compatibility).</p>
<p>That’s it for now, the new RSS feed is available here: <a href="https://blog.luevano.xyz/rss.xml">https://blog.luevano.xyz/rss.xml</a>.</p>
<p><strong>Update</strong>: Since writing this entry, <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a> has evolved quite a bit, so not everything described here is still true. For the latest updates check the newest entries or the git repository itself.</p>]]></content:encoded>
</item>
<item>
<title>Set up a Git server and cgit front-end</title>
<link>https://blog.luevano.xyz/a/git_server_with_cgit.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/git_server_with_cgit.html</guid>
<pubDate>Sun, 21 Mar 2021 19:00:29 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to create a Git server using cgit on a server running Nginx, on Arch. This is a follow up on post about creating a website with Nginx and Certbot.</description>
<content:encoded><![CDATA[<p>My git server is all I need to setup to actually <em>kill</em> my other server (I’ve been moving from servers on these last 2-3 blog entries), that’s why I’m already doing this entry. I’m basically following <a href="https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server">git’s guide on setting up a server</a> plus some specific stuff for <mark>btw i use</mark> Arch Linux (<a href="https://wiki.archlinux.org/index.php/Git_server#Web_interfaces">Arch Linux Wiki: Git server</a> and <a href="https://miracoin.wordpress.com/2014/11/25/step-by-step-guide-on-setting-up-git-server-in-arch-linux-pushable/">Step by step guide on setting up git server in arch linux (pushable)</a>).</p>
<p>Note that this is mostly for personal use, so there’s no user/authentication control other than that of normal <code>ssh</code>. And as with the other entries, most if not all commands here are run as root unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#git">Git</a></li>
<li><a href="#cgit">Cgit</a><ul>
<li><a href="#cgits-file-rendering">Cgit’s file rendering</a></li>
</ul>
</li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>I might get tired of saying this (it’s just copy paste, basically)… but you will need the same prerequisites as before (check my <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website</a> and <a href="https://blog.luevano.xyz/a/mail_server_with_postfix.html">mail</a> entries), with the extras:</p>
<ul>
<li>(Optional, if you want a “front-end”) A <strong>CNAME</strong> for “git” and (optionally) “www.git”, or some other name for your sub-domains.</li>
<li>An SSL certificate, if you’re following the other entries, add a <code>git.conf</code> and run <code>certbot --nginx</code> to extend the certificate.</li>
</ul>
<h2 id="git">Git<a class="headerlink" href="#git" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/git">Git</a> is a version control system.</p>
<p>If not installed already, install the <code>git</code> package:</p>
<pre><code class="language-sh">pacman -S git
</code></pre>
<p>On Arch Linux, when you install the <code>git</code> package, a <code>git</code> user is automatically created, so all you have to do is decide where you want to store the repositories, for me, I like them to be on <code>/home/git</code> like if <code>git</code> was a “normal” user. So, create the <code>git</code> folder (with corresponding permissions) under <code>/home</code> and set the <code>git</code> user’s home to <code>/home/git</code>:</p>
<pre><code class="language-sh">mkdir /home/git
chown git:git /home/git
usermod -d /home/git git
</code></pre>
<p>Also, the <code>git</code> user is “expired” by default and will be locked (needs a password), change that with:</p>
<pre><code class="language-sh">chage -E -1 git
passwd git
</code></pre>
<p>Give it a strong one and remember to use <code>PasswordAuthentication no</code> for <code>ssh</code> (as you should). Create the <code>.ssh/authorized_keys</code> for the <code>git</code> user and set the permissions accordingly:</p>
<pre><code class="language-sh">mkdir /home/git/.ssh
chmod 700 /home/git/.ssh
touch /home/git/.ssh/authorized_keys
chmod 600 /home/git/.ssh/authorized_keys
chown -R git:git /home/git
</code></pre>
<p>Now is a good idea to copy over your local SSH public keys to this file, to be able to push/pull to the repositories. Do it by either manually copying it or using <code>ssh</code>‘s built in <code>ssh-copy-id</code> (for that you may want to check your <code>ssh</code> configuration in case you don’t let people access your server with user/password).</p>
<p>Next, and almost finally, we need to edit the <code>git-daemon</code> service, located at <code>/usr/lib/systemd/system/</code> (called <code>git-daemon@.service</code>):</p>
<pre><code class="language-ini">...
ExecStart=-/usr/lib/git-core/git-daemon --inetd --export-all --base-path=/home/git --enable=receive-pack
...
</code></pre>
<p>I just appended <code>--enable=receive-pack</code> and note that I also changed the <code>--base-path</code> to reflect where I want to serve my repositories from (has to match what you set when changing <code>git</code> user’s home).</p>
<p>Now, go ahead and start and enable the <code>git-daemon</code> socket:</p>
<pre><code class="language-sh">systemctl start git-daemon.socket
systemctl enable git-daemon.socket
</code></pre>
<p>You’re basically done. Now you should be able to push/pull repositories to your server… except, you haven’t created any repository in your server, that’s right, they’re not created automatically when trying to push. To do so, you have to run (while inside <code>/home/git</code>):</p>
<pre><code class="language-sh">git init --bare {repo_name}.git
chown -R git:git {repo_name}.git
</code></pre>
<p><mark>Those two lines above will need to be run each time you want to add a new repository to your server</mark>. There are options to “automate” this but I like it this way.</p>
<p>After that you can already push/pull to your repository. I have my repositories (locally) set up so I can push to more than one remote at the same time (my server, GitHub, GitLab, etc.); to do so, check <a href="https://gist.github.com/rvl/c3f156e117e22a25f242">this gist</a>.</p>
<h2 id="cgit">Cgit<a class="headerlink" href="#cgit" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Cgit">Cgit</a> is a fast web interface for git.</p>
<p>This is optionally since it’s only for the web application.</p>
<p>Install the <code>cgit</code> and <code>fcgiwrap</code> packages:</p>
<pre><code class="language-sh">pacman -S cgit fcgiwrap
</code></pre>
<p>Now, just start and enable the <code>fcgiwrap</code> socket:</p>
<pre><code class="language-sh">systemctl start fcgiwrap.socket
systemctl enable fcgiwrap.socket
</code></pre>
<p>Next, create the <code>git.conf</code> as stated in my <a href="https://blog.luevano.xyz/a/website_with_nginx.html">nginx setup entry</a>. Add the following lines to your <code>git.conf</code> file:</p>
<pre><code class="language-nginx">server {
listen 80;
listen [::]:80;
root /usr/share/webapps/cgit;
server_name {yoursubdomain}.{yourdomain};
try_files $uri @cgit;
location @cgit {
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root/cgit.cgi;
fastcgi_param PATH_INFO $uri;
fastcgi_param QUERY_STRING $args;
fastcgi_param HTTP_HOST $server_name;
fastcgi_pass unix:/run/fcgiwrap.sock;
}
}
</code></pre>
<p>Where the <code>server_name</code> line depends on you, I have mine setup to <code>git.luevano.xyz</code> and <code>www.git.luevano.xyz</code>. Optionally run <code>certbot --nginx</code> to get a certificate for those domains if you don’t have already.</p>
<p>Now, all that’s left is to configure <code>cgit</code>. Create the configuration file <code>/etc/cgitrc</code> with the following content (my personal options, pretty much the default):</p>
<pre><code class="language-apache">css=/cgit.css
logo=/cgit.png
enable-http-clone=1
# robots=noindex, nofollow
virtual-root=/
repo.url={url}
repo.path={dir_path}
repo.owner={owner}
repo.desc={short_description}
...
</code></pre>
<p>Where you can uncomment the <code>robots</code> line to not let web crawlers (like Google’s) to index your <code>git</code> web app. And at the end keep all your repositories (the ones you want to make public), for example for my <a href="https://git.luevano.xyz/.dots"><em>dotfiles</em></a> I have:</p>
<pre><code class="language-apache">...
repo.url=.dots
repo.path=/home/git/.dots.git
repo.owner=luevano
repo.desc=These are my personal dotfiles.
...
</code></pre>
<p>Otherwise you could let <code>cgit</code> to automatically detect your repositories (you have to be careful if you want to keep “private” repos) using the option <code>scan-path</code> and setup <code>.git/description</code> for each repository. For more, you can check <a href="https://man.archlinux.org/man/cgitrc.5">cgitrc(5)</a>.</p>
<h3 id="cgits-file-rendering">Cgit’s file rendering<a class="headerlink" href="#cgits-file-rendering" title="Permanent link">¶</a></h3>
<p>By default you can’t see the files on the site, you need a highlighter to render the files, I use <code>highlight</code>. Install the <code>highlight</code> package:</p>
<pre><code class="language-sh">pacman -S highlight
</code></pre>
<p>Copy the <code>syntax-highlighting.sh</code> script to the corresponding location (basically adding <code>-edited</code> to the file):</p>
<pre><code class="language-sh">cp /usr/lib/cgit/filters/syntax-highlighting.sh /usr/lib/cgit/filters/syntax-highlighting-edited.sh
</code></pre>
<p>And edit it to use the version 3 and add <code>--inline-css</code> for more options without editing <code>cgit</code>‘s CSS file:</p>
<pre><code class="language-sh">...
# This is for version 2
# exec highlight --force -f -I -X -S "$EXTENSION" 2>/dev/null
# This is for version 3
exec highlight --force --inline-css -f -I -O xhtml -S "$EXTENSION" 2>/dev/null
...
</code></pre>
<p>Finally, enable the filter in <code>/etc/cgitrc</code> configuration:</p>
<pre><code class="language-apache">source-filter=/usr/lib/cgit/filters/syntax-highlighting-edited.sh
</code></pre>
<p>That would be everything. If you need support for more stuff like compressed snapshots or support for markdown, check the optional dependencies for <code>cgit</code>.</p>]]></content:encoded>
</item>
<item>
<title>Set up a Mail server with Postfix, Dovecot, SpamAssassin and OpenDKIM</title>
<link>https://blog.luevano.xyz/a/mail_server_with_postfix.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/mail_server_with_postfix.html</guid>
<pubDate>Sun, 21 Mar 2021 04:05:59 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a Mail server using Postfix, Dovecot, SpamAssassin and OpenDKIM, on Arch. This is a follow up on post about creating a website with Nginx and Certbot.</description>
<content:encoded><![CDATA[<p>The entry is going to be long because it’s a <em>tedious</em> process. This is also based on <a href="https://github.com/LukeSmithxyz/emailwiz">Luke Smith’s script</a>, but adapted to Arch Linux (his script works on debian-based distributions). This entry is mostly so I can record all the notes required while I’m in the process of installing/configuring the mail server on a new VPS of mine; <del>also I’m going to be writing a script that does everything in one go (for Arch Linux), that will be hosted <a href="https://git.luevano.xyz/server_scripts.git">here</a>.</del> <ins>I haven’t had time to do the script so nevermind this, if I ever do it I’ll make a new entry regarding it.</ins></p>
<p>This configuration works for local users (users that appear in <code>/etc/passwd</code>), and does not use any type of SQL database. Do note that I’m not running Postfix in a chroot, which can be a problem if you’re following my steps as noted by <mark><a href="https://bojanmilevski.com/">Bojan</a></mark>; in the case that you want to run in chroot then add the steps chown in the Arch wiki: <a href="https://wiki.archlinux.org/title/postfix#Postfix_in_a_chroot_jail">Postfix in a chroot jail</a>; the issue faced if following my steps and using a chroot is that there will be issues resolving the hostname due to <code>/etc/hosts</code> or <code>/etc/hostname</code> not being available in the chroot.</p>
<p>All commands executed here are run with root privileges, unless stated otherwise.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#postfix">Postfix</a></li>
<li><a href="#dovecot">Dovecot</a></li>
<li><a href="#opendkim">OpenDKIM</a><ul>
<li><a href="#opendkim-dns-txt-records">OpenDKIM DNS TXT records</a></li>
</ul>
</li>
<li><a href="#spamassassin">SpamAssassin</a></li>
<li><a href="#wrapping-up">Wrapping up</a></li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>Basically the same as with the <a href="https://blog.luevano.xyz/a/website_with_nginx.html">website with Nginx and Certbot</a>, with the extras:</p>
<ul>
<li>You will need a <strong>CNAME</strong> for “mail” and (optionally) “www.mail”, or whatever you want to call the sub-domains (although the <a href="https://tools.ietf.org/html/rfc2181#section-10.3">RFC 2181</a> states that it NEEDS to be an <strong>A</strong> record, fuck the police).</li>
<li>An SSL certificate. You can use the SSL certificate obtained following my last post using <code>certbot</code> (just create a <code>mail.conf</code> and run <code>certbot --nginx</code> again).</li>
<li>Ports <code>25</code>, <code>587</code> (SMTP), <code>465</code> (SMTPS), <code>143</code> (IMAP) and <code>993</code> (IMAPS) open on the firewall (I use <code>ufw</code>).</li>
</ul>
<h2 id="postfix">Postfix<a class="headerlink" href="#postfix" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/postfix">Postfix</a> is a “mail transfer agent” which is the component of the mail server that receives and sends emails via SMTP.</p>
<p>Install the <code>postfix</code> package:</p>
<pre><code class="language-sh">pacman -S postfix
</code></pre>
<p>We have two main files to configure (inside <code>/etc/postfix</code>): <code>master.cf</code> (<a href="https://man.archlinux.org/man/master.5">master(5)</a>) and <code>main.cf</code> (<a href="https://man.archlinux.org/man/postconf.5">postconf(5)</a>). We’re going to edit <code>main.cf</code> first either by using the command <code>postconf -e 'setting'</code> or by editing the file itself (I prefer to edit the file).</p>
<p>Note that the default file itself has a lot of comments with description on what each thing does (or you can look up the manual, linked above), I used what Luke’s script did plus some other settings that worked for me.</p>
<p>Now, first locate where your website cert is, mine is at the default location <code>/etc/letsencrypt/live/</code>, so my <code>certdir</code> is <code>/etc/letsencrypt/live/luevano.xyz</code>. Given this information, change <code>{yourcertdir}</code> on the corresponding lines. The configuration described below has to be appended in the <code>main.cf</code> configuration file.</p>
<p>Certificates and ciphers to use for authentication and security:</p>
<pre><code class="language-apache">smtpd_tls_key_file = {yourcertdir}/privkey.pem
smtpd_tls_cert_file = {yourcertdir}/fullchain.pem
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
smtp_tls_loglevel = 1
smtp_tls_CAfile = {yourcertdir}/cert.pem
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
tls_preempt_cipherlist = yes
smtpd_tls_exclude_ciphers = aNULL, LOW, EXP, MEDIUM, ADH, AECDH, MD5,
DSS, ECDSA, CAMELLIA128, 3DES, CAMELLIA256,
RSA+AES, eNULL
smtp_tls_CApath = /etc/ssl/certs
smtpd_tls_CApath = /etc/ssl/certs
smtpd_relay_restrictions = permit_sasl_authenticated, permit_mynetworks, defer_unauth_destination
</code></pre>
<p>Also, for the <em>connection</em> with <code>dovecot</code>, append the next few lines (telling postfix that <code>dovecot</code> will use user/password for authentication):</p>
<pre><code class="language-apache">smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous
</code></pre>
<p>Specify the mailbox home, this is going to be a directory inside your user’s home containing the actual mail files, for example it will end up being<code>/home/david/Mail/Inbox</code>:</p>
<pre><code class="language-apache">home_mailbox = Mail/Inbox/
</code></pre>
<p>Pre-configuration to work seamlessly with <code>dovecot</code> and <code>opendkim</code>:</p>
<pre><code class="language-apache">myhostname = {yourdomainname}
mydomain = localdomain
mydestination = $myhostname, localhost.$mydomain, localhost
milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = inet:127.0.0.1:8891
mailbox_command = /usr/lib/dovecot/deliver
</code></pre>
<p>Where <code>{yourdomainname}</code> is <code>luevano.xyz</code> in my case. Lastly, if you don’t want the sender’s IP and user agent (application used to send the mail), add the following line:</p>
<pre><code class="language-apache">smtp_header_checks = regexp:/etc/postfix/smtp_header_checks
</code></pre>
<p>And create the <code>/etc/postfix/smtp_header_checks</code> file with the following content:</p>
<pre><code class="language-coffee">/^Received: .*/ IGNORE
/^User-Agent: .*/ IGNORE
</code></pre>
<p>That’s it for <code>main.cf</code>, now we have to configure <code>master.cf</code>. This one is a bit more tricky.</p>
<p>First look up lines (they’re uncommented) <code>smtp inet n - n - - smtpd</code>, <code>smtp unix - - n - - smtp</code> and <code>-o syslog_name=postfix/$service_name</code> and either delete or uncomment them… or just run <code>sed -i "/^\s*-o/d;/^\s*submission/d;/\s*smtp/d" /etc/postfix/master.cf</code> as stated in Luke’s script.</p>
<p>Lastly, append the following lines to complete postfix setup and pre-configure for <code>spamassassin</code>.</p>
<pre><code class="language-txt">smtp unix - - n - - smtp
smtp inet n - y - - smtpd
-o content_filter=spamassassin
submission inet n - y - - smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_tls_auth_only=yes
smtps inet n - y - - smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
spamassassin unix - n n - - pipe
user=spamd argv=/usr/bin/vendor_perl/spamc -f -e /usr/sbin/sendmail -oi -f \${sender} \${recipient}
</code></pre>
<p>Now, I ran into some problems with postfix, one being <a href="https://www.faqforge.com/linux/fix-for-opensuse-error-postfixmaster-fatal-0-0-0-0smtps-servname-not-supported-for-ai_socktype/">smtps: Servname not supported for ai_socktype</a>, to fix it, as <em>Till</em> posted in that site, edit <code>/etc/services</code> and add:</p>
<pre><code class="language-apache">smtps 465/tcp
smtps 465/udp
</code></pre>
<p>Before starting the <code>postfix</code> service, you need to run <code>newaliases</code> first, but you can do a bit of configuration beforehand editing the file <code>/etc/postfix/aliases</code>. I only change the <code>root: you</code> line (where <code>you</code> is the account that will be receiving “root” mail). After you’re done, run:</p>
<pre><code class="language-sh">postalias /etc/postfix/aliases
newaliases
</code></pre>
<p>At this point you’re done configuring <code>postfix</code> and you can already start/enable the <code>postfix</code> service:</p>
<pre><code class="language-sh">systemctl start postfix.service
systemctl enable postfix.service
</code></pre>
<h2 id="dovecot">Dovecot<a class="headerlink" href="#dovecot" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Dovecot">Dovecot</a> is an IMAP and POP3 server, which is what lets an email application retrieve the mail.</p>
<p>Install the <code>dovecot</code> and <code>pigeonhole</code> (sieve for <code>dovecot</code>) packages:</p>
<pre><code class="language-sh">pacman -S dovecot pigeonhole
</code></pre>
<p>On arch, by default, there is no <code>/etc/dovecot</code> directory with default configurations set in place, but the package does provide the example configuration files. Create the <code>dovecot</code> directory under <code>/etc</code> and, optionally, copy the <code>dovecot.conf</code> file and <code>conf.d</code> directory under the just created <code>dovecot</code> directory:</p>
<pre><code class="language-sh">mkdir /etc/dovecot
cp /usr/share/doc/dovecot/example-config/dovecot.conf /etc/dovecot/dovecot.conf
cp -r /usr/share/doc/dovecot/example-config/conf.d /etc/dovecot
</code></pre>
<p>As Luke stated, <code>dovecot</code> comes with a lot of “modules” (under <code>/etc/dovecot/conf.d/</code> if you copied that folder) for all sorts of configurations that you can include, but I do as he does and just edit/create the whole <code>dovecot.conf</code> file; although, I would like to check each of the separate configuration files <code>dovecot</code> provides I think the options Luke provides are more than good enough.</p>
<p>I’m working with an empty <code>dovecot.conf</code> file. Add the following lines for SSL and login configuration (also replace <code>{yourcertdir}</code> with the same certificate directory described in the Postfix section above, note that the <code><</code> is required):</p>
<pre><code class="language-apache">ssl = required
ssl_cert = <{yourcertdir}/fullchain.pem
ssl_key = <{yourcertdir}/privkey.pem
ssl_min_protocol = TLSv1.2
ssl_cipher_list = ALL:!RSA:!CAMELLIA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4:!SHA1:!SHA256:!SHA384:!LOW@STRENGTH
ssl_prefer_server_ciphers = yes
ssl_dh = </etc/dovecot/dh.pem
auth_mechanisms = plain login
auth_username_format = %n
protocols = $protocols imap
</code></pre>
<p>You may notice we specify a file we don’t have under <code>/etc/dovecot</code>: <code>dh.pem</code>. We need to create it with <code>openssl</code> (you should already have it installed if you’ve been following this entry and the one for <code>nginx</code>). Just run (might take a few minutes):</p>
<pre><code class="language-sh">openssl dhparam -out /etc/dovecot/dh.pem 4096
</code></pre>
<p>After that, the next lines define what a “valid user is” (really just sets the database for users and passwords to be the local users with their password):</p>
<pre><code class="language-apache">userdb {
driver = passwd
}
passdb {
driver = pam
}
</code></pre>
<p>Next, comes the mail directory structure (has to match the one described in the Postfix section). Here, the <code>LAYOUT</code> option is important so the boxes are <code>.Sent</code> instead of <code>Sent</code>. Add the next lines (plus any you like):</p>
<pre><code class="language-apache">mail_location = maildir:~/Mail:INBOX=~/Mail/Inbox:LAYOUT=fs
namespace inbox {
inbox = yes
mailbox Drafts {
special_use = \Drafts
auto = subscribe
}
mailbox Junk {
special_use = \Junk
auto = subscribe
autoexpunge = 30d
}
mailbox Sent {
special_use = \Sent
auto = subscribe
}
mailbox Trash {
special_use = \Trash
}
mailbox Archive {
special_use = \Archive
}
}
</code></pre>
<p>Also include this so Postfix can use Dovecot’s authentication system:</p>
<pre><code class="language-apache">service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0660
user = postfix
group = postfix
}
}
</code></pre>
<p>Lastly (for Dovecot at least), the plugin configuration for <code>sieve</code> (<code>pigeonhole</code>):</p>
<pre><code class="language-apache">protocol lda {
mail_plugins = $mail_plugins sieve
}
protocol lmtp {
mail_plugins = $mail_plugins sieve
}
plugin {
sieve = ~/.dovecot.sieve
sieve_default = /var/lib/dovecot/sieve/default.sieve
sieve_dir = ~/.sieve
sieve_global_dir = /var/lib/dovecot/sieve/
</code></pre>
<p>Where <code>/var/lib/dovecot/sieve/default.sieve</code> doesn’t exist yet. Create the folders:</p>
<pre><code class="language-sh">mkdir -p /var/lib/dovecot/sieve
</code></pre>
<p>And create the file <code>default.sieve</code> inside that just created folder with the content:</p>
<pre><code class="language-nginx">require ["fileinto", "mailbox"];
if header :contains "X-Spam-Flag" "YES" {
fileinto "Junk";
}
</code></pre>
<p>Now, if you don’t have a <code>vmail</code> (virtual mail) user, create one and change the ownership of the <code>/var/lib/dovecot</code> directory to this user:</p>
<pre><code class="language-sh">grep -q "^vmail:" /etc/passwd || useradd -m vmail -s /usr/bin/nologin
chown -R vmail:vmail /var/lib/dovecot
</code></pre>
<p>Note that I also changed the shell for <code>vmail</code> to be <code>/usr/bin/nologin</code>. After that, to compile the configuration file run:</p>
<pre><code class="language-sh">sievec /var/lib/dovecot/sieve/default.sieve
</code></pre>
<p>A <code>default.svbin</code> file will be created next to <code>default.sieve</code>.</p>
<p>Next, add the following lines to <code>/etc/pam.d/dovecot</code> if not already present (shouldn’t be there if you’ve been following these notes):</p>
<pre><code class="language-txt">auth required pam_unix.so nullok
account required pam_unix.so
</code></pre>
<p>That’s it for Dovecot, at this point you can start/enable the <code>dovecot</code> service:</p>
<pre><code class="language-sh">systemctl start dovecot.service
systemctl enable dovecot.service
</code></pre>
<h2 id="opendkim">OpenDKIM<a class="headerlink" href="#opendkim" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/OpenDKIM">OpenDKIM</a> is needed so services like G**gle don’t throw the mail to the trash. DKIM stands for “DomainKeys Identified Mail”.</p>
<p>Install the <code>opendkim</code> package:</p>
<pre><code class="language-sh">pacman -S opendkim
</code></pre>
<p>Generate the keys for your domain:</p>
<pre><code class="language-sh">opendkim-genkey -D /etc/opendkim -d {yourdomain} -s {yoursubdomain} -r -b 2048
</code></pre>
<p>Where you need to change <code>{yourdomain}</code> and <code>{yoursubdomain}</code> (doesn’t really need to be the sub-domain, could be anything that describes your key) accordingly, for me it’s <code>luevano.xyz</code> and <code>mail</code>, respectively. After that, we need to create some files inside the <code>/etc/opendkim</code> directory. First, create the file <code>KeyTable</code> with the content:</p>
<pre><code class="language-txt">{yoursubdomain}._domainkey.{yourdomain} {yourdomain}:{yoursubdomain}:/etc/opendkim/{yoursubdomain}.private
</code></pre>
<p>So, for me it would be:</p>
<pre><code class="language-txt">mail._domainkey.luevano.xyz luevano.xyz:mail:/etc/opendkim/mail.private
</code></pre>
<p>Next, create the file <code>SigningTable</code> with the content:</p>
<pre><code class="language-txt">*@{yourdomain} {yoursubdomain}._domainkey.{yourdomain}
</code></pre>
<p>Again, for me it would be:</p>
<pre><code class="language-txt">*@luevano.xyz mail._domainkey.luevano.xyz
</code></pre>
<p>And, lastly create the file <code>TrustedHosts</code> with the content:</p>
<pre><code class="language-txt">127.0.0.1
::1
10.1.0.0/16
1.2.3.4/24
localhost
{yourserverip}
...
</code></pre>
<p>And more, make sure to include your server IP and something like <code>subdomain.domainname</code>.</p>
<p>Next, edit <code>/etc/opendkim/opendkim.conf</code> to reflect the changes (or rather, addition) of these files, as well as some other configuration. You can look up the example configuration file located at <code>/usr/share/doc/opendkim/opendkim.conf.sample</code>, but I’m creating a blank one with the contents:</p>
<pre><code class="language-apache">Domain {yourdomain}
Selector {yoursubdomain}
Syslog Yes
UserID opendkim
KeyFile /etc/opendkim/{yoursubdomain}.private
Socket inet:8891@localhost
</code></pre>
<p>Now, change the permissions for all the files inside <code>/etc/opendkim</code>:</p>
<pre><code class="language-sh">chown -R root:opendkim /etc/opendkim
chmod g+r /etc/postfix/dkim/*
</code></pre>
<p>I’m using <code>root:opendkim</code> so <code>opendkim</code> doesn’t complain about the <code>{yoursubdomani}.private</code> being insecure (you can change that by using the option <code>RequireSafeKeys False</code> in the <code>opendkim.conf</code> file, as stated <a href="http://lists.opendkim.org/archive/opendkim/users/2014/12/3331.html">here</a>).</p>
<p>That’s it for the general configuration, but you could go more in depth and be more secure with some extra configuration.</p>
<p>Now, just start/enable the <code>opendkim</code> service:</p>
<pre><code class="language-sh">systemctl start opendkim.service
systemctl enable opendkim.service
</code></pre>
<h3 id="opendkim-dns-txt-records">OpenDKIM DNS TXT records<a class="headerlink" href="#opendkim-dns-txt-records" title="Permanent link">¶</a></h3>
<p>Add the following <strong>TXT</strong> records on your domain registrar (these examples are for Epik):</p>
<ol>
<li><em>DKIM</em> entry: look up your <code>{yoursubdomain}.txt</code> file, it should look something like:</li>
</ol>
<pre><code class="language-txt">{yoursubdomain}._domainkey IN TXT ( "v=DKIM1; k=rsa; s=email; "
"p=..."
"..." ) ; ----- DKIM key mail for {yourdomain}
</code></pre>
<p>In the <strong>TXT</strong> record you will place <code>{yoursubdomain}._domainkey</code> as the “Host” and <code>"v=DKIM1; k=rsa; s=email; " "p=..." "..."</code> in the “TXT Value” (replace the dots with the actual value you see in your file).</p>
<ol start="2">
<li>
<p><em>DMARC</em> entry: just <code>_dmarc.{yourdomain}</code> as the “Host” and <code>"v=DMARC1; p=reject; rua=mailto:dmarc@{yourdomain}; fo=1"</code> as the “TXT Value”.</p>
</li>
<li>
<p><em>SPF</em> entry: just <code>@</code> as the “Host” and <code>"v=spf1 mx a:{yoursubdomain}.{yourdomain} - all"</code> as the “TXT Value”.</p>
</li>
</ol>
<p>And at this point you could test your mail for spoofing and more.</p>
<h2 id="spamassassin">SpamAssassin<a class="headerlink" href="#spamassassin" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/SpamAssassin">SpamAssassin</a> is just <em>a mail filter to identify spam</em>.</p>
<p>Install the <code>spamassassin</code> package (which will install a bunch of ugly <code>perl</code> packages…):</p>
<pre><code class="language-sh">pacman -S spamassassin
</code></pre>
<p>For some reason, the permissions on all <code>spamassassin</code> stuff are all over the place. First, change owner of the executables, and directories:</p>
<pre><code class="language-sh">chown spamd:spamd /usr/bin/vendor_perl/sa-*
chown spamd:spamd /usr/bin/vendor_perl/spam*
chwown -R spamd:spamd /etc/mail/spamassassin
</code></pre>
<p>Then, you can edit <code>local.cf</code> (located in <code>/etc/mail/spamassassin</code>) to fit your needs (I only uncommented the <code>rewrite_header Subject ...</code> line). And then you can run the following command to update the patterns and compile them:</p>
<pre><code class="language-sh">sudo -u spamd sa-update
sudo -u spamd sa-compile
</code></pre>
<p>And since this should be run periodically, create the service <code>spamassassin-update.service</code> under <code>/etc/systemd/system</code> with the following content:</p>
<pre><code class="language-ini">[Unit]
Description=SpamAssassin housekeeping
After=network.target
[Service]
User=spamd
Group=spamd
Type=oneshot
ExecStart=/usr/bin/vendor_perl/sa-update --allowplugins
SuccessExitStatus=1
ExecStart=/usr/bin/vendor_perl/sa-compile
ExecStart=/usr/bin/systemctl -q --no-block try-restart spamassassin.service
</code></pre>
<p>And you could also execute <code>sa-learn</code> to train <code>spamassassin</code>‘s bayes filter, but this works for me. Then create the timer <code>spamassassin-update.timer</code> under the same directory, with the content:</p>
<pre><code class="language-ini">[Unit]
Description=SpamAssassin housekeeping
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
</code></pre>
<p>You can now start/enable the <code>spamassassin-update</code> timer:</p>
<pre><code class="language-sh">systemctl start spamassassin-update.timer
systemctl enable spamassassin-update.timer
</code></pre>
<p>Next, you may want to edit the <code>spamassassin</code> service before starting and enabling it, because by default, it could <a href="https://rimuhosting.com/howto/memory.jsp">spawn a lot of “childs”</a> eating a lot of resources and you really only need one child. Append <code>--max-children=1</code> to the line <code>ExecStart=...</code> in <code>/usr/bin/systemd/system/spamassassin.service</code>:</p>
<pre><code class="language-ini">...
ExecStart=/usr/bin/vendor_perl/spamd -x -u spamd -g spamd --listen=/run/spamd/spamd.sock --listen=localhost --max-children=1
...
</code></pre>
<p>Finally, start and enable the <code>spamassassin</code> service:</p>
<pre><code class="language-sh">systemctl start spamassassin.service
systemctl enable spamassassin.service
</code></pre>
<h2 id="wrapping-up">Wrapping up<a class="headerlink" href="#wrapping-up" title="Permanent link">¶</a></h2>
<p>We should have a working mail server by now. Before continuing check your journal logs (<code>journalctl -xe --unit={unit}</code>, where <code>{unit}</code> could be <code>spamassassin.service</code> for example) to see if there was any error whatsoever and try to debug it, it should be a typo somewhere because all the settings and steps detailed here just worked; I literally just finished doing everything on a new server as of the writing of this text, <mark>it just werks on my machine</mark>.</p>
<p>Now, to actually use the mail service: first of all, you need a <em>normal</em> account (don’t use root) that belongs to the <code>mail</code> group (<code>gpasswd -a user group</code> to add a user <code>user</code> to group <code>group</code>) and that has a password.</p>
<p>Next, to actually login into a mail app/program, you will use the following settings, at least for <code>thunderdbird</code>(I tested in windows default mail app and you don’t need a lot of settings):</p>
<ul>
<li>* server: subdomain.domain (mail.luevano.xyz in my case)</li>
<li><strong>SMTP</strong> port: 587</li>
<li><strong>SMTPS</strong> port: 465 (I use this one)</li>
<li><strong>IMAP</strong> port: 143</li>
<li><strong>IMAPS</strong> port: 993 (again, I use this one)</li>
<li>Connection/security: SSL/TLS</li>
<li>Authentication method: Normal password</li>
<li>Username: just your <code>user</code>, not the whole email (<code>david</code> in my case)</li>
<li>Password: your <code>user</code> password (as in the password you use to login to the server with that user)</li>
</ul>
<p>All that’s left to do is test your mail server for spoofing, and to see if everything is setup correctly. Go to <a href="https://www.appmaildev.com/en/dkim">DKIM Test</a> and follow the instructions (basically click next, and send an email with whatever content to the email that they provide). After you send the email, you should see something like:</p>
<figure id="__yafg-figure-10">
<img alt="DKIM Test successful" src="https://static.luevano.xyz/images/b/mail/dkim_test_successful.png" title="DKIM Test successful">
<figcaption>DKIM Test successful</figcaption>
</figure>]]></content:encoded>
</item>
<item>
<title>Set up a website with Nginx and Certbot</title>
<link>https://blog.luevano.xyz/a/website_with_nginx.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/website_with_nginx.html</guid>
<pubDate>Fri, 19 Mar 2021 02:58:15 GMT</pubDate>
<category>Code</category>
<category>English</category>
<category>Server</category>
<category>Tools</category>
<category>Tutorial</category>
<description>How to set up a website using Nginx for web server and Certbot for SSL certificates, on Arch. This is a base for future blog posts about similar topics.</description>
<content:encoded><![CDATA[<p>These are general notes on how to setup a Nginx web server plus Certbot for SSL certificates, initially learned from <a href="https://www.youtube.com/watch?v=OWAqilIVNgE">Luke’s video</a> and after some use and research I added more stuff to the mix. And, actually at the time of writing this entry, I’m configuring the web server again on a new VPS instance, so this is going to be fresh.</p>
<p>As a side note, <mark>i use arch btw</mark> so everything here es aimed at an Arch Linux distro, and I’m doing everything on a VPS. Also note that most if not all commands here are executed with root privileges.</p>
<h2 id="table-of-contents">Table of contents<a class="headerlink" href="#table-of-contents" title="Permanent link">¶</a></h2>
<div class="toc">
<ul>
<li><a href="#table-of-contents">Table of contents</a></li>
<li><a href="#prerequisites">Prerequisites</a></li>
<li><a href="#nginx">Nginx</a></li>
<li><a href="#certbot">Certbot</a></li>
</ul>
</div>
<h2 id="prerequisites">Prerequisites<a class="headerlink" href="#prerequisites" title="Permanent link">¶</a></h2>
<p>You will need two things:</p>
<ul>
<li>A domain name (duh!). I got mine on <a href="https://www.epik.com/?affid=da5ne9ru4">Epik</a> (affiliate link, btw).<ul>
<li>With the corresponding <strong>A</strong> and <strong>AAA</strong> records pointing to the VPS’ IPs. I have three records for each type: empty string, “www” and “*” for a wildcard, that way “domain.name”, “www.domain.name”, “anythingelse.domain.name” point to the same VPS (meaning that you can have several VPS for different sub-domains). These depend on the VPS provider.</li>
</ul>
</li>
<li>A VPS or somewhere else to host it. I’m using <a href="https://www.vultr.com/?ref=8732849">Vultr</a> (also an affiliate link, btw).<ul>
<li>With <code>ssh</code> already configured both on the local machine and on the remote machine.</li>
<li>Firewall already configured to allow ports <code>80</code> (HTTP) and <code>443</code> (HTTPS). I use <code>ufw</code> so it’s just a matter of doing <code>ufw allow 80,443/tcp</code> (for example) as root and you’re golden.</li>
<li><code>cron</code> installed if you follow along (you could use <code>systemd</code> timers, or some other method you prefer to automate running commands every certain time).</li>
</ul>
</li>
</ul>
<h2 id="nginx">Nginx<a class="headerlink" href="#nginx" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Nginx">Nginx</a> is a web (HTTP) server and reverse proxy server.</p>
<p>You have two options: <code>nginx</code> and <code>nginx-mainline</code>. I prefer <code>nginx-mainline</code> because it’s the “up to date” package even though <code>nginx</code> is labeled to be the “stable” version. Install the package and enable/start the service:</p>
<pre><code class="language-sh">pacman -S nginx-mainline
systemctl enable nginx.service
systemctl start nginx.service
</code></pre>
<p>And that’s it, at this point you can already look at the default initial page of Nginx if you enter the IP of your server in a web browser. You should see something like this:</p>
<figure id="__yafg-figure-3">
<img alt="Nginx welcome page" src="https://static.luevano.xyz/images/b/nginx/nginx_welcome_page.png" title="Nginx welcome page">
<figcaption>Nginx welcome page</figcaption>
</figure>
<p>As stated in the welcome page, configuration is needed, head to the directory of Nginx:</p>
<pre><code class="language-sh">cd /etc/nginx
</code></pre>
<p>Here you have several files, the important one is <code>nginx.conf</code>, which as its name implies, contains general configuration of the web server. If you peek into the file, you will see that it contains around 120 lines, most of which are commented out and contains the welcome page server block. While you can configure a website in this file, it’s common practice to do it on a separate file (so you can scale really easily if needed for mor websites or sub-domains).</p>
<p>Inside the <code>nginx.conf</code> file, delete the <code>server</code> blocks and add the lines <code>include sites-enabled/*;</code> (to look into individual server configuration files) and <code>types_hash_max_size 4096;</code> (to get rid of an ugly warning that will keep appearing) somewhere inside the <code>http</code> block. The final <code>nginx.conf</code> file would look something like (ignoring the comments just for clarity, but you can keep them as side notes):</p>
<pre><code class="language-nginx">worker_processes 1;
events {
worker_connections 1024;
}
http {
include sites-enabled/*;
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
types_hash_max_size 4096;
}
</code></pre>
<p>Next, inside the directory <code>/etc/nginx/</code> create the <code>sites-available</code> and <code>sites-enabled</code> directories, and go into the <code>sites-available</code> one:</p>
<pre><code class="language-sh">mkdir sites-available
mkdir sites-enabled
cd sites-available
</code></pre>
<p>Here, create a new <code>.conf</code> file for your website and add the following lines (this is just the sample content more or less):</p>
<pre><code class="language-nginx">server {
listen 80;
listen [::]:80;
root /path/to/root/directory;
server_name domain.name another.domain.name;
index index.html anotherindex.otherextension;
location /{
try_files $uri $uri/ =404;
}
}
</code></pre>
<p>That could serve as a template if you intend to add more domains.</p>
<p>Note some things:</p>
<ul>
<li><code>listen</code>: we’re telling Nginx which port to listen to (IPv4 and IPv6, respectively).</li>
<li><code>root</code>: the root directory of where the website files (<code>.html</code>, <code>.css</code>, <code>.js</code>, etc. files) are located. I followed Luke’s directory path <code>/var/www/some_folder</code>.</li>
<li><code>server_name</code>: the actual domain to “listen” to (for my website it is: <code>server_name luevano.xyz www.luevano.xyz;</code> and for this blog is: <code>server_name blog.luevano.xyz www.blog.luevano.xyz;</code>).</li>
<li><code>index</code>: what file to serve as the index (could be any <code>.html</code>, <code>.htm</code>, <code>.php</code>, etc. file) when just entering the website.</li>
<li><code>location</code>: what goes after <code>domain.name</code>, used in case of different configurations depending on the URL paths (deny access on <code>/private</code>, make a proxy on <code>/proxy</code>, etc).<ul>
<li><code>try_files</code>: tells what files to look for.</li>
</ul>
</li>
</ul>
<p>Then, make a symbolic link from this configuration file to the <code>sites-enabled</code> directory:</p>
<pre><code class="language-sh">ln -s /etc/nginx/sites-available/your_config_file.conf /etc/nginx/sites-enabled
</code></pre>
<p>This is so the <code>nginx.conf</code> file can look up the newly created server configuration. With this method of having each server configuration file separate you can easily “deactivate” any website by just deleting the symbolic link in <code>sites-enabled</code> and you’re good, or just add new configuration files and keep everything nice and tidy.</p>
<p>All you have to do now is restart (or enable and start if you haven’t already) the Nginx service (and optionally test the configuration):</p>
<pre><code class="language-sh">nginx -t
systemctl restart nginx
</code></pre>
<p>If everything goes correctly, you can now go to your website by typing <code>domain.name</code> on a web browser. But you will see a “404 Not Found” page like the following (maybe with different Nginx version):</p>
<figure id="__yafg-figure-4">
<img alt="Nginx 404 Not Found page" src="https://static.luevano.xyz/images/b/nginx/nginx_404_page.png" title="Nginx 404 Not Found page">
<figcaption>Nginx 404 Not Found page</figcaption>
</figure>
<p>That’s no problem, because it means that the web server it’s actually working. Just add an <code>index.html</code> file with something simple to see it in action (in the <code>/var/www/some_folder</code> that you decided upon). If you keep seeing the 404 page make sure your <code>root</code> line is correct and that the directory/index file exists.</p>
<p>I like to remove the <code>.html</code> and trailing <code>/</code> on the URLs of my website, for that you need to add the following <code>rewrite</code> lines and modify the <code>try_files</code> line (for more: <a href="https://www.seancdavis.com/blog/remove-html-extension-and-trailing-slash-in-nginx-config/">Sean C. Davis: Remove HTML Extension And Trailing Slash In Nginx Config</a>):</p>
<pre><code class="language-nginx">server {
...
rewrite ^(/.*)\.html(\?.*)?$ $1$2 permanent;
rewrite ^/(.*)/$ /$1 permanent;
...
try_files $uri/index.html $uri.html $uri/ $uri =404;
...
</code></pre>
<h2 id="certbot">Certbot<a class="headerlink" href="#certbot" title="Permanent link">¶</a></h2>
<p><a href="https://wiki.archlinux.org/title/Certbot">Certbot</a> is what provides the SSL certificates via <a href="https://letsencrypt.org/">Let’s Encrypt</a>.</p>
<p>The only “bad” (bloated) thing about Certbot, is that it uses <code>python</code>, but for me it doesn’t matter too much. You may want to look up another alternative if you prefer. Install the packages <code>certbot</code> and <code>certbot-nginx</code>:</p>
<pre><code class="language-sh">pacman -S certbot certbot-nginx
</code></pre>
<p>After that, all you have to do now is run <code>certbot</code> and follow the instructions given by the tool:</p>
<pre><code class="language-sh">certbot --nginx
</code></pre>
<p>It will ask you for some information, for you to accept some agreements and the names to activate HTTPS for. Also, you will want to “say yes” to the redirection from HTTP to HTTPS. And that’s it, you can now go to your website and see that you have HTTPS active.</p>
<p>Now, the certificate given by <code>certbot</code> expires every 3 months or something like that, so you want to renew this certificate every once in a while. I did this before using <code>cron</code> or manually creating a <code>systemd</code> timer and service, but now it’s just a matter of enabling the <code>certbot-renew.timer</code>:</p>
<pre><code class="language-sh">systemctl start certbot-renew.timer
</code></pre>
<p>The <code>deploy-hook</code> is not needed anymore, only for plugins. For more, visit the <a href="https://wiki.archlinux.org/title/Certbot#Automatic_renewal">Arch Linux Wiki</a>.</p>]]></content:encoded>
</item>
<item>
<title>Así es raza, el blog ya tiene timestamps</title>
<link>https://blog.luevano.xyz/a/el_blog_ya_tiene_timestamps.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/el_blog_ya_tiene_timestamps.html</guid>
<pubDate>Tue, 16 Mar 2021 02:46:24 GMT</pubDate>
<category>Short</category>
<category>Spanish</category>
<category>Tools</category>
<category>Update</category>
<description>Actualización en el estado del blog y el sistema usado para crearlo.</description>
<content:encoded><![CDATA[<p>Pues eso, esta entrada es sólo para tirar update sobre mi <a href="https://blog.luevano.xyz/a/first_blog_post.html">primer post</a>. Ya modifiqué el <code>ssg</code> lo suficiente como para que maneje los <em>timestamps</em>, y ya estoy más familiarizado con este script entonces ya lo podré extender más, pero por ahora las entradas ya tienen su fecha de creación (y modificación en dado caso) al final y en el índice ya están organizados por fecha, que por ahora está algo simple pero está sencillo de extender.</p>
<p>Ya lo único que queda es cambiar un poco el formato del blog (y de la página en general), porque en un momento de desesperación puse todo el texto en justificado y pues no se ve chido siempre, entonces queda corregir eso. <em>Y aunque me tomó más tiempo del que quisiera, así nomás quedó, diría un cierto personaje.</em></p>
<p><del>El <code>ssg</code> modificado está en mis <a href="https://git.luevano.xyz/.dots">dotfiles</a> (o directamente <a href="https://git.luevano.xyz/.dots/tree/.local/bin/ssg">aquí</a>).</del>
<ins>Como al final ya no usé el <code>ssg</code> modificado, este pex ya no existe.</ins></p>
<p>Por último, también quité las extensiones <code>.html</code> de las URLs, porque se ve bien pitero, pero igual los links con <code>.html</code> al final redirigen a su link sin <code>.html</code>, así que no hay rollo alguno.</p>
<p><strong>Actualización</strong>: Ahora estoy usando mi propia solución en vez de <code>ssg</code>, que la llamé <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, de la cual empiezo a hablar <a href="https://blog.luevano.xyz/a/new_blogging_system.html">acá</a>.</p>]]></content:encoded>
</item>
<item>
<title>This is the first blog post, just for testing purposes</title>
<link>https://blog.luevano.xyz/a/first_blog_post.html</link>
<guid isPermaLink="true">https://blog.luevano.xyz/a/first_blog_post.html</guid>
<pubDate>Sat, 27 Feb 2021 13:08:33 GMT</pubDate>
<category>English</category>
<category>Short</category>
<category>Tools</category>
<category>Update</category>
<description>Just my first blog post where I state what tools I'm using to build this blog.</description>
<content:encoded><![CDATA[<p>I’m making this post just to figure out how <a href="https://www.romanzolotarev.com/ssg.html"><code>ssg5</code></a> and <a href="https://kristaps.bsd.lv/lowdown/"><code>lowdown</code></a> are supposed to work, and eventually <a href="https://www.romanzolotarev.com/rssg.html"><code>rssg</code></a>.</p>
<p>At the moment I’m not satisfied because there’s no automatic date insertion into the 1) html file, 2) the blog post itself and 3) the listing system in the <a href="https://blog.luevano.xyz/">blog homepage</a> which also has a problem with the ordering of the entries. And all of this just because I didn’t want to use Luke’s <a href="https://github.com/LukeSmithxyz/lb">lb</a> solution as I don’t really like that much how he handles the scripts (<em>but they just work</em>).</p>
<p>Hopefully, for tomorrow all of this will be sorted out and I’ll have a working blog system.</p>
<p><strong>Update</strong>: I’m now using my own solution which I called <a href="https://github.com/luevano/pyssg"><code>pyssg</code></a>, of which I talk about <a href="https://blog.luevano.xyz/a/new_blogging_system.html">here</a>.</p>]]></content:encoded>
</item>
</channel>
</rss>
|