From 7c31d84f9450aca3b366a556db85bdd494fff2f9 Mon Sep 17 00:00:00 2001 From: David Luevano Alvarado Date: Fri, 5 Mar 2021 08:39:41 -0700 Subject: Add new video tutorial notes --- blog/src/a/linux_video_notes.md | 88 ++++++++++++++ blog/src/a/shell_scripting.md | 263 ++++++++++++++++++++++++++++++++++++++++ blog/src/a/sql_video_notes.md | 2 +- 3 files changed, 352 insertions(+), 1 deletion(-) create mode 100644 blog/src/a/linux_video_notes.md create mode 100644 blog/src/a/shell_scripting.md (limited to 'blog') diff --git a/blog/src/a/linux_video_notes.md b/blog/src/a/linux_video_notes.md new file mode 100644 index 0000000..25d0a0b --- /dev/null +++ b/blog/src/a/linux_video_notes.md @@ -0,0 +1,88 @@ +# Linux tutorial video notes + +I was requested to make a summary of a video about basic Linux stuff (like the [SQL tutorial video notes](https://blog.luevano.xyz/a/sql_video_notes.html)); this time, I did most of the notes depending on the topic since I'm familiar with most of the stuff presented in the video. The video in question is: [The Complete Linux Course: Beginner to Power User!](https://www.youtube.com/watch?v=wBp0Rb-ZJak). Also, some notes were taken from [Arch Linux Wiki](https://wiki.archlinux.org/) since it's got pretty decent documentation, and, of course, general googling. + +## (Basic) commands + +A list of basic commands and small explanation (note that options are started with either `-` or `--`, depending on the program, but most of the time `-` is used for letter options and `--` for word options, `-l` vs `--list` for example): + +* `pwd`: "print working directory", full **absolute** path to the current directory. +* `cd`: "change directory", followed by the absolute or relative path of the directory to change to. + * Absolute path is started with `/`, while a relative path is started with `./` or just the name of the folder. + * Use `..` (two dots) to go up one directory. + * An abbreviation of `/home/username` is `~` (tilde). +* `ls`: "list" files and directories in current directory, or specify a directory from which to show the list after typing `ls`. Has many options, the most common ones being: + * `l`: use long listing format. + * `r` or `reverse`: reverse order while sorting. + * `s`: sort by file size, largest first. + * `a` or `all`: do not ignore entries starting with `.`. +* `mkdir`: "make directory", create a new directory with specified name. +* `touch`: create new (empty) files. +* `cp`: "copy" files or directories (using option `r` for recursive). Requires file/directory to copy and destination, separated by space. +* `mv`: "move" files or directories, also requires file/directory to copy and destination, separated by space. This is also used to **rename** files/directories. +* `rm`: "remove", followed by a file to remove it. +* `rmdir`: "remove empty directory", followed by a directory to remove it. If the directory is not empty, use `rm -r` on the directory ("remove recursive"). +* `su`: "switch user", by default to **root** user, but another one can be specified. +* `sudo`: "switch user, do", similar to `su`, but only to execute a command as **root** or the specified user. +* `clear`: clear the terminal window, a (common) keyboard shortcut is `Ctrl + l`. +* `find`: search for files/directories matching a pattern or all contents of a directory (using `.`). +* `grep`: comes from the `ed` command "g/re/p", for searching plain-text for lines that match a regular expression (regex). +* `top`: a task manager program, shows currently running commands and gives important info such as PID (process ID), user who is running that command, command name, cpu and ram usage, etc.. Some useful commands to manage programs running are: + * `pgrep`: get the PID of a running process, or a list in chronological order. + * `kill` or `pkill`: kill a running process either by PID or by name. + * `killall`: similar to `pkill`. +* `ssh`: "secure shell" is a remote login client used to connect into a remote machine and executing commands remotely, basically taking control of the remote machine. Widely used when managing servers. +* `ftp` or `sftp`: "(secure) file transfer protocol" used to transfer files from one machine to another one (usually a server). It's recommended to use `sftp` instead of `ftp` because anyone can look through the packages if it's not secured (encrypted). + +And in general, to see the options supported by almost any command, use `command -h` or `command --help`, for a quick explanation. **IMPORTANT**: Most programs have **man (manual) pages**; to access them do `man command`, this is a very powerful tool to use. + +Commands can be redirected to other commands (the output), which is powerful to create mini scripts or to achieve a goal in a single command. Most of the time the redirection can be done with the special characters `>`, `<` and most powerful, the `|` (pipe). Also, some commands accept an option to execute another command, but this depends on a command to command basis (`exec` option for `find`, for example). + +**Most terminal programs accept `Ctrl-c` or just `q` to exit the program.** + +## File permissions and ownership + +When listing files with `ls -l`, an output with file attributes (permissions) and ownership is shown, such as `drwxr-xr-x 2 user group 4096 Jul 5 21:03 Desktop`, where the first part are the attributes, and `user` and `group` the ownership info (all other info is irrelevant for now). + +File attributes (`drwxr-xr-x` in the example above) are specified by 10 (sometimes 11) characters, and can be break into 4 parts (or 5): + +* The first character is just the file type, typically `d` for directories or just `-` for files. There is `l` too, which is for **symlinks**. +* The next 3 characters represent the permissions that the **owner** has over the file. +* Next 3 the permissions that the **group** has over the file. +* Next 3 the permissions everyone else (**others**) have over the file. +* An optional `+` character that specifies whether an alternate access method applies to the file. When the character is a space, there is no alterante access method. + +Each of the three permission triads (`rwx`) can be: + +* `-` or `r`, for the first character, if the file can be **read** or directory's content can be shown. +* `-` or `w`, for the second character, if the file can be **modified** or the directory's content can be modified (create new files or folders or rename existing files or folders). +* `-` or `x`, for the third character, if the file can be **executed** or the directory can be **accessed** with `cd`. Other characters can be present, like `s`, `S`, `t` and `T` (for more: [Arch Linux Wiki: File permissions and attributes](https://wiki.archlinux.org/index.php/File_permissions_and_attributes)). + +To change attributes or ownership use `chmod` and `chown`, respectively. + +## Services + +Special type of linux process (think of a program or set of programs that run in the background waiting to be used, or doing essential tasks). There are many ways to manage (start, stop, restart, enable, disable, etc.) services, the most common way (if using `systemd`) is to just use `systemctl`. Basic usage of `systemctl` is `systemctl verb service`, where `verb` could be `start`, `enable`, `stop`, `disable`, `restart`, etc. Also, to get a general system status run `systemctl status` or just `systemctl` for a list of running **units** (a unit is an instance of a service, or a mount point or even a device or a socket). For more: [Arch Linux Wiki: systemd](https://wiki.archlinux.org/index.php/systemd). + +`systemd` also provides a way to do tasks based on a **timer**, where you can schedule from the second to the year. One could also use `cron` (using `crontab` with option `e`) to do this. These timers provide support for calendar time events, monotonic time events, and can be run asynchronously. + +## User and group management + +Most mainstream linux distributions come with a Graphic User Interface (GUI) to manage users and groups on the system. For a Command-Line Interface (CLI) just use `useradd` (with `passwd` to create a password for a given user) and `groupadd`. Also, other useful commands are `usermod`, `userdel`, `groups`, `gpasswd`, `groupdel` and more, each used for a basic management of users/groups like modification, deletion, listing (of all existing users/groups), etc.. For more: [Arch Linux Wiki: Users and groups](https://wiki.archlinux.org/index.php/users_and_groups). + +## Networking + +### Hosts file + +Located at `/etc/hosts`, serves as a translator from **hostname** (web addresses or URLs) into IP addresses (think of DNS records), meaning that any URL can be overridden to make it point to whatever IP address it's specified (only locally on the machine affected). The syntax of the file is pretty simple: first column for IP, second for hostname (URL) and third+ for aliases. + +### (Some) commands + +These commands serve the sole purpose of showing information about the network and stuff related to it: + +* `ping`: gives information about latency to a given ip/domain. +* `ifconfig`: gives similar information to `ipconfig` on windows, general info of physical network devices with their addresses and properties. An alternative could be `ip addr`, depending on the linux distribution being used and programs installed. +* `tcpdump`: "transmission control protocol dump" gives information on all "packets" being sent and received through the network. +* `netstat`: "network statistics" general statistics about network devices usage, display connections to the machine and more. +* `traceroute`: shows the route that the packets go through (how the packets jump from one server to another one) when trying to access an IP (or, for example, a website). +* `nmap`: "network mapper" explore network available hosts, opened ports, reverse DNS names, can guess the operating system of the device, it's type, MAC address and more. diff --git a/blog/src/a/shell_scripting.md b/blog/src/a/shell_scripting.md new file mode 100644 index 0000000..abcddbc --- /dev/null +++ b/blog/src/a/shell_scripting.md @@ -0,0 +1,263 @@ +# Shell scripting tutorial video notes + +Another summary, this time about shell scripting in general. And just like with the [Linux notes](https://blog.luevano.xyz/a/linux_video_notes.html), I also did most of the notes myself or with resources outside the video. The videos in question are: [The Bad Tutorials (YT): Shell Scripting Tutorials](https://www.youtube.com/playlist?list=PL7B7FA4E693D8E790) and [Automation with SCripting (YT): Complete Shell Scripting Tutorials](https://www.youtube.com/playlist?list=PL2qzCKTbjutJRM7K_hhNyvf8sfGCLklXw). Also, some notes were taken from [tutorialspoint: UNIX / LINUX Tutorial](https://www.tutorialspoint.com/unix/index.htm) and general googling. + +## Basic concepts + +A **shell** it's an **interface** between the user and the **kernel**. While the kernel it's the layer that interacts between the shell and the **hardware**. And you access the shell either via a **terminal**, or executing a **shell script**. Note that if you're using a GUI environment, you need a **terminal emulator** to actually use a terminal (most Linux distros come with everything needed, so no need to worry). + +When using a terminal a blank screen with some text and a cursor that shows you where to type will appear and depending on the shell being used (`sh`, `dash`, `ksh`, `bash`, `zsh`, `fish`, etc.) the **prompt** will be different. The most common one being of the form `user@host:~$`, which tells that the `user` is using `host` machine and the current working directory is `~` (can be `/any/path/` too), and lastly, the `$` shows the current privileges of the shell/user using the shell (a `$` for normal user and `#` for root access). + +To clear the screen use command `clear` or simply do `Ctrl + l` (most terminals let you do this) and to cancel or create a new prompt do `Ctrl + c`, this also cancels any running program that's using the terminal (typing `q` when a program is running also stops the process, sometimes). + +Also there are **POSIX** (portable operating system interface) compliant shells like `sh`, `dash`, `ksh`, etc., that have a standard syntax and are portable to any Unix system. Non POSIX compliant shells (or not necessary fully POSIX compliant) are `bash`, `zsh`, `fish`, etc., that provide a more modern syntax but lack speed on executing scripts. + +### Common commands/programs + +A list of common commands or programs with a short description (for more, do `man command` or `command -h` or `command --help`): + +* **`man`: an interface to the system reference manuals.** +* `pwd`: print name of current/working directory. +* `cd`: change the working directory. +* `ls`: list directory contents. +* `echo`: display a line of text. Also, see **escape sequences** ([Bash Prompt HOWTO: Chapter 2. Bash and Bash Prompts: 2.5. Bash Prompt Escape Sequences](https://tldp.org/HOWTO/Bash-Prompt-HOWTO/bash-prompt-escape-sequences.html)). +* `mkdir`: make directories. +* `touch`: change file timestamps (if no file exists, creates a new blank one). +* `cat`: concatenate files and print on the standard output. +* `mv`: move (rename) files. +* `rm`: remove files or directories. +* `rmdir`: remove empty directories. +* `cp`: copy files and directories. +* `ln`: make links between files (hard or soft, also known as symbolic). +* `umask`: get or set the file mode creation mask. +* `chmod`: change file mode bits (change file permissions). +* `chown`: change file owner and group. +* `wc`: print newline, word, and byte counts for each file. +* `file`: determine file type. +* `sort`: sort lines of text files. +* `cut`: remove sections from each line of files. +* `dd`: convert and copy a file (mostly used to make bootable USBs). +* `compress`: compress data. +* `gzip`, `gunzip`, `zcat`: compress or expand files. +* `uname`: print system information. +* `cal`: display a calendar. +* `date`: print or set the system date and time. +* `read`: read from standard input into shell variables (also used to read from a file). +* `tr`: translate or delete characters. +* `readonly`: set the readonly attribute for variables. +* `set`: set or unset options and positional parameters. +* `unset`: unset values and attributes of variables and functions. +* `expr`: evaluate expressions. +* `tput`, `reset`: initialize a terminal or query terminfo database (used for more complex terminal output). +* `grep`, `egrep`, `fgrep`: print lines that match patterns (usually used to find text in a file or some text). +* `sleep`: delay for a specified amount of time. +* `break`: exit from for, while, or until loop. +* `continue`: continue for, while, or until loop. +* `logname`: print user's login name. +* `write`: send a message to another user. +* `mesg`: display (or do not display) messages from other users. +* `return`: return from a function or dot script. +* `exit`: cause the sell to exit. + +And some special "commands" or "operators" (for more: [gnu: 3.6 Redirections](https://www.gnu.org/software/bash/manual/html_node/Redirections.html)): + +* `|` (pipe): used between two commands and the output from the command from the left serves as input to the command from the right. +* `>`: redirects output to a file, overwriting the file (or creating a new file). +* `>>`: redirects output to a file, appending to the file (or creating a new file). + +## Shell scripting + +A shell script is nothing more but a file that contains commands in it; they're executed in the same order they are present in the file. A shell script file is usually terminated with a `.sh` extension, independently of the shell being used, but it's not 100% necessary as in Unix systems, an extension mean nothing, other than distinction (visually) between files. Then one can just have an extension-less file as a script. **The script must have execution permissions (`chmod +x file`)**, unless `shell script` is executed in the terminal, where `shell` could be `sh`, `bash`, etc. **Comments** are created by prepending `#` to whatever the text should be a comment. + +It's common practice to have the first line as a **she-bang** (`#!`), which is just a comment telling the interpreter which shell to execute the script with (usable when having the script in your **PATH** so you only call the name of the script like any other command/program). A she-bang has the syntax `#!/path/to/shell some_other_options`, the most common she-bangs being: `#!/bin/sh`, `#!/bin/bash`, `#!/usr/bin/python`, etc. + +Also, some people argue that you shouldn't use absolute paths, since not all Unix operating systems have the same directory structure, or not all programs are going to be installed in the same folder. So a portable she-bang can be made by prepending `/usr/bin/env` and the specify the program to run, for example: `#!/usr/bin/env bash`. + +Like always... the basic "Hello, world!" script: + +```sh +#!/bin/sh +echo "Hello, world!" +``` + +Three ways of executing this script (assuming the file name is `hw`): + +1. Type in terminal `sh hw`. +2. Type in terminal `./hw`. Requires the file to have execute permissions. +3. Type in terminal `hw`. Requires the file to have execute permissions. Requires the file to be in your PATH. + +### Variables + +Variables are case sensitive, meaning that `my_var` and `MY_VAR` are different and a variable name can only contain letters and numbers (`a-z`, `A-Z` and `0-9`) or the underscore character `_`. Can't contain a space. Variables are called by prepending `$` to the variable name. + +Like in most programming languages, there are some reserved words like `if`, `select`, `then`, `until`, `while`, etc., that can't be used as variables or as values of variables. For more: [D.2 Index of Shell Reserved Words](https://www.gnu.org/software/bash/manual/html_node/Reserved-Word-Index.html). + +There is no need to specify a variable type. Anything surrounded by `"` will be treated as text. You can use booleans, numbers, text and arrays (the implementation of arrays depends on the shell being used). Make a variable readonly by calling `readonly variable_name`. Basic syntax: + +* Text variables: `var="my var"`. +* Numeric variables: `var=123`. +* Boolean variables: `var=true` and `var=false`. +* Arrays (assuming `bash` is the shell): + * `var[0]=value1`, `var[...]=...`, `var[n]=valuen`, etc. + * `var=(value1 ... valuen)` + * Access single values with `${var[index]}` and all values with `${var[*]}` or `${var[@]}`. + +There are special variables (for more. [tutorialspoint: Unix / Linux - Special Variables](https://www.tutorialspoint.com/unix/unix-special-variables.htm)): + +* `$`: represents the process ID number, or PID, of the current shell. +* `0`: the filename of the current script. +* `n`: where `n` can be any whole number, correspond to arguments passed to the script (`command arg1 arg2 arg3 argn`). +* `#:` number of arguments supplied to the script. +* `*:` all the arguments are double quoted. +* `@:` all the arguments are individually double quoted. +* `?:` exit status of the last command executed. +* `!:` process number of the last background command. + +When calling a script, you can pass optional (or required) positional arguments like: `command arg1 arg2 arg3 argn`. + +Note that a variable can also take the output of another command, one common way to do this is using `$(command)` or `` `command` ``, for example: `var="$(echo 'this is a command being executed inside the definition of a variable')"` which, since the `echo` command is being run, `var="this is a command being executed inside the definition of a variable"`, which doesn't seem like much, but there could be any command inside `$()` or `` `command` ``. Note that this is not special to defining variables, could also be used as arguments of another command. + +#### Internal Field Separator (IFS) + +This is used by the shell to determine how to do word splitting (how to recognize word boundaries). The default value for `IFS` consists of whitespace characters (space, tab and newline). This value can ve overridden by setting the variable `IFS` to something like, for example, `:`. + +### Conditionals + +#### Exit status + +Any command being run has an exit status, either `0` or `1`, if the command has been executed successfully or otherwise (an error), respectively. + +#### `if` statement + +Pretty similar to other programming languages, evaluates an expression to a `true` or `false` and executes code as specified. `if` statements can be nested, and follow normal rules of logical operations. Basic syntax is: + +```sh +#!/bin/sh +if expression +then +do_something +elif another_expression +then +do_another_thing +else +do_something_else +fi +``` + +The expression is usually wrapped around `[]` or `[[]]`, the first being POSIX compliant and the second `bash`-specific (and other shells). + +Also, some **operators** to compare things use `==` for "equals" and `>` for "greater than", for example; while in a POSIX compliant shell, `=` for "equals" and `-gt` for "greater than" has to be used. For more operators: [tutorialspoint: Unix / Linux - Shell Basic Operators](https://www.tutorialspoint.com/unix/unix-basic-operators.htm) (this also covers **logical operators** and **file test operators**). + +### Case statement + +A common good alternative to multilevel `if` statements, enables you to match several values against one variable. Basic syntax is: + +```sh +case $var in + pattern1) + do_something1 + ;; + pattern2) + subpattern1) + do_subsomething1 + ;; + subpattern2) + do_subsomething2 + ;; + *) + pattern3|pattern4|...|patternN) + do_something3 + ;; + patternM) + do_somethingM + ;; + *) + do_something_default + ;; +esac +``` + +Where the `*` pattern is not necessary but serves the same purpose as a "default" case. + +### Loops + +Loops enable execution of a set of commands repeatedly. Loops, naturally, can be nested. `expression` here (in the basic syntax examples) work the same as mentioned in the "`if` statement" section. For more: [tutorialspoint: Unix / Linux - Shell Loop Types](https://www.tutorialspoint.com/unix/unix-shell-loops.htm). + +#### Loop control + +Similar than other programming languages, there are loop controls to interrupt or continue a loop: + + * `break` statement. + * `continue` statement. + +These statements accept an argument that specify from which loop to exit/continue. + +#### `while` loop + +Enables to execute a set of commands repeatedly until some condition occurs. Basic syntax: + +```sh +#!/bin/sh +while expression +do + do_something +done +``` + +#### `until` loop + +Similar to the `while` loop, the difference is that the `while` loop is executed as long as a condition is true, but the `until` loop... until a condition is true. Basic syntax (similar to `while` loop): + +```sh +#!/bin/sh +until expression +do + do_something +done +``` + +#### `for` loop + +Operates on lists of items. It repeats a set of commands for every item in a list. Basic syntax: + +```sh +#!/bin/sh +for var in word1 word2 ... wordN +do + do_something_with_var +done +``` + +Where `var` is the current value (`word1`, `word2`, etc.) in the loop and the expression after `for` can refer to an array, or the output of a command that outputs a list of things, etc. + +#### `select` loop + +Provides an easy way to create a numbered menu from which users can select options. Basic syntax (similar to `for` loop): + +```sh +select var in word1 word2 ... wordN +do + do_something_with_var +done +``` + +### Meta characters + +Meta characters are used to execute several commands on a single line (depending on what it's needed). The most used meta characters to accomplish this are semi-colon `;`, double ampersand `&&` and double "pipe" `||`. + +* `;`: is used to finish one command (similar to some programming languages), after the command on the left of `;` is finished (whatever the exit code is), the command on the right will be executed. +* `&&`: similar to `;`, but only if the command on the left exits with code `0` (success). +* `||`: similar to `&&`, but for exit code `1`(error). + +### Functions + +Enable to break down the overall functionality of a script into smaller, logical subsections, which can then be called upon to perform their individual tasks when needed (like in any other programming language...). For more: [tutorialspoint: Unix / Linux - Shell Functions](https://www.tutorialspoint.com/unix/unix-shell-functions.htm). Basic syntax: + +```sh +#!/bin/sh +function_name () { + do_something +} +``` + +Functions can also take arguments and can access their individual arguments (each function will have a different "storage" for their arguments). Functions can also be nested. Here `exit` will not only will finish the function code, but also the shell script that called it, instead use `return` plus an exit code to just exit the function. diff --git a/blog/src/a/sql_video_notes.md b/blog/src/a/sql_video_notes.md index 0814fd9..4b37119 100644 --- a/blog/src/a/sql_video_notes.md +++ b/blog/src/a/sql_video_notes.md @@ -1,4 +1,4 @@ -# SQL tutorial videos notes +# SQL tutorial video notes I was requested to make summaries of videos about SQL, these are the notes (mostly this is a transcription of what I found useful). The videos in question are: [SQL Tutorial - Full Database Course for Beginners](https://www.youtube.com/watch?v=HXV3zeQKqGY), [MySQL Tutorial for Beginners [Full Course]](https://www.youtube.com/watch?v=7S_tz1z_5bA) and [Advanced SQL course | SQL tutorial advanced](https://www.youtube.com/watch?v=2Fn0WAyZV0E). Also, some notes were taken from [w3schools.com's SQL Tutorial](https://www.w3schools.com/sql/) and [MySQL 8.0 Reference Manual](https://dev.mysql.com/doc/refman/8.0/en/). -- cgit v1.2.3-70-g09d2