This table of contents was created using md_toc

• installed with pip install md_toc
• used with python -m md_toc github readme.md

• Table of contents
• Shell
  • Bash
    • Variables
    • Basics
• Basic UNIX tools
  • Keys
    • Cursor movement
    • Deleting
  • man
  • Navigation
  • Directories
  • File Content
  • File search
  • File permissions
  • Processes
  • Processes (Backgroun)
  • Text processing
    • Grep (search)
    • sed (stream editor / replacement)
    • awk (pattern scanning and processing)
    • networking
    • SSH
  • Archiving
  • Users
• VI(m)
  • Moving the cursor
  • Jumping the cursor
  • Command mode (default)
  • Visual mode (text selection)
  • Exiting
  • Multiple files
• Git
  • Git has three stages:
  • Workflow of git:
  • Working remotely:
  • Commands
    • Basic ones
    • Remote
    • Commits
    • Fixing mistakes
      • Mistakes not pushed yet:
      • Mistakes were pushed:
    • Branching
• Github CLI
  • GH Config
  • GH Gists
    • Flags
    • Example
  • GH Repo
    • Flags
    • View
    • Edit
    • Example workflow
    • Without github cli
• Docker
  • Basic commands
    • Managing containers
    • Managing images
    • Dockerhub
  • Tagging and pushing
    • Azure Container Registry (ACR)
  • Dockerfile
  • .dockerignore
  • Docker volumes
  • Docker compose
• MySQL
  • CLI
    • Databases
    • Tables
    • users
    • variables
    • mysqldump

• ech0 $0 - Shows the current shell we are using

Bash (Bourne Again SHell) is a scripting language and a command-line interpreter (shell).

• Variables
• Conditionals (if, case)
• Loops (for, while, until)
• Functions
• Arrays
• File I/O, process control, pipes, etc.

• MyVar=555 - Creates a variable, print it wiht echo $MyVar
  • using single quotes prevent parsing variables, double quotes is fine
• export MyVar - Exports the variable so other shells can access it
• unset MyVar - Unsets the variable

• source config.sh - Executies a file in the current shell
• bash --version - current bash version
• #!/bin/bash - she-bang top of a script, instructs a shell to run a script with bash
• # - comment

Some great resources:

• Everything Paul Cobbatin has written.
• A game enables practical learning OverTheWire Wargames Bandit

• CTRL+A - move to beginning of line
• CTRL+E - moves to end of line
• CTRL+F - moves forward one character
• CTRL+B - moves backward one character
• ALT+B - moves backward one word
• ALT+F - moves forward one word

• CTRL+W - kills the word backwards

• ALT+D - deletes the word forward

• CTRL+Y - brings back last item deleted

• CTRL+K - deletes forward to end of line

• CTRL+U - kills backward to the beginning of line

• CTRL+L - clears screen, does not clear history, clear does that

• CTRL+R - searches backward

• CTRL+Z - stops the current command, resume with fg in the foreground or bg in the background

• CTRL+C - cancels the current command

• !! - repeats the last command

• exit - logs out of current session

• pwd - Print Working Directory
• ls -la- List all files (including hidden), long format
• cd /path - Change directory
• cd ~ - Go home
• cd .. - Go back to parent dir
• cd - Back to previous directory

• mkdir -p dir/subdir - Create nested directories
• touch file.txt - Create empty file or update timestamp
• cp -r src dest - Copy recursively
• mv old new - Move or rename
• rm -rf dir - Remove recursively (cannot be undone so be careful)
• rm -i file - Interactive remove

• cat file - Concatenate/print

• cat > file - Allows typing the content that will be added to the file, CTRL + D when done.

• less file - Pager (space=page, q=quit, /search)

• head -n 20 file - First 20 lines

• tail -f file - Last lines, follow

• tail -n 50 file

• nano - opens up a basic editor

  • CTRL + O activates WriteOut for saving, followed by Enter
  • CTRL + X exits nano

• | - Pipe the stdout of a command into another command

• && - Run a command after a previous command

• ; - Same as above

• > - Adds the result of a command to a file, replaces any previous content

• >> - Appends result of a command to a file, appends to the end

• find /path -name "*.log" - Find by name
• find . -type f -mtime -1 - Modified in last 24h
• grep -r "search" . - Recursive search

Permissions: rwxrwxrwx - user/group/other

• r - read

• w - write

• x - execute

• chmod u+x myfile - Make myfile executable for user by adding x with +

• chmod u-x myfile - Remove executable for user by removing x with -

• chmod ug+w myfile - Make myfile writable for user and group.

Instead of using rwx we can also use numbers:

• 4: read permission
• 2: write permission
• 1: execute permission

For example, to add read and write permissions to user that would be

• chmod 600 myfile - 6 means 4+2 (read + write), 0 for group and others

• chmod 644 myfile - read+write for user, read for group and others

• ls -l - See permissions

• ps aux - All processes

• ps aux | grep nginx - Search for nginx

• top - Interactive process viewer (h=help)

• htop - Better top (if installed)

• kill PID

• kill -9 PID - SIGKILL

• pkill nginx - Kill by name

• df -h - Disk usage human readable

• du -sh dir - Directory size

• free -h - Memory

• uname -a - Kernel info

• lscpu - CPU info

• lsblk - Block devices (disks)

• command & # Move it to the background
• jobs # List background processes
• fg %1 # Foreground job 1
• Ctrl+Z # Suspend a job
• bg # Resume in background

• grep "error" app.log
• grep -i "error" - Case insensitive
• grep -r "TODO" . - Recursive
• grep -E "error|warn" - Extended regex

• sed 's/old/new/g' file - Replace
• sed 's/old/new/g' file > newfile - Add replacement result to a new file
• sed '1,5d' file - Delete lines 1-5
• sed '/pattern/d' file - Delete matching lines

• awk '{print $1}' file - Print first field (space/tab delimited)
• awk -F, '{print $2}' data.csv - Field separator comma
• awk '{sum += $1} END {print sum}' - Sum column

• ip addr show - Modern (replaces ifconfig)

• ip route - Routing table

• ping 8.8.8.8 - Pings a host

• curl -X POST -d '{"key":"val"}' url - Makes a post request

• netstat -tuln - Listening ports

• ss -tuln - Faster modern replacement

• dig example.com - DNS (or nslookup)

• traceroute example.com - Lists the hops made before hitting example.com

• wget url - Download

• ssh user@hostname - Connect

• ssh user@hostname -p 2222 - Connect using a port

• ssh -i /path/to/id_rsa user@hostname - Connect using a private key

  • Instead of typing ssh -i ~/.ssh/my_key.pem -p 2222 user@192.168.1.50 every time, we can add this to ~/.ssh/config file (if config doesnt exist create it)

    Host myserver
    HostName 192.168.1.50
    User user
    Port 2222
    IdentityFile ~/.ssh/my_key.pem
    

  • this allows us to connect with ssh myserver

• ssh -L 8080:127.0.0.1:3306 user@hostname - Local port forwarding, for example accessing a remote database running on port 3306 locally on port 8080

• ssh -R 9000:127.0.0.1:3000 user@hostname - Reverse port forwarding, for example sharing a local development server (e.g., port 3000) with the remote server (on port 9000)

• scp localfile.txt user@hostname:/remote/dir/ - Copy file to server

• scp user@hostname:/remote/file.txt /local/dir/ - Copy file from server

• scp -r /local/dir user@hostname:/remote/dir/ - Copy diretory to server

• sftp user@hostname - interactive transfer

ssh-keygen -t ed25519

• tar -czf archive.tar.gz dir/ - Creates a tarball

• tar -xzf archive.tar.gz - Extract

  • -c : Creates Archive
  • -x : Extract the archive
  • -f : creates archive with given filename
  • -z : zip, tells tar to use gzip
  • -t : displays or lists files in archived file
  • -u : archives and adds to an existing archive file
  • -v : Displays Verbose Information
  • -A : Concatenates the archive files
  • -j : filter archive tar file using tbzip
  • -W : Verify a archive file
  • -r : update or add file or directory in already existed .tar file

• zip -r archive.zip dir/

• unzip archive.zip

• gzip file

• gunzip file.gz

• whoami
• id
• sudo -i - Root shell
• su - user
• adduser username
• passwd username
• groups
• usermod -aG sudo username - Add to group

vi is an editor in most unix systems. VIM (VI improved) often comes with Linux distributions.

• vi or vim - Opens the editor
• vimtutor - Opens up a tutorial on how to use vim

• k - moves the cursor up
• j - moves the cursor down
• h - left
• l - right

• 0 - Jump to start of line
• $ - Jump to end of line
• e - jump to fowrad one word
• b - jump back one word
• gg - jump to top
• G - jump to bottom

• i -start insert mode
• esc - exit insert mode
• x - deletes character
• db - delete word backwards
• de - deletes word forward
• dd - deletes current line
• u - undo

• y - copy marked text
• yy - copy current line
• d - delete marked text
• dd - delete current line
• p - paste

• :w - write (save) the file but dont quit
• :wq - save and quit
• :q - quit, errors if not saved
• :q! - quit without saving

• :e filename - edits a file
• :tabe - make a new tab
• gt - switch to next tab
• gT - switch to previous tab
• ctrl+ws - split horizontally
• ctrl+wv - split vertically
• ctrl+ww - switch between windows
• ctrl+wq - quit a window

• Configuring : config, help.
• Creating : init, clone.
• Make Changes: status, diff, add, commit, reset, rm
• Branching & Merging: branch, checkout, merge, stash
• Review History: log, tag, diff, show
• Update and Publish: fetch, pull, push, remote.

- Committed : means data is safely committed.
- Modified : means you've changed the file but not committed.
- Staged : means you've marked a modified file in its current version.

- Working Area: holds the actual files, modified and unmodified
- Staging area: snapshot for next commit. Will be done by git add command.
- HEAD : Points to the last commit (and current branch).

- remote: manages set of tracked repositories.
- fetch: fetch new work from that remote server after cloning. We can later merge that repo with existing with merge command.
- pull: Automatically fetch and merge newest commit from remote server to local branch. By default, it combines fetch and merge.
- push: This will push to the remote server from local branch.

• git init - initializes a new git repo
• git clone - clones a git respositry
• git log --oneline - prints sha in one line
• git add . - adds all modified files to staging area, we can also specify which files to add by using their names

• git remote add origin - add a remote repo, name it origin
• git remote set-url origin - change the url of origin remote repo
• git push origin main - push repo to repo origin branch main

• git commit -m "message" - commit files in staging area
• git commit --ammend - change the message of last commit
• git rm --cached - opposite of git add, takes files out of staging area into working area

OBS!

If changes haven't been pushed to remote yet. Its fine to use git reset because we are still working in local commits. However, using git reset when a commit is already pushed it will be rejected because local history will be behind remote. In which case we have two options: - git push origin main --force - Replaces everything in the remote with our local build - git revert - Keeps the history in tact, makes a new commit with the fixed mistakes.

• git reset --soft HEAD~1 - undoes last commit, changes will be in staging area.
• git reset --mixed HEAD~1 - undoes last commit, changes will be in working directoy
• git reset --hard HEAD~1 - undoes last commit, changes are trashed entirely

• git revert sha- undo last commit but keeps history and adds a new "undo" commit on top of it.
• git revert --no-comit sha - same as above but keeps work in staging area instead of making a new commit, similar to reset --soft. However, the next step will be to do a new git commit -m "fixed something that was wrong before pushing.

Once a mistake has been fixed do git push origin main to deliver the fixed mistakes.

• gh --version - Veryify installation and current version
• gh auth login - Authentication

• gh gist create <file> - create a gist from a file
• gh gist list - list all gists
• gh gist view <id> - view a gist in terminal
• gh gist view <id> --web - open gist in browser
• gh gist edit <id> - edit a gist
• gh gist delete <id> - delete a gist
• gh gist clone <id> - clone gist locally

• --public - Makes it public, default is private
• --public - Make gist public
• --desc "<text>" - Add description
• --filename <name> - Name file when using stdin

Add the contents of a js file to a gist

• gh repo create <repo-name> - To create a new repo, some questions will be asked for private, public etc.

• gh repo create --help - See available commands

• gh repo view - Open details about the repo in the cli

• gh repo view --web - Open the current repo on the web

• --public - create a public repo
• --private - create a private repo
• --source <dir> - Use an existing project as source
• --push - Push the local project to the repo
• --description <text> - Sets a description to the repo
• --license - Sets a license

• gh repo view --json - A list of configurations we can view from our current repo
• gh repo view --json visibility - Shows the visibility configuration of the current repo
• gh repo view --json url - Shows the github url of the current repo
• gh repo view --json isPrivate - Alternative to visibility, but returns a true/false value

• gh repo - Opens up a selection for things we can do to edit the repo
• gh repo edit --visibility public --accept-visibility-change-consequences - Changes the repo visibility to public
• gh repo edit --description "Learning how to use typescript with react" - Changes the description of the repo

git init
git add .
git commit -m "initial commit"
gh repo create my-repo --private --source=. --push

Create repo on githubs website
git remote add origin https://github.com/username/repo.git
git push -u origin main

cat file.js | gh gist create --filename file.js --desc "debug snippet"

• docker info - shows containers, images and a bunch of other stuff.

• docker run -it nginx - Runs as interactive keeping stdin open and creates a pseudo-TT (Teletypewrites, its just a terminal).

• docker run it ... bash - Runs an interactive with a bash terminal

• docker run -d - Runs as dettached (background)

• docker run --name nginx-server nginx - Gives the container a specific name

• docker run -it alpine sh - Alpine is a small linux distro, good for linux, in here we can install node etc, but its better to just use a node image instead. Use it as a sandbox. We have to use sh because it doesnt include bash we can install it though. it also uses apk for package management.

What run does:

• Looks for a cached image, if it doesnt find it, it will pull it from Dockerhub. Then uses that image to create and start a new container.
• We can map ports by using -p flag: host:container, in for instance if our container listens to port 80 and we want to access it from 8080 insteadn its -p 8080:80.

• docker exec - executes a command from inside an already running container, for example
• docker exec -it mysql mysql -u myappuser -p - logs into mysql
• docker ls - list containers
• docker ps - same as above
• docker ls -a - list all containers, including those not running
• docker stop id - Stops a container
• docker rm id - removes a container
• docker rm id id id - removes multiple containers
• docker logs name - gets the logs of a container
• docker stats - shows container stats such as CPU usage etc..

Docker containers are often compared to virtual machines, however they are just standard processess running the host operative system (kernel when on a unix system), because of this we can just do ps aux and see the processes directly. Windows/Mac runs a mini-VM Linux with a kernel on it for Docker containers to run on.

Performance wise, this means something, when a container runs on a VM, there is a guest OS inside running its own processes, which is taxing. On native Linux the containers run along the other processes of the host, so no extra taxing.

• docker image ls - list images
• docker images - same as above
• docker pull image - pulls an image
• docker rmi image - removes an image

An image is just a static snapshot of a filesystem (binaries, libraries, config files) and JSON meta data about the image data and how to run it.

• when we run an image, docker creates a namespace and cgroups for the container (process).
• a namespace isolates its own filesystem, network and users. This way a container has no idea of other processes existing outside.
• cgroups limits the amount of resources a container can use. This allows the kernel to prevent a container from using more resources than what its allowed to use.

Docker Hub images follow this format: <username>/<repository>:<tag>

• username - Docker Hub account

• repository - the image

• tag - version

• docker tag myNodeAPI yosmel/myNodeAPI:latest - Tag an image

• docker login - login to dockerhub

• docker push yosmel/myNodeAPI:latest - Push tagged image to docker github

When using azure, instead of the Dockerhub username, we use the registry name.

• docker tag myNodeAPI mycompanyregistry.azurecr.io/web-api:latest - Tag an image
• az acr login --name mycompanyregistry - Login to acr
• docker push mycompanyregistry.azurecr.io/web-api:latest - push the image to Azure Container Registry

• FROM - The os, image to build from.
• ENV - environment vraiables
• RUN - Runs commands during the building process of the image (docker build)
• CMD - Runs commands when the container starts (docker run)
• WORKDIR - Sets the working directory for the container
• COPY - Copies files from the specified path of host to container

FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
CMD ["npm", "start"]

• docker built -t myAPI . - Builds an image from the directory the Dockerfile is located at.
• docker run -p 3000:3000 myAPI - Run the image we just built.

Keeps the image small by avoiding unecessary files

node_modules
.git
.gitignore
Dockerfile

• mysql -u root -p - access monitor as root, will prompt for password
• system clear; - clears the screen

• show databases; - show databases, all queries must end in a delimiter ; is the default.
• use database - selects a database
• create database name - creates a new database
• DROP DATABASE databaseName - Deletes a database

• show tables - show tables in a database
• describe tableName - shows the structure of a table
• CREATE TABLE City (id INT AUTO_INCREMENT, Name VARCHAR(120), PRIMARY KEY(id)); - Creates a new table
• CREATE TABLE Clinics (id INT AUTO_INCREMENT, Name VARCHAR(120), city_id INT, PRIMARY KEY(id), FOREIGN KEY(city_id) REFERENCES city(id)); - Creates a new table with a foreign key (1-M)
• CREATE TABLE comments(id INT AUTO_INCREMENT, post_id INT, user_id INT, body TEXT, PRIMARY KEY(id), FOREIGN KEY(user_id) references users(id), FOREIGN KEY(post_id) references posts(id)); - Table with 2 foreign keys, linking table (M-M)
• ALTER TABLE tableName ADD COLUMN colName VARCHAR(120); - Adds a new column to existing table
• INSERT INTO tableName (col1Name, col2Name) VALUES ('value', 'value');
• SELECT * FROM tableName - Select records from a table
• UPDATE tableName SET colName = 'newValue' WHERE colName = value - Update a record
• DELETE FROM tableName WHERE colName = value - Deletes a record
• TRUNCATE table tableName - Delete all records in a table
• DROP TABLE tableName - Deletes a table

• SELECT User,Host FROM mysql.user; - Lists all users
• CREATE USER 'username'@'localhost' IDENTIFIED BY 'password' - Creates a new user
• GRANT ALL ON database.* TO 'user'@'localhost' - Grants all access on a database to a user
• SHOW GRANTS FOR 'someuser'@'localhost'; - show grants
• DELETE FROM user WHERE user = 'dbuser' - deletes a user
• FLUSH PRIVILEGES - flush user changes after deleting a user
• ALTER USER 'dbUser'@'localhost' IDENTIFIED BY 'newPw'; - Chanes the password of a user

• SHOW VARIABLES WHERE Variable_name = 'hostname'; - shows the ip adress of the mysql host
• SHOW VARIABLES LIKE '%password%' - Show variables and policies for password
• SET GLOBAL validate_password.policy = MEDIUM; - Changes the password policy to medium

• mysqldump -u root -p databaseName > db_backup.sql - Exports a backup dump