Cheatsheet: Linux shell

Linux terminal includes myriad of opportunities by default. But sometimes we can’t figure out how to execute any operation easier. From scratch we tried to parse enourmous logs by eyes or use graphical instruments when it’s not necessary. Another problem – we’re forgetting how to type any helpful command we need. Thus there is one way to scroll Google and read the same forum branch we did before. So, it would be better to have a petty prompt with command syntax we really need. It’s useful as on the tab of your document application as well as pinned to the wall in your room.

Here is my personal educational table with Linux terminal commands I’m trying to use. I didn’t include too obvious things as “cd”, “pwd” and “shutdown -r now”. Hope every Linux begginer was learned this commands in the first couple of days. When I run command 3-5 times and frigthened to forget – this command is going to show up at table. Once in a while I’ll update this prompt. Well, follow this post if you’re interested.

System info File commands
whoami– Who you are logged in as
lsb_release -a– find out information about system
lsb_release -si– find out Linux distro name
finger user– Display information about user
uname -a– Show kernel information
printenv– list all environment variables
pwck – check /etc/passwd file correctness
pstree– processes tree visualization
last reboot– get a journal of system sessions from boot to shutdown
shutdown -r 5– Shutdown the system in 5 minutes and reboot
ls -al – Formatted listing with hidden files
tree – check current directory in tree mode
for i in G M K; do du -ah /dir| grep [0-9]$i | sort -nr -k 1; done – list all files in directory with clear output
du -a /dir | sort -n -r | head -n 10 – find 10 largest files in /dir
chmod u+s file = set SETUID bit to file (o+t makes the same thing with “sticky” bit)
chattr +i file – make file untouchable
mkdir -p /path/to/parent/child – creates parent directory if it doesn’t exist
rm -rf dir – Force remove an entire directory dir and all it’s included files and subdirectories
rm !(*.foo|*.bar) /dir – remove everything in /dir except files with .foo and .bar extension
find . -name “*.mac” | xargs rm – removes all found files with .mac extension
find . -not \( -name .svn -prune -o -name .git -prune -o -name .terraform -prune \) -type f -print0 | xargs -0 perl -pi.bak -e ‘s/ +$//’ – removes all trailing spaces in directory files
find /dir -name “*.txt” | sed “s/\.txt$//” | xargs -i echo mv {}.txt {}.bak | sh – moves all .txt files to .bak (useful for backup)
mount -o loop file.iso /mnt/cdrom – mount file.iso image in CD-ROM
dd if=image.iso of=/dev/sdc1 bs=512 count=1 – write image.iso to USB device
cd – – Move to previous working directory
dos2unix filedos.txt fileunix.txt – convert txt file format
cp file{,.bak} – equivalent of cp file file.bak, makes it shorter
ls | while read upName; do loName=`echo “${upName}” | tr ‘[:upper:]’ ‘[:lower:]’`; mv “$upName” “$loName”; done – mini-loop to rename all your files to lowercase
wc -l file – count lines in file
head -20 file | tail -20 file – Display the first/last 20 lines of the filersync -az user@remote_host:/home/user/file remote_file – sync file with remote host
find -mindepth 1 -maxdepth 1 -type d -print0 | xargs -0 rm -R – remove all subdirectories within a special directoryrsync -az user@remote_host:/home/user/file remote_file – sync file with remote host
rsync -az -e ‘ssh -p 222’ –rsync-path=”sudo rsync” user@remote_host:/home/user/file remote_file – set different ssh port and call sudo to copy from remote machine
sync – sync memory pages with disk
Hardware
hwclock – check hardware time

df -h– Show disk partitions

free– Show memory and swap usage

hdparm -Tt /dev/sda– check your disk performance

dmidecode -q – get full hardware information

lsusb -tv– check USB connections as a tree (also works with lspci)

badblocks -v /dev/sda– check disk for corrupted blocks presence

echo 1 | sudo tee /sys/bus/pci/rescan– force your system to scan PCI buses

sensors – check hardware temperature (requires lm-sensors package)

Keyboard shortcuts Text mainpulation
Ctrl + R– Allows you to type a part of the command you’re looking for and finds it

Ctrl + L– Clear the screen

Ctrl + A– Return to the start of the command you’re typing

Ctrl + E– Go to the end of the command you’re typing

Ctrl + U– Cut everything before the cursor to a special clipboard, erases the whole line

Ctrl + K– Cut everything after the cursor to a special clipboard

Ctrl + Y– Paste from the special clipboard that Ctrl + Uand Ctrl + Ksave their data to

Ctrl + T– Swap the two characters before the cursor

Ctrl + W– Delete the word / argument left of the cursor in the current line

Ctrl + D– Log out of current session, similar to exit

Ctrl + I – Horizontal tabulation

Ctrl + S– Freeze terminal in stdin

Ctrl + Q– Unfreeze terminal in stdin

paste file1 file2– print file1and file2 content as entire table

cat -n file– numerate each line in file

sort file1 file2 | uniq -d– find general lines between 2 files

truncate -s 0 filename.txt – truncate a large text file

find . -name ‘*.php’ | xargs wc -l – count lines of code in file with special extension

cloc dir/ – smart utility, can see the difference between code and comments in special file

Sed
sed ‘s/word1/word2/g’ file– simple sed manipulation with file

sed ‘s/word1/word2/g; s/word3/word4/g; s/word5/word6/g’ file– replacement with multiple patterns in file

sed ‘s/cat/dog/’ file_in >> file_out– redirect sed manipulation to file_out

sed -e ‘1d’ file – remove first line from file

sed -n ‘/word/p’ – prints only strings containing word

sed -i ‘/^$/d’ fileremove all blank lines from file

find /home/user/ -type f | xargs sed -i  ‘s/a.example.com/b.example.com/g’ – recursively substitute each mattern match in files located on /home/user

Awk
awk ‘{print $1}’ file –prints first column in filewith columns splitted by space or tab

awk -F ‘:’ ‘{print $1}’ file– use a special delimiter under -F option

awk ‘$1 ~ /^[a-c]/’ file– match a first field by regular expression

awk -F ‘:’ ‘{sum+=$3} END {print sum}’ file – sum all rows at the 3rd column

awk ‘{avg+=$2}END{print avg/NR}’ file– compute the average number of the 2nd column at file

awk ‘NR>3’ file– print everything after the 3rd line

awk -v search=”$s” ‘{print search}’ – pass shell variable to awk

awk ‘NR==FNR{arr[$0];next} $0 in arr’ file1 file2 – finds out common lines between 2 files or command outputs

SSH/FTP/Telnet Compression
scp -r root@192.168.0.10:/path/to/foo /local/path/foo– SCP from remote to local host

scp -r foo root@192.168.0.10:/some/file/foo– SCP from local to remote host

ssh-copy-id -i key.pub user@host– Add your key (file in option -i) to hostfor userto enable a keyed or passwordless login

sshfs user@host:/ /media/host -o nonempty -o reconnect– mount filesystem of remote host to your local filesystem through SSH

curlftpfs ftp-user:ftp-pass@my-ftp-location.local /mnt/my_ftp/ – mount remote filesystem by FTP

tar -czf file.tar.gz files– Create a tar with Gzip compression

tar -xzf file.tar.gz – Extract a tar using Gzip

tar -zxvf archive.tar.gz -C /dir – extract tar.gz archive to dir

unzip archive.zip -d mydir/ – extract zip archive to mydir directory

gzip file– Compresses fileand renames it to file.gz

gunzip file.gz– Decompresses file.gzback to file

Printing
lpq– Display jobs in print queue

lprm– Remove jobs from queue

lpr– Print a file

lpc– Printer control tool

printtool– Start X printer setup interface

Network GNU Screen
iwconfig – Used to set the parameters of the network interface which are specific to the wireless operation

iwlist – used to display some additional information from a wireless network interface

iwlist scan – check accessible wireless access points

route -n – show full local routing table

route add -net 0/0 192.168.0.1 – add new route to gateway 192.168.0.1

echo “1” > /proc/sys/net/ipv4/ip_forward allow IP packages forwarding

whois domain – Get whois information for domain

dig domain – Get DNS information for domain

dig +short myip.opendns.com @resolver1.opendns.com – check public ip address

netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n – IP of hosts connected to your server with number of connections

netstat -n | grep :80 |wc –l – check the number of connections with the webserver

netstat -tulpn | grep :5432 – find out which process is listening on 5432 port

route add default gw 10.20.10.254 eno2.71 – add new default route to gateway 10.20.10.254 through interface eno2.71

ip route del default via 10.20.10.254 – default route to gateway 10.20.10.254

tcpdump -n -i eth0 – discard DNS resolver at the packets capturing

tcpdump -i eth0 dst 16.181.170.246 and port 22 – capture package exchange between local and remote hosts by SSH

nmap -iL /tmp/test.txt – scan the list of hosts from the file

nmap -O –osscan-guess 192.168.0.11 – detect the OS version of remote host

nmap -p T:80 192.168.1.1 – scan 80 TCP port

nmap -sV 192.168.0.13 – detect remote services running on remote host

Ctrl + A & | – split screen vertically (| is a pipe)

Ctrl + A & S  – split screen horizontally

Ctrl + A & Q  – unsplit all sessions

Ctrl + A & Tab  – switch between sessions

Ctrl + A & Tab  – switch between sessions

Ctrl + A & с  – start a session in empty region

Ctrl + A & с  – create new window (from active region)

Ctrl + A & “backspace”  – move to previous window

Ctrl + A & [n]  – go to [n]-th window

Ctrl + A & “esc”  – switch to scroll mode

Ctrl + A & “enter” & “enter”  – switch from scroll mode

Tmux
Ctrl + B & % – split screen vertically

Ctrl + B &  – split screen horizontally

Ctrl + B & Arrow  – navigate on window panes

Ctrl + D – kill the active window pane

Ctrl + B & c  – create new window

Ctrl + B & p  – switch to the previous window

Ctrl + B & n  – switch to the next window

Ctrl + B & d  – move your session to the background

Ctrl + B & z  – resize current window to the full screen and come back the same way

tmux attach -t mysession – restore your session

tmux rename-session -t 0 database – rename your session

tmux kill-server – in case of your prefix shortcut doesn’t work

Searching Other
grep pattern file– Search for patternin file

grep -r pattern dir– Search recursively for patternin dir

locate file– Find all instances of file

find / -name filename– Starting with the root directory, look for the file called filename

find dir -user user1– find all files in dir belonged to user1

find dir -type f -mtime -10 check all files in dirwhich was modified in last 10 days

find /dir -type f -size +500M– find files larger than 500 mbytes

find /dir -mtime -1 –find all ifies changed during the day

find /dir -mtime +7 – find all files hasn’t changed during the last week

apropos keyword – list enable commands using keyword

which command (file)– Show full path name of command (file)

passwd accountname– Give accountnamea new password

fg n– Brings job nto the foreground

startx– Start the X system

gpg -c file1– encrypt file1

alias hh=’history’ – assign alias to historycommand

locale-gen ru_RU– add standard locale

locale-gen ru_RU.UTF-8– add UTF-8 locale

ldd /usr/bin/ssh– get list of libraries required to run ssh

sudo !! – run last command as a root

echo password | sudo -S command – pass password to sudo as stdin

logrotate /etc/logrotate.d/nginx – setup new service with configuration to logrotate

/msg NickServ identify AccountName Your-Password – sign in to IRC Freenode

Kernel settings
lsmod – check enabled kernel modules

sysctl -p– save all changed kernel variables permanently

watch -n1 ‘cat /proc/interrupts’– check system interrupts at the real time

Apt/dpkg Yum/DNF/RPM
apt-cache policy package– check packageversion

dpkg –configure -a && apt-get install -f – fix broken packages

dpkg-buildpackage -rfakeroot -b– build package from sources

dpkg-query -W -f=’${Installed-Size;10}\t${Package}\n’ | sort -k1,1n– prints list of packages with their size

dpkg -l – prints full list of installed packages

dpkg -s package– shows info about package

dpkg -L – list of all package files

rpm -ivh –nodeps package.rpmallows to install packagewithout dependencies checking

rpm -qg “System Environment/Daemons”– prints list of packages related to System Environment/Daemonsgroup

rpm -qa | grep package– search any package

rpm -qi packagepackageinformationrpm -qlp package– list all packagefiles

rpm -qRp package– list all packagedependencies

rpm -q -a –qf ‘%10{SIZE}\t%{NAME}\n’ | sort -k1,1n– prints list of packages with their size

yum list – check all installed packagesyum –showduplicates list httpd – list all enabled versions of httpd

yum –setopt=tsflags=noscripts remove package – remove package from YUM/RPM database; useful when you accidentally removed some files by yourself and yum finishes removal with “Error in PREUN scriptlet”

Apache Nginx
apachectl configtest– check Apache configuration syntax

apache2ctl -M– check included apache modules (deb)

httpd -M – check included apache modules (centos)

httpd -X – run Apache at the debug mode

service apache2 reload – reconfigure Apache without restart

nginx -T– check Nginx configuration syntax

nginx -V– check included Nginx modules

Ffmpeg Cisco/Allied Telesis
ffmpeg -i some.ts – discover streams and pids of file

ffmpeg -re -i video.mp4 -vcodec libx264 -acodec aac -f flv rtmp://login:password@server_name/pull/mystream – make rtmp stream from video.mp4 and redirect it to server_name authenticated by login and password

ffmpeg -i rtmp://10.20.10.51:1935/app/channel -strict -2 test.mp4 – record HLS stream to file

ffmpeg -i udp://223.255.16.1:5004 – check RTP stream parameters

ffmpeg -i “concat:file1.ts|file2.ts|file3.ts” -c copy output.ts – concatenate multiple files into single one

ffmpeg -re -i file.ts -vcodec libx264 -acodec aac -f mpegts “udp://234.5.5.5:5000” – TS file multicast streaming

ffmpeg -i “udp://239.255.16.1:5004?fifo_size=1000000&overrun_nonfatal=1” -vcodec libx264 -acodec aac -c:s copy -strict -2 -map 0:p:1001 -f mpegts some.ts – map a special program 1001 from multicast to transport stream file with additional buffer options

ffmpeg -i input.mkv -map 0:3 -map 0:2 -map 0:1 -map 0:0 -c copy output.mkv – map special streams (0:0-0:3) from video file to another one

add firewall policy=name rule=2 interface=eth0 protocol=tcp port=1723 ip=192.168.1.10 gblip=56.23.149.10 gblport=1723 action=allow– add new NAT rule to router (just a template)

copy running-config startup-config– save all configuration changes

add firewall policy=name proxy=http interface=eth0 gblinterface=eth0 direction=both ip=192.168.0.13– create HTTP proxy

Date/time settings Daemon Management
ln -s /usr/share/zoneinfo/Europe/Moscow /etc/localtime– change timezone on machine

date– Show the current date and time

cal– show up shell calendar

time– check command/script execution time

ntpdate ru.pool.ntp.org – sync your time with ntp server (ntpd service should be stopped at this moment)

watch -n seconds script – repeat script every n seconds

zdump timezone– check current time in timezone

systemctl list-units – check all system services

service –status-all– check all system services at “init.d” system

update-rc.d nginx defaults – add nginx daemon to startup on Ubuntu

Wget/curl
curl -o /path/to/directory url:// – download a file from url://  to special directory

wget -c file– Continue a stopped download

Git Proxmox VE
git config –global user.name “John Doe”– primary user setting

git config –global user.name “johndoe@mail.com” – primary user setting

git rev-list –count master – count number of commits at the special branch

git rev-list –count – count number of commits for all branches in repository

git rm -r –cached . – apply .gitignore changes to ignore already commited giles

pct enter 100– connect to shell of container 100

pct start 100 – switch on container 100

Sysstat/sar Streaming
sar -P ALL 20 10– track all cores in CPU between 20 seconds 10 times

sar -r 20 10– track memory usage between 20 seconds 10 times

sar -S 20 10– track swap usage

sar -n DEV 20 10– check up network traffic by devices

iostat -d 2 10– real-time device report: interval 2 seconds, prints 10 times

pidstat -r– show full processes list with memory usage statistics

sar -r -f /var/log/sa/sa$(date +%d)– check memory usage for current day (useful for crontab rule)

dvbsnoop -s ts -if test.ts 0x135 > forum.bin – get information about particular PID 0x135 of transport stream in binary format

dvbsnoop -s ts -n 1 -if full.ts – get PAT table from transport stream

dvbsnoop -s ts -n 1 -nph -if full.ts – get PMT table from transport stream

tsinfo -v -max 60000 1080i.mpg – get PAT and PMT tables from MPEG-TS files

tsplay destination.ts 234.5.5.5:5000 – direct multicast streaming of destination.ts file

tcpdump -i eth4 -c 100 dst host 233.48.70.53 and port 1234 and multicast -w test.ts – record a multicast stream from udp://233.48.70.53:1234 to transport stream file through a network interface eth4

socat -u UDP4-RECV:5000,ip-add-membership=233.48.70.183:10.1.43.100 CREATE:output.ts – record a multicast stream from udp://233.48.70.183:5000 to transport stream file through a network interface with IP address 10.1.43.10

vlc -vvv udp://@239.10.10.12:1001 –program=1600 – play a special program of multicast stream from VLC

Process management Python
kill -TERM 98989 – correct 98989 process termination

fg n – Brings job n to the foreground

ps aux | sort -nk +4 | tail – quickly look which process consumes the most part of memory

ldd /usr/bin/ssh – get list of libraries required to run ssh

ps -efL | grep PID – get the threads list of process identified by PID

strace -p PID – trace current process identified by PID

strace -e open ls – trace only open system calls

strace -t man strace – get a timestamp of each system call

strace -r pwd – get a relative timestamp of system calls (since 0)

strace -c whoami – get a summary table of process system calls

virtualenv -p /usr/bin/python3 myproject/ – set Python virtual environment with 3rd version

source bin/activate – activate virtual environment

deactivate – deactivate virtual environment

Last update: 7 August 2016.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s