Linux Terminal Basics

Graphical User Interface (GUI) has been useful and easy to use for all types of computer users. We all use it every day to get things done. However, the computing history has the provision to do command-line instructions also to do the same things. Linux terminal is one such command line interface. Let us take a few examples where one beats the other.

Why command line ?

Suppose you wanted to rename 3 files by including their date of creation. You can easily do that in GUI. What if you had to do the same for 100 files?. Would you be still happy to do it via GUI, selecting each file, right-click, rename and type?.

Commandline tools give us the option to do in single-line instructions. Both command-line and GUI have their pros and cons. It is up to us to identify the ideal tool for the scenario and use it.

The command-line tool in Linux is often called a terminal. When it comes to the professional skill set of any programmer/machine learning/artificial intelligence enthusiast, he/she needs to have at least some familiarity with command-line usage. In this blog post, we will see the minimum required basics that will help someone in this technology space.

Some examples of terminals include Python shell, command prompt in Windows, Console in a browser where you can execute Javascript, Database terminals. So we are already familiar with the terminals in one way or other. A historical look at the terminal usage shows some interesting aspects.

Although these days terminal is a piece of software it originally meant a physical piece of hardware for interfacing with the computer. The computing device and the hardware interface were often different. The computing part was too big to fit in a table. So the immediate interaction of a user was with the physical hardware like a typewriter which is used to instruct the computer.

A similar theme we might see these days could be the fact that most of the AI/ML engines run on the cloud and our laptop can be thought of as an interface to the cloud. Another situation is where you have to leverage a high-performance computing or parallel computing facility, you might be remotely instructing the supercomputing facility from your laptop.

When it comes to the Linux terminal we have a piece of software via which we can give commands to the operating system and get the tasks done. Some key plus points of command-line interface compared to GUI is that are:

• Programmable
• Efficient in terms of Bandwidth
• You can work remotely

The second and third points are obvious when we think about it in terms of using a cloud service or supercomputing facility.

Now let us get started with the actual terminal usage practice

Linux Terminal

Launching Terminal in Ubuntu

We can launch the terminal in Ubuntu by pressing Control+ALT+T. There is also the GUI way of doing this. But the whole point of learning terminal is to realize its use-cases and power. So we will stick to the keyboard shortcut way.

Just to keep in mind that Unix and Linux programmers over the years have written many different shell programs like Bourne shell, C shell, Korn shell, Bash shell, etc. The one we will be using is the Bash shell.

Upon opening the terminal we will get a window with a prompt. The general form of the prompt will be like ‘user@host$’. This means the user has been logged into the host computer and the computer is at its home directory ready to receive instructions.

Basic Commands and its Usage

Zoom In and Out: Press Control and + to make fonts larger, Control and – to make fonts smaller

Date: The command date gives the present date and time in the system.

sajil@sajil-KY652AA-ACJ-CQ3050IL:~$ date
Wed Nov 25 17:02:04 IST 2015

Mathematical Expression: We can evaluate mathematical expressions in the terminal, just like a calculator.

sajil@sajil-KY652AA-ACJ-CQ3050IL:~$ expr 10 + 21 
31 

Printing Message: We can display custom messages as outputs by using the echo command. The command ‘echo’ can be used to output a message to the terminal. This feature is especially useful when writing bash script files (a file containing a set of bash commands to accomplish a relatively complex task).

sajil@sajil-KY652AA-ACJ-CQ3050IL:~$ echo Namaste 
Namaste 

user@host~$ echo Have a good day
Have a good day

Clearing Terminal: At any point, we can clear the terminal screen using the clear command.

sajil@sajil-KY652AA-ACJ-CQ3050IL:~$ clear

Present Working Directory: To know the present working location we can use the pwd command.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ pwd
/home/sajil

# ~ denotes home directory
$ cd ~

# . denote current directory
$ cd .

Navigating File Structures and Folders

Listing Files and Directories: To list out all files and folders in a directory, the ls command is used.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ ls
11.pdf
Activity files
Adder.ipynb
Desktop
Dist3D

Here we can see it lists in addition to the PDF and Ipython files the folders also.

List with details: list out files with additional details like file size,users, modified date,time, permissions, etc.

# General format is
$ ls -<flag> <filename/directory>

# List files with details
$ ls -l

# 1st column: permission
# 2nd column: type (1 for file, 2 for directory)
# 3rd column: owner
# 4th column: group
# 5th column: size
# 6-8th column: modified time
#9 th column: name

# To list all files including hidden files
$ ls -a

# Show folders with a slash and executables with a star
$ ls -F 

# sort by size (big to small)
$ ls -S

sort by time (from new to old)
$ ls -t 

# Combine arguements
$ ls -la
# -h : human readable
# -r : reverse order

This could be the most frequent usage you ever going to encounter. To move inside a directory you can type.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ cd documents

To move one step out/parent folder type cd<space>.. To step back to the root folder type cd<space>/

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ cd ..
sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ cd /

Additional change directory commands:

# To home folder
$ cd ~ 

# To last used directory
$ cd - 

Directory and File Management

# To create a directory
$ mkdir yourfoldername

# To remove/delete a file
$ rm filename

# Remove directory recursively
# rm -<flag> <directory path>
# -r: recursively (Remove directory and their contents)
# -i: interactively (Ask to confirm if it before removing a file)
# -v: verbose (Show the removing progress)
# -f: force (Remove without asking)


$ rm -r foldername

# Copy a file from source to destination
# cp -<flag> <source> <destination>
# -r: recursively
# -i: interactively
# -v: verbose

$ cp source destination

# Copy whole folder
$ cp -r sourcedirectory destinationdirectory

# To cut and paste to another location
$ mv source destination

# To create shortcut
$ ln -s file shortcutname

To denote space between file or folder names: The character ‘\<space>’ is used to denote actual space between names of files or folders.

$ mkdir commandline\ workshop

# To create an empty file
$ touch new.txt

# Logging command line outputs to files
$ echo hello>new.txt

# Display/concatenate files and print on the standard output
$ cat filename 

# Disply contents of file pagewise/controlled (press q to exit, up/down keys to scroll)
$ less filename
$ more filename

# Display first k lines (10 lines by default) of a file
$ head filename 
$ head -2 filename

# Display last k lines(10 lines by default) of a file
$ tail filename
$ tail -2 filename

# Command line editor
$ nano filename

# Sort lines in a file

$  sort -<flag> <filename>

# Flag options
# -k col1,col2 : Specify a key to do the sorting, col1 and col2 are used to indicate the starting filed indices and ending field indices 
# -t: specify the delimiter
# -n: sort based on numerical value (default: alphabetically)
# -r: reverse order
# -u: only unique lines
# -b: Ignore blanks at the start of the line
# -o: specify the output file (default is the standard output)

Auto-Completion

Another key feature that is very useful to the terminal is its auto-completion feature. Suppose you want to navigate inside into a set of folders. You don’t have to type the name of all the files and folders. You can simply type the starting 2-3 letters and press Tab. If there is only one matching file or folder starting with that name the terminal completes the typing for you. Otherwise, it lists out all the matches, with that information you can add one more letter to the starting and press Tab to complete.

Accessing previous commands: Also the up and down arrows on the keyboard lets you recall all the past commands used so that if you want to use it again with minor modifications, just select from the past and use it with the auto-completion feature.

Moving Files: You can move files from one location to another using the ‘mv’ command. The usage is simply mv<space>Source<space>Destination.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ mv *.txt ../

Here I am moving all text files from the present directory to the home directory. The ‘*’ here represents any character. This way the source here represents all files with ‘.txt’ as an extension. The destination here is ‘../’ which means step back to the home location where the terminal opens up by default and move all these files to there.

History: If you want to get a history of all commands used so far, you can use the history command.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ history

File Permissions

The change mode command is used to alter the file permission settings. The general format is chmod ugo filename (u:user, g:group, o:others). values of u,g,and o goes between 0 to 7.

$ chmod [user/group/others/all] [+-] [permission] <file/directory name>

# r : read, w : write, x : executable

# You can read and write
$ chmod 600 filename 

# You can read, write, and execute (e.g. scripts)
$ chmod 700 filename 

# You can read, write, and execute, and everyone else can read and execute (e.g. programs to share)
$ chmod 755 filename 

# Alternate was to set all permissions (read, write, execute) to all
$ chmod a+x 777 filename

RegEx Use

You can write regular expression patterns to select from files

# Concatenates files whose name starts with chr
$ cat chr* 

# List details of only PDF files
$ ls *.pdf

#  List files that contains chr in filename
$ ls *chr* 

System Commands

Run as Admin: In situations where you want to run code with admin privileges, you can prepend with the keyword ‘sudo’. The terminal will ask for a password before executing it.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ sudo snap install vlc

Command Documentation: To check the Manuel/documentation for any Bash command prepend with the keyword ‘man’ with that command.

sajil@sajil-KY652AA-ACJ- CQ3050IL:~$ man ls

# To show current date and time
$ date

# To show current user logged in 
$ whoami 

# Show disk memory usage
$ df 

# Show directory space usage
$ du 

Listing out the structure of files and folders: We know that we can organize files inside folders or subfolders according to our convenience. The command ‘tree’ lets us visualize the hierarchical structure in a visual form.

sajil@sajil-KY652AA-ACJ-CQ3050IL:~/a$ tree
├ ── 1
└── 2
├ ── 3
├ ── 4
└── 5
└── 6
6 directories, 0 files

Creating and deleting folders: Folders/directories can be created or deleted using ‘mkdir’ and ‘rmdir’ commands. The other most frequent commands are changing directory and exiting from a directory.

# Creating Directories
sajil@sajil-KY652AA-ACJ-CQ3050IL:~/a$ mkdir directoryname

# Changing Directories
sajil@sajil-KY652AA-ACJ-CQ3050IL:~/a$ cd directoryname

# Removing Directories
sajil@sajil-KY652AA-ACJ-CQ3050IL:~/a$ rmdir directoryname

# Exiting from current Directory
sajil@sajil-KY652AA-ACJ-CQ3050IL:~/a$ cd ..

Logging Output

Saving Output of a Command to Text File: We can log the output messages given by a command to a text file using a ‘>’ character. The usage is as follows.

# > : output to the file (if file exists, it will overwrite it)
# >>: append the output to the end of a file (if file doesn’t exist, it will create the file)

$ ls >outputlog.txt

Here the command ‘ls’ will list out all files and folders in the current directory which is written into a text file named ‘output.txt’.

Alias: You can give alternate names to specific commands

# $alias<space><customname>="<actualcommand>"
$ alias listout="ls"

Process Management

# Display system usages and running programs (press q to exit)
$ top

# Display currently active process
$ ps 

# Stop a particular process with process id
$ kill pid

Networking

# Check Web Connection
$ ping google.com

# Download file from URL
$ wget url 

$ curl url

# Remote log in
$ ssh user@host 

# Remote log in with specific port
$ ssh -p port user@host

# Download and Upload SCP

# To upload file to remote server
$ scp <local file> username@hoffman2.idre.ucla.edu:<path>

# To download file from remote server to local computer
$ scp username@hoffman2.idre.ucla.edu:<path> .

# Exit or logout
$ exit

$ logout

File Compression

# Compressing files
$ gzip file.txt

# Uncompressing files
$ gzip -d file.txt.gz
$ unzip text.zip

# Ccreate a tar named file.tar containing files
$ tar cf outputfile_or_folder.tar filename_or_foldername

# Extract tar files
$ tar xf file.tar 

# Create tar file with Gzip
$ tar czf zipfile.tar.gz inputfile

# Extract tar file with Gzip
$ tar xzf file.tar.gz 

Searching Files

# Listing specific type of files
$ ls *.pdf

# Finding files
$ find . -name “*.txt“

Pipeline

Combine multiple commands together
$ command1 | command2 | command3
# The command1’s output will be redirected to command2’s input, and so on

# Color Output
$ echo Hello|grep --color el

# Getting Arguements
$ echo a b c d e f |xargs -n 3

Bash Scripting

Creating Bash Script: We can combine many of these commands and create a bash script. This can be done by writing the commands in a file with the extension ‘sh’ and changing its administrative permission as executable. To run this file we will call its name prepending a ‘./’ characters.

$ gedit new.sh
    echo first command
    echo second command
    echo third command

$ chmod a+x new.sh 
# or chmod 755 new.sh

$./new.sh >outputlog.txt

Here the first commands invoke the inbuilt text editor in Ubuntu to create a text file with ‘sh’ extension. In the next step, a bunch of simple echo print commands is written onto that file. The ‘chmod’ command and its permission setting make the file executable. The last step runs the executable file and logs its terminal outputs to another text file.

Exiting From Terminal: The final command to exit from the terminal can be done by simply typing ‘exit’.

$ exit

Concluding Thoughts

In this post, we have seen the most frequent commands which a developer or programmer might encounter while working with terminal commands. A combination of two or more of such commands might make many tasks easier which otherwise might be harder to do in a GUI. The usage of such commands creatively and practice of it is crucial to be productive in your workspace. I hope these examples will get start you to learn about the first steps, start realizing its power and save you a lot of your time from mundane tasks.