# Command Line and the Unix Shell¶

learning-objectives

• Know the command line’s utility
• Use basic navigation commands to interact with the file system
• Recognize the similarities between shell commands and iRODS
• Describe resources to continue learning command line applications

## Why should you care about the shell?¶

At a high level, computers do four things:

1. run programs
2. store data
3. communicate with each other
4. interact with users

The graphical user interface (GUI)

The GUI is the most widely used way to interact with computers

• Often the first way people learn to interact with a computer
• Give instructions with mouse (ex. run a program, create a folder/directory, move files, etc.)
• Difficult to scale since it requires human interaction

The shell - a command-line interface (CLI).

Less common way to interact with personal computers. Enables remote access to servers and analyses.

• Commands are reproducible

Example

If we have to copy the third line of each of a thousand text files stored in thousand different folders/directories and paste it into a single file line by line.

• Using the traditional GUI approach will take several hours to do this.
• Using the shell this will only take a couple of minutes (at most).

The Shell is a program which runs other programs rather than doing calculations itself.

• programs can be as complicated as a climate modeling software
• as simple as a program that creates a new folder/directory
• simple programs used to perform stand alone tasks are usually refered to as commands.
• most popular Unix shell is Bash, (the Bourne Again SHell).
• Bash is the default shell on most modern implementations of Unix

## Setting up Our Shell Environment:¶

Launch JupyterLab VICE Application:

Search for the application Jupyter Lab SciPy Notebook Latest here:

Setup Outputs:

Memory Allocation:

Launch Application on DE2:

Launch the Shell in JupyterLab:

Command Prompt:

|Jupyter Shell Prompt|

Optional: Change to JupyterLab Dark Theme

|Dark Theme|

Question

In small, groups help each other get the shell prompt open inside JupyterLab.

If you all have JupyterLab’s shell launched, please discuss the following question.

1. How does learning aspects of the shell help us create FAIR data?
2. Do you think we can use the shell to do data tasks on the CyVerse Data Store?

## Introduction to Shell Commands¶

Print working directory (pwd)

Directories are equivalent to folders on a desktop computer, but are like an address for the computer’s file system.

To find “where” you are we can use the command pwd:

$pwd /home/jovyan/work  List files and directories (ls) We can see the contents of our current directory by running ls. $ ls


Your results may be slightly different depending on your operating system and any system modifications.

With the option, or flag, -F we can differentiate between files and folders. All folders will appear in the shell prompt with a / appended to their names.

$ls -F data-shell/ downloads.stderr.log downloads.stdout.log  Changing Directories After running pwd we know we’re in the directory /home/joyvan/work. How can we change our location to the data-shell directory? The cd or change directory command. $ cd data-shell


Making Directories

To make directories in the command line we can use the command mkdir

$mkdir thesis$ ls -F
creatures/  data/  molecules/  north-pacific-gyre/  notes.txt  pizza.cfg
solar.pdf  thesis/  writing/


Now let’s change into our directory thesis/

$cd thesis$ ls -F


When we run ls -F we don’t see any output in the terminal.

So let’s make some files in this directory.

Creating Files in the Shell

Good Names for Files and Directories

1. Don’t use spaces.

Spaces are used to separate arguments on the command line it is better to avoid them in names of files and directories. You can use - or _ instead (e.g. north-pacific-gyre/ rather than north pacific gyre/).

2. Don’t begin the name with - (dash).

Commands treat names starting with - as options or flags.

3. Stick with letters, numbers, . (period or ‘full stop’), - (dash) and _ (underscore).

If you need to refer to names of files or directories that already have spaces or other special characters, you should surround the name in quotes (“”).

$nano draft.txt  Let’s type in a few lines of text. Once we’re happy with our text, we can press Ctrl+O (press the Ctrl or Control key and, while holding it down, press the O key) to write our data to disk (we’ll be asked what file we want to save this to: press Return to accept the suggested default of draft.txt). Once our file is saved, we can use Ctrl-X to quit the editor and return to the shell. In nano, along the bottom of the screen you’ll see ^G Get Help ^O WriteOut. This means that you can use Control-G to get help and Control-O to save your file. nano doesn’t leave any output on the screen after it exits, but ls now shows that we have created a file called draft.txt: $ ls
draft.txt


We can also use the touch command to create files.

$touch my_file.txt  How large is my_file.txt?? What command can we use to check this? Getting help with commands We can pass a –help option to the command, such as: $ ls --help


The man command

Note

The man command will only work in the command line with certain systems.

The GNU website manual will be useful if this is not available (like in our Docker container).

$man ls  This will open the manual in your terminal with a description of the ls command and its options and, if you’re lucky, some examples of how to use it. To navigate through the man pages, you may use and to move line-by-line, or try B and Spacebar to skip up and down by a full page. To quit the man pages, press q. Manual pages on the web GNU provides links to its manuals including the core GNU utilities , which covers many commands introduced within this lesson. ## Accessing the Data Store via the Shell¶ We can access the data store with a command line toolset called iRODS. iRODS allows us to move files from the Discovery Environment container to other directories in the Data Store. Configuring iRODS First let’s configure iRODS with the command iinit $ iinit

Enter the host name (DNS) of the server to connect to: data.cyverse.org
Enter the port number: 1247


Working Directory on the Data Store

$ipwd /iplant/home/<Your Cyverse Username>  Changing Directories in the Data Store $ icd /iplant/home/shared/foss/spring-2021


Optional: Create a folder in the data store

This can be done via the web GUI, like we did earlier. Or you can use the command line and iRODS to make a folder with the same name as your CyVerse username in the path /iplant/home/shared/foss/spring-2021. For example after using icd above, I would write the following:

$ipwd /iplant/home/shared/foss/spring-2021$ imkdir rbartelme
\$ ils
/iplant/home/shared/foss/spring-2021:
C- /iplant/home/shared/foss/spring-2021/data-shell
C- /iplant/home/shared/foss/spring-2021/rbartelme


FAIR Data Challenge

In small groups determine how you can use irsync with the flags -rKva to transfer the folder molecules from the data-shell directory to the data store folder you created in the foss directory in the Data Store.

Hint

• You need to add i: to the beginning of the file/folder argument to tell iRODS

the location of items in the data store. - irsync behaves differently with / after the folder name than without the slash.

## Further Practice¶

Note

This is was just a brief summary of how to use the command line. There is much, much more you can do.

For more information check out the Software Caprentry Shell Lesson<https://swcarpentry.github.io/shell-novice/>_ page. This lesson uses the folder data-shell which is already in your Jupyter Lab instance. Work through these lessons on your own or in groups as homework.