Bash Basics Exercises

Bash Basics Exercises#

These exercise should help you either reviewing or learning new shell commands. The exercises can be done localy on Linux, Mac or Windows 11 bash console. You can also open this Jupyter Notebook directly in Binder or google collab and do the exercises. In this case be aware that the system runs on Linux, which is important when using paths.

Linux and Mac use / instead of \ as an example.

But first, we go through a quick explanation on how to use BASH in a Jupyter Notebook on a Linux or Mac system.

Using Bash in Jupyter Notebook#

Depending on what system you use it is important to first figure out the current path on the file system, for that run the next cell.

%%bash
pwd

To run a single command you may add an exclamation mark to the front of the command. As seen as in the example below ! is added to the command echo. After that just run the cell and it should give the output, in this case “hello world”.

!echo "hello world!"

hello world!

However, that won’t work if for example you want to run multiple commands that talk to each other, since each command is executed in a separate process:

!export RAND=42
!echo $RAND

Nothing is printed, since RAND was defined in a separate process which has already exited.

But, if your notebook code cell starts with %%bash, it’s all executed as one script!

%%bash

export RAND=42
echo $RAND

42

You can also pass Python variables into your Shell commands, and vise versa!

# You can also pass python variables into your shell commands, and vise versa!
python_var = []

output_of_ls_as_python_var = !ls -a
print(output_of_ls_as_python_var)

['.', '..', 'bash_advanced.ipynb', 'bash_basics.ipynb', 'bash_tools.ipynb', 'config.ipynb', 'error_handling.ipynb', 'git_advanced.ipynb', 'intro_make.ipynb', 'packaging.ipynb', 'provenance.ipynb', 'python_cli.ipynb', 'python_refresher.ipynb', 'snakemake.md', 'solutions', 'testing.ipynb', 'version_control.ipynb', 'work_teams.ipynb']

Basic Exercises#

The exercises below involve creating and moving new files, as well as considering hypothetical files. Please note that if you create or move any files or directories in your Zipf’s Law project, you may want to reorganize your files following the outline at the beginning of the next chapter. If you accidentally delete necessary files, you can start with a fresh copy of the data files by following the instructions in Section Getting Started.

1 Exploring more ls flags#

What does the command ls do when used with the -l option?

What happens if you use two options at the same time, such as ls -l -h?

2 Listing recursively and by time#

The command ls -R lists the contents of directories recursively, which means the subdirectories, sub-subdirectories, and so on at each level are listed. The command ls -t lists things by time of last change, with most recently changed files or directories first.

In what order does ls -R -t display things? Hint: ls -l uses a long listing format to view timestamps.

3 Absolute and relative paths#

Starting from your current directory, which of the following commands could you use to navigate to your home directory, which is /Users/*YOUR-USERNAME or on Linux and Mac /home/*YOUR-USERNAME* ?

  1. cd .

  2. cd /

  3. cd /home/amira

  4. cd ../..

  5. cd ~

  6. cd home

  7. cd ~/data/..

  8. cd

  9. cd ..

  10. cd ../.

4 Relative path resolution#

Using the filesystem shown below, if pwd displays /Users/sami, what will ls -F ../backup display?

  1. ../backup: No such file or directory

  2. final original revised

  3. final/ original/ revised/

  4. data/ analysis/ doc/

Filesystem/exercise

5 ls reading comprehension#

Using the filesystem shown above, if pwd displays /Users/backup, and -r tells ls to display things in reverse order, what command(s) will result in the following output:

doc/ data/ analysis/

  1. ls pwd

  2. ls -r -F

  3. ls -r -F /Users/backup

6 Creating files a different way#

What happens when you execute touch my_file.txt? (Hint: use ls -l to find information about the file)

When might you want to create a file this way?

7 Using rm safely#

What would happen if you executed rm -i my_file.txt on the file created in the previous exercise?

Why would we want this protection when using rm?

8 Moving to the current folder#

After running the following commands, Amira realizes that she put the (hypothetical) files chapter1.txt and chapter2.txt into the wrong folder:

$ ls -F

  data/  docs/

$ ls -F data

README.md			frankenstein.txt		sherlock_holmes.txt
chapter1.txt		jane_eyre.txt			time_machine.txt
chapter2.txt		moby_dick.txt
dracula.txt			sense_and_sensibility.txt

$ cd docs

Fill in the blanks to move these files to the current folder (i.e., the one she is currently in):

$ mv ___/chapter1.txt  ___/chapter2.txt ___

9 Renaming files#

Suppose that you created a plain-text file in your current directory to contain a list of the statistical tests you will need to do to analyze your data, and named it: statstics.txt

After creating and saving this file you realize you misspelled the filename! You want to correct the mistake, which of the following commands could you use to do so?

  1. cp statstics.txt statistics.txt

  2. mv statstics.txt statistics.txt

  3. mv statstics.txt .

  4. cp statstics.txt .

10 Moving and copying#

Assuming the following hypothetical files, what is the output of the closing ls command in the sequence shown below?

$ pwd

/Users/amira/data

$ ls

books.dat

$ mkdir doc
$ mv books.dat doc/
$ cp doc/books.dat ../books-saved.dat
$ ls

  1. books-saved.dat doc

  2. doc

  3. books.dat doc

  4. books-saved.dat

11 Copy with multiple filenames#

This exercise explores how cp responds when attempting to copy multiple things.

What does cp do when given several filenames followed by a directory name?

$ mkdir backup
$ cp dracula.txt frankenstein.txt backup/

What does cp do when given three or more filenames?

$ cp dracula.txt frankenstein.txt jane_eyre.txt

12 List filenames matching a pattern#

When run in the data directory of your project directory, which ls command(s) will produce this output?

jane_eyre.txt   sense_and_sensibility.txt

  1. ls ??n*.txt

  2. ls *e_*.txt

  3. ls *n*.txt

  4. ls *n?e*.txt

13 Organizing directories and files#

Amira is working on a project and she sees that her files aren’t very well organized:

$ ls -F

books.txt    data/    results/   titles.txt

The books.txt and titles.txt files contain output from her data analysis.

What command(s) does she need to run to produce the output shown?

$ ls -F

data/   results/

$ ls results

books.txt    titles.txt

14 Reproduce a directory structure#

You’re starting a new analysis, and would like to duplicate the directory structure from your previous experiment so you can add new data.

Assume that the previous experiment is in a folder called 2023-05-18, which contains a data folder that in turn contains folders named raw and processed that contain data files. The goal is to copy the folder structure of 2023-05-18/data into a folder called 2023-05-20 so that your final directory structure looks like this:

	2024-05-20/
	└── data
	    ├── processed
	    └── raw

Which of the following commands would achieve this objective?

What would the other commands do?

# Set 1
$ mkdir 2023-05-20
$ mkdir 2023-05-20/data
$ mkdir 2023-05-20/data/processed
$ mkdir 2023-05-20/data/raw

# Set 2
$ mkdir 2023-05-20
$ cd 2023-05-20
$ mkdir data
$ cd data
$ mkdir raw processed

# Set 3
$ mkdir 2023-05-20/data/raw
$ mkdir 2023-05-20/data/processed

# Set 4
$ mkdir 2023-05-20
$ cd 2023-05-20
$ mkdir data
$ mkdir raw processed

15 Wildcard expressions#

Wildcard expressions can be very complex, but you can sometimes write them in ways that only use simple syntax, at the expense of being a bit more verbose. In your data/ directory, the wildcard expression [st]*.txt matches all files beginning with s or t and ending with .txt.

Imagine you forgot about this.

  1. Can you match the same set of files with basic wildcard expressions that do not use the [] syntax? Hint: You may need more than one expression.

  2. Under what circumstances would your new expression produce an error message where the original one would not?

16 Removing unneeded files#

Suppose you want to delete your processed data files, and only keep your raw files and processing script to save storage. The raw files end in .txt and the processed files end in .csv.

Which of the following would remove all the processed data files, and only the processed data files?

  1. rm ?.csv

  2. rm *.csv

  3. rm * .csv

  4. rm *.*

17 Other wildcards#

The shell provides several wildcards beyond the widely used *.

To explore them, explain in plain language what (hypothetical) files the expression novel-????-[ab]*.{txt,pdf} matches and why.

18 Finding files with different properties#

The find command can be given criteria called “tests” to locate files with specific attributes, such as creation time, size, or ownership.

Use man find to explore these, then write a single command using -type, -mtime, and -user to find all files in or below your Desktop directory that are owned by you and were modified in the last 24 hours.

Explain why the value for -mtime needs to be negative.

Contents