Environment Setup

Prepare your system before the course begins

Part 1Prerequisites & Access Part 2Connecting & Modules Part 3File Transfer Part 4IGV Part 5Conda Environment Part 6Quick Reference Part 7Linux Cheatsheet

Prerequisites

Before You Begin — Check Your Ibex Access

Complete both checks below before the course starts. If either fails, follow the instructions to resolve it in advance.

1. Verify you can submit jobs

Log in to Ibex and run a quick test job. If it queues and completes, your account has compute access.

ssh your_username@ilogin.ibex.kaust.edu.sa

# Submit a short test job
sbatch --wrap="hostname" --time=00:01:00 --mem=1G

# Check the output (replace JOBID with the number printed above)
cat slurm-JOBID.out
No compute access?

If you receive a permission error or your jobs are rejected, you need to request access before the course. Submit a request through the Ibex HPC portal. Ask your PI to approve the request.

2. Check available disk space

The course dataset and conda environment require at least 200 GB of free space in your Ibex home directory.

# Check your current usage and quota on /ibex/user
quota -s /ibex/user/$USER

# Or check with df
df -h /ibex/user/$USER
Less than 200 GB free?

Free up space by removing unused files, old conda environments, or temporary outputs before the course:

# List your largest directories
du -sh /ibex/user/$USER/* | sort -rh | head -20

Connecting to Ibex

ssh your_username@ilogin.ibex.kaust.edu.sa

Bioinformatics Tools — Use the Module System

All bioinformatics tools used in the labs (FastQC, fastp, STAR, Salmon, samtools, MultiQC, Nextflow, etc.) are pre-installed on Ibex. Load them on demand — no local installation needed.

# Search for a tool
module avail toolName

# Load a tool
module load toolName

# Example
module avail salmon
module load salmon

Working with Files Between Your Computer and Ibex

You will regularly need to transfer files to and from Ibex — uploading scripts, downloading results, and opening output files locally. Choose the method that suits your operating system.

Command Line: SCP and Rsync

Works on macOS, Linux, and Windows (WSL2 or Git Bash). Run these commands on your local machine.

Replace username with your Ibex username in all commands below.

Copy a file to Ibex

# Upload a single file
scp local_file.txt username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/

# Upload an entire folder
scp -r local_folder/ username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/

Download a file from Ibex

# Download a single file
scp username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/pca_plot.pdf ./

# Download an entire results folder
scp -r username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/ ./

Sync a folder with rsync (recommended for large transfers)

Rsync only transfers new or changed files, making it much faster than scp for repeated syncs.

# Sync local folder → Ibex (upload)
rsync -avz --progress local_folder/ \
    username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/

# Sync Ibex results → local folder (download)
rsync -avz --progress \
    username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/ \
    ./results/

macOS: Browse Ibex Files in Finder

macOS Finder can connect to Ibex over SMB with no extra software. Your Ibex directory mounts as a network drive and you can open files directly in any application.

Open "Connect to Server"

In Finder, go to Go → Connect to Server… (or press ⌘ Cmd + K).

Enter the server address

Type the following in the address bar and click Connect:

smb://samba.ibex.kaust.edu.sa/weka_user/umerhm
Save it for next time

Click the + button next to the address bar to add this server to your Favourite Servers list. You can then connect with one click in future.

Authenticate and browse

Enter your Ibex username and password when prompted. Ibex will mount as a drive in Finder's sidebar under Locations. Navigate to /ibex/user/username/workshop to access your files. You can open, copy, and drag files between Finder and your local Desktop just like any other folder.

Disconnect when done

Right-click the mounted drive in the Finder sidebar and choose Eject, or press ⌘ Cmd + E while the drive is selected.

Windows: MobaXterm

MobaXterm is a free terminal for Windows that includes a built-in SSH client, SFTP file browser, and X11 display — all in one application.

Install MobaXterm

  1. Download the free Home Edition from mobaxterm.mobatek.net
  2. Run the installer (or use the portable version — no installation required)

Connect to Ibex

  1. Open MobaXterm and click SessionSSH
  2. Enter ilogin.ibex.kaust.edu.sa as the remote host
  3. Enter your username and click OK
  4. MobaXterm automatically opens an SFTP panel on the left side of the terminal window showing your Ibex files

Transfer files with the SFTP panel

  • Upload: drag files from Windows Explorer into the SFTP panel, or use the upload button (arrow pointing up)
  • Download: right-click a file in the SFTP panel and choose Download, or drag it to your desktop
  • Navigate: click folders in the SFTP panel just like Windows Explorer
  • Edit a file: right-click → Open with to open a remote file in any local application; MobaXterm automatically re-uploads it when you save
Save your session

In MobaXterm, right-click your connection in the Sessions sidebar and choose Save session so you don't need to re-enter the host and username each time.

IGV — Install on Your Local Computer

The Integrative Genomics Viewer (IGV) is a desktop application for visualizing BAM files and genomic data. Install it on your laptop before Lab 4.

Download and install

  1. Go to igv.org/doc/desktop
  2. Download the installer for your operating system:
    • macOS: download the .app zip, unzip, and drag IGV to your Applications folder
    • Windows: download the .zip, unzip, and run igv.bat
    • Linux: download the .zip, unzip, and run igv.sh
  3. Launch IGV and verify it opens. On first launch it will download a small genome index — an internet connection is required.
Java requirement

The IGV installer bundles its own Java runtime — you do not need to install Java separately. If you use the "command line" version instead, Java 11+ is required.

Conda Environment

A single rnaseq conda environment provides JupyterLab, all Python packages used in the notebooks, and all R/Bioconductor packages used in the R labs. Install it on Ibex for running analyses on the cluster, and optionally on your laptop for the R/Python visualization labs that do not require HPC resources.

Step 1 — Install Miniforge (skip if conda/mamba is already installed)

Miniforge is a minimal conda installer that uses the community-maintained conda-forge channel by default and includes mamba (a fast dependency solver) out of the box.

Linux & Ibex (x86_64)

# Download the Miniforge installer for Linux (64-bit)
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
    -O ~/miniforge3_install.sh

# Run the installer in batch mode (no interactive prompts)
bash ~/miniforge3_install.sh -b -p ~/miniforge3

# Activate conda for your current shell session
source ~/miniforge3/bin/activate

# Initialise conda so it loads automatically in future sessions
conda init bash

# Reload your shell config
source ~/.bashrc

# Verify (mamba is included with Miniforge)
mamba --version

macOS

Apple Silicon (M1/M2/M3) or Intel?

Run uname -m in your terminal. Output arm64 = Apple Silicon; x86_64 = Intel. Use the matching installer below.

# Apple Silicon (M1/M2/M3)
curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh \
    -o ~/miniforge3_install.sh

# Intel Mac — use this line instead:
# curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-x86_64.sh \
#     -o ~/miniforge3_install.sh

# Run the installer
bash ~/miniforge3_install.sh -b -p ~/miniforge3

# Activate conda for your current shell session
source ~/miniforge3/bin/activate

# Initialise conda — macOS defaults to zsh; add bash if you also use it
conda init zsh

# Reload your shell config
source ~/.zshrc

# Verify
mamba --version

Windows

Recommended: use WSL2 (Windows Subsystem for Linux)

WSL2 gives you a full Ubuntu environment on Windows and is the most compatible option for bioinformatics work. Once inside WSL2, follow the Linux & Ibex instructions above. If you prefer a native Windows install, download the .exe GUI installer from github.com/conda-forge/miniforge/releases (Miniforge3-Windows-x86_64.exe) and run it — then use Miniforge Prompt or Anaconda Prompt for the steps below.

Step 2 — Create the rnaseq environment

This command is the same on all platforms (Linux, macOS, Windows WSL2). It installs JupyterLab, Python analysis packages, and R/Bioconductor packages for all course labs. Expect it to take 10–20 minutes on first install.

mamba create -n rnaseq -c conda-forge -c bioconda \
    python=3.11 \
    jupyterlab \
    ipykernel \
    pandas \
    numpy \
    matplotlib \
    seaborn \
    scikit-learn \
    scipy \
    pydeseq2 \
    adjusttext \
    r-base \
    bioconductor-deseq2 \
    bioconductor-edger \
    bioconductor-tximport \
    bioconductor-tximeta \
    bioconductor-genomicfeatures \
    bioconductor-clusterprofiler \
    bioconductor-org.hs.eg.db \
    bioconductor-fgsea \
    bioconductor-complexheatmap \
    bioconductor-enhancedvolcano \
    bioconductor-apeglm \
    r-pheatmap \
    r-ggplot2 \
    r-dplyr \
    r-tidyr \
    r-ggrepel \
    -y
Package overview
Package Used for
jupyterlab / ipykernelInteractive Python notebooks (Labs 5, 7)
pandas / numpyLoading quant.sf files, matrix operations
matplotlib / seabornPCA plots, heatmaps, volcano plots
scikit-learn / scipyPCA decomposition, pairwise distances
pydeseq2Differential expression in Python
adjustTextNon-overlapping labels in volcano plots
r-baseR interpreter
bioconductor-deseq2 / bioconductor-edgerDifferential expression analysis (Lab 7)
bioconductor-tximport / bioconductor-tximetaImport Salmon output into R (Labs 5, 7)
bioconductor-genomicfeaturesBuild tx2gene mapping from GTF
bioconductor-clusterprofiler / bioconductor-fgseaGO and pathway enrichment, GSEA (Lab 8)
bioconductor-complexheatmap / r-pheatmapHeatmap visualisation
bioconductor-enhancedvolcanoVolcano plots (Lab 7)
bioconductor-apeglmLog fold-change shrinkage (Lab 7)
r-ggplot2 / r-dplyr / r-tidyr / r-ggrepelGeneral-purpose R plotting and data wrangling

Step 3 — Activate and verify

Activation syntax is identical on all platforms.

conda activate rnaseq

# Verify Python packages
python -c "import pandas, numpy, matplotlib, seaborn, sklearn, pydeseq2; print('Python packages OK')"

# Verify Jupyter
jupyter lab --version

# Verify R packages
Rscript -e "library(DESeq2); library(tximport); library(clusterProfiler); cat('R packages OK\n')"
Running JupyterLab
  • On Ibex (recommended): use the Ibex Open OnDemand portal — go to Interactive Apps → JupyterLab, request resources, and click Connect. No terminal setup needed.
  • On your laptop: activate the environment and run jupyter lab — it opens automatically in your browser.

Quick Reference: Daily Commands

# Connect to Ibex
ssh username@ilogin.ibex.kaust.edu.sa

# Start interactive session
srun --pty --time=4:00:00 --mem=8G --cpus-per-task=4 bash

# Navigate to workspace
cd /ibex/user/$USER/workshop

Linux Command Line Cheatsheet

Navigation

# Print current directory
pwd

# List files and directories
ls              # Basic listing
ls -l           # Detailed listing (permissions, size, date)
ls -la          # Include hidden files (starting with .)
ls -lh          # Human-readable file sizes

# Change directory
cd /path/to/dir     # Go to specific directory
cd ~                # Go to home directory
cd ..               # Go up one level
cd -                # Go to previous directory

File Operations

# Create files and directories
mkdir mydir             # Create directory
mkdir -p dir1/dir2      # Create nested directories
touch file.txt          # Create empty file

# Copy files
cp file.txt backup.txt          # Copy file
cp -r mydir/ mydir_backup/      # Copy directory recursively

# Move/rename files
mv oldname.txt newname.txt      # Rename file
mv file.txt /path/to/dest/      # Move file

# Remove files
rm file.txt             # Delete file
rm -r mydir/            # Delete directory recursively
rm -i file.txt          # Ask before deleting

Viewing Files

# Display file contents
cat file.txt            # Show entire file
head file.txt           # Show first 10 lines
head -n 20 file.txt     # Show first 20 lines
tail file.txt           # Show last 10 lines
tail -f logfile.txt     # Follow file updates in real-time
less file.txt           # View with scrolling (q to quit)

# View compressed files
gzip -dc file.gz         # View gzipped file (works on macOS and Linux)
zless file.gz           # View gzipped file with scrolling

File Information

# Count lines, words, characters
wc file.txt             # Lines, words, characters
wc -l file.txt          # Lines only

# File size
du -h file.txt          # Size of file
du -sh mydir/           # Total size of directory

# Disk space
df -h                   # Disk usage summary

Searching

# Find files
find . -name "*.fastq"          # Find by name pattern
find . -type f -size +100M      # Find files larger than 100MB

# Search file contents
grep "pattern" file.txt         # Lines containing pattern
grep -r "pattern" mydir/        # Search recursively in directory
grep -i "pattern" file.txt      # Case-insensitive search
grep -c "pattern" file.txt      # Count matching lines

Pipes and Redirection

# Redirect output
command > file.txt      # Write output to file (overwrite)
command >> file.txt     # Append output to file
command 2> error.txt    # Redirect errors to file

# Pipes (send output to another command)
cat file.txt | head             # First 10 lines
cat file.txt | grep "pattern"   # Filter lines
cat file.txt | wc -l            # Count lines
cat file.txt | sort | uniq      # Sort and remove duplicates

Compression

# Gzip
gzip file.txt           # Compress (creates file.txt.gz)
gunzip file.txt.gz      # Decompress

# Tar archives
tar -cvf archive.tar dir/       # Create archive
tar -xvf archive.tar            # Extract archive
tar -czvf archive.tar.gz dir/   # Create compressed archive
tar -xzvf archive.tar.gz        # Extract compressed archive

Permissions

# View permissions
ls -l file.txt          # Shows -rw-r--r-- format

# Change permissions
chmod +x script.sh      # Make executable
chmod 755 script.sh     # rwx for owner, rx for others

# Change ownership
chown user:group file.txt

Process Management

# View running processes
ps                      # Your processes
ps aux                  # All processes
top                     # Interactive process viewer (q to quit)
htop                    # Better interactive viewer (if installed)

# Control processes
command &               # Run in background
Ctrl+C                  # Stop current process
Ctrl+Z                  # Suspend current process
bg                      # Resume suspended process in background
kill PID                # Terminate process by ID
killall processname     # Terminate by name

Useful Shortcuts

# Keyboard shortcuts
Ctrl+C          # Cancel current command
Ctrl+L          # Clear screen
Ctrl+A          # Move cursor to start of line
Ctrl+E          # Move cursor to end of line
Ctrl+R          # Search command history
Tab             # Auto-complete file/directory names
Up/Down         # Navigate command history

# History
history                 # Show command history
!123                    # Run command #123 from history
!!                      # Repeat last command