Environment Setup
Prepare your system before the course begins
Prerequisites
- An active Ibex HPC account. Apply through Ibex HPC portal
- A laptop with a terminal application (macOS Terminal, Linux shell, or Windows with WSL2)
- Familiarity with the Linux command line
Before You Begin — Check Your Ibex Access
Complete both checks below before the course starts. If either fails, follow the instructions to resolve it in advance.
1. Verify you can submit jobs
Log in to Ibex and run a quick test job. If it queues and completes, your account has compute access.
ssh your_username@ilogin.ibex.kaust.edu.sa
# Submit a short test job
sbatch --wrap="hostname" --time=00:01:00 --mem=1G
# Check the output (replace JOBID with the number printed above)
cat slurm-JOBID.out
If you receive a permission error or your jobs are rejected, you need to request access before the course. Submit a request through the Ibex HPC portal. Ask your PI to approve the request.
2. Check available disk space
The course dataset and conda environment require at least 200 GB of free space in your Ibex home directory.
# Check your current usage and quota on /ibex/user
quota -s /ibex/user/$USER
# Or check with df
df -h /ibex/user/$USER
Free up space by removing unused files, old conda environments, or temporary outputs before the course:
# List your largest directories
du -sh /ibex/user/$USER/* | sort -rh | head -20Connecting to Ibex
ssh your_username@ilogin.ibex.kaust.edu.sa
Bioinformatics Tools — Use the Module System
All bioinformatics tools used in the labs (FastQC, fastp, STAR, Salmon, samtools, MultiQC, Nextflow, etc.) are pre-installed on Ibex. Load them on demand — no local installation needed.
# Search for a tool
module avail toolName
# Load a tool
module load toolName
# Example
module avail salmon
module load salmon
Working with Files Between Your Computer and Ibex
You will regularly need to transfer files to and from Ibex — uploading scripts, downloading results, and opening output files locally. Choose the method that suits your operating system.
Command Line: SCP and Rsync
Works on macOS, Linux, and Windows (WSL2 or Git Bash). Run these commands on your local machine.
username with your Ibex username in all commands below.
Copy a file to Ibex
# Upload a single file
scp local_file.txt username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/
# Upload an entire folder
scp -r local_folder/ username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/
Download a file from Ibex
# Download a single file
scp username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/pca_plot.pdf ./
# Download an entire results folder
scp -r username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/ ./
Sync a folder with rsync (recommended for large transfers)
Rsync only transfers new or changed files, making it much faster than scp for repeated syncs.
# Sync local folder → Ibex (upload)
rsync -avz --progress local_folder/ \
username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/
# Sync Ibex results → local folder (download)
rsync -avz --progress \
username@ilogin.ibex.kaust.edu.sa:/ibex/user/username/workshop/results/ \
./results/
macOS: Browse Ibex Files in Finder
macOS Finder can connect to Ibex over SMB with no extra software. Your Ibex directory mounts as a network drive and you can open files directly in any application.
Open "Connect to Server"
In Finder, go to Go → Connect to Server… (or press ⌘ Cmd + K).
Enter the server address
Type the following in the address bar and click Connect:
smb://samba.ibex.kaust.edu.sa/weka_user/umerhm
Click the + button next to the address bar to add this server to your Favourite Servers list. You can then connect with one click in future.
Authenticate and browse
Enter your Ibex username and password when prompted. Ibex will mount as a drive in Finder's sidebar under Locations. Navigate to /ibex/user/username/workshop to access your files. You can open, copy, and drag files between Finder and your local Desktop just like any other folder.
Disconnect when done
Right-click the mounted drive in the Finder sidebar and choose Eject, or press ⌘ Cmd + E while the drive is selected.
Windows: MobaXterm
MobaXterm is a free terminal for Windows that includes a built-in SSH client, SFTP file browser, and X11 display — all in one application.
Install MobaXterm
- Download the free Home Edition from mobaxterm.mobatek.net
- Run the installer (or use the portable version — no installation required)
Connect to Ibex
- Open MobaXterm and click Session → SSH
- Enter
ilogin.ibex.kaust.edu.saas the remote host - Enter your username and click OK
- MobaXterm automatically opens an SFTP panel on the left side of the terminal window showing your Ibex files
Transfer files with the SFTP panel
- Upload: drag files from Windows Explorer into the SFTP panel, or use the upload button (arrow pointing up)
- Download: right-click a file in the SFTP panel and choose Download, or drag it to your desktop
- Navigate: click folders in the SFTP panel just like Windows Explorer
- Edit a file: right-click → Open with to open a remote file in any local application; MobaXterm automatically re-uploads it when you save
In MobaXterm, right-click your connection in the Sessions sidebar and choose Save session so you don't need to re-enter the host and username each time.
IGV — Install on Your Local Computer
The Integrative Genomics Viewer (IGV) is a desktop application for visualizing BAM files and genomic data. Install it on your laptop before Lab 4.
Download and install
- Go to igv.org/doc/desktop
- Download the installer for your operating system:
- macOS: download the
.appzip, unzip, and drag IGV to your Applications folder - Windows: download the
.zip, unzip, and runigv.bat - Linux: download the
.zip, unzip, and runigv.sh
- macOS: download the
- Launch IGV and verify it opens. On first launch it will download a small genome index — an internet connection is required.
The IGV installer bundles its own Java runtime — you do not need to install Java separately. If you use the "command line" version instead, Java 11+ is required.
Conda Environment
A single rnaseq conda environment provides JupyterLab, all Python packages used in the notebooks, and all R/Bioconductor packages used in the R labs. Install it on Ibex for running analyses on the cluster, and optionally on your laptop for the R/Python visualization labs that do not require HPC resources.
Step 1 — Install Miniforge (skip if conda/mamba is already installed)
Miniforge is a minimal conda installer that uses the community-maintained conda-forge channel by default and includes mamba (a fast dependency solver) out of the box.
Linux & Ibex (x86_64)
# Download the Miniforge installer for Linux (64-bit)
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
-O ~/miniforge3_install.sh
# Run the installer in batch mode (no interactive prompts)
bash ~/miniforge3_install.sh -b -p ~/miniforge3
# Activate conda for your current shell session
source ~/miniforge3/bin/activate
# Initialise conda so it loads automatically in future sessions
conda init bash
# Reload your shell config
source ~/.bashrc
# Verify (mamba is included with Miniforge)
mamba --version
macOS
Run uname -m in your terminal. Output arm64 = Apple Silicon; x86_64 = Intel. Use the matching installer below.
# Apple Silicon (M1/M2/M3)
curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh \
-o ~/miniforge3_install.sh
# Intel Mac — use this line instead:
# curl -L https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-x86_64.sh \
# -o ~/miniforge3_install.sh
# Run the installer
bash ~/miniforge3_install.sh -b -p ~/miniforge3
# Activate conda for your current shell session
source ~/miniforge3/bin/activate
# Initialise conda — macOS defaults to zsh; add bash if you also use it
conda init zsh
# Reload your shell config
source ~/.zshrc
# Verify
mamba --version
Windows
WSL2 gives you a full Ubuntu environment on Windows and is the most compatible option for bioinformatics work. Once inside WSL2, follow the Linux & Ibex instructions above. If you prefer a native Windows install, download the .exe GUI installer from github.com/conda-forge/miniforge/releases (Miniforge3-Windows-x86_64.exe) and run it — then use Miniforge Prompt or Anaconda Prompt for the steps below.
Step 2 — Create the rnaseq environment
This command is the same on all platforms (Linux, macOS, Windows WSL2). It installs JupyterLab, Python analysis packages, and R/Bioconductor packages for all course labs. Expect it to take 10–20 minutes on first install.
mamba create -n rnaseq -c conda-forge -c bioconda \
python=3.11 \
jupyterlab \
ipykernel \
pandas \
numpy \
matplotlib \
seaborn \
scikit-learn \
scipy \
pydeseq2 \
adjusttext \
r-base \
bioconductor-deseq2 \
bioconductor-edger \
bioconductor-tximport \
bioconductor-tximeta \
bioconductor-genomicfeatures \
bioconductor-clusterprofiler \
bioconductor-org.hs.eg.db \
bioconductor-fgsea \
bioconductor-complexheatmap \
bioconductor-enhancedvolcano \
bioconductor-apeglm \
r-pheatmap \
r-ggplot2 \
r-dplyr \
r-tidyr \
r-ggrepel \
-y
| Package | Used for |
|---|---|
jupyterlab / ipykernel | Interactive Python notebooks (Labs 5, 7) |
pandas / numpy | Loading quant.sf files, matrix operations |
matplotlib / seaborn | PCA plots, heatmaps, volcano plots |
scikit-learn / scipy | PCA decomposition, pairwise distances |
pydeseq2 | Differential expression in Python |
adjustText | Non-overlapping labels in volcano plots |
r-base | R interpreter |
bioconductor-deseq2 / bioconductor-edger | Differential expression analysis (Lab 7) |
bioconductor-tximport / bioconductor-tximeta | Import Salmon output into R (Labs 5, 7) |
bioconductor-genomicfeatures | Build tx2gene mapping from GTF |
bioconductor-clusterprofiler / bioconductor-fgsea | GO and pathway enrichment, GSEA (Lab 8) |
bioconductor-complexheatmap / r-pheatmap | Heatmap visualisation |
bioconductor-enhancedvolcano | Volcano plots (Lab 7) |
bioconductor-apeglm | Log fold-change shrinkage (Lab 7) |
r-ggplot2 / r-dplyr / r-tidyr / r-ggrepel | General-purpose R plotting and data wrangling |
Step 3 — Activate and verify
Activation syntax is identical on all platforms.
conda activate rnaseq
# Verify Python packages
python -c "import pandas, numpy, matplotlib, seaborn, sklearn, pydeseq2; print('Python packages OK')"
# Verify Jupyter
jupyter lab --version
# Verify R packages
Rscript -e "library(DESeq2); library(tximport); library(clusterProfiler); cat('R packages OK\n')"
- On Ibex (recommended): use the Ibex Open OnDemand portal — go to Interactive Apps → JupyterLab, request resources, and click Connect. No terminal setup needed.
- On your laptop: activate the environment and run
jupyter lab— it opens automatically in your browser.
Quick Reference: Daily Commands
# Connect to Ibex
ssh username@ilogin.ibex.kaust.edu.sa
# Start interactive session
srun --pty --time=4:00:00 --mem=8G --cpus-per-task=4 bash
# Navigate to workspace
cd /ibex/user/$USER/workshop
Linux Command Line Cheatsheet
Navigation
# Print current directory
pwd
# List files and directories
ls # Basic listing
ls -l # Detailed listing (permissions, size, date)
ls -la # Include hidden files (starting with .)
ls -lh # Human-readable file sizes
# Change directory
cd /path/to/dir # Go to specific directory
cd ~ # Go to home directory
cd .. # Go up one level
cd - # Go to previous directory
File Operations
# Create files and directories
mkdir mydir # Create directory
mkdir -p dir1/dir2 # Create nested directories
touch file.txt # Create empty file
# Copy files
cp file.txt backup.txt # Copy file
cp -r mydir/ mydir_backup/ # Copy directory recursively
# Move/rename files
mv oldname.txt newname.txt # Rename file
mv file.txt /path/to/dest/ # Move file
# Remove files
rm file.txt # Delete file
rm -r mydir/ # Delete directory recursively
rm -i file.txt # Ask before deleting
Viewing Files
# Display file contents
cat file.txt # Show entire file
head file.txt # Show first 10 lines
head -n 20 file.txt # Show first 20 lines
tail file.txt # Show last 10 lines
tail -f logfile.txt # Follow file updates in real-time
less file.txt # View with scrolling (q to quit)
# View compressed files
gzip -dc file.gz # View gzipped file (works on macOS and Linux)
zless file.gz # View gzipped file with scrolling
File Information
# Count lines, words, characters
wc file.txt # Lines, words, characters
wc -l file.txt # Lines only
# File size
du -h file.txt # Size of file
du -sh mydir/ # Total size of directory
# Disk space
df -h # Disk usage summary
Searching
# Find files
find . -name "*.fastq" # Find by name pattern
find . -type f -size +100M # Find files larger than 100MB
# Search file contents
grep "pattern" file.txt # Lines containing pattern
grep -r "pattern" mydir/ # Search recursively in directory
grep -i "pattern" file.txt # Case-insensitive search
grep -c "pattern" file.txt # Count matching lines
Pipes and Redirection
# Redirect output
command > file.txt # Write output to file (overwrite)
command >> file.txt # Append output to file
command 2> error.txt # Redirect errors to file
# Pipes (send output to another command)
cat file.txt | head # First 10 lines
cat file.txt | grep "pattern" # Filter lines
cat file.txt | wc -l # Count lines
cat file.txt | sort | uniq # Sort and remove duplicates
Compression
# Gzip
gzip file.txt # Compress (creates file.txt.gz)
gunzip file.txt.gz # Decompress
# Tar archives
tar -cvf archive.tar dir/ # Create archive
tar -xvf archive.tar # Extract archive
tar -czvf archive.tar.gz dir/ # Create compressed archive
tar -xzvf archive.tar.gz # Extract compressed archive
Permissions
# View permissions
ls -l file.txt # Shows -rw-r--r-- format
# Change permissions
chmod +x script.sh # Make executable
chmod 755 script.sh # rwx for owner, rx for others
# Change ownership
chown user:group file.txt
Process Management
# View running processes
ps # Your processes
ps aux # All processes
top # Interactive process viewer (q to quit)
htop # Better interactive viewer (if installed)
# Control processes
command & # Run in background
Ctrl+C # Stop current process
Ctrl+Z # Suspend current process
bg # Resume suspended process in background
kill PID # Terminate process by ID
killall processname # Terminate by name
Useful Shortcuts
# Keyboard shortcuts
Ctrl+C # Cancel current command
Ctrl+L # Clear screen
Ctrl+A # Move cursor to start of line
Ctrl+E # Move cursor to end of line
Ctrl+R # Search command history
Tab # Auto-complete file/directory names
Up/Down # Navigate command history
# History
history # Show command history
!123 # Run command #123 from history
!! # Repeat last command