Lab 1: Warm-up for Large-scale Analysis & Genomic File Formats

Create project directories

# Activate your environment
conda activate genomics

# Create main project directory
mkdir ~/genomics
cd ~/genomics

# Create subdirectories for each analysis step
mkdir subsampled_data
mkdir trimmed_data
mkdir references
#you can create multiple folders in the same command:
mkdir -p qc_reports/fastqc_raw qc_reports/fastqc_trimmed qc_reports/fastp
mkdir alignment
mkdir salmon_quant
mkdir counts
mkdir -p results/tables results/figures results/enrichment
mkdir logs

# View the structure
tree

Step 1: Navigate to NCBI GEO

NCBI GEO (Gene Expression Omnibus) is a public repository for gene expression data, including RNA-seq datasets.

1. Open your web browser and go to https://www.ncbi.nlm.nih.gov/geo/
2. You will see the GEO homepage with a search box at the top

Step 2: Search for the Dataset

1. In the search box, type the accession number: GSE136366
2. Click "Search" or press Enter
3. The search results will show the dataset: "TDP-43 regulates LC3ylation..."
4. Click on the accession number GSE136366 to view the full record

Step 3: Explore the GEO Series Page

The GEO Series page contains important information about the dataset:

• Title: Full study title and description
• Summary: Experimental design and objectives
• Overall design: Sample information and conditions
• Contributors: Authors and their affiliations
• Citation: Link to the published paper
• Samples: List of all individual samples (GSM accessions)

Step 4: Find the SRA Data

To download the raw FASTQ files, we need to access the SRA (Sequence Read Archive):

1. Scroll down to the "Relations" section on the GEO page
2. Look for the link labeled "SRA" with accession SRP219898
3. Click on the SRA link to navigate to the SRA page

Alternatively, scroll to the "Samples" section and click on any sample (e.g., GSM4047073) to see its individual SRA run.

Step 5: Navigate the SRA Run Selector

The SRA Run Selector allows you to view and download individual sequencing runs:

1. On the SRA page, click "Send results to Run selector" (if not already there)
2. You will see a table listing all 6 runs (SRR10045016 - SRR10045021)
3. The table shows important metadata:
   • Run: The SRR accession number
   • Sample: Sample name from the study
   • LibraryLayout: PAIRED (paired-end sequencing)
   • Bases: Total sequencing bases
   • Spots: Number of reads

Step 6: Identify Sample-to-Accession Mapping

From the SRA Run Selector, you can determine which SRR accession corresponds to each sample:

Step 7: Download Options

There are several ways to download the FASTQ files, including:

Step 1: Navigate to Ensembl

1. Open your browser and go to https://www.ensembl.org
2. The homepage shows featured species and a search box
3. Click on "Human" in the popular species section, or search for "Homo sapiens"

Step 2: Access the FTP Download Site

Reference files are available through Ensembl's FTP server:

1. From the Ensembl homepage, click "Downloads" in the top navigation menu
2. Select "Download data via FTP"
3. Alternatively, go directly to: https://ftp.ensembl.org/pub/

Step 3: Understand the FTP Directory Structure

The Ensembl FTP site is organized by release version and data type:

ftp.ensembl.org/pub/
├── release-110/              # Release version (we use 110)
│   ├── fasta/                # Sequence files
│   │   └── homo_sapiens/
│   │       ├── dna/          # Genomic DNA sequences
│   │       ├── cdna/         # Transcriptome (cDNA)
│   │       └── pep/          # Protein sequences
│   ├── gtf/                  # Gene annotation files
│   │   └── homo_sapiens/
│   └── gff3/                 # Alternative annotation format
└── current_fasta/            # Symlink to latest release

Step 4: Find the Genome FASTA Files

To download genomic DNA sequences:

1. Navigate to: release-110/fasta/homo_sapiens/dna/
2. You will see several file types:
   • Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz - (primary assembly)
   • Homo_sapiens.GRCh38.dna.toplevel.fa.gz - Includes alternate haplotypes
   • Homo_sapiens.GRCh38.dna.chromosome.*.fa.gz - Individual chromosomes
3. For this workshop, we download: Homo_sapiens.GRCh38.dna.chromosome.11.fa.gz

Download reference genome - chr11 only

cd ~/genomics/references

# Download human chromosome 11 sequence
echo "Downloading chromosome 11 genome..."
wget https://ftp.ensembl.org/pub/release-110/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.chromosome.11.fa.gz
gunzip Homo_sapiens.GRCh38.dna.chromosome.11.fa.gz

Step 5: Find the GTF Annotation Files

Gene annotations provide coordinates and metadata for genes, transcripts, and exons:

1. Navigate to: release-110/gtf/homo_sapiens/
2. The main annotation file is: Homo_sapiens.GRCh38.110.gtf.gz
3. You may also see:
   • Homo_sapiens.GRCh38.110.chr.gtf.gz - Chromosomes only (no scaffolds)
   • Homo_sapiens.GRCh38.110.abinitio.gtf.gz - Ab initio predictions

Download reference genome and annotation

cd ~/genomics/references
# Download full GTF annotation
echo "Downloading gene annotation..."
wget https://ftp.ensembl.org/pub/release-110/gtf/homo_sapiens/Homo_sapiens.GRCh38.110.gtf.gz
gunzip Homo_sapiens.GRCh38.110.gtf.gz

# Extract chromosome 11 annotations only
echo "Extracting chr11 annotations..."
grep -E "^#|^11	" Homo_sapiens.GRCh38.110.gtf > Homo_sapiens.GRCh38.110.chr11.gtf
                        
                    

Step 6: Find the Transcriptome (cDNA) Files

The transcriptome contains sequences of all annotated transcripts:

1. Navigate to: release-110/fasta/homo_sapiens/cdna/
2. Download: Homo_sapiens.GRCh38.cdna.all.fa.gz
3. This file is used for transcript-level quantification with tools like Salmon and Kallisto

Download reference genome and annotation

cd ~/genomics/references
# Download transcriptome and extract chr11 transcripts
echo "Downloading transcriptome..."
wget https://ftp.ensembl.org/pub/release-110/fasta/homo_sapiens/cdna/Homo_sapiens.GRCh38.cdna.all.fa.gz
gunzip -c Homo_sapiens.GRCh38.cdna.all.fa.gz | awk '/^>/ {keep = /chromosome:GRCh38:11:/} keep' > transcriptome_chr11.fa
                

Step 7: Check Release Notes

Before downloading, it's good practice to check release notes for any important changes:

1. Go to Ensembl News
2. Review what's new in your chosen release
3. Check for known issues or updates to gene annotations

Explore FASTA format (genome sequence)

cd ~/genomics/references

# View the first 10 lines of the genome file
head -10 Homo_sapiens.GRCh38.dna.chromosome.11.fa

# The first line starts with > and contains the sequence name
# Subsequent lines contain the DNA sequence (A, T, G, C, N)

# Count total bases (excluding header)
grep -v "^>" Homo_sapiens.GRCh38.dna.chromosome.11.fa | tr -d '\n' | wc -c

# Chromosome 11 is approximately 135 million base pairs

>chromosome_name description
ATGCATGCATGCATGCATGCATGCATGC...
GCTAGCTAGCTAGCTAGCTAGCTAGCTA...

Explore GTF format (gene annotation)

cd ~/genomics/references

# View header comments and first few data lines
head -50 Homo_sapiens.GRCh38.110.chr11.gtf

# Count different feature types
echo "Feature types in GTF:"
cut -f3 Homo_sapiens.GRCh38.110.chr11.gtf | grep -v "^#" | sort | uniq -c | sort -rn

# Count genes on chromosome 11
echo "Number of genes:"
awk '$3 == "gene"' Homo_sapiens.GRCh38.110.chr11.gtf | wc -l

# Find a specific gene (e.g., TARDBP - the TDP-43 gene, though it's on chr1)
# Let's look at a chr11 gene: INS (insulin)
grep "gene_name \"INS\"" Homo_sapiens.GRCh38.110.chr11.gtf | head -5

Download the FASTQ raw files

# The data is available under the following path

cd ~/genomics

# Run wget to download
wget -bc https://zenodo.org/records/18462432/files/raw_data.zip

unzip raw_data.zip

# Verify the files
ls -lh raw_data/

# You should see 12 files (6 samples × 2 paired-end files):
# KO_1_SRR10045016_1.fastq.gz  KO_1_SRR10045016_2.fastq.gz
# KO_2_SRR10045017_1.fastq.gz  KO_2_SRR10045017_2.fastq.gz
# KO_3_SRR10045018_1.fastq.gz  KO_3_SRR10045018_2.fastq.gz
# WT_1_SRR10045019_1.fastq.gz  WT_1_SRR10045019_2.fastq.gz
# WT_2_SRR10045020_1.fastq.gz  WT_2_SRR10045020_2.fastq.gz
# WT_3_SRR10045021_1.fastq.gz  WT_3_SRR10045021_2.fastq.gz