NLP Course, 2026

Current curriculum covers topic from basic NLP techinques to the most modern ones, that may be helpful for custom training of LLMs:

• NLP Basics: tokenization, text preprocessing, text representations
• Text & Language Models: embeddings, n-gram models, RNNs, LSTMs, seq2seq, attention
• Transformers & LLMs: Transformer, pre-training (MLM/CLM), prompting, fine-tuning, PEFT
• Scaling & Optimization: : distributed training, MoE, KV-cache, Flash Attention, efficient inference, quantization
• Retrieval & Agents: Information Retrieval, RAG, agent-based systems
• Post-training: alignment, RLHF, DPO

Course Staff

• German Gritsai @grgera
• Anastasiia Vozniuk @natriistorm
• Ildar Khabutdinov @depinwhite

Materials

Week #	Date	Topic	Lecture	Seminar	Additional	Recording
1	February 10	Intro to NLP & Tokenization	slides	ipynb	materials	TBA

Homeworks

TBA

Game Rules

Final mark = 0.3 × (oral answer grade) + 0.7 × (average score for practical assignments)

Both oral exam and homeworks are blocking parts, you need to pass both parts to pass the course.

Prerequisities

• Probability Theory + Statistics
• Machine Learning
• Python Python guide
• Basic knowledge on NLP

We expect students to know basics of Natural Language Processing, as the course focuses on more advanced topics. When you unsure about the basics, we recommned to read these lectures / materials:

1. Course from Lena Voita
2. Speech and Language Processing by Jurafsky and Martin
3. Stanford CS 224n
4. Great blog on Transformer & BERT