Incremental Learning of Sparse Attention Patterns in Transformers

The analysis/ folder contains notebooks for regenerating the paper plots from the W&B project r-alvarezlucendo16/incremental-learning.

Installation

uv sync

# List available experiments
bash run.sh

# Run a specific experiment
bash run.sh <experiment_name>

Experiments are configured using Hydra with configs located in conf/.

Experiment configs in conf/experiments/ override base settings from conf/train.yaml
Component configs can be customized: model/, dataset/, optimizer/, scheduler/, loss/

Name		Name	Last commit message	Last commit date
Latest commit History 39 Commits
.vscode		.vscode
analysis		analysis
conf		conf
figures		figures
src		src
tests		tests
.gitignore		.gitignore
.python-version		.python-version
README.md		README.md
ode.py		ode.py
pyproject.toml		pyproject.toml
run.sh		run.sh
train.py		train.py
uv.lock		uv.lock