Skip to content

Adding LuMamba to BioFoundation#9

Open
danaebroustail wants to merge 7 commits intopulp-bio:mainfrom
danaebroustail:lumamba
Open

Adding LuMamba to BioFoundation#9
danaebroustail wants to merge 7 commits intopulp-bio:mainfrom
danaebroustail:lumamba

Conversation

@danaebroustail
Copy link
Copy Markdown

Overview:
Added dataset, pretrain and finetune configs and tasks (LeJEPA pretraining), docs, logo, and model files for LuMamba.

Changes in existing parts of the repo:
Overlapping changes that might need an extra look:

  • LUNA's multiloader_data_module.py: added option to track t-SNE of TUAB and TUAR during pre-training.
    • Should not affect regular LUNA pre-training, and can be commented out of LuMamba pretrain config.
  • Config defaults.yaml: added a hydra dir option, to change hydra output logs directory.
  • LUNA's finetune_task_LUNA.py: added integration for LuMamba, is consistent with LUNA.
    • Included bug fix for "model." prefixes for LUNA/LuMamba checkpoint loading (.ckpt and .safetensors files). Consistent with LUNA safetensors behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant