Draft: Resolve "Konfigurieren für llama3:70b und slrum" (!3) · Merge requests · Stephan Mitte / Re-CHESS

Open Erik Jonas Hartnick requested to merge 3-konfigurieren-fur-llama3-70b-und-slrum into 2-llama3-2-konfigurationstest 2 months ago

Mar 31, 2025
- Edit README.md · d5a52e8c
  Erik Jonas Hartnick authored 2 weeks ago
  
  d5a52e8c
- Added commented out `DATA_PATH` for full BIRD dev set to .env-rechess-bird-sds · 907940da
  Erik Jonas Hartnick authored 2 weeks ago
  
  907940da
- Added commented out Llama3-70B to retrieve_entity.py · b236282c
  Erik Jonas Hartnick authored 2 weeks ago
  
  b236282c
- Added commented out Llama3-70B to preprocess.py · d98f6f91
  Erik Jonas Hartnick authored 2 weeks ago
  
  d98f6f91
Mar 05, 2025

Changed minor things in scripts · 7d99559a

Erik Jonas Hartnick authored 1 month ago

- informative part at the start
- CUDA_VISIBLE devices now exported
- vllm proper context length now also on download, shouldn't matter much since on execution, it was correct already

7d99559a

Mar 03, 2025
- implement sampling count · 240c74d3
  stepmitt authored 1 month ago
  
  240c74d3
- Added sds in data subdir · d21b36cc
  Erik Jonas Hartnick authored 1 month ago
  
  d21b36cc
- Fixed some configs, run script things · 89868cdc
  Erik Jonas Hartnick authored 1 month ago
  
  89868cdc
Feb 21, 2025

Gotten closer to running it on McG, vllm nl2sql model uses too much memory. · 2190c706

Erik Jonas Hartnick authored 1 month ago

- added slurm log files to .gitignore
- renamed env file, setup to run with bird/sds
- added vllm and its outlines dependency to requirements as mentioned in model_deployment.txt
- renamed run script to bird-oss (open source software, refers to open source models)
- changed cleanerscript to produce output
- changed copyrepo.sh to allow for parallel execution of models and CHESS code
- moved parallel sections of copyrepo to separate scripts

2190c706

Feb 18, 2025

Added some config for running on GPU-Cluster, progress on Slurm · 0cc9ec68

Erik Jonas Hartnick authored 1 month ago

- needs fixed slurm scripts and docs
- needs more config and docs matching to Slurm script
- setup and run scripts for McGarret should be separated

0cc9ec68

Feb 10, 2025
- Prepared "dry run" version to test batch script · b07ae7af
  Erik Jonas Hartnick authored 2 months ago
  
  b07ae7af
Feb 09, 2025
- Added untested draft for sbatch script · efe19616
  Erik Jonas Hartnick authored 2 months ago
  
  efe19616

Draft: Resolve "Konfigurieren für llama3:70b und slrum"

Merge request reports

Activity