Skip to content
Snippets Groups Projects

Draft: Resolve "Konfigurieren für llama3:70b und slrum"

Open Draft: Resolve "Konfigurieren für llama3:70b und slrum"
3-konfigurieren-fur-llama3-70b-und-slrum into 2-llama3-2-konfigurationstest
Open Erik Jonas Hartnick requested to merge 3-konfigurieren-fur-llama3-70b-und-slrum into 2-llama3-2-konfigurationstest
Viewing commit 2190c706
Prev Next
Show latest version
9 files
+ 183
− 47
Compare changes
  • Side-by-side
  • Inline
Files
9
  • 2190c706 
    - added slurm log files to .gitignore
- renamed env file, setup to run with bird/sds
- added vllm and its outlines dependency to requirements as mentioned in model_deployment.txt
- renamed run script to bird-oss (open source software, refers to open source models)
- changed cleanerscript to produce output
- changed copyrepo.sh to allow for parallel execution of models and CHESS code
- moved parallel sections of copyrepo to separate scripts
CHESS/run/run_main_ir_ss_cg_bird-oss.sh 0 → 100755
+ 11
− 0
source .env
data_mode=$DATA_MODE # Options: 'dev', 'train'
data_path=$DATA_PATH # UPDATE THIS WITH THE PATH TO THE TARGET DATASET
config="./run/configs/CHESS_IR_SS_CG_BIRD_OSS.yaml"
num_workers=1 # Number of workers to use for parallel processing, set to 1 for no parallel processing
python3 -u ./src/main.py --data_mode ${data_mode} --data_path ${data_path} --config "$config" \
--num_workers ${num_workers} --pick_final_sql true