Skip to content
Snippets Groups Projects
Commit 996a27bf authored by Riko Corwin Uphoff's avatar Riko Corwin Uphoff
Browse files

Updated pretraining scripts

parent 9538fe00
Branches
No related tags found
No related merge requests found
Pipeline #25320 passed
Stage: build
Hide whitespace changes
Inline Side-by-side
scripts/shell/pretrain_60m.sh
+ 1
− 0
View file @ 996a27bf
...@@ -7,6 +7,7 @@ do ...@@ -7,6 +7,7 @@ do
--mode pretraining \ --mode pretraining \
--optimizer "$optimizer" \ --optimizer "$optimizer" \
--model llama_60m \ --model llama_60m \
--dataset c4 \
--batch_size 512 \ --batch_size 512 \
--num_epochs 1 \ --num_epochs 1 \
--num_training_tokens 1310000000 \ --num_training_tokens 1310000000 \
......
scripts/shell/pretrain_7b.sh
+ 1
− 0
View file @ 996a27bf
...@@ -7,6 +7,7 @@ do ...@@ -7,6 +7,7 @@ do
--mode pretraining \ --mode pretraining \
--optimizer "$optimizer" \ --optimizer "$optimizer" \
--model llama_7b \ --model llama_7b \
--dataset c4 \
--batch_size 512 \ --batch_size 512 \
--num_epochs 1 \ --num_epochs 1 \
--num_training_tokens 13100000 \ --num_training_tokens 13100000 \
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment