Skip to content
Snippets Groups Projects
Commit 4861ec0f authored by Erik Jonas Hartnick's avatar Erik Jonas Hartnick :dragon_face:
Browse files

Added config/changed preprocessing to work with ollama

- added .env config for test with Llama3.2 for pool PC (TODO: rename to .env.llama3.2.example)
- added langchain-ollama python package and changed Chat and Embeddings calls accordingly (mainly constructors)
- switched preprocessing to ollama3.2, model needs to be configured in CHESS/src/database_utils/db_catalog/preprocess.py
- added engine config (models their config for each agent, preprocessing (embedding api, generating tokens) does not use this
- changed retrieve_entity.py to also use OllamaEmbeddings, because Embeddings are configured separately
- adapted run scripts to work with configured env
- adapted tool/agent configs to work with new ollama config
- added a place to put ollama and models locally, more or less to prepare different places (data buckets on server)
- added some things to gitignore
- TODO: documentation of the changes and configs above
- TODO: documentation on how and where to get the datasets and where to put them
parent 34fd8830
Branches
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment