Skip to content
Snippets Groups Projects
  • Erik Jonas Hartnick's avatar
    4861ec0f
    Added config/changed preprocessing to work with ollama · 4861ec0f
    Erik Jonas Hartnick authored
    - added .env config for test with Llama3.2 for pool PC (TODO: rename to .env.llama3.2.example)
    - added langchain-ollama python package and changed Chat and Embeddings calls accordingly (mainly constructors)
    - switched preprocessing to ollama3.2, model needs to be configured in CHESS/src/database_utils/db_catalog/preprocess.py
    - added engine config (models their config for each agent, preprocessing (embedding api, generating tokens) does not use this
    - changed retrieve_entity.py to also use OllamaEmbeddings, because Embeddings are configured separately
    - adapted run scripts to work with configured env
    - adapted tool/agent configs to work with new ollama config
    - added a place to put ollama and models locally, more or less to prepare different places (data buckets on server)
    - added some things to gitignore
    - TODO: documentation of the changes and configs above
    - TODO: documentation on how and where to get the datasets and where to put them
    4861ec0f
    History
    Added config/changed preprocessing to work with ollama
    Erik Jonas Hartnick authored
    - added .env config for test with Llama3.2 for pool PC (TODO: rename to .env.llama3.2.example)
    - added langchain-ollama python package and changed Chat and Embeddings calls accordingly (mainly constructors)
    - switched preprocessing to ollama3.2, model needs to be configured in CHESS/src/database_utils/db_catalog/preprocess.py
    - added engine config (models their config for each agent, preprocessing (embedding api, generating tokens) does not use this
    - changed retrieve_entity.py to also use OllamaEmbeddings, because Embeddings are configured separately
    - adapted run scripts to work with configured env
    - adapted tool/agent configs to work with new ollama config
    - added a place to put ollama and models locally, more or less to prepare different places (data buckets on server)
    - added some things to gitignore
    - TODO: documentation of the changes and configs above
    - TODO: documentation on how and where to get the datasets and where to put them
Code owners