happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/examples/agents
History
ochafik ffc74360e2 agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
..
hermes_function_calling@b4f757e27d agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
README.md agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
fastify-requirements.txt agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
fastify.py agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
requirements.txt agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00
run_sandboxed_tools.sh agents: scripts to run scripts as sandboxed fastapi servers 2024-04-27 23:14:11 +01:00

README.md

Edit examples/agents/hermes_function_calling/utils.py:

log_folder = os.environ.get('LOG_FOLDER', os.path.join(script_dir, "inference_logs"))

Then run:

REQUIREMENTS_FILE=<( cat examples/agents/hermes_function_calling/requirements.txt | grep -vE "bitsandbytes|flash-attn" ) \
  examples/agents/run_sandboxed_tools.sh \
    examples/agents/hermes_function_calling/functions.py \
    -e LOG_FOLDER=/data/inference_logs