happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/examples/server/public_simplechat
History
HanishKVC 5a5f6ab848 SimpleChat: Update notes a bit. Try keep browser happy 
Avoid browser quirk mode with DOCTYPE.

Help with accessibility a bit by specifying the language explicitly.

Specify the char encoding explicitly, inturn utf-8 is a safe bet,
even with intermixing of languages if reqd in future.

Add a cache-control http-equiv meta tag, which in all probability
will be ignored.

Defer js loading and execution, just for fun and future, not that
critical here as it stands now.
 		2024-05-19 01:59:25 +05:30
..
readme.md SimpleChat: Update notes a bit. Try keep browser happy 2024-05-19 01:59:25 +05:30
simplechat.css SimpleChat:CSS: Allow for chat div to be scrollable 2024-05-18 22:11:59 +05:30
simplechat.html SimpleChat: Update notes a bit. Try keep browser happy 2024-05-19 01:59:25 +05:30
simplechat.js SimpleChat:JS: bottom of element visible, Set focus to user input 2024-05-18 22:59:21 +05:30
simplechat.sh SimpleChat:sh: Add simple shell script to run python3 http.server 2024-05-18 00:36:23 +05:30

readme.md

SimpleChat

by Humans for All.

overview

This simple web frontend, allows triggering/testing the server's /completions or /chat/completions endpoints in a simple way with minimal code from a common code base. And also allows trying to maintain a basic back and forth chatting to an extent.

User can set a system prompt, as well as try and chat with the model over a series of back and forth chat messages.

NOTE: Given that the idea is for basic minimal testing, it doesnt bother with any model context length and culling of old messages from the chat.

NOTE: It doesnt set any parameters other than temperature for now. However if someone wants they can update the js file as needed.

usage

first run examples/server

  • bin/server -m path/model.gguf

next run this web front end in examples/server/public_simplechat

  • ./simplechat.sh
  • this uses python3's http.server to host this web front end

Open this simple web front end from your local browser as noted in the message printed when simplechat.sh is run

Once inside

  • Select between chat and completion mode. By default it is set to chat mode.
  • If you want to provide a system prompt, then enter it first, before entering any user query.
    • you cant change the system prompt, after it is has been submitted once along with user query.
    • you cant set a system prompt, after you have submitted any user query
  • Enter your query and either press enter or click on the submit button
  • Wait for the logic to communicate with the server and get the response.
    • the user is not allowed to enter any fresh query during this time.
    • the user input box will be disabled and a working message will be shown in it.
  • just refresh the page, to reset wrt the chat history and or system prompt and start afresh.

Devel note

Sometimes the browser may be stuborn with caching of the file, so your updates to html/css/js may not be visible. Also remember that just refreshing/reloading page in browser or for that matter clearing site data, dont directly override site caching in all cases. Worst case you may have to change port.