happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/examples/server/public_simplechat
History
HanishKVC 6597fafeae SimpleChat: Make vertical layout better responsive (flex based) 
Also needed to make things cleaner and properly usable whether
landscape or portrait, after changing to multiline textarea rather
than single line user input.

Avoid hardcoding the chat-till-now display area height, instead
make it a flex-growable within a flex column of ui elements within
a fixed vertical area.
 		2024-05-20 16:31:13 +05:30
..
readme.md SimpleChat: Make vertical layout better responsive (flex based) 2024-05-20 16:31:13 +05:30
simplechat.css SimpleChat: Make vertical layout better responsive (flex based) 2024-05-20 16:31:13 +05:30
simplechat.html SimpleChat: Make vertical layout better responsive (flex based) 2024-05-20 16:31:13 +05:30
simplechat.js SimpleChat: textarea for multiline user chat, inturn shift+enter 4 enter 2024-05-20 11:59:07 +05:30
simplechat.sh SimpleChat:sh: Add simple shell script to run python3 http.server 2024-05-18 00:36:23 +05:30

readme.md

SimpleChat

by Humans for All.

overview

This simple web frontend, allows triggering/testing the server's /completions or /chat/completions endpoints in a simple way with minimal code from a common code base. And also allows trying to maintain a basic back and forth chatting to an extent.

User can set a system prompt, as well as try and chat with the model over a series of back and forth chat messages.

The UI follows a responsive web design so that the layout can adapt to available display space in a usable enough manner, in general.

NOTE: Given that the idea is for basic minimal testing, it doesnt bother with any model context length and culling of old messages from the chat.

NOTE: It doesnt set any parameters other than temperature for now. However if someone wants they can update the js file as needed.

usage

first run examples/server

  • bin/server -m path/model.gguf

next run this web front end in examples/server/public_simplechat

  • ./simplechat.sh
  • this uses python3's http.server to host this web front end

Open this simple web front end from your local browser as noted in the message printed when simplechat.sh is run

Once inside

  • Select between chat and completion mode. By default it is set to chat mode.
  • If you want to provide a system prompt, then ideally enter it first, before entering any user query.
    • if handle_systemprompt_begin is used
      • you cant change the system prompt, after it is has been submitted once along with user query.
      • you cant set a system prompt, after you have submitted any user query
    • if handle_systemprompt_anytime is used
      • one can change the system prompt any time during chat, by changing the contents of system prompt.
      • inturn the updated/changed system prompt will be inserted into the chat session.
      • this allows for the subsequent user chatting to be driven by the new system prompt set above.
  • Enter your query and either press enter or click on the submit button
  • Wait for the logic to communicate with the server and get the response.
    • the user is not allowed to enter any fresh query during this time.
    • the user input box will be disabled and a working message will be shown in it.
  • just refresh the page, to reset wrt the chat history and or system prompt and start afresh.

Devel note

Sometimes the browser may be stuborn with caching of the file, so your updates to html/css/js may not be visible. Also remember that just refreshing/reloading page in browser or for that matter clearing site data, dont directly override site caching in all cases. Worst case you may have to change port.

Concept of multiple chat sessions with same or different system prompts, as well as saving and restoring of those across browser usage sessions, can be woven around the SimpleChat class and its instances relatively easily, however given the current goal of keeping this simple, it has not been added, for now.

By switching between handle_systemprompt_begin/anytime, one can control whether one can change the system prompt, anytime during the conversation or only at the beginning.