SimpleChatTC:Readme: Updated wrt new relativelyProper toolCallsHS
Also update the sliding window context size to last 9 chat messages so that there is a sufficiently large context for multi turn tool calls based adjusting by ai and user, without needing to go full hog, which has the issue of overflowing the currently set context window wrt the loaded ai model.
This commit is contained in:
parent
152deb5d5a
commit
61b70bfa5d
|
|
@ -136,8 +136,9 @@ Once inside
|
|||
called, then the ai response might include request for tool call.
|
||||
* the SimpleChat client will show details of the tool call (ie tool name and args passed) requested
|
||||
and allow the user to trigger it as is or after modifying things as needed.
|
||||
NOTE: Tool sees the original tool call only, for now
|
||||
* inturn returned / generated result is placed into user query entry text area with approriate tags
|
||||
ie <tool_response> generated result </tool_response>
|
||||
ie <tool_response> generated result with meta data </tool_response>
|
||||
* if user is ok with the tool response, they can click submit to send the same to the GenAi/LLM.
|
||||
User can even modify the response generated by the tool, if required, before submitting.
|
||||
|
||||
|
|
@ -193,7 +194,7 @@ It is attached to the document object. Some of these can also be updated using t
|
|||
sent back to the ai model, under user control.
|
||||
|
||||
as tool calling will involve a bit of back and forth between ai assistant and end user, it is
|
||||
recommended to set iRecentUserMsgCnt to 5 or more, so that enough context is retained during
|
||||
recommended to set iRecentUserMsgCnt to 10 or more, so that enough context is retained during
|
||||
chatting with ai models with tool support.
|
||||
|
||||
apiEP - select between /completions and /chat/completions endpoint provided by the server/ai-model.
|
||||
|
|
@ -239,7 +240,7 @@ It is attached to the document object. Some of these can also be updated using t
|
|||
be set if needed using the settings ui.
|
||||
|
||||
iRecentUserMsgCnt - a simple minded SlidingWindow to limit context window load at Ai Model end.
|
||||
This is set to 5 by default. So in addition to latest system message, last/latest iRecentUserMsgCnt
|
||||
This is set to 10 by default. So in addition to latest system message, last/latest iRecentUserMsgCnt
|
||||
user messages after the latest system prompt and its responses from the ai model will be sent
|
||||
to the ai-model, when querying for a new response. Note that if enabled, only user messages after
|
||||
the latest system message/prompt will be considered.
|
||||
|
|
@ -325,7 +326,7 @@ work.
|
|||
|
||||
ALERT: The simple minded way in which this is implemented, it can be dangerous in the worst case,
|
||||
Always remember to verify all the tool calls requested and the responses generated manually to
|
||||
ensure everything is fine, during interaction with ai modles with tools support.
|
||||
ensure everything is fine, during interaction with ai models with tools support.
|
||||
|
||||
#### Builtin Tools
|
||||
|
||||
|
|
@ -358,9 +359,11 @@ that should be sent back to the ai model, in your constructed code.
|
|||
Update the tc_switch to include a object entry for the tool, which inturn includes
|
||||
* the meta data as well as
|
||||
* a reference to the handler and also
|
||||
the handler should take toolCallId, toolName and toolArgs and pass these along to
|
||||
web worker as needed.
|
||||
* the result key (was used previously, may use in future, but for now left as is)
|
||||
|
||||
#### Mapping tool calls and responses to normal assistant - user chat flow
|
||||
#### OLD: Mapping tool calls and responses to normal assistant - user chat flow
|
||||
|
||||
Instead of maintaining tool_call request and resultant response in logically seperate parallel
|
||||
channel used for requesting tool_calls by the assistant and the resulstant tool role response,
|
||||
|
|
@ -375,10 +378,14 @@ NOTE: This flow tested to be ok enough with Gemma-3N-E4B-it-Q8_0 LLM ai model fo
|
|||
given the way current ai models work, most of them should understand things as needed, but need
|
||||
to test this with other ai models later.
|
||||
|
||||
TODO: Need to think later, whether to continue this simple flow, or atleast use tool role wrt
|
||||
TODO:OLD: Need to think later, whether to continue this simple flow, or atleast use tool role wrt
|
||||
the tool call responses or even go further and have the logically seperate tool_calls request
|
||||
structures also.
|
||||
|
||||
DONE: rather both tool_calls structure wrt assistant messages and tool role based tool call
|
||||
result messages are generated as needed.
|
||||
|
||||
|
||||
#### ToDo
|
||||
|
||||
WebFetch and Local web proxy/caching server
|
||||
|
|
@ -386,6 +393,7 @@ WebFetch and Local web proxy/caching server
|
|||
Try and trap promises based flows to ensure all generated results or errors if any are caught
|
||||
before responding back to the ai model.
|
||||
|
||||
Trap error responses.
|
||||
|
||||
### Debuging the handshake
|
||||
|
||||
|
|
|
|||
|
|
@ -219,7 +219,7 @@ let gUsageMsg = `
|
|||
<ul class="ul2">
|
||||
<li> submit tool response placed into user query textarea</li>
|
||||
</ul>
|
||||
<li> Default ContextWindow = [System, Last4 Query+Resp, Cur Query].</li>
|
||||
<li> Default ContextWindow = [System, Last9 Query+Resp, Cur Query].</li>
|
||||
<ul class="ul2">
|
||||
<li> ChatHistInCtxt, MaxTokens, ModelCtxt window to expand</li>
|
||||
</ul>
|
||||
|
|
@ -983,7 +983,7 @@ class Me {
|
|||
this.bCompletionFreshChatAlways = true;
|
||||
this.bCompletionInsertStandardRolePrefix = false;
|
||||
this.bTrimGarbage = true;
|
||||
this.iRecentUserMsgCnt = 5;
|
||||
this.iRecentUserMsgCnt = 10;
|
||||
/** @type {Object<string, number>} */
|
||||
this.sRecentUserMsgCnt = {
|
||||
"Full": -1,
|
||||
|
|
@ -991,6 +991,7 @@ class Me {
|
|||
"Last1": 2,
|
||||
"Last2": 3,
|
||||
"Last4": 5,
|
||||
"Last9": 10,
|
||||
};
|
||||
this.apiEP = ApiEP.Type.Chat;
|
||||
/** @type {Object<string, string>} */
|
||||
|
|
|
|||
Loading…
Reference in New Issue