Users of recent_chat updated to work with ChatMessageEx
As part of same recent_chat_ns also added, for the case where the
array of chat messages can be passed as is ie in the chat mode,
provided it has only the network handshake representation of the
messages.
Simplify Add semantic by expecting any validation of stuff before
adding to be done by the callers of Add and not by add itself.
Also update it to expect ChatMessageEx object
Update all users of add to follow the new syntax and semantic.
Remove the old and ununsed AddSysPromptOnlyAtBegin helper
GetSystemLatest and its users updated wrt ChatMessageEx.
RecentChat updated wrt ChatMessageEx. Also now irrespective of
whether full history is being retrieved or only a subset, both
cases refer to the ChatMessageEx instances in SimpleChat.xchat
without creating new instances of anything.
Use the equivalent update_stream directly added to ChatMessageEx.
update_stream is also more generic to some extent and also directly
implemented by the ChatMessageEx class.
Rename ChatMessage to ChatMessageEx.
Add typedefs for NSToolCall and NSChatMessage, they represent the
way the corresponding data is structured in network hs.
Add logic to build the ChatMessageEx from data got over network in
streaming mode.
As the tool calling, if enabled, will need access to last few
user query and ai assistant responses (which will also include
in them the tool call requests and the corresponding results),
so that the model can build answers based on its tool call reqs
and got responses, and also given that most of the models these
days have sufficiently large context windows, so the sliding
window context implemented by SimpleChat logic has been increased
by default to include last 4 query and their responses roughlty.
Had forgotten to specify type as module wrt web worker, in order
to allow it to import the toolsconsole module.
Had forgotten to maintain the id of the timeout handler, which is
needed to clear/stop the timeout handler from triggering, if tool
call response is got well in time.
As I am currently reverting the console redirection at end of
handling a tool call code in the web worker message handler, I
need to setup the redirection each time. Also I had forgotten
to clear the console.log capture data space, before a new tool
call code is executed, this is also fixed by this change.
TODO: Need to abort the tool call code execution in the web worker
if possible in future, if the client / browser side times out
waiting for tool call response, ie if the tool call code is taking
up too much time.
tools manager/module
* setup the web worker that will help execute the tool call related
codes in a js environment that is isolated from the browsers main
js environment
* pass the web worker to the tool call providers, for them to use
* dont wait for the result from the tool call, as it will be got
later asynchronously through a message
* allow users of the tools manager to register a call back, which
will be called when ever a message is got from the web worker
containing response wrt previously requested tool call execution.
simplechat
* decouple toolcall response handling and toolcall requesting logic
* setup a timeout to take back control if tool call takes up too
much time. Inturn help alert the ai model, that the tool call
took up too much time and so was aborted, by placing a approriate
tagged tool response into user query area.
* register a call back that will be called when response is got
asynchronously wrt anye requested tool calls.
In turn take care of updating the user query area with response
got wrt the tool call, along with tool response tag around it.
These no longer need to worry about
* setting up the console.log related redirection to capture
the generated outputs, nor about
* setting up a dynamic function for executing the needed
tool call related code
The web worker setup to help run tool calls in a relatively
isolated environment independent of the main browser env,
takes care of these.
One needs to only worry about getting the handle to the
web worker to use and inturn pass the need code wrt the
tool call to it.
The request for code to run as well as the resultant response data
both need to follow a structured object convention, so that it is
easy to map a request and the corresponding response to some extent.
Try ensure as well as verify that original console.log is saved
and not overwritten. Throw an exception if things seem off wrt
same.
Also ensure to add a newline at end of console.log messages
Instead of automatically calling any requested tool by the GenAi
/ llm, that is from the tail end of the handle user submit btn
click,
Now if the GenAi/LLM has requested any tool to be called, then
enable the Tool Run related UI elements and fill them with the
tool name and tool args.
In turn the user can verify if they are ok with the tool being
called and the arguments being passed to it. Rather they can
even fix any errors in the tool usage like the arithmatic expr
to calculate that is being passed to simple_calculator or the
javascript code being passed to run_javascript_function_code
If user is ok with the tool call being requested, then trigger
the same.
The results if any will be automatically placed into the user
query text area.
User can cross verify if they are ok with the result and or
modify it suitabley if required and inturn submit the same to
the GenAi/LLM.
Instead of automatically calling the requested tool with supplied
arguments, rather allow user to verify things before triggering the
tool.
NOTE: User already provided control over tool_response before
submitting it to the ai assistant.
So that when tool handler writes the result to the tc_switch, it
can make use of the same, to write to the right location.
NOTE: This also fixes the issue with I forgetting to rename the
key in js_run wrt writing of result.
Move tool calling logic into tools module.
Try trap async promise failures by awaiting results of tool calling
and putting full thing in an outer try catch. Have forgotten the
nitty gritties of JS flow, this might help, need to check.
As output generated by any tool/function call is currently placed
into the TextArea provided for End user (for their queries), bcas
the GenAi (engine/LLM) may be expecting the tool response to be
sent as a user role data with tool_response tag surrounding the
results from the tool call. So also now at the end of submit btn
click handling, the end user input text area is not cleared, if
there was a tool call handled, for above reasons.
Also given that running a simple arithmatic expression in itself
doesnt generate any output, so wrap them in a console.log, to
help capture the result using the console.log trapping flow that
is already setup.
Checks for toolname to be defined or not in the GenAi's response
If toolname is set, then check if a corresponding tool/func exists,
and if so call the same by passing it the GenAi provided toolargs
as a object.
Inturn the text generated by the tool/func is captured and put
into the user input entry text box, with tool_response tag around
it.
The implementations of javascript and simple_calculator now use
provided helpers to trap console.log messages when they execute
the code / expression provided by GenAi and inturn store the
captured log messages in the newly added result key in tc_switch
This should help trap the output generated by the provided code
or expression as the case maybe and inturn return the same to the
GenAi, for its further processing.
Previously if content was empty, it would have always sent the
toolcall info related version even if there was no toolcall info
in it. Fixed now to return empty string, if both content and
toolname are empty.
As there could be failure wrt getting the response from the ai
server some where in between a long response spread over multiple
parts, the logic uses the latestResponse to cache the response
as it is being received. However once the full response is got,
one needs to transfer it to a new instance of AssistantResponse
class, so that latestResponse can be cleared, while the new
instance can be used in other locations in the flow as needed.
Achieve the same now.
Switch oneshot handler to use AssistantResponse, inturn currenlty
only handle the normal content in the response.
TODO: If any tool_calls in the oneshot response, it is currently
not handled.
Inturn switch the generic/toplevel handle response logic to use
AssistantResponse class, given that both oneshot and the
multipart/streaming flows use/return it.
Inturn add trimmedContent member to AssistantResponse class and
make the generic handle response logic to save the trimmed content
into this. Update users of trimmed to work with this structure.
Make latestResponse into a new class based type instance wrt
ai assistant response, which is what it represents.
Move clearing, appending fields' values and getting assistant's
response info (irrespective of a content or toolcall response)
into this new class and inturn use the same.
I was wrongly checking for finish_reason to be non null, before
trying to extract the genai content/toolcalls, have fixed this
oversight with the new flow in progress.
I had added few debug logs to identify the above issue, need to
remove them later. Note: given that debug logs are disabled by
replacing the debug function during this program's initialisation,
which I had forgotten about, I didnt get the debug messages and
had to scratch my head a bit, before realising this and the other
issue ;)
Also either when I had originally implemented simplechat 1+ years
back, or later due to changes on the server end, the streaming
flow sends a initial null wrt the content, where it only sets the
role. This was not handled in my flow on the client side, so a
null was getting prepended to the chat messages/responses from the
server. This has been fixed now in the new generic flow.
Update response_extract_stream to check for which field is being
currently streamed ie is it normal content or tool call func name
or tool call func args and then return the field name and extracted
value.
Previously it was always assumed that only normal content will be
returned.
Currently it is assumed that the server will only stream one of the
3 supported fields at any time and not more than one of them at the
same time.
TODO: Have to also add logic to extract the reasoning field later,
ie wrt gen ai models which give out their thinking.
Have updated append_response to expect both the key and the value
wrt the latestResponse object, which it will be manipualted.
Previously it was always assumed that content is what will be got
and inturn appended.
Changed latestResponse type to an object instead of a string.
Inturn it contains entries for content, toolname and toolargs.
Added a custom clear logic due to the same and used it to replace
the previously simple assigning of empty string to latestResponse.
For now in all places where latestReponse is used, I have replaced
with latestReponse.content.
Next need to handle identifying the field being streamed and inturn
append to it. Also need to add logic to call tool, when tool_call
triggered by genai.
Define the meta that needs to be passed to the GenAi Engine.
Define the logic that implements the tool call, if called.
Implement the flow/structure such that a single tool calls
implementation file can define multiple tool calls.
Enable streaming by default, to check the handshake before going
on to change the code, given that havent looked into this for more
than a year now and have been busy with totally different stuff.
Also updated the user messages used for testing a bit
hanishkvc