happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

SimpleChatTC: twins wrt streamed response handling 

As there could be failure wrt getting the response from the ai
server some where in between a long response spread over multiple
 parts, the logic uses the latestResponse to cache the response
as it is being received. However once the full response is got,
one needs to transfer it to a new instance of AssistantResponse
class, so that latestResponse can be cleared, while the new
instance can be used in other locations in the flow as needed.

Achieve the same now.
This commit is contained in:
hanishkvc 2025-10-11 04:03:53 +05:30
parent 53f85d09be
commit 383c19c99b
1 changed files with 11 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
tools/server/public_simplechat/simplechat.js
View File

@ -39,8 +39,16 @@ class ApiEP {
class AssistantResponse { class AssistantResponse {
constructor() { constructor(content="", toolname="", toolargs="", trimmedContent="") {
this.response = { content: "", toolname: "", toolargs: "", trimmedContent: "" }; this.response = { content: content, toolname: toolname, toolargs: toolargs, trimmedContent: trimmedContent };
}
/**
* Create a new instance from an existing instance
* @param {AssistantResponse} old
*/
static newFrom(old) {
return new AssistantResponse(old.response.content, old.response.toolname, old.response.toolargs, old.response.trimmedContent)
} }
clear() { clear() {
@ -466,7 +474,7 @@ class SimpleChat {
} }
} }
console.debug("DBUG:SC:PART:Full:", this.latestResponse.content_equiv()); console.debug("DBUG:SC:PART:Full:", this.latestResponse.content_equiv());
return this.latestResponse; return AssistantResponse.newFrom(this.latestResponse);
} }
/** /**