happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

SimpleChatTCRV:Markdown:List items without list marker cleanup 

If lines immidately follows a list item, without the list marker
at their begining, but with a offset matching the list item, then
these lines will be appended to that list item.

If a empty line is there between a list item and a new line with
some content, but without a list marker

* if the content offset is less than the last list item, then
  unwind the lists before such a line.

* if the content offset is larger than the last list item, then
  the line will be added as a new list item at the same level
  as the last list item.

* if the content offset is same as the last list tiem, then
  unwind the list by one level and then insert this line as a
  new list item at this new unwound level.
This commit is contained in:
hanishkvc 2025-11-27 14:46:04 +05:30
parent 82d436b537
commit 6358a2083d
2 changed files with 38 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
tools/server/public_simplechat/docs/changelog.md
View File

@ -303,6 +303,7 @@ Chat Session specific settings
* A simple minded basic Markdown to Html logic with support for below to some extent
* headings, horiz line,
* lists (ordered, unordered, intermixed at diff leves)
accomodate lines without list markers inbetween list items to some extent, hopefully in a sane way.
* tables, fenced code blocks, blockquotes
* Rename fetch_web_url_raw to fetch_url_raw, avoids confusion and matchs semantic of access to local and web.
@ -342,7 +343,3 @@ in turn using openai file - file-data type sub block within content array or so
See why the ai streamed response not showing up in TCExternalAi chat session ui, even thou the content is getting
appended to its DivStream. IE why it is hidden.
Markdown if a line which doesnt have any list marker appears at the same offset level as the last list item,
that too after a new line before this ambiguous line, then maybe pop out 1 level wrt the list.

47
tools/server/public_simplechat/typemd.mjs
View File

@ -130,14 +130,24 @@ export class MarkDown {
/**
* Process list one line at a time.
* * Account for ordered lists as well as unordered lists, including intermixing of the lists.
* at different list hierarchy levels.
* * Allow a list item line to be split into multiple lines provided the split lines retain
* the same or more line offset compared to the starting line of the item to which they belong.
* * if there is a empty line in between, then the new line will be treated as a new item.
* * allows for empty lines inbetween items.
* * currently there is no limit on the number of empty lines.
* but may bring in a limit later.
*
* Account for ordered lists as well as unordered lists, including intermixing of the lists.
* * inturn at different list hierarchy levels.
*
* Allow a list item line to be split into multiple lines provided the split lines retain
* the same or more line offset compared to the starting line of the item to which they belong.
* * these following split lines wont have the list marker in front of them.
*
* Allows for empty lines inbetween items (ie lines with list marker)
* * currently there is no limit on the number of empty lines, but may bring in a limit later.
*
* If empty line between a list item and new line with some content, but without a list marker
* * if content offset less than last list item, then unwind the lists before such a line.
* * if content offset larger than last list item, then line will be added as new list item
* at the same level as the last list item.
* * if content offset same as last list item, then unwind list by one level and insert line
* as a new list item at this new unwound level.
*
* @param {string} line
*/
process_list(line) {
@ -168,6 +178,7 @@ export class MarkDown {
return true
} else {
if (this.in.list.offsets.length > 0) {
if (emptyTracker.cur > 0) {
// skip empty line
return true
@ -180,8 +191,24 @@ export class MarkDown {
if (matchOffset[1].length < lastOffset) {
return false
}
this.extend_else_appendnew(matchOffset[2], "</li>\n", '<li>', emptyTracker)
return true
if (emptyTracker.prev == 0) {
if (this.html.endsWith("</li>\n")) {
this.extend(matchOffset[2], "</li>\n")
return true
}
} else {
if (matchOffset[1].length > lastOffset) {
this.appendnew(matchOffset[2], "<li>", "</li>\n")
return true
}
let uw = this.unwind_list(lastOffset-1)
if (uw.remaining > 0) {
this.appendnew(matchOffset[2], "<li>", "</li>\n")
return true
}
}
return false
}
}
return false