As c doesnt have the concept of pass by reference, and inturn the
existing c api uses pointers wrt llama chat message structure, so
switching to same wrt chat_tmpl_apply logics.
Also fix a oversight in previous commit and add the remaining logic.
Initial skeletons
Update existing logics to help with same. Also the inbetween helper
was having a bad signature wrt returning status and data, thats also
fixed.
While sending the current chat session along with new user query
to the model, many models expect that a tag be added at the end
to indicate that user is expecting the model to respond, this
flags allows for the same.
Add a c api wrapper for a single message tagging scenario.
Inturn to match convention followed by existing chat_apply_template
code, make it return the size expected of the tagged message string
buffer. Update internal single logic to help with same.
Explicitly check if tmpl specified is available in the loaded json
or not and then return a error if not found.
Fix a oversight wrt key name.
Add a alert in case if passed meta json file contains begin(BoS)
wrt assistant role, similar to check for end (EoS) wrt user role.
Bcas normally both (ie EoS wrt User and BoS wrt Assistant) shouldnt
be needed.
Update main wrt begin & prefix and suffix & end addition.
Wrt llama2
* add bos wrt llama2 system and user begins, but not assistant
* split system suffix into suffix and end, and add systemuser-system
flags so that end can be avoided wrt system+user message combo
* add eos wrt assistant end
* With these potentially this should work with main and server flows
Wrt llama3
* add empty begin, end fields and systemuser-system flags
* This should potentially work with main and server flows
Move helpers to the begining, so can avoid adding prototype
declerations/function signatures to the begining
Get the char * wrt string data in the c++ string.
Also fix a oversight wrt begin, when flag based begin adding control
was introduced.
NOTE: Currently system role suffix/end conditional adding always
triggered, if 1st system prompt seen or additional system prompt
is seen.
Now there is a simple and extended version of returning tagged
messages.
The extended version returns the tagged string, as well as the
details of the parts that make up that tagged message interms of
the type of parts and the lengths of the parts.
Now there is a simple and extended version of returning tagged
message wrt a single role and its content.
The extended version returns the tagged string, as well as the
details of the parts that make up that tagged message interms of
the type of parts and the lengths of the parts.
Dump shows user->begin.
chat-template-apply[-single] updated to work with begin and prefix
TODO: need to wrap begin in a try-catch, so that irrespective of
role, begin+prefix will work, irrespoective of whether that role
has a begin entry or not.
Was looking at the tokenized vector, and noticed that the EOS
mentioned by existing chat_apply_template of llama.cpp, is different
from what I noticed in tokenizer_config.json of deepseek llm, so
I have added two entries
* "deepseek-alt" which matches llama.cpp's chat_apply_template and
* "deepseek" which matches that in tokenizer_config.json.
This impacts the assistant suffix and reverse prompt entries.
CasOfThis: Need to look into other entries which I added previously
at a later time. However as the default logic should be picking the
EOS from model file, so I assume reverse-prompt being outofsync,
may not matter beyond a limit, potentially.
rename because they return value of specified key.
[main] update metaok to take template-id, so that one can cross
check that all needed entries are there wrt that template-id in
the chaton-meta-json file
HanishKVC