C++17 provides a good enough variant as a standard feature, and
chaton uses the same at its core, instead of rolling out its own
struct of union based variant. And given that currently chaton
is part of common library and not the base llama library, so limit
the use of c++17 to common library. Initially while experimenting,
had set the flag for full llama, limitting it for now.
Also by now most embedded targets should be potentially having c++
compilers and libraries with support for c++17 features. So chances
are it is a ok enough path to take.
Rename chaton-meta hpp to cpp and include this cpp file which brings
in the compile time built-in global chaton configurable template data
into the common library, and avoid the nop hpp file references.
Update chaton.hpp to not include the meta-cpp, instead just make a
reference to the global ChatTemplates instance, so that the hpp can
be used as a header file proper.
Avoid pragma once in the chaton-meta.cpp, including the script, which
helps create it.
Any program which wants to use json file to update/extend the
chaton's configurable template data, can include this new file
chaton_json.hpp, to get the reqd functionality.
Update chaton_meta_ok, _chaton_meta_validate_dump and
chaton_meta_load_json to either work with a passed ChatTemplates
instance, or fallback to the compiled-in global instance of same.
The initial version was rooted around a json object, while the new
version is rooted around a MapOfMapOfVariant (GroupKV), which could
be preloaded with chat templates info at compile time itself and
used as is. Or optionally one could allow the configurable template
data to be extended/updated at runtime from a text(/SimpCfg)/json
file.
GroupKV dump adds needed ":" seperator on its own, so calling
functions can just pass the tag string they want in the log without
worrying about any demarkation.
Given that now the multi chat templating logic itself is used to
apply chat templating/tagging to a single chat message, so give
flexibility of deciding whether global tags if any should be
applied or not wrt the core tagging logic.
examples/main inturn updated to not apply global tags if any wrt
the system message. Also the user messages already dont apply
global tags if any, as its currently implemented to build on the
existing in-prefix/suffix and anitprompt flow.
To avoid having to duplicate any hardcoding in future, wrt any new
model/chat-template-standard, at multiple locations, remove the
single message templating code with a wrapper which does the same
but using the multi-msg templating helper.
Reuse the code already moved into GroupKV
Add explicit get and set wrt int32_t, which was added after move
to GroupKV wrt basic MapOfMapOfVariant logic.
The json library generates less informative exception message,
which doesnt help one identify which key is missing, so switch to
the new json_get_str helper added in the last commit. It generates
more informative exception message.
Also has dump was using get_value calls with fallback to default,
so it wasnt identifying the missed field.
Have fixed both of those. Also reconverted meta json file.
Misc: interesting avesham and aattam
This should allow for using this generic chat templating code flow
along with the included chat template data, without needing to
load any json file at runtime.
However If user wants to change the already included chat template
data, or add new chat template standard/model related data, one can
explicitly load json file.
TODO: Need to cross check this flow once, but logically should work
WIP:NOTE:
Initial go converting from json driven flow to ChatTemplatesGroupKV
related flow done. Needs to be tested.
A optional helper added to load ChatTemplates from a specified
json file.
Need to add a compile time initialized MapOfMapOfVariants wrt
the chat template details of models/standards already known
to the program. So that one can use the llama.cpp and this new
chat template logic, even without json dependency, if one doesnt
want to.
Manually iterate the json object items using begin-end explicitly,
because the implicit iteration for loop related helpers for the
used json lib gives only the values and not a key-value pair.
So that no need to explicitly specify <int64_t> or LL wrt int
literals, which dont need 64bit space by default.
Which also means one shouldnt/cant mix up type of value stored and
default type specified when getting.
HanishKVC
Radoslav Gerganov
Johannes Gäßler