So that no need to explicitly specify <int64_t> or LL wrt int
literals, which dont need 64bit space by default.
Which also means one shouldnt/cant mix up type of value stored and
default type specified when getting.
Rename all the log messages to have GKV and not SC.
The log messages in get_vector made conditional to GKV_DEBUG, this
was missed out earlier in simpcfg itself.
This should pave way for having a default chat templates dataset
in the code, without needing to load it from a config file, if
one doesnt want to.
TODO: allow for loading config from json into simpcfg, so that
a program which uses llama.cpp can decide, whether it is ok with
what is already there in the internal dataset, or allow for loading
template info at runtime using the simpcfg's simple text file or
additionally include the json code to load template info at runtime
from json file.
It appears like std::format is not supported in older g++/lib still
in wide use like current debian stable, so avoiding same wrt direct
library use.
Allow for empty VAARGS
NOTE: However test program mode of the same uses cout and format
Have merged master branch has of 20240510IST12XY with chaton_v3
branch.
As part of same had to update the flow in examples/main/main.cpp
wrt conversion related commit in master branch and my chaton related
commits in this branch.
If the tagged message will be of 0 length, ensure that the passed
dest char* array, has null inserted appropriately.
Check that user has passed a non-null pNumParts.
Dont hard code int32_t size, pick using sizeof
so that the size of the elements is explicit and fixed, so that
it is inturn in sync with the fixed int size specified wrt the
c-api, even with any c compilers with different idea about int.
avoid some ununsed vars, need to update compile flags later to
enable corresponding warnings.
With this and past few commits, now there is simple yet sufficient
support to help move multi-level-hierarchy config files into the
SimpCfg's simple physically 1-level, but if reqd logically multi
level hierarchy flow.
B4 this series of commits also one could have still achieved this,
but there would have been bit more effort needed.
Use the commonality between Indian languages to show mixup issue
with the simple minded trim_dump logic and how trim_oversmart
could potentially avoid that.
Given that I am using valid strings to show the pitfalls of fixed
native char size driven logic, so no need to keep the dump and
oversmart flows seperate, so merge into a common loop.
Update the notes to match the templated flow now and some of the
nitty gritties involved.
Update DumpHexString to be templated.
Split check nonenglish flow wrt trim dumb and oversmart testing,
so that things with work with one, but not the other can be
differentiated in the flow.
The constructor method doesnt convert wstring to string, when it
involves non-english chars which will encode to multibyte chars
in utf8. even thou it does work for the already utf8 u8string.
wcstombs doesnt seem to work for non english chars, when the
locale is set to the default c, need to change to something like
en_US.UTF-8, to allow it to do the conversion properly.
Seperate out the checks wrt different string types.
Add a wstring_basic, which verifies that wstring iterator handles
non english chars propery or atleast better.
Without using imbue, I couldnt get non-english wstrings to print
on mac. Need to check on linux also.
Also avoid the uint8_t typecasting, given that wchar isnt 8bit
HanishKVC
Johannes Gäßler
Dawid Potocki