model-conversion : add token ids to prompt token output [no ci] (#17863)

This commit adds the token ids to the printed prompt outputs.

The motivation for this is that is can be useful to see the actual token
ids alongside the token strings for debugging.
This commit is contained in:
Daniel Bevenius 2025-12-08 17:13:08 +01:00 committed by GitHub
parent 951520ddb0
commit 2fa51c19b0
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions

View File

@ -144,7 +144,7 @@ int main(int argc, char ** argv) {
return 1;
}
std::string s(buf, n);
printf("%s", s.c_str());
printf("%s (%d)", s.c_str(), id);
}
printf("\n");