2 Commits

Author	SHA1	Message	Date
Luca Versari	9c3f969405	Implement the Griffin model. ... Also implement support for some model variations: - Local attention. - Add support for biases. - Use RoPE only on half vectors. - Support different order of QKV weights. Co-authored-by: Andrey Mikhaylov <amik@google.com> Co-authored-by: Martin Bruse <zondolfin@gmail.com> Co-authored-by: Zoltan Szabadka <szabadka@google.com>	2024-04-08 21:45:54 +02:00
Zoltan Szabadka	b670d43e4f	Add standalone tool to compress weights. ... Co-authored-by: Eugene Kliuchnikov <eustas@google.com>	2024-04-03 14:54:08 +00:00