happyz
/
llama.cpp
mirror of https://github.com/ggerganov/llama.cpp.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
llama.cpp/gguf-py
History
Aleksei Nikiforov d82b7a7c1d
gguf-py : fix passing non-native endian tensors (editor-gui and new-metadata) (#17553) 
gguf_new_metadata.py reads data from reader.
Reader doesn't byteswap tensors to native endianness.
But writer does expect tensors in native endianness to convert them
into requested endianness.

There are two ways to fix this: update reader and do conversion to native endianness and back,
or skip converting endianness in writer in this particular USE-case.

gguf_editor_gui.py doesn't allow editing or viewing tensor data.
Let's go with skipping excessive byteswapping.

If eventually capability to view or edit tensor data is added,
tensor data should be instead byteswapped when reading it.
 		2025-11-28 20:53:01 +01:00
..
examples Refactor gguf scripts to improve metadata handling (#11909) 2025-02-26 08:04:48 -05:00
gguf gguf-py : fix passing non-native endian tensors (editor-gui and new-metadata) (#17553) 2025-11-28 20:53:01 +01:00
tests gguf-py : add Numpy MXFP4 de/quantization support (#15111) 2025-08-08 17:48:26 -04:00
LICENSE gguf : make gguf pip-installable 2023-08-25 09:26:05 +03:00
README.md gguf-py : GGUF Editor GUI - Python + Qt6 (#12930) 2025-04-18 20:30:41 +02:00
pyproject.toml gguf-py : make sentencepiece optional (#14200) 2025-06-19 15:56:12 +02:00

README.md

gguf

This is a Python package for writing binary files in the GGUF (GGML Universal File) format.

See convert_hf_to_gguf.py as an example for its usage.

Installation

pip install gguf

Optionally, you can install gguf with the extra 'gui' to enable the visual GGUF editor.

pip install gguf[gui]

API Examples/Simple Tools

examples/writer.py — Generates example.gguf in the current directory to demonstrate generating a GGUF file. Note that this file cannot be used as a model.

examples/reader.py — Extracts and displays key-value pairs and tensor details from a GGUF file in a readable format.

gguf/scripts/gguf_dump.py — Dumps a GGUF file's metadata to the console.

gguf/scripts/gguf_set_metadata.py — Allows changing simple metadata values in a GGUF file by key.

gguf/scripts/gguf_convert_endian.py — Allows converting the endianness of GGUF files.

gguf/scripts/gguf_new_metadata.py — Copies a GGUF file with added/modified/removed metadata values.

gguf/scripts/gguf_editor_gui.py — Allows for viewing, editing, adding, or removing metadata values within a GGUF file as well as viewing its tensors with a Qt interface.

Development

Maintainers who participate in development of this package are advised to install it in editable mode:

cd /path/to/llama.cpp/gguf-py

pip install --editable .

Note: This may require to upgrade your Pip installation, with a message saying that editable installation currently requires setup.py. In this case, upgrade Pip to the latest:

pip install --upgrade pip

Automatic publishing with CI

There's a GitHub workflow to make a release automatically upon creation of tags in a specified format.

  1. Bump the version in pyproject.toml.
  2. Create a tag named gguf-vx.x.x where x.x.x is the semantic version number.
git tag -a gguf-v1.0.0 -m "Version 1.0 release"
  1. Push the tags.
git push origin --tags

Manual publishing

If you want to publish the package manually for any reason, you need to have twine and build installed:

pip install build twine

Then, follow these steps to release a new version:

  1. Bump the version in pyproject.toml.
  2. Build the package:
python -m build
  1. Upload the generated distribution archives:
python -m twine upload dist/*

Run Unit Tests

From root of this repository you can run this command to run all the unit tests

python -m unittest discover ./gguf-py -v

TODO

  • Include conversion scripts as command line entry points in this package.