Table of Contents

Here are some open tasks. If you'd like to pick up a task, open an issue (if the issue doesn't exist, if the issue does exist add a comment that you'll be working on it) and/or join discord here https://discord.gg/H5jCBAWxAe for discussion.

Incremental Library Improvements

MQA Implementation for 2B models (WIP at https://github.com/google/gemma.cpp/pull/114)
Refactor terminal repl code out of run.cc into a reusable header
Toy example of an alternative model in examples/

Large Refactors

Pub/Sub message queue as interface between application and runtime for I/O, introspection
Error handling as return values so that applications can handle failure states in their own ways.
Move model configuration/selection to runtime
Add GPU backend (webgpu?)

Google Workflow Items

Incorporate copybara:end into copybara transformations

Features / Research

%save / %load KV cache (see comment https://github.com/google/gemma.cpp/issues/60#issuecomment-1971268148)
self-extend capability https://github.com/google/gemma.cpp/issues/60
self-consistency decoding https://arxiv.org/abs/2203.11171
start work towards webgpu backend
FTXUI based tui