gemma.cpp

Commit Graph

Author	SHA1	Message	Date
Zoltan Szabadka	df01700b54	Move the backpropagation code to its own directory	2024-06-04 10:20:16 +00:00
Zoltan Szabadka	36e4d8bbfe	Add first version of backpropagation support. This is still in progress / experimental, currently it is only implemented for normal gemma MQA attention layers, and no parallelism is added yet for backward pass. Since we need to remember all activations from all layers, the forward pass was also reimplemented with a new activation data structure.	2024-06-04 08:37:49 +00:00

Author

SHA1

Message

Date

Zoltan Szabadka

df01700b54

Move the backpropagation code to its own directory

2024-06-04 10:20:16 +00:00

Zoltan Szabadka

36e4d8bbfe

Add first version of backpropagation support.

This is still in progress / experimental, currently it is only
implemented for normal gemma MQA attention layers, and no
parallelism is added yet for backward pass.

Since we need to remember all activations from all layers, the
forward pass was also reimplemented with a new activation data
structure.

2024-06-04 08:37:49 +00:00

2 Commits