gemma.cpp

Commit Graph

Select branches

Hide Pull Requests

dev

main

test_648971168

test_683370269

test_824820179

test_836654012

test_840724686

test_841765739

test_875649021

test_882132164

test_897620425

test_904484786

#102

#105

#107

#108

#109

#110

#111

#113

#114

#115

#116

#117

#118

#119

#120

#122

#123

#124

#125

#126

#127

#128

#129

#130

#131

#132

#133

#136

#137

#138

#139

#14

#140

#141

#142

#144

#145

#146

#147

#148

#149

#150

#151

#154

#155

#156

#157

#157

#158

#159

#160

#162

#163

#165

#166

#167

#168

#168

#169

#170

#172

#173

#174

#175

#176

#177

#178

#179

#180

#181

#182

#183

#184

#186

#187

#188

#189

#190

#191

#192

#194

#195

#199

#200

#201

#202

#203

#204

#205

#206

#207

#208

#209

#210

#211

#212

#213

#214

#215

#216

#217

#218

#219

#22

#220

#222

#223

#224

#225

#226

#227

#228

#229

#230

#231

#232

#233

#234

#235

#236

#237

#238

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#249

#250

#250

#251

#253

#254

#255

#256

#257

#258

#259

#26

#260

#261

#262

#263

#264

#265

#266

#267

#268

#269

#270

#271

#272

#273

#274

#275

#277

#279

#280

#281

#282

#284

#286

#287

#287

#288

#289

#290

#291

#292

#293

#294

#295

#296

#297

#298

#299

#3

#300

#302

#303

#304

#305

#306

#307

#308

#309

#310

#311

#312

#313

#314

#315

#316

#317

#319

#32

#320

#321

#322

#323

#324

#325

#326

#327

#328

#329

#33

#330

#331

#333

#334

#335

#336

#337

#338

#339

#34

#342

#343

#344

#345

#346

#347

#348

#349

#35

#350

#351

#352

#353

#354

#355

#356

#357

#358

#359

#36

#360

#361

#362

#363

#364

#366

#367

#368

#369

#371

#372

#374

#375

#376

#377

#378

#379

#38

#380

#381

#382

#383

#386

#387

#388

#389

#390

#391

#392

#393

#394

#395

#396

#397

#398

#399

#40

#400

#402

#403

#404

#405

#406

#407

#408

#409

#41

#410

#411

#412

#413

#413

#414

#415

#416

#417

#418

#419

#42

#420

#421

#422

#424

#425

#426

#427

#428

#430

#431

#431

#432

#433

#434

#435

#436

#437

#438

#439

#440

#441

#442

#443

#444

#445

#446

#447

#448

#449

#450

#451

#452

#453

#454

#455

#456

#457

#458

#459

#460

#461

#462

#463

#464

#465

#466

#467

#468

#469

#47

#470

#471

#472

#473

#474

#475

#476

#477

#478

#479

#480

#481

#482

#483

#484

#485

#486

#487

#488

#489

#490

#491

#492

#493

#494

#495

#496

#497

#498

#499

#500

#502

#503

#504

#505

#506

#507

#509

#510

#511

#512

#513

#514

#515

#516

#517

#519

#520

#521

#523

#524

#525

#526

#527

#528

#529

#53

#530

#532

#534

#535

#536

#537

#538

#539

#541

#542

#545

#546

#547

#548

#549

#55

#550

#552

#553

#554

#555

#556

#557

#558

#559

#56

#561

#562

#563

#564

#565

#566

#567

#569

#570

#571

#572

#573

#574

#575

#576

#579

#58

#580

#581

#583

#584

#585

#586

#587

#588

#589

#590

#591

#592

#593

#594

#595

#596

#597

#598

#599

#6

#600

#601

#602

#603

#604

#605

#606

#607

#609

#609

#61

#610

#610

#611

#612

#613

#614

#615

#616

#617

#618

#619

#620

#621

#622

#624

#624

#626

#627

#628

#629

#63

#630

#633

#634

#635

#636

#637

#637

#638

#639

#640

#641

#642

#643

#644

#645

#646

#647

#649

#65

#650

#651

#652

#653

#654

#655

#656

#657

#658

#659

#66

#660

#663

#664

#666

#667

#668

#669

#67

#670

#671

#672

#673

#674

#675

#676

#677

#678

#679

#68

#680

#682

#683

#684

#685

#686

#687

#689

#69

#690

#691

#692

#693

#694

#695

#696

#697

#698

#699

#700

#701

#702

#703

#704

#705

#706

#707

#708

#709

#71

#710

#711

#712

#713

#714

#715

#716

#717

#718

#719

#720

#721

#722

#723

#724

#725

#727

#728

#729

#730

#731

#732

#733

#734

#735

#736

#737

#738

#739

#74

#740

#741

#742

#743

#744

#745

#746

#747

#747

#748

#749

#75

#750

#751

#752

#753

#754

#755

#756

#757

#759

#76

#760

#761

#763

#764

#765

#766

#767

#768

#769

#77

#770

#770

#771

#772

#773

#774

#775

#776

#777

#779

#78

#780

#781

#781

#782

#783

#784

#785

#786

#787

#788

#789

#79

#790

#791

#792

#793

#794

#794

#795

#796

#797

#798

#799

#800

#801

#802

#803

#804

#805

#806

#807

#808

#809

#81

#810

#811

#812

#813

#814

#815

#816

#818

#819

#82

#820

#821

#822

#823

#824

#825

#827

#828

#829

#83

#830

#831

#832

#833

#834

#835

#836

#837

#838

#839

#840

#841

#842

#843

#844

#845

#846

#847

#847

#848

#849

#85

#850

#851

#852

#854

#855

#856

#857

#858

#859

#86

#86

#860

#861

#862

#863

#864

#865

#865

#866

#867

#868

#868

#869

#87

#870

#871

#872

#873

#874

#876

#877

#878

#879

#880

#881

#882

#883

#884

#886

#889

#890

#891

#892

#893

#894

#895

#896

#897

#899

#899

#9

#900

#900

#92

#93

#94

#95

#96

#97

#98

v0.1.0

v0.1.1

v0.1.2

v0.1.3

v0.1.4

fcf5c1af88 Merge pull request #114 from ufownl:experimental Copybara-Service 2024-03-22 05:36:07 -0700
b9829ccc3c revert hello_world gemma git tag weedge 2024-03-22 10:40:12 +0800
2bdcf6f426 remove const argv weedge 2024-03-22 10:36:56 +0800
61e031fe98 Towards building tests without GUnit Refs #29 Jan Wassenberg 2024-03-21 19:26:27 -0700
24add61dd9 Fix SFP/NUQ for bf16 rounding in Highway SFP: Avoid rounding twice, and more robust TestDot. NUQ: also more robust SNR, minor touchups to header. Jan Wassenberg 2024-03-21 19:05:44 -0700
9c66eca963 change main use char** argv weedge 2024-03-21 21:26:20 +0800
9b7804c75b change main use char** argv weedge 2024-03-21 21:22:25 +0800
7165806216 change example helloworld gemma.cpp git tag weedge 2024-03-21 19:56:53 +0800
73db33580b fix: change hello_world cmake compiler use -std=c++17 && argv add const weedge 2024-03-21 19:50:50 +0800
90b0e9fd7a Refactor the implementation of `Attention` RangerUFO 2024-03-21 14:40:56 +0800
a135bc1e47 Fix build for RPi, missing hn::. Refs #112, thanks long568 Jan Wassenberg 2024-03-21 04:07:12 +0100
ba86c8d590 Remove obsolete copybara tags, faster bazel builds (debug) Jan Wassenberg 2024-03-20 19:31:23 +0100
f8baac80f9 Fix msan error, uninitialized model_training Jan Wassenberg 2024-03-20 05:12:06 +0100
52940d435f Connect "--weights" parameter to Gemma Eric Ye 2024-03-20 00:07:47 +0100
89be4c3de8 No public description Eric Ye 2024-03-19 23:35:58 +0100
30b8a3c1ac Fix build for RPi, missing hn::. Refs #112, thanks long568 Jan Wassenberg 2024-03-20 20:07:12 -0700
546519c855

Added a missing space in app.h Ville Sundell 2024-03-21 00:39:45 +0200
06cea2bcdb Remove obsolete copybara tags, faster bazel builds (debug) Jan Wassenberg 2024-03-20 19:31:23 +0100
edaafe335f Fix msan error, uninitialized model_training Jan Wassenberg 2024-03-20 05:12:06 +0100
e2a04b79ed Connect "--weights" parameter to Gemma Eric Ye 2024-03-20 00:07:47 +0100
ffd02c59ad No public description Eric Ye 2024-03-19 23:35:58 +0100
7d5364bb80 Remove obsolete copybara tags, faster bazel builds (debug) Jan Wassenberg 2024-03-20 11:31:23 -0700
b20e490dc3 Remove obsolete copybara tags, faster bazel builds (debug) Jan Wassenberg 2024-03-19 21:30:10 -0700
8fc6959950 Move conditional branch out of `pos2` loop RangerUFO 2024-03-20 23:50:14 +0800
c75d2eb635 Add the missing `HWY_ATTR` of `ProjKV` RangerUFO 2024-03-20 23:21:43 +0800
ce32f4db81 Streamline the implementation RangerUFO 2024-03-20 22:39:31 +0800
11d9c51473 Fix msan error, uninitialized model_training Jan Wassenberg 2024-03-20 05:12:06 +0100
6865819bb7 Connect "--weights" parameter to Gemma Eric Ye 2024-03-20 00:07:47 +0100
fdc3812446 No public description Eric Ye 2024-03-19 23:35:58 +0100
6923aec853 Add MQA support RangerUFO 2024-03-20 18:14:09 +0800
130e1f678f Adjust vocab size to be the same as gemma_pytorch RangerUFO 2024-03-19 22:00:52 +0800
5e0cafbdc2 Fix msan error, uninitialized model_training Jan Wassenberg 2024-03-19 21:12:06 -0700
fdb1091b9c Connect "--weights" parameter to Gemma Eric Ye 2024-03-19 16:07:47 -0700
4400842337 Minor refactor in Softmax enum-class 2024-03-20 00:20:14 +0800
858d5b08c2 Use highway in AddFrom, MulBy, MulByConst, MulByConstAndAdd, create_distribution enum-class 2024-03-19 08:38:09 +0800
720f609d84 Merge pull request #102 from google:experimental Copybara-Service 2024-03-18 10:56:52 -0700
810b5a0cc2 Update README with more details on contributing code, add experimental/ directory, add READMEs for subdirectories, clean up DEVELOPER notes austinvhuang 2024-03-15 14:10:24 -0400
8fb44ed6dd Merge pull request #97 from ufownl:feature/topk_flag v0.1.1 Copybara-Service 2024-03-14 07:34:55 -0700
a2ef389897 Merge pull request #98 from zeerd:patch-2 Copybara-Service 2024-03-14 07:21:23 -0700
46c1aca304

Add missing log that point to a failed Generation Charles Chan 2024-03-14 10:03:25 +0800
1408784d7b Remove unused ascii banner string Eric Ye 2024-03-13 12:21:06 -0700
a0f316d853 Merge pull request #95 from google:conversion Copybara-Service 2024-03-13 09:37:36 -0700
f520e5c25c

Remove WIP messages. pculliton 2024-03-13 11:36:19 -0400
83ec42954f Allow changing k parameter of `SampleTopK` as a compiler flag RangerUFO 2024-03-13 13:55:37 +0800
e0b912fc46

Merge branch 'google:main' into main Sascha Ronnie Daoudia 2024-03-13 05:12:16 +0100
5fa2eb1a86 Use bf16-rounded sqrt for scaling embeddings to match Gemma Thanks Daniel & Michael Han for pointing this out. https://unsloth.ai/blog/gemma-bugs Jan Wassenberg 2024-03-12 19:14:31 -0700
0221956b2e Merge pull request #87 from google:refactor-tidy Copybara-Service 2024-03-12 16:10:47 -0700
b6831a2256 Fixed 7B conversion. Phil Culliton 2024-03-12 21:12:28 +0000
72247614bb fix prefill feedback off-by-1, update fetch commit hash austinvhuang 2024-03-12 15:10:44 -0400
4aa8d0584e Merge branch 'dev' into refactor-tidy austinvhuang 2024-03-12 15:01:46 -0400
a9aa63fd2e Fix bazel build attempt #2 (remove //base through automation change). https://github.com/google/gemma.cpp/actions/runs/8252325981/job/22571549312 Austin Huang 2024-03-12 10:42:50 -0700
9345b0aed5 Fix bazel build failure: https://github.com/google/gemma.cpp/actions/runs/8252325981/job/22571549312 Austin Huang 2024-03-12 09:40:34 -0700
ccd055e06b Merge pull request #82 from google:examples Copybara-Service 2024-03-12 09:24:24 -0700
0d406061c0 Detect and print build type. Refs #88 Jan Wassenberg 2024-03-11 21:57:35 -0700
a9be065c28 Merge pull request #77 from enum-class:rmsnorm Copybara-Service 2024-03-11 06:25:57 -0700
6712f07ee7 Merge branch 'dev' into rmsnorm enum-class 2024-03-11 20:13:18 +0800
683c1b5c32

Merge branch 'google:main' into rmsnorm Saman 2024-03-11 20:01:16 +0800
60d054e041 move arg definitions out of gemma.h to app.h austinvhuang 2024-03-10 23:49:25 -0400
e577198fc0 Merge pull request #78 from enum-class:rmsnorm2 Copybara-Service 2024-03-10 16:14:44 -0700
415464b047 fix CMakeLists typo austinvhuang 2024-03-10 15:41:17 -0400
5d323c00fe fix tokenizer scope austinvhuang 2024-03-10 13:23:16 -0400
0fc80fad05 libgemma refactor - review changes austinvhuang 2024-03-10 12:55:08 -0400
0e1aefdac1

Merge branch 'google:main' into main Sascha Ronnie Daoudia 2024-03-10 11:59:40 +0100
cc5c24c4f8 remove app.h dependency + fix bazel build austinvhuang 2024-03-08 18:06:43 -0500
8c7b2cf61b add README, license to hello_world austinvhuang 2024-03-08 17:59:54 -0500
571a5449c4 update commit hash for gemma lib austinvhuang 2024-03-08 17:33:33 -0500
03147effbd update loader arg names: cache -> compressed_weights, model -> weights austinvhuang 2024-03-08 17:32:36 -0500
dfd2fdc1dd Decouple gemma constructor from loader args, update hello_world example, add convenience version of constructor (no uncompressed weights) austinvhuang 2024-03-08 17:26:03 -0500
42e53e2da8 [WIP] simplify hello world example, add convenience function. TODO: update git hash in CMakeLists.txt of hello world after push austinvhuang 2024-03-08 14:55:35 -0500
fce5c8c967 Avoid fadvise on older Android. Fixes #84 Jan Wassenberg 2024-03-07 22:18:46 -0800
b67e28d1a0 [WIP] remove args from GetWeights, GetCompressedWeights austinvhuang 2024-03-08 00:00:11 -0500
2161908f50 Added 7B support and args parsing. Still todo: more testing of 7B conversion. Phil Culliton 2024-03-07 22:34:14 +0000
3df06f64c2

Merge pull request #81 from ufownl/feature/separated_kvcache Austin Huang 2024-03-07 10:10:11 -0500
0f92464f14

Auto-labels 'Gemma' on 'gemma' issues/PRs. Shivam Mishra 2024-03-07 14:45:56 +0530
170a9b4690 Make `CreateKVCache` a free function rather than a method RangerUFO 2024-03-07 14:08:48 +0800
b841612e8c Separate KV cache from GemmaImpl RangerUFO 2024-03-05 17:50:24 +0800
6c0388e049 [WIP] refine Runtime struct definition austinvhuang 2024-03-07 01:14:07 -0500
49e654258d [WIP] clean up hello_world #includes and CMakeLists.txt austinvhuang 2024-03-07 01:04:25 -0500
e781007836 [WIP] Remove InferenceArgs from hello_world example, fix ordering of LoaderArgs validation, revert ReplGemma EOT token behavior austinvhuang 2024-03-06 23:21:13 -0500
7042316013 [WIP] update GemmaInterface, Gemma, and Generate input parameter specs to remove InferenceArgs. TODO: update hello_world example after git commit hash is available for fetching austinvhuang 2024-03-06 22:22:59 -0500
0f6a4b49d5 [WIP] quality tweaks - for constants, defer float cast and use double for intermediate computations, add `model` to EOT token austinvhuang 2024-03-06 15:34:11 -0500
5b9d8a9936 [WIP] dev/examples branch merge austinvhuang 2024-03-06 15:10:48 -0500
10f7a086aa [WIP] decouple GemmaImpl from CLI args austinvhuang 2024-03-06 15:06:41 -0500
3cdd5e524a Fix loop iteration in GeluMulToBF16 Also attempt to speed up builders (parallel) Jan Wassenberg 2024-03-05 23:00:09 -0800
843d9b0e1f fix for-loop bounderies enum-class 2024-03-06 08:25:50 +0800
c8b9675898 Merge pull request #74 from osanseviero:patch-1 Copybara-Service 2024-03-05 12:49:09 -0800
c93e1a1e4d Resolved layer ordering, reshaping, MQA->MHA, and quantization. Works only for 2B. Phil Culliton 2024-03-05 17:54:55 +0000
bc845515b7 fix style, add kCamelCase style for constexpr in clang-tidy enum-class 2024-03-05 20:45:30 +0800
5f016fb433 use hwy/simd for RMSNorm(f, bf, f) calculation enum-class 2024-03-05 17:53:52 +0800
507d64e3e6 use hwy/simd for SquaredL2 calculation enum-class 2024-03-05 17:37:09 +0800
bb9b023502 Support Bazel builds. Fixes #16 Also fix nuq/sfp-inl: warning, cast, and disable SCALAR Jan Wassenberg 2024-03-04 22:06:51 -0800
90dd6c1f07 decoupling from args.h Rliop913 2024-03-05 01:42:12 +0900
8c857b957e

Update README.md Omar Sanseviero 2024-03-04 12:58:49 +0100
cd7468199c Merge pull request #65 from enum-class:narrowing-issues Copybara-Service 2024-03-03 18:51:59 -0800
c378ac2c56 [WIP] hello world example working. TODO: refactor interfaces to decouple arguments austinvhuang 2024-03-03 11:36:48 -0500
39cd59caec [WIP] create skeleton for example frontend application austinvhuang 2024-03-03 10:33:29 -0500
90c99c330d use hwy::SignedFromSize traits enum-class 2024-03-03 10:56:46 +0800
3c69695c1e transformations and validations (wip) austinvhuang 2024-03-02 14:46:51 -0500
7d7d43e661 converter transformations (wip) austinvhuang 2024-03-02 08:11:55 -0500
86761dc113

Update README.md Omar Sanseviero 2024-03-01 23:44:38 +0100