17 KiB

Raw Blame History

Memos Codebase Guide for AI Agents

This document provides comprehensive guidance for AI agents working with the Memos codebase. It covers architecture, workflows, conventions, and key patterns.

Project Overview

Memos is a self-hosted knowledge management platform built with:

Backend: Go 1.25 with gRPC + Connect RPC
Frontend: React 18.3 + TypeScript + Vite 7
Databases: SQLite (default), MySQL, PostgreSQL
Protocol: Protocol Buffers (v2) with buf for code generation
API Layer: Dual protocol - Connect RPC (browsers) + gRPC-Gateway (REST)

Architecture

Backend Architecture

cmd/memos/              # Entry point
└── main.go             # Cobra CLI, profile setup, server initialization

server/
├── server.go           # Echo HTTP server, healthz, background runners
├── auth/               # Authentication (JWT, PAT, session)
├── router/
│   ├── api/v1/        # gRPC service implementations
│   │   ├── v1.go      # Service registration, gateway & Connect setup
│   │   ├── acl_config.go   # Public endpoints whitelist
│   │   ├── connect_services.go  # Connect RPC handlers
│   │   ├── connect_interceptors.go # Auth, logging, recovery
│   │   └── *_service.go    # Individual services (memo, user, etc.)
│   ├── frontend/       # Static file serving (SPA)
│   ├── fileserver/     # Native HTTP file serving for media
│   └── rss/           # RSS feed generation
└── runner/
    ├── memopayload/    # Memo payload processing (tags, links, tasks)
    └── s3presign/     # S3 presigned URL management

store/                  # Data layer with caching
├── driver.go           # Driver interface (database operations)
├── store.go           # Store wrapper with cache layer
├── cache.go           # In-memory caching (instance settings, users)
├── migrator.go        # Database migrations
├── db/
│   ├── db.go          # Driver factory
│   ├── sqlite/        # SQLite implementation
│   ├── mysql/         # MySQL implementation
│   └── postgres/      # PostgreSQL implementation
└── migration/         # SQL migration files (embedded)

proto/                  # Protocol Buffer definitions
├── api/v1/           # API v1 service definitions
└── gen/               # Generated Go & TypeScript code

Frontend Architecture

web/
├── src/
│   ├── components/     # React components
│   ├── contexts/       # React Context (client state)
│   │   ├── AuthContext.tsx      # Current user, auth state
│   │   ├── ViewContext.tsx      # Layout, sort order
│   │   └── MemoFilterContext.tsx # Filters, shortcuts
│   ├── hooks/          # React Query hooks (server state)
│   │   ├── useMemoQueries.ts    # Memo CRUD, pagination
│   │   ├── useUserQueries.ts    # User operations
│   │   ├── useAttachmentQueries.ts # Attachment operations
│   │   └── ...
│   ├── lib/            # Utilities
│   │   ├── query-client.ts  # React Query v5 client
│   │   └── connect.ts       # Connect RPC client setup
│   ├── pages/          # Page components
│   └── types/proto/    # Generated TypeScript from .proto
├── package.json        # Dependencies
└── vite.config.mts     # Vite config with dev proxy

plugin/                 # Backend plugins
├── scheduler/         # Cron jobs
├── email/            # Email delivery
├── filter/           # CEL filter expressions
├── webhook/          # Webhook dispatch
├── markdown/         # Markdown parsing & rendering
├── httpgetter/        # HTTP fetching (metadata, images)
└── storage/s3/       # S3 storage backend

Key Architectural Patterns

1. API Layer: Dual Protocol

Connect RPC (Browser Clients):

Protocol: connectrpc.com/connect
Base path: /memos.api.v1.*
Interceptor chain: Metadata → Logging → Recovery → Auth
Returns type-safe responses to React frontend
See: server/router/api/v1/connect_interceptors.go:177-227

gRPC-Gateway (REST API):

Protocol: Standard HTTP/JSON
Base path: /api/v1/*
Uses same service implementations as Connect
Useful for external tools, CLI clients
See: server/router/api/v1/v1.go:52-96

Authentication:

JWT Access Tokens (V2): Stateless, 15-min expiration, verified via AuthenticateByAccessTokenV2
Personal Access Tokens (PAT): Stateful, long-lived, validated against database
Both use Authorization: Bearer <token> header
See: server/auth/authenticator.go:17-166

2. Store Layer: Interface Pattern

All database operations go through the Driver interface:

type Driver interface {
    GetDB() *sql.DB
    Close() error

    IsInitialized(ctx context.Context) (bool, error)

    CreateMemo(ctx context.Context, create *Memo) (*Memo, error)
    ListMemos(ctx context.Context, find *FindMemo) ([]*Memo, error)
    UpdateMemo(ctx context.Context, update *UpdateMemo) error
    DeleteMemo(ctx context.Context, delete *DeleteMemo) error

    // ... similar methods for all resources
}

Three Implementations:

store/db/sqlite/ - SQLite (modernc.org/sqlite)
store/db/mysql/ - MySQL (go-sql-driver/mysql)
store/db/postgres/ - PostgreSQL (lib/pq)

Caching Strategy:

Store wrapper maintains in-memory caches for:
- Instance settings (instanceSettingCache)
- Users (userCache)
- User settings (userSettingCache)
Config: Default TTL 10 min, cleanup interval 5 min, max 1000 items
See: store/store.go:10-57

3. Frontend State Management

React Query v5 (Server State):

All API calls go through custom hooks in web/src/hooks/
Query keys organized by resource: memoKeys, userKeys, attachmentKeys
Default staleTime: 30s, gcTime: 5min
Automatic refetch on window focus, reconnect
See: web/src/lib/query-client.ts

React Context (Client State):

AuthContext: Current user, auth initialization, logout
ViewContext: Layout mode (LIST/MASONRY), sort order
MemoFilterContext: Active filters, shortcut selection, URL sync

4. Database Migration System

Migration Flow:

preMigrate: Check if DB exists. If not, apply LATEST.sql
checkMinimumUpgradeVersion: Reject pre-0.22 installations
applyMigrations: Apply incremental migrations in single transaction
Demo mode: Seed with demo data

Schema Versioning:

Stored in instance_setting table (key: bb.general.version)
Format: major.minor.patch
Migration files: store/migration/{driver}/{version}/NN__description.sql
See: store/migrator.go:21-414

5. Protocol Buffer Code Generation

Definition Location: proto/api/v1/*.proto

Regeneration:

cd proto && buf generate

Generated Outputs:

Go: proto/gen/api/v1/ (used by backend services)
TypeScript: web/src/types/proto/api/v1/ (used by frontend)

Linting: proto/buf.yaml - BASIC lint rules, FILE breaking changes

Development Commands

Backend

# Start dev server
go run ./cmd/memos --mode dev --port 8081

# Run all tests
go test ./...

# Run tests for specific package
go test ./store/...
go test ./server/router/api/v1/test/...

# Lint (golangci-lint)
golangci-lint run

# Format imports
goimports -w .

# Run with MySQL/Postgres
DRIVER=mysql go run ./cmd/memos
DRIVER=postgres go run ./cmd/memos

Frontend

# Install dependencies
cd web && pnpm install

# Start dev server (proxies API to localhost:8081)
pnpm dev

# Type checking
pnpm lint

# Auto-fix lint issues
pnpm lint:fix

# Format code
pnpm format

# Build for production
pnpm build

# Build and copy to backend
pnpm release

Protocol Buffers

# Regenerate Go and TypeScript from .proto files
cd proto && buf generate

# Lint proto files
cd proto && buf lint

# Check for breaking changes
cd proto && buf breaking --against .git#main

Key Workflows

Adding a New API Endpoint

Define in Protocol Buffer:
- Edit proto/api/v1/*_service.proto
- Add request/response messages
- Add RPC method to service
Regenerate Code:
```
cd proto && buf generate
```
Implement Service (Backend):
- Add method to server/router/api/v1/*_service.go
- Follow existing patterns: fetch user, validate, call store
- Add Connect wrapper to server/router/api/v1/connect_services.go (optional, same implementation)
If Public Endpoint:
- Add to server/router/api/v1/acl_config.go:11-34
Create Frontend Hook (if needed):
- Add query/mutation to web/src/hooks/use*Queries.ts
- Use existing query key factories

Database Schema Changes

Create Migration Files:

store/migration/sqlite/0.28/1__add_new_column.sql
store/migration/mysql/0.28/1__add_new_column.sql
store/migration/postgres/0.28/1__add_new_column.sql

Update LATEST.sql:
- Add change to store/migration/{driver}/LATEST.sql
Update Store Interface (if new table/model):
- Add methods to store/driver.go:8-71
- Implement in store/db/{driver}/*.go
Test Migration:
- Run go test ./store/test/... to verify

Adding a New Frontend Page

Create Page Component:
- Add to web/src/pages/NewPage.tsx
- Use existing hooks for data fetching
Add Route:
- Edit web/src/App.tsx (or router configuration)

Use React Query:

import { useMemos } from "@/hooks/useMemoQueries";
const { data, isLoading } = useMemos({ filter: "..." });

Use Context for Client State:

import { useView } from "@/contexts/ViewContext";
const { layout, toggleSortOrder } = useView();

Testing

Backend Tests

Test Pattern:

func TestMemoCreation(t *testing.T) {
    ctx := context.Background()
    store := test.NewTestingStore(ctx, t)

    // Create test user
    user, _ := createTestUser(ctx, store, t)

    // Execute operation
    memo, err := store.CreateMemo(ctx, &store.Memo{
        CreatorID: user.ID,
        Content:  "Test memo",
        // ...
    })
    require.NoError(t, err)
    assert.NotNil(t, memo)
}

Test Utilities:

store/test/store.go:22-35 - NewTestingStore() creates isolated DB
store/test/store.go:37-77 - resetTestingDB() cleans tables
Test DB determined by DRIVER env var (default: sqlite)

Running Tests:

# All tests
go test ./...

# Specific package
go test ./store/...
go test ./server/router/api/v1/test/...

# With coverage
go test -cover ./...

Frontend Testing

TypeScript Checking:

cd web && pnpm lint

No Automated Tests:

Frontend relies on TypeScript checking and manual validation
React Query DevTools available in dev mode (bottom-left)

Code Conventions

Go

Error Handling:

Use github.com/pkg/errors for wrapping: errors.Wrap(err, "context")
Return structured gRPC errors: status.Errorf(codes.NotFound, "message")

Naming:

Package names: lowercase, single word (e.g., store, server)
Interfaces: Driver, Store, Service
Methods: PascalCase for exported, camelCase for internal

Comments:

Public exported functions must have comments (godot enforces)
Use // for single-line, /* */ for multi-line

Imports:

Grouped: stdlib, third-party, local
Sorted alphabetically within groups
Use goimports -w . to format

TypeScript/React

Components:

Functional components with hooks
Use useMemo, useCallback for optimization
Props interfaces: interface Props { ... }

State Management:

Server state: React Query hooks
Client state: React Context
Avoid direct useState for server data

Styling:

Tailwind CSS v4 via @tailwindcss/vite
Use clsx and tailwind-merge for conditional classes

Imports:

Absolute imports with @/ alias
Group: React, third-party, local
Auto-organized by Biome

Important Files Reference

Backend Entry Points

File	Purpose
`cmd/memos/main.go`	Server entry point, CLI setup
`server/server.go`	Echo server initialization, background runners
`store/store.go`	Store wrapper with caching
`store/driver.go`	Database driver interface

API Layer

File	Purpose
`server/router/api/v1/v1.go`	Service registration, gateway setup
`server/router/api/v1/acl_config.go`	Public endpoints whitelist
`server/router/api/v1/connect_interceptors.go`	Connect interceptors
`server/auth/authenticator.go`	Authentication logic

Frontend Core

File	Purpose
`web/src/lib/query-client.ts`	React Query client configuration
`web/src/contexts/AuthContext.tsx`	User authentication state
`web/src/contexts/ViewContext.tsx`	UI preferences
`web/src/contexts/MemoFilterContext.tsx`	Filter state
`web/src/hooks/useMemoQueries.ts`	Memo queries/mutations

Data Layer

File	Purpose
`store/memo.go`	Memo model definitions, store methods
`store/user.go`	User model definitions
`store/attachment.go`	Attachment model definitions
`store/migrator.go`	Migration logic
`store/db/db.go`	Driver factory
`store/db/sqlite/sqlite.go`	SQLite driver implementation

Configuration

Backend Environment Variables

Variable	Default	Description
`MEMOS_MODE`	`dev`	Mode: `dev`, `prod`, `demo`
`MEMOS_PORT`	`8081`	HTTP port
`MEMOS_ADDR`	``	Bind address (empty = all)
`MEMOS_DATA`	`~/.memos`	Data directory
`MEMOS_DRIVER`	`sqlite`	Database: `sqlite`, `mysql`, `postgres`
`MEMOS_DSN`	``	Database connection string
`MEMOS_INSTANCE_URL`	``	Instance base URL

Frontend Environment Variables

Variable	Default	Description
`DEV_PROXY_SERVER`	`http://localhost:8081`	Backend proxy target

CI/CD

GitHub Workflows

Backend Tests (.github/workflows/backend-tests.yml):

Runs on go.mod, go.sum, **.go changes
Steps: verify go mod tidy, golangci-lint, all tests

Frontend Tests (.github/workflows/frontend-tests.yml):

Runs on web/** changes
Steps: pnpm install, lint, build

Proto Lint (.github/workflows/proto-linter.yml):

Runs on .proto changes
Steps: buf lint, buf breaking check

Linting Configuration

Go (.golangci.yaml):

Linters: revive, govet, staticcheck, misspell, gocritic, etc.
Formatter: goimports
Forbidden: fmt.Errorf, ioutil.ReadDir

TypeScript (web/biome.json):

Linting: Biome (ESLint replacement)
Formatting: Biome (Prettier replacement)
Line width: 140 characters
Semicolons: always

Common Tasks

Debugging Database Issues

Check connection string in logs
Verify store/db/{driver}/migration/ files exist
Check schema version: SELECT * FROM instance_setting WHERE key = 'bb.general.version'
Test migration: go test ./store/test/... -v

Debugging API Issues

Check Connect interceptor logs: server/router/api/v1/connect_interceptors.go:79-105
Verify endpoint is in acl_config.go if public
Check authentication via auth/authenticator.go:133-165
Test with curl: curl -H "Authorization: Bearer <token>" http://localhost:8081/api/v1/...

Debugging Frontend State

Open React Query DevTools (bottom-left in dev)
Inspect query cache, mutations, refetch behavior
Check Context state via React DevTools
Verify filter state in MemoFilterContext

Running Tests Against Multiple Databases

# SQLite (default)
DRIVER=sqlite go test ./...

# MySQL (requires running MySQL server)
DRIVER=mysql DSN="user:pass@tcp(localhost:3306)/memos" go test ./...

# PostgreSQL (requires running PostgreSQL server)
DRIVER=postgres DSN="postgres://user:pass@localhost:5432/memos" go test ./...

Plugin System

Backend supports pluggable components in plugin/:

Plugin	Purpose
`scheduler`	Cron-based job scheduling
`email`	SMTP email delivery
`filter`	CEL expression filtering
`webhook`	HTTP webhook dispatch
`markdown`	Markdown parsing (goldmark)
`httpgetter`	HTTP content fetching
`storage/s3`	S3-compatible storage

Each plugin has its own README with usage examples.

Performance Considerations

Backend

Database queries use pagination (limit, offset)
In-memory caching reduces DB hits for frequently accessed data
WAL journal mode for SQLite (reduces locking)
Thumbnail generation limited to 3 concurrent operations

Frontend

React Query reduces redundant API calls
Infinite queries for large lists (pagination)
Manual chunks: utils-vendor, mermaid-vendor, leaflet-vendor
Lazy loading for heavy components

Security Notes

JWT secrets must be kept secret (MEMOS_MODE=prod generates random secret)
Personal Access Tokens stored as SHA-256 hashes in database
CSRF protection via SameSite cookies
CORS enabled for all origins (configure for production)
Input validation at service layer
SQL injection prevention via parameterized queries

17 KiB Raw Blame History

Memos Codebase Guide for AI Agents

Project Overview

Architecture

Backend Architecture

Frontend Architecture

Key Architectural Patterns

1. API Layer: Dual Protocol

2. Store Layer: Interface Pattern

3. Frontend State Management

4. Database Migration System

5. Protocol Buffer Code Generation

Development Commands

Backend

Frontend

Protocol Buffers

Key Workflows

Adding a New API Endpoint

Database Schema Changes

Adding a New Frontend Page

Testing

Backend Tests

Frontend Testing

Code Conventions

Go

TypeScript/React

Important Files Reference

Backend Entry Points

API Layer

Frontend Core

Data Layer

Configuration

Backend Environment Variables

Frontend Environment Variables

CI/CD

GitHub Workflows

Linting Configuration

Common Tasks

Debugging Database Issues

Debugging API Issues

Debugging Frontend State

Running Tests Against Multiple Databases

Plugin System

Performance Considerations

Backend

Frontend

Security Notes

17 KiB

Raw Blame History