Skip to content

feat: In-memory TTL cache to reduce API calls by 50-90%#19

Open
romancircus wants to merge 1 commit intotacticlaunch:mainfrom
romancircus:feat/in-memory-cache
Open

feat: In-memory TTL cache to reduce API calls by 50-90%#19
romancircus wants to merge 1 commit intotacticlaunch:mainfrom
romancircus:feat/in-memory-cache

Conversation

@romancircus
Copy link
Copy Markdown

Summary

  • Adds a transparent in-memory TTL cache layer that wraps LinearService, reducing redundant Linear API calls by 50-90%
  • The Linear SDK eagerly resolves all relationships (team, assignee, project, cycle, parent, labels) for each entity, causing N+1 query patterns — e.g. getIssues(25) makes ~151 API calls
  • This is especially problematic when used with AI coding assistants (Claude Code, Cursor, etc.) that make frequent repeated read calls within short windows, quickly exhausting the 5,000 req/hr rate limit

What's Included

File Purpose
src/services/cache.ts Zero-dependency TTL cache with LRU eviction
src/services/cached-linear-service.ts Extends LinearService — caches all reads, invalidates on writes
src/tools/definitions/cache-tools.ts linear_cacheStats and linear_clearCache MCP tool definitions
src/tools/handlers/cache-handlers.ts Handler implementations for cache tools
src/index.ts Updated to use CachedLinearService (opt-out via env var)

Design Decisions

  • Enabled by default — disable with LINEAR_CACHE_ENABLED=false
  • Per-resource TTLs — stable resources (teams, org, workflow states) cached for 1 hour; volatile resources (issues, search) cached for 1-2 minutes
  • Write-through invalidation — every write method invalidates related cache entries by prefix
  • Zero new dependencies — pure Map-based implementation
  • Backward compatibleCachedLinearService extends LinearService, same API surface
  • Max 500 entries (configurable via LINEAR_CACHE_MAX_SIZE) with LRU eviction

TTL Configuration

Resource TTL Rationale
Teams, Org, Workflow States 1 hour Rarely change
Users, Labels, Viewer 30 min Stable within sessions
Projects, Cycles, Initiatives 5 min Moderate change frequency
Issue lists 2 min Frequently updated
Single issues, Search, Comments 1 min Need freshness

Test plan

  • tsc --noEmit passes with zero errors
  • Full build (tsc) succeeds
  • Smoke tested: cache hit/miss tracking, stats reporting, LRU eviction
  • Deployed and running in production with Claude Code MCP integration
  • Unit tests for CacheManager (happy to add if desired)

🤖 Generated with Claude Code

The Linear SDK eagerly resolves all relationships for each entity,
causing N+1 query patterns. For example, getIssues(25) makes ~151
API calls (1 list + 25 * 6 relationship fetches). This quickly
exhausts Linear's 5,000 requests/hour rate limit when used with
AI coding assistants that make frequent read calls.

This adds a transparent caching layer with:
- Per-resource TTLs (teams: 1hr, issues: 2min, search: 1min)
- LRU eviction when cache reaches max size (default: 500)
- Automatic invalidation on write operations
- Two new MCP tools: linear_cacheStats and linear_clearCache
- Opt-out via LINEAR_CACHE_ENABLED=false env var
- Configurable max size via LINEAR_CACHE_MAX_SIZE env var
- Zero new dependencies (pure Map-based implementation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@beautyfree
Copy link
Copy Markdown
Member

@romancircus sounds fair, but you need to add information about the new ENV and TTL to the README

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants