Skip to content

edimuj/tokenlean

Repository files navigation

Tokenlean mascot - a squirrel collecting tokens

tokenlean

60+ CLI tools + MCP server that let AI agents understand codebases without burning tokens

npm version npm downloads license

WhyInstallQuick ReferenceAll ToolsAI IntegrationWorkflowsLanguages


Zero confignpm i -g tokenlean and you're done · Token-conscious — every tool outputs only what's needed · Fast — ripgrep-powered with disk caching · Multi-language — JS/TS, Python, Go, Rust, Ruby · MCP native — structured tool access, no shell overhead · Minimal deps — installs in seconds


The Problem

If you've ever checked your usage quota at 2 PM and decided whether to debug now or wait for the reset — this is for you. Enterprise teams with unlimited API keys don't feel the burn. Solo devs, small teams, anyone on a Pro subscription does. Every cat of a 2000-line file when you needed one function, every 300-line test run when only 3 lines failed, every full directory read to find a signature — that's your working day getting shorter.

tokenlean fixes this:

Instead of... Use Savings
Reading a 500-line file for signatures tl symbols ~90% fewer tokens
Reading all files to find exports tl exports ~95% fewer tokens
Guessing what a change might break tl impact Know for sure
Reading a file to extract one function tl snippet ~85% fewer tokens
Running npm test and reading all output tl run "npm test" Errors only
Scanning long logs for real failures tl tail app.log Errors/warns + dedupe

How much are you wasting?

Find out in one command, no install needed:

npx tokenlean audit --all --savings --claudecode
Summary (348 Claude Code sessions)
  Opportunities:
  Category                Count  Actual     Saveable   Suggestion
  ----------------------------------------------------------------------------
  read-large-file          75x    253.4k      202.7k   -> tl symbols + tl snippet
  build-test-output        34x     28.2k       18.3k   -> tl run
  tail-command              7x      4.8k        3.3k   -> tl tail
  curl-command             13x      3.2k        2.3k   -> tl browse
  cat-large-file            1x      1.1k         902   -> tl symbols + tl snippet
  webfetch                  4x      1.2k         823   -> tl browse
  head-command             11x      3.8k         759   -> Read tool (with limit)

  Still saveable:     243.6k of 363.9k (67%)

  Already saved by tokenlean:
  Tool              Count  Compressed   Raw estimate   Saved
  ------------------------------------------------------------------
  tl snippet          233x      215.7k            2.2M   1.9M
  tl symbols           93x       59.0k          295.0k   236.0k
  tl run               98x       28.7k           82.0k   53.3k

  Tokens saved:       2.2M (424 uses)
  Capture rate:       90% of potential savings realized

tokenlean demo — tl structure, tl symbols, and tl exports in action

Install

npm install -g tokenlean

Requires Node.js >= 18, ripgrep for search tools, and git for history tools.

Use tl as the global entry point — one command, many subcommands:

tl                        # List all available commands
tl doctor                 # Verify Node.js, ripgrep, git, hooks, and config
tl doctor --agents        # Check MCP, hooks, skills, and project instructions
tl update                 # Update the global tokenlean install
tl completions bash|zsh   # Tab completions for subcommands and flags
Development setup
git clone https://github.com/edimuj/tokenlean.git
cd tokenlean
npm link

Quick Reference

# What's in this file?           tl symbols src/auth.ts
# Functions only (dir/multi-file) tl symbols src/ --filter function
# Extract just one function      tl snippet handleSubmit
# What does this module export?  tl exports src/lib/
# How many tokens will this cost? tl context src/api/
# What's the project shape?      tl structure

# What depends on this file?     tl impact src/auth.ts
# How complex is this code?      tl complexity src/auth.ts
# Where are the tests?           tl related src/Button.tsx

# What changed recently?         tl diff --staged
# Is it safe to commit?          tl guard
# Find real usage examples       tl example useAuth
# Summarize noisy logs            tl tail logs/app.log
# What's the tech stack?         tl stack
# Workflow-ready briefing        tl pack refactor src/auth.ts
# What should I run next?       tl advise "debug npm test"

Every tool supports -l N (limit lines), -t N (limit tokens), -j (JSON output), -q (quiet), and -h (help).

See all tools for the complete reference.

AI Agent Integration

Add tokenlean instructions to your AI tool's config with a single command:

AI Tool Command
Claude Code tl prompt >> CLAUDE.md
Codex tl prompt --codex >> AGENTS.md
Cursor tl prompt --minimal >> .cursorrules
GitHub Copilot tl prompt >> .github/copilot-instructions.md
Windsurf tl prompt --minimal >> .windsurfrules

MCP Server — expose tokenlean as native MCP tools for structured, faster access with no CLI overhead:

# Add to any project's .mcp.json (Claude Code, Codex, etc.)
{
  "mcpServers": {
    "tokenlean": { "command": "tl mcp" }
  }
}

# Or pick just the tools you need
{ "command": "tl mcp", "args": ["--tools", "symbols,snippet,run"] }

MCP tools include the core context reducers (tl_symbols, tl_snippet, tl_run, tl_impact, tl_browse, tl_tail, tl_guard, tl_diff) plus workflow routers (tl_advise, tl_pack) and briefing tools (tl_analyze, tl_related, tl_context, tl_structure, tl_entry). Structured JSON in/out — no bash command construction, no argument parsing, no stdout scraping.

Hooks — automatically nudge agents toward token-efficient tool usage:

tl hook install claude-code    # Hard PreToolUse nudges for Claude Code
tl hook install codex          # Hard PreToolUse nudges for Codex CLI
tl hook status --all           # Check hook adapters
tl hook run -j                 # Structured policy decision for adapters/MCP
tl audit --all --savings       # Measure actual savings across sessions
tl audit --all --plan          # Turn audit findings into prioritized fixes

See measuring token savings for full audit and hook setup details.

Agent Skills

Ready-made workflows following the Agent Skills open format. Each workflow ships in Claude Code and Codex variants with runtime-specific wording, but the underlying method is the same: gather narrow context first, then act.

Workflow Use it when... Claude Code Codex
Code review Reviewing a PR or local diff with risk-first context gathering code-review code-review
Explore codebase Understanding an unfamiliar repo without reading everything explore-codebase explore-codebase
Safe refactor Renaming, moving, extracting, or reshaping shared code safe-refactor safe-refactor
Add feature Implementing behavior after locating existing patterns add-feature add-feature
Debug bug Reproducing, tracing, fixing, and verifying defects debug-bug debug-bug
Debug performance Measuring before optimizing, then confirming wins debug-performance debug-performance
Write tests Adding behavior-focused tests that match project conventions write-tests write-tests
Upgrade deps Auditing usage and changelogs before dependency bumps upgrade-deps upgrade-deps
Migrate framework Running incremental migrations in dependency-safe batches migrate-framework migrate-framework

Install the variant for your agent:

# Copy all workflows
cp -r node_modules/tokenlean/skills/claude/* ~/.claude/skills/
cp -r node_modules/tokenlean/skills/codex/* ~/.codex/skills/

# Or copy one workflow
cp -r node_modules/tokenlean/skills/claude/code-review ~/.claude/skills/
cp -r node_modules/tokenlean/skills/codex/code-review ~/.codex/skills/

When working from a clone, replace node_modules/tokenlean with the local repo path.

Design Principles

  1. Single purpose — one tool, one job
  2. Minimal output — show what's needed, nothing more
  3. Composable — every tool supports -j for JSON piping
  4. Fast — no heavy parsing, no external services, aggressive caching
  5. Multi-language — JS/TS first, expanding to Python, Go, Rust, Ruby

More

  • All tools — complete tool reference with examples
  • Workflows — task-oriented recipes (refactoring, PR review, releases, etc.)
  • Language support — compatibility matrix across languages
  • Configuration.tokenleanrc.json schema and caching

Also by Exelerus

Project Description
claude-rig Run multiple Claude Code profiles side-by-side — isolate or share settings, plugins, MCPs per rig
agent-awareness Modular awareness plugins for AI coding agents
claude-mneme Persistent memory for Claude Code — context across sessions
claude-simple-status Minimal statusline — model, context usage, quota at a glance
vexscan-claude-code Security scanner for untrusted plugins, skills, MCPs, and hooks

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines on adding tools, code style, and submitting PRs.

License

MIT © Edin Mujkanovic

About

Lean CLI tools for AI agents and developers - reduce context, save tokens

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Contributors