In-sandbox linter/analyzer execution

[haasonsaas]

Problem

CodeRabbit runs 40+ linters and analyzers in a secure sandbox — they don't parse existing CI output, they actually execute the tools. This gives them findings even if the project hasn't configured CI linting. DiffScope has ESLint and Semgrep plugins but doesn't run a broad set of tools in-sandbox.

What CodeRabbit Runs

Category	Tools
Security	Gitleaks, TruffleHog, Semgrep, OpenGrep, OSV-Scanner, Checkov, Trivy
JS/TS	ESLint, Biome, oxlint
Python	Ruff, Pylint, Flake8
Go	golangci-lint
Rust	Clippy
Ruby	RuboCop, Brakeman
PHP	PHPStan, PHPMD, PHPCS
Java	PMD
Kotlin	detekt
C/C++	Cppcheck, Clang-Tidy
Swift	SwiftLint
Shell	ShellCheck
SQL	SQLFluff
IaC	TFLint, Checkov, Trivy, Hadolint
Markup	markdownlint, HTMLHint, YAMLlint, LanguageTool
AST	ast-grep
CI/CD	actionlint, CircleCI validator

Each tool is individually configurable and their findings are:

1. Injected into the LLM review prompt as additional context
2. Used to reduce LLM false positives (LLM agrees/disagrees with deterministic findings)
3. Posted directly when confidence is high (no LLM overhead)

Proposed Solution

Phase 1: Sandbox Framework

pub struct ToolSandbox {
    workdir: PathBuf,
    timeout: Duration,
    tools: Vec<Box<dyn AnalysisTool>>,
}

#[async_trait]
trait AnalysisTool {
    fn name(&self) -> &str;
    fn supported_languages(&self) -> &[&str];
    fn is_available(&self) -> bool;  // check if binary exists
    async fn run(&self, files: &[PathBuf], workdir: &Path) -> Result<Vec<Finding>>;
}

Phase 2: Priority Tool Integration

Start with tools that provide the highest value and are commonly available:

1. Clippy (Rust) — already available in Rust projects
2. ESLint/Biome (JS/TS) — existing plugin, enhance
3. Ruff (Python) — fast, single binary
4. Gitleaks (secrets) — ties into Built-in secrets detection scanner #20
5. ShellCheck (shell scripts) — single binary
6. actionlint (GitHub Actions) — catches CI misconfigs

Phase 3: Auto-Detection

• Detect project languages from file extensions
• Check which tools are available on PATH
• Auto-run applicable tools without configuration
• Respect existing tool configs (.eslintrc, clippy.toml, ruff.toml)

Configuration

tools:
  auto_detect: true  # detect and run available tools
  enabled:
    - clippy
    - ruff
    - gitleaks
    - shellcheck
  disabled:
    - pylint  # too noisy for this project
  timeout: 60  # seconds per tool

Docker Integration

For the Docker deployment, bundle common tools in the image so they're always available.

Priority

Medium — quality floor + value-add. Self-hosted users get free linting even if they haven't set up CI. Deterministic findings complement LLM review.