CLAUDE.md · diff
git:20260122.d22b047 to git:20260329.4d470e1
86 added, 952 removed. Audit A to A.
- # CLAUDE.md - Smart Tree v6.5.1 Comprehensive Development Guide
-
- **Location**: `/aidata/aye/smart-tree`
- **Language**: 100% Rust (174 source files)
- **Binary Size**: ~29MB core (TUI/Dashboard optional)
- **Latest Version**: 6.5.1
-
- ## What's New in 6.5.1
-
- - **Security Scanner**: `--security-scan` detects supply chain attack patterns (IPFS/IPNS injection, fake verification, claude-flow references)
- - **Aggressive Scanning**: Security scan ignores .gitignore to scan node_modules and hidden directories
- - **Risk Classification**: Findings categorized as Critical/High/Medium/Low with context-aware adjustment
- - **Actionable Recommendations**: Scanner provides cleanup steps and remediation guidance
-
- ## Previous: 6.2.0
-
- - **Session Persistence**: `SessionStart`/`SessionEnd` hooks auto-save and restore context
- - **Smart Restore**: `--claude-restore` only shows relevant, recent context (24h window)
- - **Feature Gates**: TUI and Dashboard now optional (`--features tui`, `--features dashboard`)
-
- ## Quick Start for Returning Claude Instances
-
- If this is your second+ session on smart-tree, use this section:
-
- ```bash
- # Get back up to speed (5 seconds)
- st -m context . # See what changed since last time
- st -m quantum . # Super-compressed project overview
-
- # Build & Test (follows pre-commit checklist automatically)
- ./scripts/manage.sh test # Runs: cargo fmt + clippy + cargo test
-
- # Run MCP server (for Claude Desktop integration)
- ./scripts/manage.sh mcp-run
-
- # Build with optional features
- cargo build --release # Core only (29MB)
- cargo build --release --features tui # With spicy TUI
- cargo build --release --features dashboard # With egui dashboard
- cargo build --release --features full # Everything
- ```
+ # CLAUDE.md
- ---
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
- **Smart Tree** is a lightning-fast, AI-friendly directory visualization tool built in Rust.
-
- **Key Metrics**:
- - 10-24x faster than traditional `tree` command
- - 30+ MCP tools for AI integration
- - Multiple compression formats (AI, Quantum, Marqant)
- - Spicy TUI mode for interactive exploration
- - Real-time SSE streaming support
- - 25+ output format types
-
- **Architecture**: Single binary (`st`) that can also output `mq` (Marqant compression) and `m8` (MEM8 consciousness tools).
-
- ---
-
- ## Essential Build & Test Commands
-
- ### Pre-Commit Checklist (REQUIRED)
-
- Before any git commit, always run:
-
- ```bash
- ./scripts/manage.sh test
- ```
-
- This runs (in order):
- 1. `cargo test` - All unit tests
- 2. `cargo clippy -- -D warnings` - Strict linting (no warnings allowed!)
- 3. `cargo fmt -- --check` - Format verification
-
- **Why this matters**: Smart Tree has strict quality gates. Clippy warnings will fail CI/CD.
-
- ### Standard Build Commands
-
- ```bash
- # Release build (10x faster runtime, what you almost always want)
- cargo build --release
-
- # Debug build (slower runtime, better debugging symbols)
- cargo build
-
- # Release build with specific features
- cargo build --release --features "std"
-
- # Quick format + lint check
- cargo fmt && cargo clippy -- -D warnings
- ```
-
- ### Test-Specific Commands
-
- ```bash
- # Run all tests with output (great for debugging)
- cargo test -- --nocapture
-
- # Test a specific module
- cargo test scanner # Tests in scanner.rs
- cargo test formatters # Tests in formatters/
-
- # Single test by name
- cargo test test_quantum -- --exact --nocapture
-
- # Integration tests
- cargo test --test mcp_integration -- --nocapture
-
- # Benchmarks (if any)
- cargo bench # Criterion benchmarks if configured
- ```
-
- ### Using manage.sh (Recommended)
-
- The project includes a powerful `scripts/manage.sh` that handles most tasks:
-
- ```bash
- # Interactive menu
- ./scripts/manage.sh menu # Launch interactive setup
-
- # Build
- ./scripts/manage.sh build release # Release mode (can also specify debug)
- ./scripts/manage.sh build # Interactive menu
-
- # Test (recommended way)
- ./scripts/manage.sh test # Runs full suite + clippy + fmt
-
- # Run the tool
- ./scripts/manage.sh run # Interactive arguments
- ./scripts/manage.sh run -- . -d 3 # Direct args: analyze . with depth 3
-
- # Install binary
- ./scripts/manage.sh install # Install to /usr/local/bin (may need sudo)
-
- # Version management
- ./scripts/manage.sh bump patch # v5.4.0 → v5.4.1
- ./scripts/manage.sh bump minor # v5.4.0 → v5.5.0
- ./scripts/manage.sh release v6.0.0 "Major feature release"
-
- # MCP/Server commands
- ./scripts/manage.sh mcp-run # Run as MCP stdio server
- ./scripts/manage.sh mcp-config # Show Claude Desktop config
- ./scripts/manage.sh mcp-tools # List all 30+ tools
-
- # Demos
- ./scripts/manage.sh demo-stream # Show streaming mode
- ./scripts/manage.sh demo-search # Show content search
- ./scripts/manage.sh demo-relations # Show code relations
- ```
-
- Use `-n` flag for non-interactive mode:
- ```bash
- NON_INTERACTIVE=true ./scripts/manage.sh build release
- ```
-
- ---
-
- ## Architecture Overview
-
- ### Core Design Pattern
-
- Smart Tree uses a **Scanner → Formatter** pipeline:
-
- 1. **Scanner** (`src/scanner.rs`): Traverses directories, collects metadata
- - Respects `.gitignore` patterns
- - Handles symlinks safely
- - Permission-aware (marks inaccessible with `*`)
- - ~1800 lines of core logic
-
- 2. **Formatters** (`src/formatters/`): Convert scan results to output
- - 25+ different formats supported
- - Each formatter implements `Formatter` trait
- - Composable (formatters can wrap other formatters)
-
- 3. **MCP Server** (`src/mcp/`): Exposes tools via Model Context Protocol
- - JSON-RPC stdio server
- - 30+ tools for AI assistants
- - Consciousness persistence (`~/.mem8/`)
-
- ### Directory Structure
-
- ```
- src/
- ├── main.rs # CLI entry point (clap 4.5)
- ├── lib.rs # Module declarations
- ├── scanner.rs # Directory traversal (core logic)
- ├── terminal.rs # Smart Tree Terminal Interface
- │
- ├── formatters/ # 25+ output formats
- │ ├── mod.rs # Formatter trait definition
- │ ├── classic.rs # Traditional tree view
- │ ├── ai.rs # Token-efficient AI format (80% smaller!)
- │ ├── quantum.rs # Wave-based compression (100x!)
- │ ├── marqant.rs # Markdown compression
- │ ├── hex.rs # Hex-encoded AI format
- │ ├── json.rs, csv.rs, tsv.rs # Data formats
- │ ├── stats.rs, digest.rs # Metadata summaries
- │ ├── semantic.rs # Semantic grouping
- │ ├── relations.rs # Code relationship analysis
- │ ├── sse.rs # Server-Sent Events streaming
- │ └── ... (18 more formatters)
- │
- ├── mcp/ # Model Context Protocol
- │ ├── mod.rs # Server entry point
- │ ├── tools_consolidated.rs # 30+ tool definitions
- │ ├── smart_edit.rs # AST-aware code editing
- │ ├── unified_watcher.rs # Real-time file monitoring
- │ ├── consciousness/ # Memory persistence
- │ └── ... (20+ supporting modules)
- │
- ├── spicy_tui.rs # Interactive TUI mode
- ├── spicy_fuzzy.rs # Fuzzy matching for TUI
- │
- ├── compression_manager.rs # Global compression control
- ├── content_detector.rs # Detects file types
- ├── relations.rs # Code relationship analyzer
- ├── semantic.rs # Semantic analysis
- ├── inputs.rs # Universal input adapters
- ├── tokenizer.rs # Smart tokenization
- │
- └── bin/ # Additional binaries
- ├── mq.rs # Marqant markdown compressor
- ├── m8.rs # MEM8 consciousness tools
- ├── tree.rs # Tree command alternative
- └── import_claude_memories.rs # Claude context import
- ```
-
- ### Key File Count & Sizes
-
- - **Total Source Files**: 173 Rust files
- - **Main Binary Target**: `st` (11MB release)
- - **Additional Binaries**: `mq`, `m8`, `tree`
- - **Tests**: 10 integration test files
- - **Dependencies**: ~140 (see Cargo.toml)
-
- ### Critical Dependencies
-
- ```toml
- # CLI & Async
- clap 4.5 # Argument parsing
- tokio 1.42 # Async runtime with full features
- async-trait # Async trait support
-
- # Data Processing
- serde/serde_json # Serialization
- regex/globset # Pattern matching & gitignore support
- walkdir 2.5 # Directory traversal
- rayon # Parallel processing
-
- # Compression & Encoding
- flate2 # Zlib compression
- base64 # Base64 encoding
- bincode # Binary serialization
-
- # AI/ML Integration
- tree-sitter + 10 language modules # AST parsing
- fuzzy-matcher # Fuzzy search for TUI
- syntect # Syntax highlighting
-
- # Terminal UI
- ratatui 0.25 # Terminal UI framework (TUI mode)
- crossterm # Terminal manipulation
- termimad # Markdown rendering
- colored # Color output
-
- # MCP/Server
- axum 0.7 # Web framework (WebSocket support)
- hyper 1.7 # HTTP protocol
- reqwest # HTTP client
-
- # File operations
- notify 6.1 # File system watching
- gix 0.73 # Git operations (will become g8t)
-
- # Utilities
- chrono # Date/time
- humansize 2.1 # Human-readable sizes
- uuid # Unique identifiers
- ```
-
- ---
-
- ## Testing Patterns & Practices
-
- ### Test Organization
-
- ```
- tests/
- ├── test_unified_integration.rs # Integration tests
- ├── test_smart_edit.rs # Smart Edit tool tests
- ├── test_claude_integration.rs # Claude-specific integration
- ├── mcp_integration.rs # MCP server tests
- └── ...
- ```
-
- ### How to Run Tests
-
- ```bash
- # All tests (REQUIRED before commit)
- cargo test
-
- # Specific test file
- cargo test --test test_smart_edit
-
- # Single test
- cargo test test_scanner_basic -- --exact --nocapture
-
- # With logging
- RUST_LOG=debug cargo test -- --nocapture
-
- # Benchmarks
- cargo bench
- ```
-
- ### Testing Philosophy
-
- - **No Mock Data**: Smart Tree tests use real file system operations
- - **Permission Handling**: Tests verify `*` markers for inaccessible directories
- - **Large Directory Tests**: Tests include 1M+ file scenarios
- - **Async Testing**: Uses tokio runtime for async code
- - **Property-Based**: Some tests use quickcheck-style randomization
-
- ### Common Test Patterns You'll See
-
- ```rust
- #[test]
- fn test_scanner_respects_gitignore() {
- // Create temp directory with .gitignore
- // Verify ignored files are marked
- }
-
- #[tokio::test]
- async fn test_mcp_tool_execution() {
- // Async test for MCP tools
- }
-
- #[test]
- fn test_formatter_output() {
- // Verify formatter produces expected format
- }
- ```
-
- ---
-
- ## Formatter Architecture (Core to Smart Tree)
-
- ### Implementing a New Formatter
-
- All formatters implement the `Formatter` trait:
-
- ```rust
- pub trait Formatter {
- fn format(&self, nodes: &[FileNode]) -> Result<String>;
- fn supports_streaming(&self) -> bool { false }
- }
- ```
-
- **Location**: `src/formatters/mod.rs` (trait definition)
-
- **To add a new formatter**:
-
- 1. Create `src/formatters/my_format.rs`:
- ```rust
- use crate::scanner::FileNode;
- pub struct MyFormatter;
-
- impl Formatter for MyFormatter {
- fn format(&self, nodes: &[FileNode]) -> Result<String> {
- // Implement your format
- Ok(String::new())
- }
- }
- ```
-
- 2. Add to `src/formatters/mod.rs`:
- ```rust
- pub mod my_format;
- pub use my_format::MyFormatter;
- ```
-
- 3. Add to CLI in `src/main.rs`:
- ```rust
- #[derive(ValueEnum)]
- enum FormatterType {
- MyFormat,
- // ... others
- }
- ```
-
- 4. Wire it in the format dispatch:
- ```rust
- FormatterType::MyFormat => {
- let formatter = MyFormatter;
- formatter.format(&nodes)?
- }
- ```
-
- 5. Test it:
- ```bash
- cargo test formatters::
- ./scripts/manage.sh test
- ```
-
- ### Key Formatters
-
- | Formatter | Purpose | Token Efficiency |
- |-----------|---------|------------------|
- | `classic` | Traditional tree view | 100% (baseline) |
- | `ai` | AI-optimized output | 20% (5x smaller!) |
- | `quantum` | Wave compression | <1% (100x+!) |
- | `hex` | Hex-encoded for AI | 30% |
- | `json` | Machine-readable | 60% |
- | `stats` | Metadata only | 10% |
- | `digest` | Ultra-compact summary | 5% |
- | `semantic` | Grouped by type/purpose | 15% |
- | `relations` | Code dependencies | 25% |
- | `sse` | Real-time streaming | N/A (streaming) |
- | `marqant` | Markdown compression | 10-30% |
-
- **Performance Tips**:
- - Use `--mode quantum` for massive directories (100x compression!)
- - Use `--mode ai --compress` for LLM context (80% reduction)
- - Use `--mode digest` for pre-analysis (5% of original)
- - Use `--stream` for directories >100k files
-
- ---
-
- ## MCP (Model Context Protocol) Integration
-
- ### Running as MCP Server
-
- ```bash
- # Start MCP server (stdio-based)
- ./scripts/manage.sh mcp-run
-
- # Or directly
- cargo run --release -- --mcp
-
- # List all available tools
- ./scripts/manage.sh mcp-tools
-
- # Show Claude Desktop config
- ./scripts/manage.sh mcp-config
- ```
-
- ### Available MCP Tools (30+)
-
- **Project Analysis**:
- - `overview` - Quick 3-level project scan
- - `find` - Powerful file discovery (type, pattern, recent, large)
- - `search` - Content search with line context
- - `analyze` - Deep analysis (git, statistics, semantic)
- - `analyze:git_status` - Git-aware directory analysis
-
- **Code Editing**:
- - `edit` - AST-aware code editing (90% token reduction!)
- - `edit:get_functions` - List all functions in file
- - `edit:insert_function` - Add new function
- - `edit:remove_function` - Remove function
- - `edit:smart_edit` - Multiple edits at once
-
- **Memory & Context**:
- - `context:gather_project` - Full project context
- - `context:collaboration_rapport` - Session history
- - `context:suggest_insights` - AI suggestions
- - `history:get_file` - File change history
- - `history:get_project` - Project audit trail
- - `memory:anchor` - Save breakthrough insight
- - `memory:find` - Recall saved insights
-
- **Advanced Features**:
- - `sse` - Real-time directory monitoring
- - `hooks` - Claude Code hook management (enable/disable/test)
- - `server_info` - Smart Tree capabilities
- - `verify_permissions` - Check directory access
- - `unified_watcher` - Monitor multiple directories
-
- ### MCP Tool Usage Examples
-
- ```javascript
- // Get quick overview
- overview {mode:'quick', path:'.', depth:2}
-
- // Find Rust files
- find {type:'code', languages:['rust'], path:'src'}
-
- // Search for TODOs
- search {keyword:'TODO', include_content:true}
-
- // Edit with AST awareness
- edit {operation:'get_functions', file_path:'src/main.rs'}
-
- // Gather full context
- context {operation:'gather_project', project_path:'.'}
-
- // Enable Claude Code hooks
- hooks {operation:'set', hook_type:'UserPromptSubmit', enabled:true}
-
- // Real-time monitoring
- sse {path:'./src', heartbeat_interval:30}
- ```
-
- ### How MCP Tools Are Implemented
-
- **Location**: `src/mcp/tools_consolidated.rs` (main tool implementations)
-
- Each tool is a JSON-RPC method that:
- 1. Receives parameters as JSON
- 2. Executes the operation
- 3. Returns results as JSON
-
- The server reads JSON-RPC requests from stdin and writes responses to stdout.
-
- ---
-
- ## Performance & Optimization
-
- ### Why Smart Tree is Fast
-
- 1. **Single-pass directory traversal**: `walkdir` crate does smart iteration
- 2. **Parallel processing**: `rayon` for independent operations
- 3. **Smart memory management**: `unsafe` blocks used sparingly, with comments
- 4. **Format-specific optimizations**: Each formatter optimizes for its use case
- 5. **Streaming mode**: Doesn't load everything in memory (`--stream` flag)
-
- ### Benchmarking
-
- ```bash
- # Run benchmarks (if configured in Cargo.toml)
- cargo bench
-
- # Manual timing
- time st . -m hex > /dev/null
- time st . -m quantum > /dev/null
- time st . -m ai > /dev/null
-
- # Large directory test
- time st /usr -m hex --depth 2 > /dev/null
- ```
-
- ### Expected Performance
-
- | Directory Size | Time | vs tree | Speedup |
- |---|---|---|---|
- | 100 files | 2ms | 15ms | 7.5x |
- | 10K files | 35ms | 450ms | 12.8x |
- | 100K files | 198ms | 4.8s | 24.2x |
- | 1M files | 1.9s | 45s | 23.7x |
-
- **Key insight**: Speedup increases with directory size (better cache locality).
-
- ---
-
- ## Feature Flags & Configuration
-
- ### Cargo Features
-
- ```toml
- [features]
- default = ["std"]
- std = [] # Standard library (always enabled)
- alloc = [] # Allocator-specific (experimental)
- mem8 = [] # MEM8 consciousness integration
-
- # Optional features (disabled by default for smaller binary)
- tui = ["ratatui", "crossterm", "syntect", "artem"] # Spicy TUI mode
- dashboard = ["egui", "eframe", "egui_extras", "winit"] # egui dashboard
- candle = ["candle-core", "candle-transformers", ...] # Local LLM
- full = ["tui", "dashboard", "candle"] # Everything
- ```
-
- ### Build Commands by Feature
-
- ```bash
- cargo build --release # Core only (~29MB)
- cargo build --release --features tui # With spicy TUI
- cargo build --release --features dashboard # With egui dashboard
- cargo build --release --features full # Everything
- ```
-
- ### Environment Variables
-
- ```bash
- # Logging
- export RUST_LOG=debug # Show debug logs
- export RUST_LOG=info # Info level
- export RUST_LOG=trace # Trace (very verbose)
-
- # Smart Tree specific
- export ST_DEFAULT_DEPTH=5 # Default tree depth
- export ST_COLOR=always # Force colors
- export ST_NO_ICONS=1 # Disable emoji icons
- export ST_MAX_FILE_SIZE=10M # Skip large files
-
- # Build
- export CARGO_BUILD_JOBS=4 # Parallel jobs during build
- ```
-
- ### Config File
-
- Create `~/.config/smart-tree/config.toml`:
-
- ```toml
- [display]
- default_depth = 5
- show_hidden = false
- use_icons = true
- color_mode = "auto"
-
- [performance]
- max_buffer_size = "100MB"
- thread_count = 8
- use_streaming = true
-
- [mcp]
- enabled = true
- port = 3000
- ```
-
- ---
-
- ## Development Workflows
-
- ### Adding a New Command-Line Option
-
- 1. Edit `src/main.rs` - Update `Cli` struct (clap parsing)
- 2. Add logic to handle the new flag
- 3. Test with: `cargo run --release -- --help`
- 4. Update docs in `README.md` and `docs/st-cheetsheet.md`
-
- ### Debugging with Smart Tree Itself
-
- ```bash
- # Context on your changes
- st -m context .
-
- # Relationships in modified files
- st -m relations --focus src/main.rs
-
- # Search for your changes
- st --search "let new_var" -m hex
-
- # Watch directory for changes (SSE)
- st --sse
- ```
-
- ### Running Tests During Development
-
- ```bash
- # Quick test (no full cleanup)
- cargo test scanner::test_
-
- # With backtrace for panics
- RUST_BACKTRACE=1 cargo test
-
- # Test a specific feature
- cargo test --features "std" --test mcp_integration
- ```
-
- ### Updating Dependencies
-
- ```bash
- # Check for outdated dependencies
- cargo outdated
-
- # Update all (carefully!)
- cargo update
-
- # Update specific crate
- cargo update --package serde
-
- # Then test!
- ./scripts/manage.sh test
- ```
-
- ---
-
- ## Code Patterns & Best Practices
-
- ### Error Handling
-
- Smart Tree uses `anyhow::Result<T>` throughout:
-
- ```rust
- use anyhow::{Context, Result};
-
- fn process_file(path: &Path) -> Result<String> {
- let content = std::fs::read_to_string(path)
- .context("Failed to read file")?;
- Ok(content)
- }
- ```
-
- **Always add context** with `.context()` - helps users debug.
-
- ### String vs str
-
- - Use `&str` for function parameters
- - Use `String` for owned values
- - Never `.clone()` when you can use references
-
- ```rust
- // Good
- fn analyze(path: &str) -> Result<()> { }
-
- // Avoid
- fn analyze(path: String) -> Result<()> { }
- ```
-
- ### Async Code
-
- Smart Tree uses `tokio` for async:
-
- ```rust
- #[tokio::main]
- async fn main() {
- // Async entry point
- }
-
- // In libraries
- pub async fn scan_async(path: &Path) -> Result<Vec<FileNode>> {
- // Async operation
- }
- ```
-
- ### Avoiding Panics
-
- - Never unwrap unless you're 100% sure (use `.expect("reason")` instead)
- - Use `?` operator for error propagation
- - Mark intentional panics with comments explaining why
-
- ```rust
- // Good
- let val = map.get("key").context("missing key")?;
-
- // Avoid
- let val = map.get("key").unwrap();
-
- // Acceptable with comment
- let val = config.port.expect("port required in config");
- ```
-
- ### Comments & Documentation
-
- Smart Tree has a fun commenting style (the "Cheet" persona). Keep it:
-
- ```rust
- // Fun but informative comments explaining the why, not the what
- // Example from scanner.rs:
- // "You've found scanner.rs, the intrepid explorer and engine room of st."
-
- /// Documentation for public items (use ///)
- /// Multiple lines are fine for complex stuff
- pub fn do_something() { }
- ```
-
- ---
-
- ## Debugging & Troubleshooting
-
- ### Common Issues
-
- **Issue**: Tests fail with permission errors
- ```bash
- # Solution: Run with elevated permissions or use test isolation
- sudo cargo test
- # Or check test directory permissions
- ls -la /tmp/st_test_*
- ```
-
- **Issue**: MCP server won't start
- ```bash
- # Solution: Check for port conflicts
- lsof -i :3000
- # Or use debug logging
- RUST_LOG=debug st --mcp
- ```
-
- **Issue**: Slow compile times
- ```bash
- # Solution: Use incremental compilation
- export CARGO_INCREMENTAL=1
- # Or use mold linker (faster than GNU ld)
- # Install: sudo apt install mold
- # Use: RUSTFLAGS="-C link-arg=-fuse-ld=mold" cargo build
- ```
-
- **Issue**: Binary won't run after build
- ```bash
- # Verify it was built
- ls -lh target/release/st
-
- # Try running directly
- ./target/release/st --version
-
- # Check dependencies
- ldd target/release/st
- ```
-
- ### Enabling Debug Output
-
- ```bash
- # Maximum verbosity
- RUST_LOG=trace cargo run --release -- . -m hex
-
- # Specific module
- RUST_LOG=st::scanner=debug cargo run --release -- .
-
- # With backtraces
- RUST_BACKTRACE=full cargo run -- .
- ```
-
- ### Profiling Performance
-
- ```bash
- # Use Linux perf (if available)
- cargo build --release
- perf record -g ./target/release/st . -m quantum > /dev/null
- perf report
-
- # Or use flamegraph (install: cargo install flamegraph)
- cargo flamegraph --bin st -- . -m quantum
- ```
-
- ---
-
- ## Git Workflow & Commits
-
- ### Pre-Commit Requirements (STRICT)
-
- Before every commit:
-
- ```bash
- ./scripts/manage.sh test
- ```
-
- This must pass with **zero clippy warnings**.
-
- ### Commit Message Format
-
- Follow Conventional Commits:
-
- ```
- feat: Add quantum compression mode
- fix: Handle symlinks in scanner
- docs: Update README with examples
- refactor: Simplify formatter trait
- perf: Optimize tree traversal
-
- Body (optional):
- - List changes
- - Explain why
- ```
+ Smart Tree (`st`) is a fast, AI-friendly directory visualization tool written in Rust. It provides 30+ MCP tools for AI assistants and is 10-24x faster than traditional `tree`.
- ### Creating Releases
+ ## Build & Development Commands
```bash
- # Bump version (updates Cargo.toml + CLAUDE.md)
- ./scripts/manage.sh bump patch # v5.4.0 → v5.4.1
- ./scripts/manage.sh bump minor # v5.4.0 → v5.5.0
- ./scripts/manage.sh bump major # v5.4.0 → v6.0.0
-
- # Build and create GitHub release
- ./scripts/manage.sh release v5.5.0 "New quantum features"
-
- # This will:
- # 1. Create git tag
- # 2. Build release artifacts
- # 3. Upload to GitHub
- # 4. Generate release notes
+ cargo build # Debug build
+ cargo build --release # Release build (LTO enabled, stripped)
+ cargo test # Run all tests
+ cargo test --release # Run performance tests
+ cargo test <test_name> # Run a single test
+ cargo run --bin st -- --help # Run locally without installing
+ cargo fmt # Format code
+ ./scripts/build-and-install.sh # Build and install locally (clears shell cache)
```
- ---
-
- ## Project Philosophy & Vision
-
- ### Core Principles
-
- 1. **Smallest and Fastest**: Every kilobyte counts, every millisecond matters
- 2. **AI-First Design**: Optimize for AI context windows (not human readability)
- 3. **Constraints = Creativity**: Limited memory? Use wave patterns. Limited tokens? Use compression.
- 4. **SID/VIC-II Wisdom**: Work within hardware constraints to achieve magic
- 5. **The Cheet's Humor**: Keep development fun and memorable
-
- ### Why Different Output Modes Matter
-
- - **classic**: Human-readable (for terminals)
- - **ai**: 80% smaller (for LLM context)
- - **quantum**: 100x+ compression (for massive projects)
- - **hex/json**: Machine parseable
- - **sse**: Real-time monitoring
-
- Each mode is optimized for its audience.
-
- ### Future Direction (Roadmap)
+ After rebuilding, if `st --version` hangs, run `hash -r` to clear your shell's binary cache.
- - Integration with MEM8 consciousness system
- - WebRTC interface for real-time collaboration
- - Python/JS SDKs for broader ecosystem
- - Kubernetes operators for deployment
- - Hot Tub visualization dashboard
+ ## Architecture
- ---
+ **Thin-client + daemon model**: The `st` CLI binary is a thin client that spawns an async daemon (`std`) for heavy lifting and persistent state. The daemon exposes an HTTP API on port **8420** (default).
- ## Quick Reference Card
+ ### CLI → Daemon Flow
- ### Commands You'll Use Most
+ Most `st` commands follow this path:
+ 1. `main.rs` parses CLI args, initializes logging
+ 2. **Exclusive modes** run locally and exit: `--mcp`, `--version`, `--completions`, `--man`, `--security-scan`, `--terminal`, `--dashboard`, `--http-daemon`, `--guardian-daemon`, daemon control flags
+ 3. **Everything else** (the common case): auto-starts daemon if needed, builds a `CliScanRequest` (defined in `daemon_cli.rs`), sends HTTP POST to `/cli/scan`, prints response
- ```bash
- # Development loop
- cargo test && cargo fmt # Quick test
- ./scripts/manage.sh test # Full checklist (do this before commits!)
+ ### Daemon Communication
- # Running st
- st . # Classic view
- st --mode quantum . # Compressed
- st --mode ai --compress . # AI-optimized
- st --mcp # Start MCP server
- st --spicy # Interactive TUI
+ - **CLI ↔ Daemon**: HTTP/JSON on port 8420, authenticated via Bearer token stored at `~/.st/daemon.token`
+ - **Daemon ↔ Daemon**: `st-protocol` binary wire protocol (6502-inspired opcodes) over Unix sockets — this is NOT used for CLI-daemon communication
+ - **Auto-spawn**: `DaemonClient::ensure_running()` spawns daemon in background with `setsid` (Unix) or `DETACHED_PROCESS` (Windows), retries with exponential backoff (5 attempts, 100ms→1600ms)
- # MCP tools
- st -m overview . # Quick scan
- st -m find --type rs . # Find Rust files
- st -m search "TODO" . # Search content
- st -m analyze --mode semantic . # Semantic analysis
+ ### MCP Integration (two modes)
- # Building/Installing
- ./scripts/manage.sh build # Build
- ./scripts/manage.sh install # Install to /usr/local/bin
- ./scripts/manage.sh release v5.5.0 "Notes" # Release
+ - **Stdio** (`st --mcp`): JSON-RPC 2.0 over stdin/stdout, used by Claude Desktop and other MCP clients. Entry: `McpServer::run_stdio()` in `src/mcp/mod.rs`
+ - **HTTP** (via daemon): REST endpoints at `/mcp/tools/list`, `/mcp/tools/call`, etc. Entry: `src/web_dashboard/mcp_http.rs`
+ - Tool consolidation: Config flag `use_consolidated_tools` reduces 50+ tools to ~15 grouped tools to save context tokens
- # Debugging
- st -m context . > context.txt # Save context
- RUST_LOG=debug st --mcp # Debug logging
- ```
+ ### Binaries (`src/bin/`)
+ - **`st`** — Main CLI (thin client), entry point in `src/main.rs`
+ - **`std`** — Always-on daemon service
+ - **`mq`** — Marqant markdown compression tool
+ - **`m8`** — Memory management tool
+ - **`n8x`** — Nexus Agent (tree orchestrator)
+ - **`import-claude-memories`** — Memory import utility
- ### Pre-Commit Checklist
+ ### Key Modules
- ```bash
- # Always do this before git commit
- ./scripts/manage.sh test
+ | Domain | Key Files | Purpose |
+ |--------|-----------|---------|
+ | Scanner | `scanner.rs`, `scanner_interest.rs`, `scanner_safety.rs`, `scanner_state.rs` | Filesystem traversal, interest scoring, change detection |
+ | Formatters | `src/formatters/` | Output format handlers (classic, AI-optimized, quantum, markdown, JSON, CSV) |
+ | MCP | `mcp.rs`, `src/mcp/` | Model Context Protocol integration (30+ tools, stdio/HTTP) |
+ | Daemon | `daemon.rs`, `daemon_client.rs`, `daemon_cli.rs` | HTTP API server for AI context |
+ | Proxy | `src/proxy/` | LLM proxy with OpenAI-compatible API |
+ | Memory | `mem8/`, `memory_manager.rs`, `mega_session_manager.rs`, `m8_*.rs` | Wave-based memory system, session persistence |
+ | TUI | `spicy_tui_enhanced.rs`, `spicy_fuzzy.rs`, `src/terminal/` | Interactive terminal UIs |
+ | Web | `src/web_dashboard/` | Browser-based dashboard with real PTY |
+ | Security | `security_scan.rs`, `ai_guardian.rs` | Supply chain scanning, prompt injection protection |
+ | Smart Tools | `src/smart/` | Context-aware AI tools |
+ | Collaboration | `src/collab/` | Multi-AI collaboration features |
+ | Compression | `compression_manager.rs`, `dynamic_tokenizer.rs`, `st_tokenizer.rs` | Smart compression, tokenization |
+ | CLI | `cli.rs` | clap-based argument parsing (60+ options) |
+ | Config | `config.rs` | Configuration management |
- # Expected output:
- # ✅ cargo test: All tests passed
- # ✅ cargo clippy: No warnings (strict mode)
- # ✅ cargo fmt: Code formatted
- # Then you can commit!
- ```
+ ### Workspace Members
+ - **`st-protocol/`** — Binary wire protocol (6502-inspired) for daemon-to-daemon communication only
+ - **`expert_prompt_engineer/`** — Prompt engineering workspace
+ - **`marqant/`** — Git submodule for markdown compression (excluded from workspace)
- ---
+ ## Configuration & Environment
- ## Useful Files to Know About
+ ### Config files
+ - `~/.st/config.toml` — Main config (API keys, model preferences, daemon settings, safety)
+ - `~/.st/daemon.token` — Auth token (auto-generated on first daemon run)
+ - `.aye_consciousness.m8` — Per-directory consciousness state (persistent AI context)
- | File | Purpose |
- |------|---------|
- | `src/main.rs` | CLI entry point (clap 4.5 argument parsing) |
- | `src/lib.rs` | Module declarations (65+ modules!) |
- | `src/scanner.rs` | Core directory traversal engine |
- | `src/formatters/mod.rs` | Formatter trait & registry |
- | `src/mcp/mod.rs` | MCP server implementation |
- | `scripts/manage.sh` | All build/test/release commands |
- | `Cargo.toml` | Dependencies & build config |
- | `tests/` | Integration tests |
- | `docs/INDEX.md` | Full documentation index |
- | `README.md` | Project overview |
+ ### Key environment variables
+ | Variable | Purpose | Default |
+ |----------|---------|---------|
+ | `ST_TOKEN_PATH` | Override auth token file location | `~/.st/daemon.token` |
+ | `ST_DEFAULT_MODE` | Default output format | `smart` |
+ | `ST_SESSION_AWARE` | Enable MCP session-aware init | disabled |
+ | `MCP_DEBUG` | Show MCP startup messages | disabled |
+ | `RUST_LOG` | Tracing log level | `info` |
- ---
+ ### Systemd services
+ - `systemd/smart-tree-daemon@.service` — User-level daemon (one per user)
+ - `systemd/smart-tree-daemon.service` — System-level daemon
+ - `systemd/smart-tree-guardian.service` — System-wide protection daemon
- ## Need More Info?
+ ## Code Conventions
- - **Full Docs**: `st --help` or `docs/INDEX.md`
- - **MCP Tools**: `./scripts/manage.sh mcp-tools`
- - **Examples**: `./scripts/manage.sh examples`
- - **Cheat Sheet**: `st --cheet`
- - **Code Comments**: Start with "The Cheet" persona comments in source
+ - **Error handling**: `anyhow::Result<T>` throughout; `thiserror` for typed errors
+ - **Async**: `tokio` runtime with full features
+ - **HTTP**: `axum` framework
+ - **Git ops**: `gix` crate
+ - **Serialization**: `serde` for all data structures
+ - **Terminal UI**: `ratatui` + `crossterm`
+ - **Commit messages**: `type: short description` (types: feat, fix, docs, style, refactor, test, chore)
- ---
+ ## Feature Flags
- **Made with Rust by the 8b-is team**
- *"Smallest and fastest."* - The Cheet
+ - Default features: none (minimal build)
+ - `candle` — Local LLM support
+ - `voice` — Voice features
+ - `full` — All features enabled