git:20260325.d6b2de5 to git:20260403.152cab1
55 added, 483 removed. Audit A to A.
# LLM CLI Module
- ## Overview
+ ## Purpose
- The `infrastructure/llm/cli/` directory contains the command-line interface for the LLM (Large Language Model) infrastructure. This thin orchestrator provides terminal access to LLM functionality, wrapping the core LLM modules with user-friendly command-line operations.
+ `infrastructure/llm/cli/` is the thin command-line wrapper around the local Ollama-backed LLM modules. It should stay focused on parsing arguments, loading config, and printing results.
- ## Directory Structure
+ ## Layout
```text
infrastructure/llm/cli/
- ├── AGENTS.md # This technical documentation
- ├── __init__.py # Package initialization
- └── main.py # CLI implementation
- ```
-
- ## CLI Architecture
-
- ### Thin Orchestrator Pattern
-
- The CLI follows the **thin orchestrator pattern**:
- - **Business Logic**: All LLM operations in `infrastructure.llm.core/`
- - **Orchestration**: CLI handles argument parsing and I/O
- - **Dependencies**: Imports from core LLM modules
- - **Error Handling**: Graceful failure with informative messages
-
- ### Main CLI Implementation (`main.py`)
-
- **Command-line interface providing access to LLM functionality:**
-
- #### Core Features
- - **Query Commands**: Interactive LLM queries with various modes
- - **Connection Testing**: Ollama server connectivity verification
- - **Model Management**: Available model discovery and selection
- - **Template Application**: Pre-built research prompt templates
- - **Configuration**: Environment-based and command-line configuration
-
- #### Supported Commands
-
- **1. Query Command**
- ```bash
- # Basic query
- python3 -m infrastructure.llm.cli query "What is machine learning?"
-
- # Response modes
- python3 -m infrastructure.llm.cli query --short "Summarize this concept"
- python3 -m infrastructure.llm.cli query --long "Explain in detail"
- python3 -m infrastructure.llm.cli query --structured --schema research.json
-
- # Streaming output
- python3 -m infrastructure.llm.cli query --stream "Write a poem"
-
- # Generation control
- python3 -m infrastructure.llm.cli query --temperature 0.0 --seed 42 --max-tokens 500 "Deterministic response"
- ```
-
- **2. Connection Testing**
- ```bash
- # Check Ollama availability
- python3 -m infrastructure.llm.cli check
-
- # Expected output:
- # ✓ Connected to Ollama at http://localhost:11434
- # ✓ Default model 'gemma3:4b' is available
- ```
-
- **3. Model Discovery**
- ```bash
- # List available models
- python3 -m infrastructure.llm.cli models
-
- # Expected output:
- # Available models:
- # • gemma3:4b (4.0B parameters, 128K context)
- # • llama3.2:3b (3.2B parameters, 128K context)
- # • codellama:7b (7.3B parameters, 16K context)
- ```
-
- **4. Template Application**
- ```bash
- # List available templates
- python3 -m infrastructure.llm.cli template --list
-
- # Apply research template
- python3 -m infrastructure.llm.cli template summarize_abstract --input "Abstract text here..."
-
- # Template with custom parameters
- python3 -m infrastructure.llm.cli template literature_review --papers "paper1,paper2" --focus "methodology"
- ```
-
- ## Command Reference
-
- ### Query Command Options
-
- | Option | Short | Description | Default |
- |--------|-------|-------------|---------|
- | `--short` | `-s` | Short response (< 150 tokens) | False |
- | `--long` | `-l` | Long response (> 500 tokens) | False |
- | `--structured` | | JSON-structured response | False |
- | `--stream` | | Stream response in real-time | False |
- | `--model` | `-m` | Override default model | Auto-detected |
- | `--temperature` | `-t` | Sampling temperature (0.0-2.0) | 0.7 |
- | `--max-tokens` | | Maximum tokens to generate | Mode-dependent |
- | `--seed` | | Random seed for reproducibility | None |
- | `--schema` | | JSON schema file for structured mode | None |
- | `--system-prompt` | | Custom system prompt | Default research assistant |
-
- ### Global Options
-
- | Option | Description | Default |
- |--------|-------------|---------|
- | `--verbose`, `-v` | Enable verbose logging | False |
- | `--quiet`, `-q` | Suppress non-essential output | False |
- | `--config` | Path to config file | Environment-based |
-
- ## Implementation Details
-
- ### Command Structure
-
- **Main Entry Point:**
- ```python
- def main():
- """Main CLI entry point."""
- parser = argparse.ArgumentParser(
- description="LLM CLI for research assistance",
- formatter_class=argparse.RawDescriptionHelpFormatter
- )
-
- # Global options
- parser.add_argument('--verbose', '-v', action='store_true')
- parser.add_argument('--quiet', '-q', action='store_true')
-
- # Subcommands
- subparsers = parser.add_subparsers(dest='command', help='Available commands')
-
- # Query command
- query_parser = subparsers.add_parser('query', help='Query the LLM')
- # ... query options ...
-
- # Check command
- check_parser = subparsers.add_parser('check', help='Check Ollama connection')
-
- # Models command
- models_parser = subparsers.add_parser('models', help='List available models')
-
- # Template command
- template_parser = subparsers.add_parser('template', help='Apply research templates')
-
- # Parse arguments
- args = parser.parse_args()
-
- # Execute command
- if args.command == 'query':
- query_command(args)
- elif args.command == 'check':
- check_command(args)
- # ... other commands ...
- ```
-
- ### Query Command Implementation
-
- **Query Execution Flow:**
- 1. **Configuration Loading**: Load from environment and apply overrides
- 2. **Model Selection**: Auto-detect best model or use specified model
- 3. **Connection Verification**: Ensure Ollama server is available
- 4. **Option Building**: Construct `GenerationOptions` from arguments
- 5. **Query Execution**: Call appropriate LLMClient method
- 6. **Output Handling**: Display results with appropriate formatting
-
- **Mode-Specific Logic:**
- ```python
- def query_command(args):
- """Handle query command execution."""
- # Load configuration
- config = LLMConfig.from_env()
-
- # Apply command-line overrides
- if args.model:
- config = config.with_overrides(default_model=args.model)
-
- # Auto-select best model if none specified
- if not args.model and is_ollama_running():
- best_model = select_best_model()
- if best_model:
- config = config.with_overrides(default_model=best_model)
-
- # Create client
- client = LLMClient(config)
-
- # Verify connection
- if not client.check_connection():
- print("❌ Error: Cannot connect to Ollama server", file=sys.stderr)
- print("💡 Start Ollama with: ollama serve", file=sys.stderr)
- sys.exit(1)
-
- # Build generation options
- options = GenerationOptions(
- temperature=args.temperature,
- max_tokens=get_max_tokens_for_mode(args),
- seed=args.seed if args.seed is not None else None,
- # ... other options
- )
-
- # Execute query based on mode
- if args.stream:
- stream_response(client, args.prompt, args, options)
- else:
- execute_query(client, args.prompt, args, options)
- ```
-
- ### Response Mode Handling
-
- **Short Mode (< 150 tokens):**
- ```python
- if args.short:
- response = client.query_short(args.prompt, options=options)
- # Additional validation for short responses
- ```
-
- **Long Mode (> 500 tokens):**
- ```python
- if args.long:
- response = client.query_long(args.prompt, options=options)
- # Additional validation for responses
- ```
-
- **Structured Mode (JSON):**
- ```python
- if args.structured:
- schema = load_json_schema(args.schema)
- response = client.query_structured(args.prompt, schema=schema, options=options)
- # JSON validation and pretty printing
- ```
-
- **Streaming Mode:**
- ```python
- if args.stream:
- for chunk in client.stream_query(args.prompt, options=options):
- print(chunk, end="", flush=True)
- print() # Final newline
+ ├── AGENTS.md
+ ├── README.md
+ ├── __init__.py
+ └── main.py
```
- ## Configuration Integration
-
- ### Environment Variable Support
-
- **CLI Respects All LLM Environment Variables:**
- ```bash
- # Model and connection
- OLLAMA_HOST=http://localhost:11434 OLLAMA_MODEL=gemma3:4b python3 -m infrastructure.llm.cli query "test"
-
- # Generation parameters
- LLM_TEMPERATURE=0.0 LLM_SEED=42 python3 -m infrastructure.llm.cli query "deterministic output"
-
- # Context and limits
- LLM_CONTEXT_WINDOW=131072 LLM_MAX_TOKENS=4096 python3 -m infrastructure.llm.cli query "long context"
- ```
+ ## Entry Point
- ### Command-Line Overrides
+ Run the CLI with:
- **CLI Arguments Override Environment:**
```bash
- # Override model selection
- python3 -m infrastructure.llm.cli query --model llama3.2:3b "test query"
-
- # Override generation parameters
- python3 -m infrastructure.llm.cli query --temperature 0.0 --max-tokens 100 "constrained response"
- ```
-
- ## Error Handling and User Experience
-
- ### Connection Error Handling
-
- **Graceful Connection Failures:**
- ```python
- # Connection check with helpful error messages
- if not client.check_connection():
- print("❌ Error: Cannot connect to Ollama server", file=sys.stderr)
- print("💡 Start Ollama with: ollama serve", file=sys.stderr)
- print("💡 Check status with: python3 -m infrastructure.llm.cli check", file=sys.stderr)
- sys.exit(1)
- ```
-
- ### Model Availability Checking
-
- **Automatic Model Selection:**
- ```python
- # Auto-discover best available model
- if is_ollama_running():
- best_model = select_best_model()
- if best_model:
- print(f"📋 Using model: {best_model}")
- config = config.with_overrides(default_model=best_model)
- else:
- print("⚠️ No suitable models found. Install with: ollama pull gemma3:4b")
+ uv run python -m infrastructure.llm.cli
```
- ### Input Validation
+ ## Implementation Surface
- **Prompt Validation:**
- ```python
- if not args.prompt or not args.prompt.strip():
- print("❌ Error: Query prompt cannot be empty", file=sys.stderr)
- sys.exit(1)
+ `main.py` defines:
- if len(args.prompt.strip()) < 3:
- print("⚠️ Warning: Very short prompt may produce poor results")
- ```
+ - `CLIError`
+ - `query_command(args: argparse.Namespace) -> None`
+ - `check_command(args: argparse.Namespace) -> None`
+ - `models_command(args: argparse.Namespace) -> None`
+ - `template_command(args: argparse.Namespace) -> None`
+ - `create_parser() -> argparse.ArgumentParser`
+ - `main() -> None`
- ## Output Formatting
+ ## Commands
- ### Response Display
+ ### `query`
- **Standard Output:**
- ```python
- # Direct output for piping/scripting
- print(response)
- ```
+ Uses `OllamaClientConfig.from_env()` and `LLMClient` to send a prompt to the configured Ollama endpoint.
- **Verbose Mode:**
- ```python
- if args.verbose:
- print(f"📋 Model: {client.config.default_model}")
- print(f"🌡️ Temperature: {options.temperature}")
- print(f"🎯 Max tokens: {options.max_tokens}")
- print(f"🔢 Seed: {options.seed}")
- print("-" * 50)
- print(response)
- ```
+ Flags:
- ### Structured Output
+ - `--short`
+ - `--long`
+ - `--stream`
+ - `--model`
+ - `--temperature`
+ - `--max-tokens`
+ - `--seed`
- **JSON Mode Formatting:**
- ```python
- if args.structured:
- try:
- # Parse and validate JSON
- data = json.loads(response)
- # Pretty print for readability
- print(json.dumps(data, indent=2))
- except json.JSONDecodeError:
- print("❌ Error: Response is not valid JSON", file=sys.stderr)
- print(response, file=sys.stderr)
- sys.exit(1)
- ```
+ ### `check`
- ## Integration with Core Modules
+ Verifies that the configured Ollama endpoint is reachable.
- ### Dependencies
+ ### `models`
- **Core LLM Infrastructure:**
- - `infrastructure.llm.core.client` - LLMClient for queries
- - `infrastructure.llm.core.config` - Configuration management
- - `infrastructure.llm.utils.ollama` - Ollama utilities
- - `infrastructure.llm.templates` - Template system
+ Lists available models from the local Ollama daemon.
- **Shared Infrastructure:**
- - `infrastructure.core.logging.utils` - Logging integration
- - `infrastructure.core.exceptions` - Error handling
+ ### `template`
- ### Import Structure
+ Applies a prompt template from `infrastructure.llm.templates`. Supports `--list` and `--input`.
- **Clean Separation:**
- ```python
- # Core functionality imports
- from infrastructure.llm.core.client import LLMClient
- from infrastructure.llm.core.config import LLMConfig, GenerationOptions
+ ## Dependencies
- # Utility imports
- from infrastructure.llm.utils.ollama import select_best_model, is_ollama_running
+ The CLI imports:
- # Shared infrastructure
- from infrastructure.core.logging.utils import get_logger
- from infrastructure.core.exceptions import LLMConnectionError
- ```
+ - `LLMClient`
+ - `GenerationOptions`
+ - `OllamaClientConfig`
+ - `select_best_model`
+ - `is_ollama_running`
+ - `TEMPLATES`
+ - `get_template`
## Testing
- ### CLI Testing
-
- CLI behavior is tested without mocks, using real command invocations:
-
- - **Pure logic tests** cover argument parsing and option wiring.
- - **Subprocess tests** invoke the CLI via `uv run python -m infrastructure.llm.cli ...` and assert on exit codes and stdout/stderr.
- - **Integration tests** that require a running Ollama instance are marked with `@pytest.mark.requires_ollama` and may be skipped in CI when the service is unavailable.
-
- ## Performance Considerations
-
- ### Response Time Optimization
-
- **Connection Reuse:**
- - LLMClient maintains connection across queries
- - Avoids reconnection overhead
- - Context preservation for follow-up queries
-
- **Streaming for Large Responses:**
- - Real-time output for long responses
- - Reduced memory usage for large content
- - User feedback during generation
-
- ### Resource Management
-
- **Memory Efficiency:**
- - Streaming responses prevent large memory allocation
- - Context pruning for long conversations
- - Automatic cleanup of resources
-
- ## Troubleshooting
-
- ### Common Issues
-
- **Connection Problems:**
- ```bash
- # Check Ollama status
- python3 -m infrastructure.llm.cli check
-
- # If not running:
- ollama serve # Start in another terminal
-
- # Check model availability
- python3 -m infrastructure.llm.cli models
- ```
-
- **Model Issues:**
- ```bash
- # Install required model
- ollama pull gemma3:4b
-
- # Check model loading
- ollama ps
- ```
-
- **Configuration Issues:**
- ```bash
- # Check environment variables
- env | grep -E "(OLLAMA|LLM)_"
-
- # Test with explicit configuration
- OLLAMA_HOST=http://localhost:11434 python3 -m infrastructure.llm.cli check
- ```
-
- ### Debug Mode
-
- **Verbose Operation:**
- ```bash
- # Enable debug logging
- LOG_LEVEL=0 python3 -m infrastructure.llm.cli query --verbose "debug test"
-
- # Check detailed connection info
- python3 -c "
- from infrastructure.llm.core.client import LLMClient
- client = LLMClient()
- print(f'Connected: {client.check_connection()}')
- print(f'Available models: {client.get_available_models()}')
- "
- ```
-
- ## Usage Examples
-
- ### Basic Research Assistance
-
- **Quick Queries:**
- ```bash
- # Define a concept
- python3 -m infrastructure.llm.cli query --short "What is a neural network?"
-
- # Get detailed explanation
- python3 -m infrastructure.llm.cli query --long "Explain backpropagation algorithm"
-
- # Structured analysis
- python3 -m infrastructure.llm.cli query --structured "Analyze this dataset" --schema analysis_schema.json
- ```
-
- ### Research Workflow Integration
-
- **Manuscript Review:**
- ```bash
- # Review paper section
- cat manuscript/methodology.md | python3 -m infrastructure.llm.cli query "Review this methodology section for clarity and completeness"
-
- # Generate abstract summary
- python3 -m infrastructure.llm.cli template summarize_abstract --input "$(cat manuscript/abstract.md)"
- ```
-
- ### Development Assistance
-
- **Code Documentation:**
- ```bash
- # Generate docstring
- python3 -m infrastructure.llm.cli query "Generate a Python docstring for this function" --input "$(cat src/analysis.py)"
- ```
-
- ## Future Enhancements
-
- ### Planned Features
-
- **CLI Capabilities:**
- - **Interactive Mode**: Multi-turn conversations
- - **Batch Processing**: Multiple queries from file
- - **Output Formatting**: Markdown, JSON, CSV export
- - **Template Management**: Custom template creation and management
- - **History**: Query history and favorites
-
- **Integration Improvements:**
- - **Pipe Support**: Accept input from stdin
- - **File Output**: Save responses to files
- - **Progress Indication**: Progress bars for long operations
- - **Error Recovery**: Automatic retry with backoff
+ - Deterministic tests should cover parser wiring and command behavior.
+ - Real-daemon smoke tests should use `@pytest.mark.requires_ollama`.
+ - End-to-end CLI tests should run through `uv run python -m infrastructure.llm.cli ...`.
- ## See Also
+ ## Related Docs
- **Related Documentation:**
- - [`../core/AGENTS.md`](../core/AGENTS.md) - Core LLM functionality
- - [`../templates/AGENTS.md`](../templates/AGENTS.md) - Template system
- - [`../utils/AGENTS.md`](../utils/AGENTS.md) - LLM utilities
+ - `README.md`
+ - `../README.md`
+ - `../AGENTS.md`
+ - `../core/README.md`
+ - `../utils/README.md`
+ - `../../../docs/operational/troubleshooting/llm-review.md`
- **System Documentation:**
- - [`../../../AGENTS.md`](../../../AGENTS.md) - LLM module overview
- - [`../../../infrastructure/core/AGENTS.md`](../../../infrastructure/core/AGENTS.md) - Core infrastructure
- - [`../../../docs/operational/troubleshooting/llm-review.md`](../../../docs/operational/troubleshooting/llm-review.md) - LLM troubleshooting guide