# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

This is **gurume** - a Python library and MCP (Model Context Protocol) server for searching restaurants on Tabelog using web scraping. It provides both a Python library (`gurume`) and an MCP server (`gurume`) for integration with AI assistants.

**See also**: [IDEAS.md](IDEAS.md) - Feature ideas, improvements, and development notes

## Development Commands

### Testing
```bash
# Run all tests with coverage
make test
# or
uv run pytest -v -s --cov=src tests

# Run specific test file
uv run pytest tests/test_models.py -v
uv run pytest tests/test_restaurant.py -v
uv run pytest tests/test_search.py -v
```

### Code Quality
```bash
# Run linter
make lint
# or
uv run ruff check .

# Run type checker (重要！每次修改程式碼後都要執行)
make type
# or
uv run ty check .

# 完整檢查流程
uv run ruff check .  # 程式碼風格檢查
uv run ty check .    # 型別檢查
uv run pytest -v     # 執行測試
```

### Publishing
```bash
make publish  # Builds wheel and publishes to PyPI
```

### Running Examples
```bash
# Basic search example
uv run python examples/basic_search.py
```

### Running MCP Server
```bash
# Via entry point (requires installation)
uv run gurume mcp

# Local development
uv run --directory /path/to/gurume gurume mcp

# Direct execution
uv run python -m gurume.server
```

## Architecture

### Core Modules (src/gurume/)

The codebase has the following main files:

1. **restaurant.py** - Core scraping and search implementation
   - `Restaurant` dataclass: Represents restaurant data (name, rating, reviews, prices, etc.)
   - `RestaurantSearchRequest` dataclass: Builds search URLs and scrapes Tabelog
   - `SortType` enum: Search sorting options (STANDARD, RANKING, REVIEW_COUNT, NEW_OPEN)
   - `PriceRange` enum: Extensive lunch/dinner price ranges (B001-B012, C001-C012)
   - `query_restaurants()` function: Cached convenience function for quick searches
   - Key methods: `_build_params()`, `_parse_restaurants()`, `search_sync()`, `search()`
   - **Area filtering**: Uses area slug paths (e.g., `/tokyo/rstLst/`) for accurate prefecture-level filtering
   - **Cuisine filtering**: Supports `genre_code` parameter for precise cuisine type filtering (e.g., `RC0107` for すき焼き)
   - **Caching and retry**: Integrated with cache.py and retry.py via `use_cache` and `use_retry` parameters (both default to True)

2. **cache.py** - Caching system for HTTP responses (🆕)
   - `CacheEntry` dataclass: Stores data with timestamp and TTL
   - `MemoryCache` class: In-memory cache with LRU eviction
     - Default: 1000 entries max, 1 hour TTL
     - Methods: `get()`, `set()`, `clear()`, `size()`
     - Automatic oldest entry eviction when full
   - `FileCache` class: File-based persistent cache
     - Stores as JSON with SHA256 hashed filenames
     - Useful for development and debugging
   - Global cache functions:
     - `get_cache()`: Returns global cache instance
     - `set_cache()`: Sets cache backend (memory or file)
     - `generate_cache_key()`: Creates consistent keys from URL + params
     - `cached_get()`: Get from cache with force refresh option
     - `cache_set()`: Store to cache with custom TTL
     - `clear_cache()`: Clear all entries
   - **Integration**: Automatically used in `RestaurantSearchRequest.search_sync()` and `.search()` with 30-min TTL

3. **retry.py** - Retry and resilience mechanisms (🆕)
   - `is_retryable_error()`: Checks if exception should trigger retry
   - `handle_http_errors()`: Converts HTTP errors to custom exceptions
   - `fetch_with_retry()`: Sync fetch with automatic retry
     - Max 3 attempts with exponential backoff (1s, 2s, 4s)
     - Retries on: ConnectError, TimeoutException, NetworkError, 5xx errors
     - Raises: RateLimitError (429), NetworkError (other errors)
   - `fetch_with_retry_async()`: Async version with manual retry loop
   - Configuration: `DEFAULT_MAX_ATTEMPTS=3`, `DEFAULT_MIN_WAIT=1`, `DEFAULT_MAX_WAIT=10`
   - Uses `tenacity` library for retry decorators with exponential backoff and jitter
   - **Integration**: Automatically used in `RestaurantSearchRequest.search_sync()` and `.search()` when `use_retry=True`

4. **search.py** - Higher-level search API with metadata and pagination
   - `SearchRequest` dataclass: Wraps `RestaurantSearchRequest` with pagination support
   - `SearchResponse` dataclass: Structured response with status and metadata
   - `SearchMeta` dataclass: Total counts, page info, navigation flags
   - `SearchStatus` enum: SUCCESS, NO_RESULTS, ERROR
   - Supports multi-page scraping via `max_pages` parameter
   - Both sync (`search_sync()`) and async (`search()`) methods
   - **Area filtering**: Automatically uses area slug paths when prefecture can be mapped
   - **Cuisine filtering**: Supports `genre_code` parameter for URL path construction with cuisine types

5. **area_mapping.py** - Area name to URL slug mapping
   - `PREFECTURE_MAPPING`: Maps all 47 prefectures to Tabelog URL slugs
   - `CITY_MAPPING`: Maps major cities to URL slugs (tokyo, osaka, kyoto, etc.)
   - `get_area_slug()`: Converts area names (東京都, 大阪府) to URL slugs (tokyo, osaka)
   - **Purpose**: Enables accurate area filtering by using Tabelog's path-based filtering

6. **genre_mapping.py** - Cuisine type to genre code mapping
   - `GENRE_CODE_MAPPING`: Maps 45+ Japanese cuisine types to Tabelog genre codes (RC codes)
   - `get_genre_code()`: Converts cuisine name (すき焼き) to genre code (RC0107)
   - `get_genre_name_by_code()`: Reverse lookup from genre code to cuisine name
   - `get_all_genres()`: Returns list of all supported cuisine types
   - **Purpose**: Enables accurate cuisine filtering via URL path-based genre codes
   - **Coverage**: Supports common Japanese cuisine types (和食, 洋食, 中華, 焼肉, 鍋, 居酒屋, カレー, カフェ, etc.)

7. **suggest.py** - Area and keyword suggestion API integration
   - `AreaSuggestion` dataclass: Represents area/station suggestions from Tabelog
   - `KeywordSuggestion` dataclass: Represents keyword suggestions (cuisine types, restaurant names, combinations)
   - `get_area_suggestions()`: Sync function to get area suggestions
   - `get_area_suggestions_async()`: Async function to get area suggestions
   - `get_keyword_suggestions()`: Sync function to get keyword suggestions (🆕)
   - `get_keyword_suggestions_async()`: Async function to get keyword suggestions (🆕)
   - **API**: Uses `https://tabelog.com/internal_api/suggest_form_words`
     - `sa=query`: Area suggestions (prefecture, city, station)
     - `sk=query`: Keyword suggestions (cuisine, restaurant, combinations) (🆕)
   - **Response datatypes**:
     - Area: `AddressMaster`, `RailroadStation`
     - Keyword: `Genre2` (cuisine type), `Restaurant` (restaurant name), `Genre2 DetailCondition` (cuisine + condition)

8. **cli.py** - Command-line interface using Typer and Rich
   - `app`: Main Typer application instance
   - `search()`: Search restaurants with various filters (area, keyword, cuisine, query, sort, limit, output format)
   - `list_cuisines()`: Display all supported cuisine types in a table
   - `tui()`: Launch the interactive TUI
   - **Output formats**: table (Rich table), json (structured JSON), simple (plain text)
   - **Features**: Color-coded output, error handling, cuisine auto-detection, genre code filtering, AI natural language parsing
   - **CLI options**: -a/--area, -k/--keyword, -c/--cuisine, -q/--query (🆕), -s/--sort, -n/--limit, -o/--output
   - **Natural language query (-q)**: Uses `llm.parse_user_input()` to parse user input in multiple languages (Chinese, Japanese, English)
   - **Auto-detection**: Automatically detects cuisine types in keyword and converts to genre_code for precise filtering
   - Entry point: `gurume search` or direct execution

9. **tui.py** - Interactive terminal UI using Textual framework
   - `TabelogApp`: Main TUI application class
   - `SearchPanel`: Search input panel with area, keyword, and sorting options
   - `ResultsTable`: DataTable for displaying restaurant results
   - `DetailPanel`: Panel showing detailed restaurant information
   - `AreaSuggestModal`: Modal popup for area suggestions (F2 key)
   - `GenreSuggestModal`: Modal popup for static cuisine type list (F3 key, when keyword is empty)
   - `KeywordSuggestModal`: Modal popup for dynamic keyword suggestions (F3 key, when keyword has content) (🆕)
   - **Intelligent F3 behavior** (🆕):
     - Empty keyword → Shows `GenreSuggestModal` with 45+ static cuisine types (fast, no API call)
     - Non-empty keyword → Calls API and shows `KeywordSuggestModal` with dynamic suggestions (cuisine types, restaurant names, combinations)
     - Icon differentiation: 🍜 (Genre2/cuisine), 🏪 (Restaurant), 🔖 (combinations)
   - **Auto-detection**: Automatically detects cuisine names in keyword input and converts to genre_code
   - **Smart linking**: AI parse (F4) automatically triggers area suggest (F2) or genre suggest (F3) after parsing
   - Features: RadioButton sorting, two-column layout, async worker management, context-aware autocomplete
   - Keybindings: F2 (area suggest), F3 (intelligent keyword/genre suggest), F4 (AI parse), s (search), r (results), d (detail), q (quit)
   - Entry point: `python -m gurume.tui` or `uv run gurume tui`

10. **server.py** - MCP (Model Context Protocol) server implementation using FastMCP (🔄 Refactored)
    - `mcp`: FastMCP server instance (high-level API)
    - **Tools** (4 total, all with `tabelog_` prefix):
      - `tabelog_search_restaurants`: Search restaurants with area, keyword, cuisine, sort, limit parameters
      - `tabelog_list_cuisines`: Get all 45+ cuisine types with genre codes
      - `tabelog_get_area_suggestions`: Get area/station suggestions from Tabelog API
      - `tabelog_get_keyword_suggestions`: Get keyword/cuisine/restaurant suggestions from Tabelog API
    - **Architecture** (符合 SKILL.md 規範):
      - ✅ **FastMCP API**: Uses `@mcp.tool()` decorator for automatic schema generation
      - ✅ **Pydantic Output Schemas**: `RestaurantOutput`, `CuisineOutput`, `SuggestionOutput` with Field descriptions
      - ✅ **Tool Annotations**: All tools have proper hints (readOnly, idempotent, openWorld)
      - ✅ **Comprehensive error handling**: Try-except with actionable error messages (ValueError for validation, RuntimeError for execution)
      - ✅ **Auto schema generation**: Input schemas automatically generated from type hints
      - ✅ **Auto serialization**: Returns Pydantic models, FastMCP handles JSON serialization
    - **Design principles**:
      - ✅ Zero configuration (no API keys required)
      - ✅ Type-safe with full type hints
      - ✅ Read-only operations (all tools marked with readOnlyHint=True)
      - ✅ Structured output for machine readability
    - **Implementation details**:
      - Uses `@mcp.tool(annotations=ToolAnnotations(...))` for tool registration
      - Docstrings automatically become tool descriptions (supports markdown formatting)
      - Parameter validation with clear error messages
      - Async-first design for all I/O operations
    - **Transport**: stdio (standard input/output)
    - **Entry point**: `gurume mcp` command or `python -m gurume.server`
    - **File size**: 505 lines (refactored from 286 lines with low-level Server API)

11. **__init__.py** - Public API exports
   - Exports: `Restaurant`, `RestaurantSearchRequest`, `SearchRequest`, `SearchResponse`, `SortType`, `PriceRange`, `query_restaurants`, `AreaSuggestion`, `get_area_suggestions`, `get_area_suggestions_async`, `get_genre_code`, `get_genre_name_by_code`, `get_all_genres`

### API Patterns

The library provides **three levels of abstraction**:

1. **Quick function** (simplest): `query_restaurants()` - cached, sync-only
2. **Core request** (flexible): `RestaurantSearchRequest` - full control, sync/async
3. **Advanced search** (metadata): `SearchRequest` - pagination, status, metadata

### Important Implementation Details

- **Web scraping**: Uses `httpx` for requests and `BeautifulSoup` (lxml parser) for HTML parsing
- **CSS selectors**: Relies on specific Tabelog CSS classes (`list-rst`, `c-rating__val`, etc.)
- **HTML format handling**: Supports multiple Tabelog HTML formats for backward compatibility:
  - New format: `<div class="list-rst__area-genre"> [縣] 市區 / 類型</div>` (優先)
  - Old format: `<span class="list-rst__area-genre">地區、駅名 距離</span>` (fallback)
  - Genre extraction: Parses from area-genre field and separate `list-rst__genre` element
- **Area filtering** (CRITICAL):
  - **Problem**: Tabelog's `/rst/rstsearch?sa=area` endpoint DOES NOT filter by area - always returns national results
  - **Solution**: Use path-based URLs (e.g., `/tokyo/rstLst/` instead of `/rst/rstsearch?sa=東京`)
  - **Implementation**: `area_mapping.py` maps prefecture names to URL slugs, automatically used in search methods
  - **Limitation**: Only works for 47 prefectures + major cities; city/station-level filtering not supported
  - **Fallback**: If area cannot be mapped to slug, falls back to `/rst/rstsearch` (returns national results)
- **Cuisine filtering** (CRITICAL):
  - **Problem**: Using `keyword` parameter for cuisine types returns broad results (restaurants mentioning the cuisine, not specialists)
  - **Solution**: Use genre codes in URL path (e.g., `/mie/rstLst/RC0107/` for すき焼き specialists in 三重)
  - **Implementation**: `genre_mapping.py` maps cuisine names to genre codes, automatically used in search methods
  - **URL patterns**: Three combinations supported:
    - Area + Cuisine: `/area_slug/rstLst/GENRE_CODE/` (most precise)
    - Area only: `/area_slug/rstLst/`
    - Cuisine only: `/rstLst/GENRE_CODE/`
  - **Coverage**: 45+ common Japanese cuisine types with RC codes (RC0107-RC2101)
  - **Auto-detection**: TUI automatically detects cuisine names in keyword input and converts to genre_code
- **Keyword/Cuisine suggestion API** (NEW - discovered 2025-12-29):
  - **Endpoint**: `https://tabelog.com/internal_api/suggest_form_words`
  - **Parameters**:
    - `sa=query`: Area suggestions (prefecture, station, city)
    - `sk=query`: Keyword suggestions (cuisine, restaurant names, combinations) (🆕)
  - **Response structure**: JSON array with `name`, `datatype`, `id_in_datatype`, optional `lat`/`lng`
  - **datatypes** for keyword (`sk`):
    - `Genre2`: Cuisine type (すき焼き, 寿司, ラーメン)
    - `Restaurant`: Restaurant name (和田金, すきやき割烹 美川)
    - `Genre2 DetailCondition`: Cuisine + condition (すき焼き ランチ, 寿司 接待)
  - **Usage in TUI**: F3 key intelligently switches between static genre list (empty keyword) and dynamic API suggestions (non-empty keyword)
- **Error handling**: Gracefully skips malformed items during parsing (try/except with continue)
- **Caching**: `query_restaurants()` uses `@cache` decorator for performance
- **User-Agent**: All requests include browser User-Agent to avoid bot detection
- **Date/time format**: Validates `YYYYMMDD` for dates, `HHMM` for times in `__post_init__`
- **String cleaning**: Auto-strips whitespace from `area` and `keyword` parameters

## Testing Strategy

- **Location**: All tests in `tests/` directory
- **Current coverage**: 71% overall, 87-94% on core modules (target: 90%+)
- **Total tests**: 101 tests (all passing ✓)
- **Test types**:
  - Unit tests: Models, validation, parameter building, caching, retry logic
  - Function tests: HTTP requests, HTML parsing, error handling
  - Integration tests: End-to-end search workflows
- **New test files** (🆕):
  - `test_cache.py`: 12 tests for MemoryCache, FileCache, and cache helpers
  - `test_retry.py`: 11 tests for retry logic, error handling, and exponential backoff
- **Mocking**: Mock `httpx.get` and `httpx.AsyncClient` for HTTP tests (no real network calls)
- **Async**: Uses `pytest-asyncio` with `AsyncMock` for async operations
- **Performance**: All tests run in ~6 seconds with mocking (no actual retry delays)
- **Note**: TUI (tui.py) and CLI (cli.py) currently have 0% coverage (UI modules)

## Type Annotations

- **Required**: Full type hints on all functions
- **Style**: Built-in types only (e.g., `list[str]`, `dict[str, Any]`, `X | None`)
- **Checker**: Uses `ty` (configured in Makefile and pyproject.toml)
- **py.typed marker**: Present in `src/gurume/py.typed` for library consumers

## Code Style

- **Formatter/Linter**: ruff with specific rules (see pyproject.toml)
- **Line length**: 120 characters max
- **Import sorting**: Single-line imports enforced (`force-single-line = true`)
- **Ignored**: E501, C901 per pyproject.toml
- **Selected rules**: B (bugbear), C (comprehensions), E/W (pycodestyle), F (pyflakes), I (isort), N (naming), SIM (simplify), UP (pyupgrade)

## Pre-commit Hooks

Configured in `.pre-commit-config.yaml`:
1. Basic checks (YAML, EOF, trailing whitespace)
2. Ruff (lint + format)
3. MyPy (type checking)
4. UV lock file sync

## Key Constraints & Considerations

- **Web scraping fragility**: Tabelog's HTML structure may change without notice
- **Rate limiting**: No built-in rate limiting - users must implement for production use
- **Legal compliance**: Library is for educational/research purposes - respect robots.txt and ToS
- **Fallback selectors**: Parser includes backup CSS selectors for robustness
- **MCP server** (🆕): Fully implemented in `server.py` with 4 tools, zero-config design, no API key required

## Dependencies

**Production**:
- `beautifulsoup4` - HTML parsing
- `lxml` - BeautifulSoup parser backend
- `httpx` - HTTP client (sync + async)
- `loguru` - Logging
- `mcp[cli]` - MCP server framework
- `typer` - CLI framework
- `tenacity` - Retry logic with exponential backoff (🆕)

**Development**:
- `pytest`, `pytest-asyncio`, `pytest-cov` - Testing
- `ruff` - Linting/formatting
- `ty` - Type checking