CLAUDE.md · diff

git:20260305.a7f1ee7 to git:20260306.d94d5f5

4 added, 4 removed. Audit A to A.

# CLAUDE.md

This file provides guidance to Claude Code when working in this repository.

## Project Summary

Web Forager — a search-and-fetch toolkit for AI agents with three MCP capabilities:

- Web search via DuckDuckGo (ddgs)

- News search via DuckDuckGo (ddgs)

- - URL fetch + conversion via Jina Reader

+ - URL fetch + conversion via direct HTTP/trafilatura (Jina Reader as fallback)

Also includes five Agent Skills (in `skills/`) that orchestrate search + fetch into

specialized workflows: web-research, fact-check, news-monitor, competitive-intel, tech-advisor.

Implements MCP over STDIO using FastMCP.

## Key Paths

- `src/web_forager/cli.py`: CLI entry point (serve/search/news/fetch/version)

- `src/web_forager/duckduckgo_search.py`: DuckDuckGo web search implementation

- `src/web_forager/duckduckgo_news.py`: DuckDuckGo news search implementation

- - `src/web_forager/jina_fetch.py`: Jina Reader fetch implementation

+ - `src/web_forager/web_fetch.py`: URL fetch (direct HTTP + trafilatura, Jina Reader fallback)

- `src/web_forager/server.py`: MCP server setup

- `src/web_forager/_version.py`: generated by setuptools_scm (do not edit)

- `skills/`: Agent Skills (each has a SKILL.md following the Agent Skills spec)

## Common Commands

### Setup

```bash

uv pip install -e .

uv pip install -e ".[dev]"

```

### Run

```bash

web-forager serve

web-forager serve --debug

```

### CLI Smoke Tests

```bash

web-forager search "your search query" --max-results 5 --safesearch moderate

web-forager news "your search query" --max-results 10

web-forager fetch "https://example.com" --format markdown --with-images

web-forager fetch "https://example.com" --format json

web-forager version

```

### Quality

```bash

black src

isort src

mypy src

pytest

pytest --cov=web_forager

```

## Workflow Guidelines

- Keep changes small and aligned with existing patterns.

- Read relevant files before editing; avoid speculative edits.

- Do not edit generated files (notably `src/web_forager/_version.py`).

- Do not add or expose secrets in code or docs.

- If you change CLI flags or MCP tool behavior, update README.md accordingly.

- Prefer running targeted tests for the change when possible.

## Notes

- Search and news search use the `ddgs` package (renamed from `duckduckgo-search`).

- - Fetch uses Jina Reader at `https://r.jina.ai/`.

- - Skills work standalone (ddgs + Jina HTTP API) and prefer MCP tools when available.

+ - Fetch tries direct HTTP + trafilatura first, falls back to Jina Reader at `https://r.jina.ai/`.

+ - Skills work standalone (ddgs + direct HTTP/Jina) and prefer MCP tools when available.