--- name: gpt-image-markup-edit description: >- Markup-guided image edits with GPT Image 2.6 on varg: draw circles, boxes, arrows and short notes on a transparent PNG the same size as the source (scripts/markup.py turns a JSON list of shapes into that PNG, a preview and the prompt lines), upload it, then edit with provider_options.fal.markup_image_url and a prompt written per colour ("Red circle: … Blue circle: … Change nothing outside the marks."). Includes the measured example (two changes, the rest of the frame kept), how to check what changed, and when to use a black-and-white mask_url instead. Use when someone wants to change one or several specific spots in an image, says "circle it", "only change this part", "fix this word", "add a hat here", or an edit keeps changing things it should not. Not for whole-image restyles (gpt-image-2-6 recipes), animation (gpt-image-gif) or video edits (varg-ai). license: MIT metadata: author: vargHQ version: "1.0.0" openclaw: requires: anyBins: - curl - python3 homepage: https://varg.ai/how-to compatibility: >- Needs a VARG_API_KEY with curl (see the varg-ai skill) or the varg MCP connection (https://mcp.varg.ai/mcp). One markup edit costs 7 credits (high quality + one reference, measured 2026-11-05). scripts/markup.py needs Python 3 and Pillow; the diff check uses ImageMagick 7. allowed-tools: Bash(curl:*) Bash(jq:*) Bash(python3:*) Bash(magick:*) Read Write Edit --- # Markup edits with GPT Image 2.6 Instead of describing where a change goes, you draw it. GPT Image 2.6 takes a second, transparent PNG with only your marks on it, and the prompt says what each mark means. OpenAI: "You can now send a marked-up image to the API to show exactly where you want an edit" ([@OpenAIDevs](https://x.com/OpenAIDevs/status/2118124761867845847)). OpenAI's guide: keep the original as the first input, put the annotations on "a separate transparent PNG", send it as the markup, and describe the changes in the prompt; "Markup applies to the first input image. Additional images serve as references" ([OpenAI image guide](https://developers.openai.com/api/docs/guides/image-generation)). On varg the field is `provider_options.fal.markup_image_url`; fal: "Transparent PNG containing only annotations (circles, arrows, notes) that guide the edit of the first image", and it "must match input image dimensions". ## The measured example (2026-11-05 on varg) Idea by @IHayato, who drew a red and a blue circle on an illustration and asked 「赤丸にサンタ帽、青丸の本はプレゼントに」 ("a Santa hat in the red circle; the book in the blue circle becomes a present"): "face, hair ornament and clothes stayed the same, only the inside of the circles was redrawn, in a little over 20 seconds" (translated from [the post](https://x.com/IHayato/status/2118151619212890291)). Ours, with `gpt_image_2_6`: | Source (1024×768) | Markup layer | After | |---|---|---| | [cosy reading nook](https://s3.varg.ai/files/194b94d0-58ff-4e8b-91de-4f1959d04013/file_736079abc4e4.png) | [two ellipses, red and blue, 10 px](https://s3.varg.ai/files/194b94d0-58ff-4e8b-91de-4f1959d04013/file_af0957634427.png) | [Santa hat on the cat, the book is a wrapped present](https://s3.varg.ai/files/194b94d0-58ff-4e8b-91de-4f1959d04013/file_f74cc278406f.png) | Prompt: `Red circle: Santa hat on the cat. Blue circle: book → wrapped present. Change nothing outside the circles.` 7 credits. Outside the circles, 2.4% of pixels changed by more than 10% (mean change 6 of 255); inside the circles 14.1% did (measured with the mask from `markup.py --mask`, grown by 20 px). The full-frame diff (`magick compare`, 10% fuzz) flags 3.6% of pixels: the hat, the present, and light speckle on the pillow, the plant and the mug. ## 1. Draw the markup: scripts/markup.py Write the marks as JSON. Pixel coordinates of the source, or fractions (0-1) of its width and height: ```json { "units": "px", "width": 10, "shapes": [ {"type": "ellipse", "box": [81, 329, 329, 521], "color": "red", "label": "Santa hat on the cat"}, {"type": "ellipse", "box": [621, 433, 979, 597], "color": "blue", "label": "book → wrapped present"} ] } ``` ```bash python3 scripts/markup.py nook.png examples/nook.json -o nook_markup.png --preview nook_preview.png ``` ``` markup nook_markup.png (1024x768, RGBA, transparent outside the marks) preview nook_preview.png prompt: Red circle: Santa hat on the cat. Blue circle: book → wrapped present. Change nothing outside the marks. ``` - Shapes: `ellipse` and `rect` (`box: [x0, y0, x1, y1]`), `arrow` (`from`, `to`), `text` (`at`, `text`). - `label` goes into the printed prompt only. `note` (1-3 words) is also drawn on the markup next to the shape, in the shape's colour with a white outline. `examples/notes.json` shows a box with a note and an arrow. - Colours: red, blue, green, yellow, magenta, orange, cyan, white, black or `#RRGGBB`. Use one colour per change and colours that stand out from the image. - Stroke width defaults to 1.3% of the short side (10 px at 768), the width of the measured example. - The markup is always the source's exact size and fully transparent outside the marks. Open the preview before you upload: a circle that misses the object misleads the edit. Tested 2026-11-05: `examples/nook.json` on the source above reproduces the example markup (same bounding box, 91% of its marked pixels overlap; the difference is in the stroke edges); `examples/notes.json` draws a box, an arrow and two notes. ## 2. Upload both and edit Upload the source and the markup (`create_upload` on the MCP, or `POST /v2/files`; see [gpt-image-2-6](../gpt-image-2-6/) section 6), then: ```json call_tool({ "tool": "image", "input": { "model": "gpt_image_2_6", "prompt": "Red circle: Santa hat on the cat. Blue circle: book → wrapped present. Change nothing outside the circles.", "files": [{"url": "https://s3.varg.ai/files/194b94d0-58ff-4e8b-91de-4f1959d04013/file_736079abc4e4.png"}], "provider_options": {"fal": { "markup_image_url": "https://s3.varg.ai/files/194b94d0-58ff-4e8b-91de-4f1959d04013/file_af0957634427.png", "quality": "high", "reasoning_effort": "low" }} }, "wait": true }) ``` - The source must be the **first** file; more files are references (e.g. a photo of the exact hat). - `reasoning_effort`: OpenAI: "For most edits, start with low complexity. If a scene has several elements or constraints to juggle, try medium or high" ([@OpenAIDevs](https://x.com/OpenAIDevs/status/2118124761867845847)). `quality` is separate: keep `high` for the final. - With `files` the call goes to the edit route by itself. For a chain of edits on one image, pin `fal:openai/gpt-image-2.6/sunburst/edit` (OpenAI says Sunburst keeps details more consistent across edits). ## 3. Write the prompt per colour - One line per mark: ` : .` Name the colour and the shape exactly as drawn. - Say what the thing becomes, not how to draw it: "book → wrapped present", "the word SALE → SOLD". - End with the boundary: `Change nothing outside the circles.` (the line in the measured example). - Do all changes in one pass. Each extra edit round costs detail: faces changed from the second edit on ([@k_matsumaru](https://x.com/k_matsumaru/status/2118186638228816322)), wear showed from about the third ([@NeneneAI](https://x.com/NeneneAI/status/2118165875979407628)). Go back to the original rather than editing an edit. ## 4. Check what changed ```bash magick compare -fuzz 10% -metric AE before.png after.png diff.png # prints changed pixels; red in diff.png ``` Red outside your marks means the model touched more than you asked. A few percent of speckle is normal (3.6% in our example, mostly texture); a moved face or a changed sign is not. Rerun with a tighter mark, or with a mask (section 5). ## 5. Markup or mask? | | `markup_image_url` | `mask_url` | |---|---|---| | What you send | Transparent PNG with circles, arrows, notes | Black-and-white PNG, same size: "White marks regions to edit and black marks regions to keep" (fal) | | Best for | Several separate changes in one pass; "this thing → that thing"; rough areas | One exact region with hard edges: replace a background, a sky, a label area, the inside of a frame | | The prompt says | What each colour means | What fills the white area | | Precision | Guidance | Also guidance: "Masking with GPT Image is entirely prompt-based. The model uses the mask as guidance, but may not follow its exact shape with complete precision" (OpenAI guide) | You cannot send both in one call (fal). `markup.py --mask mask.png` writes the filled shapes as white on black from the same JSON, so you can try both from one drawing. We measured the markup route; we did not run a `mask_url` edit for this skill. ## 6. Failure → fix | What you see | Fix | |---|---| | The marks themselves are drawn into the result | Send the marks only as `markup_image_url`, never painted onto the source | | `markup_image_url` rejected | The markup is not the source's exact size, not a PNG with transparency, or the call also has `mask_url` | | The edit lands next to the mark | Tighter shape around the object; an arrow pointing at it; the object named in the line | | Things outside the marks change | `Change nothing outside the …`; fewer marks per pass; the diff check; a mask for hard edges | | A face changes after several rounds | Edit from the original; all changes in one pass; Sunburst for chains | ## Related skills [gpt-image-2-6](../gpt-image-2-6/) (model guide, prices, recipes), [gpt-image-gif](../gpt-image-gif/) and [gpt-image-sticker-pack](../gpt-image-sticker-pack/) (fix a still before you animate it), [gpt-image-sprite-sheet](../gpt-image-sprite-sheet/) (circle one broken frame in a sheet).