A Menu Without Photos Doesn't Sell

Dishes with photos get ordered more than dishes without — every delivery platform that has published data says so. Our app shows travelers restaurant menus (translated, allergens flagged), and a menu where each dish has a real photo of that dish at that restaurant is obviously better than a wall of text.

The photos already exist. Open any restaurant's Google Maps listing: the "Food & drink" tab has years of guest photos, taken in the real lighting at the real portion sizes. The problem is that none of them are labeled. Photo #47 is a carbonara, but nothing in any API says it belongs next to Spaghetti alla Carbonara — €14.

So the task is a join: photos × menu → {photoUrl, dishName}. Trivial for a human, impossible at thousands of restaurants. Here's how we built it, and what actually broke along the way.

Getting the photos out of Google Maps

There is no official API for a place's photo gallery — the Places API returns a handful of photos with no categories. Scraping the Maps UI with headless Playwright works, until it doesn't: signed-out browsers increasingly get Google's "limited view", which hides the photo gallery entirely. Your scraper doesn't error; it just finds zero photos.

Our scraper handles the cooperative case (drive the gallery tabs, regex photo URLs out of the DOM) and falls back to Outscraper's API when Google won't cooperate. Two details cost us real debugging time:

Photo URLs come in two generations. Classic place photos live under /p/, current ones under /gps-cs-s/ — and avatars under /a/ must not match:

PHOTO_URL_RE = re.compile(
    r"https?://[a-z0-9.-]*googleusercontent\.com/(?:p|gps-cs-s)/[^\s\"')]+")

Canonical URLs serve the original file. A googleusercontent URL without a size suffix returns the full-resolution upload — multi-megabyte originals you're about to send to a vision model (we hard-reject anything over 10 MB). Append a bounded width first:

def sized_url(url: str) -> str:
    if "googleusercontent.com/p/" in url and "=" not in url.rsplit("/", 1)[-1]:
        return url + "=w1280"
    return url

The matching call

Matching itself is one Gemini vision request: a batch of photos plus the menu as a numbered list, asking for a JSON array of {imageIndex, dishName}. We batch 9 photos per call — large enough to amortize the menu tokens across photos, small enough that the response never hits output limits.

The prompt is short. The load-bearing lines are the rules:

Rules:
- Match based on visual appearance of the food/drink in the photo
- If a photo does not clearly match any menu item, set dishName to null
- Use the exact "name" field value from the menu list for dishName
- Return ONLY valid JSON, no markdown

We also set response_mime_type="application/json" so the model returns JSON instead of prose. Inputs are clamped defensively — 200 menu items max, 300 chars per field — because user-supplied menus will eventually contain a 40 KB description.

Never trust the model

Vision models want to be helpful, and "helpful" here means hallucinating. Ask for a dish name and you'll sometimes get one that isn't on the menu — a plausible Italian dish the model invented from the photo. A photo matched to nothing is fine; a soup labeled as steak is how an app loses a user. So every answer is validated against the actual menu:

valid_names = {it["name"] for it in menu_items}
dish = by_index.get(i)
if dish is not None and dish not in valid_names:
    dish = None  # hallucinated name — treat as no match

This is the single highest-value line in the pipeline. The prompt already says "use the exact name" — the model mostly complies, and mostly isn't good enough.

Two more failure modes worth stealing the fixes for:

Truncated JSON. Long responses get cut mid-array by the output-token limit (one 8-photo menu produced a truncated 218 KB response). json.loads throws, and a naive rfind("},") salvage fails when the cutoff lands inside a nested array. Our parser tracks bracket depth (string- and escape-aware), keeps every complete top-level object, and closes the array — so a 95%-complete response yields 95% of the results instead of zero.

Transient 503s vs permanent 4xxs. "Model overloaded" spikes are real and need a real pause — we retry with 5 s and 20 s backoffs. But a 4xx (bad key, bad request) is retried by nobody sane:

if attempt == len(backoffs) or type(e).__name__ == "ClientError":
    raise
time.sleep(backoffs[attempt])

One more operational lesson: pin model IDs, avoid -latest aliases. Google retired the entire gemini-2.5-* family for generateContent — calls started 404ing while ListModels still advertised the models. An alias like flash-latest can silently move under you and change output shape and cost; a pinned ID at least fails loudly.

The output

One record per photo, boring on purpose:

{ "photoUrl": "https://lh3.googleusercontent.com/p/...",
  "dishName": "Spaghetti alla Carbonara",
  "matched": true }

At the default gemini-3.5-flash, matching a photo costs a fraction of a cent.

Run it yourself

The pipeline is published as public Apify actors — bring your own free Google AI Studio key:

• Restaurant Dish Photo Matcher — menu items + photo URLs (or just a Google Maps place URL) in, matched photos out.
• No structured menu yet? The Google Maps Menu Scraper builds one from a Maps listing, and the AI Restaurant Menu Parser builds one from menu photos.
• No API keys at all? Use the managed twins with keys included: Photo Matcher, Menu Scraper, Menu Parser.

Your guests already did the photo shoot. The join is the product.

Travel Eat helps travelers see what they can actually eat — menus scanned, translated, and allergen-checked in 36 languages.