πŸ”§ Behind the Scenes

Google Maps Put Its Photo Gallery Behind Sign-In. It Broke Our Menu Pipeline.

Our Playwright scraper went from full photo galleries to ~10 photos per restaurant. A debugging post-mortem with the actual code: what broke, what we tested, the proxy bug that wasted an afternoon, and the API that replaced the browser.

Travel Eat TeamJuly 9, 20267 min read
engineeringgoogle-mapsscrapingplaywrightmenu-dataapifybehind-the-scenes
Google Maps open on a phone, showing a restaurant listing with its rating

Travel Eat shows you a restaurant's menu before you walk in β€” dishes, prices, EU-14 allergens, translated. The pipeline behind it is short:

  1. Find the restaurant's menu photos on Google Maps (guests and owners upload them constantly).
  2. Send those photos to Gemini with a structured-output prompt.
  3. Get back JSON: one object per dish, with name, price, ingredients, allergens.

Step 1 was a Playwright scraper. This is the story of how Google killed it, what we tried, and what replaced it β€” with the actual code, because "we fixed it" posts without code are useless.

How the scraper worked

A Google Maps place used to expose a tabbed photo gallery: All, Menu, Food & drink, By owner. That's a role="tablist" in the DOM, so the scraper clicked into the gallery, matched tab labels against a category map, and scrolled each tab while harvesting image URLs:

# Ordered tab-label patterns -> our category buckets. First match wins per tab.
TAB_CATEGORY_MAP = [
    (re.compile(r"^(menu|menΓΊ|carta)$", re.I), ["menu"]),
    (re.compile(r"^(food\s*&\s*drink|comida\s*y\s*bebida)$", re.I), ["dish", "beverage"]),
    (re.compile(r"^(food|comida)$", re.I), ["dish"]),
    (re.compile(r"^(drink|drinks|bebida|bebidas)$", re.I), ["beverage"]),
]

The harvest itself is a page-side script that grabs every googleusercontent.com place-photo URL from <img> tags and CSS background-images, then a scroll loop that stops after four idle rounds (no new URLs, no scroll-height change):

() => {
    const urls = new Set();
    const root = document.querySelector("div[role='main']") || document;
    root.querySelectorAll("img").forEach(img => {
        const src = img.currentSrc || img.src || "";
        if (/googleusercontent\.com\/(p|gps-cs-s)\//.test(src)) urls.add(src);
    });
    return Array.from(urls);
}

One detail in that regex is itself a 2026 change: place photos used to live under lh3.googleusercontent.com/p/…, and new ones now live under a /gps-cs-s/… path. If your scraper filters on the old /p/ pattern only, you silently lose every recently uploaded photo. (Avatars live under /a/ β€” don't match those.)

Harvested URLs carry a size suffix (=w408-h272-k-no) that we strip to get a canonical, resolution-independent URL:

def normalize_url(url: str) -> str:
    """Strip trailing size hints (e.g. =w408-h272-k-no) -> canonical URL."""
    return re.sub(r"=[wh]\d+(?:-[a-z0-9-]+)*$", "", url)

This worked for a long time. A busy place like Katz's Delicatessen has hundreds of tagged photos, and the Menu tab alone reliably gave us everything Gemini needed.

The break: ~10 photos, no tabs, no error

The pipeline didn't crash. It kept returning results β€” just capped at roughly ten photos per place, for every place, with no category tabs. For a restaurant with 40 menu photos and 300 food photos, we got a ten-photo sliver with no way to page deeper.

Opening Google Maps in a clean signed-out browser confirmed it: the tabbed gallery is gone for signed-out sessions. Google calls this the "limited view." It made the news in February 2026 when signed-out users suddenly lost reviews and photos β€” 9to5Google and gHacks covered it, and Google walked it back for regular users within days. Notably, Google's own pop-up at the time said you might see the limited view "if unusual traffic is detected on your network."

That wording matters, because for automated sessions it never went away. Which brings us to what we tested.

Ruling out the usual suspects

"Unusual traffic detected" sounds like IP reputation, so that was our first theory. The test ladder:

  1. Different countries. Spanish datacenter IPs and US IPs: same limited view.
  2. US residential proxies. Traffic indistinguishable from a home connection: same ~10-photo cap.
  3. Fresh browser contexts, en-US locale, human-like pacing, real viewport. No difference.

Result 2 and 3 together are the important ones. Residential IPs plus a real Chromium with careful pacing is the ceiling of "look human" β€” and it changed nothing. This is not bot detection you can engineer around; for automated signed-out sessions, the full gallery is behind a Google sign-in, period.

We considered signing in with real Google accounts and rejected it immediately. That path means maintaining a pool of accounts that Google actively detects and bans, solving sign-in challenges, and rotating sessions β€” an operational treadmill where every step is fragile and none of it is your product.

The afternoon we lost to a proxy URL

While testing residential proxies we hit a bug that had nothing to do with Google but chose the worst week to exist. Symptom: every page loads blank β€” no error, no timeout, just an empty document.

The cause: passing an authenticated proxy URL (http://user:pass@proxy.example:8000) directly as Playwright's server field. Playwright silently fails the proxy auth. Credentials must be split into separate fields:

def playwright_proxy(proxy_url: str | None) -> dict | None:
    """Playwright needs credentials split out β€” passing a URL with embedded
    user:pass as `server` fails auth silently (blank pages)."""
    if not proxy_url:
        return None
    u = urlparse(proxy_url)
    proxy = {"server": f"{u.scheme}://{u.hostname}:{u.port}"}
    if u.username:
        proxy["username"] = u.username
        proxy["password"] = u.password or ""
    return proxy

If you're debugging blank pages behind an authenticated proxy in Playwright: it's this. You're welcome to the afternoon we didn't get back.

The fix: stop scraping, buy the data

The photos still exist β€” Google just won't render the gallery for us. So we stopped asking the browser and started asking Outscraper, whose photos-v3 API returns a place's full photo set with Google's own photo tags. That last part is the killer feature: instead of scraping tabs to categorise photos, we request exactly the tab we want, server-side:

_API = "https://api.app.outscraper.com/maps/photos-v3"

url = (f"{_API}?query={quote(place_id)}&photosLimit={limit}"
       f"&language=en&async=false&tag=menu")

Three quirks we learned from live payloads, so you don't have to:

  • photo_tags is a repr-string, not a list. The field arrives as the literal string "['menu']". We normalise with ast.literal_eval before matching:

    def _parse_tags(raw_tags) -> list[str]:
        """photo_tags is usually a repr-string like "['menu']" β€” normalise."""
        if isinstance(raw_tags, str):
            try:
                raw_tags = ast.literal_eval(raw_tags)
            except (ValueError, SyntaxError):
                return []
        return [str(t).strip().lower() for t in (raw_tags or [])]
    
  • You must pass tag=menu to get menu photos. An untagged fetch rarely surfaces them even at photosLimit=300 β€” the results are dominated by general food and interior shots. The tag parameter mirrors Google's photo tabs and returns menu photos directly.

  • Use photo_url_big for OCR. The payload offers several URL variants; photo_url_big is the highest-resolution one, and menu text is exactly the workload where resolution decides whether Gemini can read the prices.

Large places flip the API into async mode (status: "Pending" plus a results_location URL), so the client polls before parsing. The browser scraper still exists as a keyless fallback β€” it opens the lightbox and arrow-keys through whatever the limited view exposes β€” but we're honest about its ceiling: about ten uncategorised photos.

One downstream lesson, since you'll hit it too

More photos per place exposed a second failure mode: send eight menu photos to Gemini in one call and a long menu overflows the model's max output tokens. We got a truncated 218 KB JSON response from a single Katz's menu. Two mitigations, both boring:

  • Chunk the images β€” three per call, dedupe dishes across chunks by original name.
  • Salvage truncated JSON β€” when a response is cut mid-array, parse up to the last complete object instead of throwing the whole response away.

The pipeline is public

Everything above runs Travel Eat in production, and we've published it as three Apify actors:

Don't want to create any API keys? Each actor has a managed twin with Gemini and Outscraper keys included β€” Menu Scraper, Menu Parser, Photo Matcher β€” at a slightly higher per-result price, zero configuration.

If your Google Maps photo scraper still runs but suddenly returns about ten photos per place with no gallery tabs: it's not your proxies, it's not your fingerprint, and no amount of pacing will fix it. It's the limited view β€” and the way out is an API, not a better browser.

Travel Eat scans and translates restaurant menus and flags allergens in 36 languages.

T

Travel Eat Team

Contributing writer at Travel Eat. Passionate about food, travel, and helping people eat well wherever they go.

Ready to Eat Confidently Anywhere?

Download Travel Eat and discover amazing food wherever you travel.

Download for iOS