2026-09-15 — Google · Tutorials

Tracking Google AI Overviews at scale with async batches and webhooks

At the time of writing the Google Search endpoint is listed as coming soon. The request and fields below are its documented contract; the Google Search page shows its current availability.

A handful of keywords fit in a script that polls. A tracker with hundreds of keywords across markets doesn’t: polling that many tasks burns your rate limit and adds delay between a task finishing and you reading it. This post builds the scaled version with the SDKs — batches to submit, a webhook to collect.

Building the batch

Each task is one keyword in one market, with include.aioverview requested and an idempotency key derived from what it means, so a re-run of the day’s submission never double-queues a keyword:

import { Client } from "<sdk-package>"; // see the quickstart for the package name

const client = new Client({ apiKey: process.env.API_KEY!, baseUrl: process.env.API_URL! });

function buildTasks(keywords: string[], countries: string[], day: string, webhookUrl: string) {
  return keywords.flatMap((query) =>
    countries.map((country) => ({
      taskType: "GOOGLE" as const,
      payload: { query, country, include: { aioverview: { markdown: true } } },
      idempotencyKey: `${query}-${country}-${day}`,
      webhook: { url: webhookUrl },
    })),
  );
}

const tasks = buildTasks(keywords, ["US", "GB"], "2026-09-15", "https://your-app.com/hooks/aioverview");
const batch = await client.createBatch(tasks);
for (const item of batch.results) {
  if (!item.success) console.warn(tasks[item.index].idempotencyKey, item.error.code);
}

from <sdk_package> import Client  # see the quickstart for the package name

client = Client(os.environ["API_KEY"], base_url=os.environ["API_URL"])

def build_tasks(keywords: list, countries: list, day: str, webhook_url: str) -> list:
    return [
        {
            "taskType": "GOOGLE",
            "payload": {"query": query, "country": country, "include": {"aioverview": {"markdown": True}}},
            "idempotencyKey": f"{query}-{country}-{day}",
            "webhook": {"url": webhook_url},
        }
        for query in keywords
        for country in countries
    ]

tasks = build_tasks(keywords, ["US", "GB"], "2026-09-15", "https://your-app.com/hooks/aioverview")
batch = client.create_batch(tasks)
failed = [r for r in batch["results"] if not r["success"]]

createBatch / create_batch take 1 to 500 tasks in one call and refuse a larger list before sending it; chunk a bigger keyword list yourself. Each item in results answers by its index in input order, so a failure (VALIDATION_ERROR, RESOURCE_ALREADY_EXISTS or INSUFFICIENT_CREDITS) doesn’t cost you the rest of the batch — see batches.

Collecting by webhook

Each finished task is delivered to the webhook URL as the same JSON GET /v1/async/task/{id} returns. Verify the SDK’s helper before trusting a delivery, then read the AI Overview fields:

import express from "express";
import { verifyWebhook } from "<sdk-package>";

const app = express();
app.post("/hooks/aioverview", express.raw({ type: "application/json" }), async (req, res) => {
  if (!(await verifyWebhook(req.body, req.get("webhook-signature"), process.env.WEBHOOK_SECRET!))) return res.sendStatus(400);
  const delivery = JSON.parse(req.body.toString("utf8"));
  if (delivery.task.status === "COMPLETED") {
    const overview = delivery.response.result.aioverview; // null when Google showed none
    if (overview) store(delivery.task.idempotencyKey, overview.sources, overview.citationPills, overview.relatedLinks);
  }
  res.sendStatus(204);
});

from flask import Flask, abort, request
from <sdk_package> import verify_webhook  # see the quickstart for the package name

app = Flask(__name__)

@app.post("/hooks/aioverview")
def aioverview():
    raw = request.get_data()
    if not verify_webhook(raw, request.headers.get("Webhook-Signature"), os.environ["WEBHOOK_SECRET"]):
        abort(400)
    delivery = request.get_json()
    task = delivery["task"]
    if task["status"] == "COMPLETED":
        overview = delivery["response"]["result"]["aioverview"]
        if overview:
            store(task["idempotencyKey"], overview.get("sources", []), overview.get("citationPills", []))
    return "", 204

Three rules keep this correct at scale, all covered in the webhooks guide: verify the raw body before parsing, answer 2xx within 15 seconds and do slow work after, and deduplicate by task.id since deliveries are retried independently and can repeat.

Reading the result

aioverview is null when no AI Overview was shown for that keyword and market — record that as a data point, not a gap. Otherwise:

• sources: the source rail, each with position, url, label and description.
• citationPills: the inline citation chips; a chip citing several sources shares one citationPillId across entries, so group by it to rebuild the chip.
• relatedLinks: a chip’s “View related links” items that aren’t in sources, grouped the same way.

Join sources against result.organicResults (also on the same response, alongside the overview) to compare organic rank with AI Overview citation for the same keyword.

Reconciling and scheduling

Webhook deliveries retry with backoff for up to 10 attempts, then stop; after each run’s window closes, poll the tasks your batch created but never received a delivery for with GET /v1/async/task/{id} and file those too. Trigger the whole submission from any scheduler once per period — a run costs keywords × markets × the Google engine’s price, and the pricing page has the current add-on cost for include.aioverview.

Keep reading