How to Use AI to Generate Keyword Clusters

Key takeaways

• Cluster by semantic similarity and SERP overlap, not shared words — "best CRM for freelancers" and "CRM tools for solo consultants" belong together even though they share almost no vocabulary.
• One cluster should map to exactly one URL. If an AI tool hands you 40 clusters for a site with 10 real topics, your similarity threshold is too tight.
• Validate every AI-generated cluster against actual Google search results before you write anything — embeddings are a starting point, not ground truth.

Why keyword clustering matters more than keyword lists

A keyword list tells you what people type. A cluster tells you what they mean. The distinction matters because Google and AI answer engines both evaluate pages at the topic level, not the keyword level — a page ranking for "project management software for agencies" will often also rank for "best PM tool for agency teams" without ever targeting that exact phrase, because the underlying intent and entities overlap. Google's Search Central documentation describes this directly: ranking systems match queries to content based on meaning and context, and a single well-structured page is expected to satisfy dozens of phrasing variants.

The practical failure mode we see constantly in SaaS content: founders build a spreadsheet of 200 keywords, assign each one its own blog post, and end up with eight articles that all answer the same question in slightly different words. Google indexes them, can't tell which one deserves to rank, and splits authority across all eight — a textbook case of keyword cannibalization. Clustering fixes this before a single word gets written, by forcing you to decide up front how many distinct intents actually exist in your niche.

What an AI keyword cluster actually is

A cluster is a group of keywords that would all be satisfied by the same page. Two reliable tests:

• Semantic test: the keywords produce embedding vectors with cosine similarity above roughly 0.80–0.85 (this threshold varies by model and niche, so treat it as a starting point to tune, not a law).
• SERP test: when you search both keywords, at least 3 of the top 10 ranking URLs overlap. This is the heuristic serious SEOs have used for years, and it's a better ground truth than embeddings alone because it reflects what Google has already decided, rather than what a language model thinks is similar.

AI is genuinely good at the first test and mediocre at the second unless you feed it real SERP data. That's the single most important thing to understand before you automate this: a language model clustering keywords from its training knowledge is guessing at search behavior, while a model clustering based on actual SERP overlap is reading reality.

The actual workflow, step by step

1. Generate a seed list. Pull 100–500 keywords from Google Keyword Planner, Search Console queries, or competitor site pages. Don't filter by volume yet — low-volume long-tail terms often reveal the intent clusters you'd otherwise miss.

2. Embed every keyword. Run each phrase through an embedding model. OpenAI's text-embedding-3-small outputs a 1536-dimension vector per string; you don't need anything larger for keyword clustering, and the smaller model is roughly 5x cheaper per token than the large variant, which matters once you're processing thousands of terms monthly.

3. Cluster the vectors. Hierarchical agglomerative clustering tends to outperform k-means for keyword work because you don't have to pre-specify the number of clusters — you set a distance threshold instead and let the algorithm find natural groupings. K-means forces every keyword into a cluster even when it's an outlier that deserves its own page.

4. Cross-check against SERPs. For each proposed cluster, pull the top 10 results for two or three representative keywords. If the ranking pages differ substantially, split the cluster. This step is the one most AI clustering tools skip entirely, and it's the reason their output looks plausible but produces cannibalization in practice.

5. Label each cluster by intent, not topic. "Keyword clustering tools" and "how to cluster keywords for free" look similar semantically but split into commercial-investigation and informational intent respectively. They need different pages even if they're otherwise close in vector space.

6. Assign one owned URL per cluster. This becomes your content map. A cluster with no existing page is a content gap; a cluster mapping to two or three existing pages is a cannibalization problem to merge or redirect, something worth running through a proper content audit spreadsheet before you publish anything new on top of it.

What breaks when founders automate this blindly

The most common failure we see with indie hackers using AI for clustering is trusting the model's confidence. Ask an LLM to "cluster these 150 keywords into topics" with no SERP grounding and it will produce a tidy, convincing list every time — tidy because language models are built to produce coherent-sounding output, not because the clusters reflect how Google actually groups the queries. We've watched tools confidently merge "pricing page examples" with "how to price a SaaS product," which share vocabulary but serve completely different searchers: one wants design inspiration, the other wants a pricing strategy framework.

The second failure is cluster granularity drift. Run the same keyword set through an LLM twice with slightly different prompts and you'll often get different numbers of clusters — 12 one time, 19 the next — because "how many topics exist in this list" isn't a deterministic question for a generative model the way a distance-threshold clustering algorithm makes it deterministic. If your clustering step isn't reproducible, your content calendar isn't either, and that inconsistency compounds every time you re-run the process for a new batch of keywords.

The fix in both cases is the same: use the LLM for labeling and intent classification, where its language understanding genuinely adds value, and use a real clustering algorithm (hierarchical clustering, DBSCAN, or even simple cosine-similarity thresholding) for the grouping itself, where you need consistency more than creativity.

Turning clusters into a content plan

Once you have clean clusters, structure them as pillar-and-cluster content rather than a flat list of blog posts. One pillar page targets the broad cluster theme; supporting pages target the specific sub-intents that didn't pass the SERP-overlap test for merging. Internally link every supporting page back to the pillar using the exact cluster keyword as anchor text where it reads naturally — this is also how you reinforce topical relevance for AI answer engines, which tend to cite pages that sit inside a clearly interlinked topic cluster rather than isolated one-off posts.

A practical sizing rule: if a cluster has fewer than 3 keywords with meaningful search volume, fold it into the nearest pillar as an H2 section instead of giving it a standalone URL. Thin, single-keyword pages are exactly the kind of content that gets consolidated in Google's periodic quality updates, and maintaining dozens of near-duplicate thin pages creates ongoing audit overhead for no ranking benefit.

For naming the actual articles once clusters are mapped, the headline still does a lot of work — a cluster correctly identified but given a weak, keyword-stuffed title underperforms a less rigorously clustered topic with a title that earns clicks, so it's worth pairing this process with deliberate work on writing titles that actually get clicked.

Where this applies beyond blog posts

Keyword clustering isn't only a blogging exercise. SaaS products with public documentation can apply the same method to support and API content — grouping "authentication error 401," "how to get an API key," and "token expired fix" into one cluster reveals that they belong on a single troubleshooting page rather than three disconnected doc entries, which is the same logic that applies when structuring API documentation that's built to rank instead of just being technically accurate.

Seolyn's own agent runs this clustering step before it drafts anything, specifically because skipping it is the single biggest cause of the cannibalization problem described above — generating content first and discovering the overlap later costs far more time than spending an hour clustering up front.

Frequently Asked Questions

Q: What's the difference between keyword clustering and keyword grouping?

Grouping usually means sorting keywords by shared words or manual categories, while clustering uses a quantitative method — embedding similarity or SERP overlap — to group keywords by actual search intent, even when the wording is completely different.

Q: Can I generate keyword clusters without coding?

Yes — tools like Keyword Insights, free embedding-based clustering scripts run through no-code platforms, or spreadsheet add-ons that call an embeddings API can do this without you writing clustering code yourself, though you still need to manually validate clusters against live SERPs.

Q: How many keywords should be in a single cluster?

There's no fixed number, but clusters with fewer than 3 meaningful keywords are usually better absorbed into a related pillar page's subheadings rather than published as a standalone page, since thin single-keyword pages rarely sustain rankings on their own.

Q: Does AI clustering replace manual SERP research?

No. AI clustering is fastest for the first pass of grouping hundreds of keywords by meaning, but SERP overlap is still the most reliable confirmation that Google treats two keywords as the same intent, so the two methods work best combined rather than one replacing the other.

Q: How often should keyword clusters be updated?

Revisit clusters whenever you do a content audit or roughly every two to three months for an active site, since new SERP features, competitor content, and search behavior shift which pages overlap and can quietly split or merge clusters you mapped earlier.