# TF-IDF for SEO: What It Measures, What It Misses, How to Use It

> TF-IDF weighs a word by how often it appears in a text, discounted by how many texts in a set contain it. In SEO it is a diagnostic, not a ranking factor: comparing your page with pages that rank shows the terms, and so the subtopics, you have yet to cover. Use the gaps to write missing sections, never to repeat words.

Source: https://tonaily.com/guides/tf-idf-seo/
Last updated: 2026-11-06
Checked against sources on: 2026-11-06

## What is TF-IDF in SEO?

TF-IDF (term frequency–inverse document frequency) is a score that weighs a word by how often it appears in one text, discounted by how many texts in a set contain it. In SEO it is used as a diagnostic: compare your page with pages that rank, and the heavy terms they share that you lack point to subtopics you have yet to cover.

A content manager has a 1,400-word article on meta descriptions stuck in position 14. She pastes it next to the page in position 3 into a [free TF-IDF tool](/resources/tools/tf-idf-analyzer/). The gap list starts with "pixel", "truncates", "rewrites". Her article never mentions that Google rewrites snippets or that the cut-off is measured in pixels. The fix is not to sprinkle those three words: it is to write the two sections they stand for.

That is the whole method in one example. The rest of this guide explains the formula, works through it with real numbers, and is honest about what TF-IDF cannot tell you.

## How is TF-IDF calculated?

TF-IDF multiplies two numbers: term frequency (how present a word is in this text) and inverse document frequency (how rare it is across the set of texts). A word scores high when it is frequent in one document and found in few others.

The idea of inverse document frequency goes back to Karen Spärck Jones in 1972, who, as the [Wikipedia article on tf–idf](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) puts it, "conceived a statistical interpretation of term-specificity called Inverse Document Frequency (idf)." The standard form is:

```txt title="TF-IDF, standard form"
tf(t, d)      = occurrences of term t in document d / total terms in d
idf(t)        = log( N / df(t) )
tfidf(t, d)   = tf(t, d) × idf(t)

N     = number of documents in the set
df(t) = number of documents that contain t
```

Why discount common words? The [scikit-learn documentation](https://scikit-learn.org/stable/modules/feature_extraction.html) explains it plainly:

```quote source="scikit-learn, Tf–idf term weighting" href="https://scikit-learn.org/stable/modules/feature_extraction.html"
In a large text corpus, some words will be very present (e.g. "the", "a", "is" in English) hence carrying very little meaningful information about the actual contents of the document.
```

The IDF does exactly that: a word present in every document gets log(N / N) = log(1) = 0, so its weight is zero however often it appears. A word present in only one document keeps its full weight.

## A worked example, step by step

Take four pages that rank for "how to write a meta description". After removing stop words, page A has 400 terms. We look at three of them, using the natural logarithm (ln).

| Term | Times in page A | tf in page A | Pages containing it (of 4) | idf = ln(4 / df) | TF-IDF |
|---|---|---|---|---|---|
| description | 24 | 24 / 400 = 0.060 | 4 | ln(1) = 0 | **0** |
| snippet | 8 | 8 / 400 = 0.020 | 1 | ln(4) ≈ 1.386 | **0.0277** |
| pixel | 4 | 4 / 400 = 0.010 | 2 | ln(2) ≈ 0.693 | **0.0069** |

Three lessons in that table:

1. **The most repeated word scores zero.** "Description" appears 24 times, but every page uses it, so it says nothing about what makes page A different. That is the opposite of what keyword density would tell you.
2. **A word used by one page only scores highest.** "Snippet" is page A's distinctive vocabulary in this set.
3. **The score is relative.** 0.0277 means nothing on its own. It only says "snippet" weighs about four times more than "pixel" in this page, in this set of four.

### What changes in a two-text comparison

When you compare only your page and one competitor, the standard formula breaks down: with N = 2, a term present in both texts gets ln(2 / 2) = 0, and libraries usually smooth the formula to avoid zeros and divisions by zero. The scikit-learn default, `smooth_idf=True`, "adds '1' to the numerator and denominator as if an extra document was seen containing every term in the collection exactly once":

```txt title="Smoothed IDF (scikit-learn default)"
idf(t) = log( (1 + n) / (1 + df(t)) ) + 1
```

With two texts (n = 2) and the natural logarithm, a term in one text gets ln(3 / 2) + 1 ≈ 1.41, a term in both gets ln(3 / 3) + 1 = 1. That is the formula our [TF-IDF tool](/resources/tools/tf-idf-analyzer/) uses. Say the competitor's text has 150 terms after stop words, and "truncates" appears 3 times in it and never in yours: tf = 3 / 150 = 0.02, weight = 0.02 × 1.41 ≈ 0.028. The tool marks it High priority (above 0.01).

Notice what this means: with only two texts, the IDF can only take two values. The ranking is mostly driven by term frequency in the competitor's text, and the IDF simply boosts terms you do not use. That is fine for a gap analysis, as long as you know it.

## What can TF-IDF actually tell you about your content?

TF-IDF tells you which words carry weight in a text compared to others. Read across several pages that rank, it reveals the vocabulary a topic tends to require, and the gaps in yours. It does not tell you whether your page is good, true or useful.

What it does well:

- **Spot missing subtopics.** A heavy term missing from your page usually stands for a question you have not answered: "pixel" stands for "how long should it be", "rewrites" for "why Google ignores mine".
- **Spot thin treatment.** A term you share but use far less than the competitor can mean you mention a point without explaining it. The tool lists these as weaker terms.
- **Spot your own angle.** Terms only you use are either your added value (your method, your data) or off-topic drift. Both are worth knowing.

It is cheap, fast and explainable: you can see why a term ranks where it does, which is not true of most "content scores".

## What can't TF-IDF tell you?

TF-IDF counts words. It does not understand meaning, order, intent, accuracy or quality, and it says nothing about how Google ranks pages.

- **No meaning.** "car" and "cars" are two different terms; "price" and "cost" are unrelated. In the free tool, terms are single words, with no stemming and no phrases, so "meta description" is counted as "meta" and "description".
- **No word order.** "Brazil traveler to USA" and "USA traveler to Brazil" produce identical counts. Google used exactly that kind of query to show what BERT changed: in its 2019 announcement it explains that "The word “to” and its relationship to the other words in the query are particularly important to understanding the meaning."
- **No quality signal.** A page can match the vocabulary of the top results and still be wrong, outdated or copied. Google's helpful content questions ask whether a page provides "original information, reporting, research, or analysis", and whether content that draws on other sources avoids "simply copying or rewriting those sources". No word count answers that.
- **No ranking model.** A competitor may rank for reasons that have nothing to do with words: links, brand, freshness, site reputation.

### Does Google use TF-IDF?

Google does not say so. Its [guide to its ranking systems](https://developers.google.com/search/docs/appearance/ranking-systems-guide), which covers "some of our more notable ranking systems", does not mention TF-IDF. For language, it describes AI systems: BERT, neural matching, passage ranking and RankBrain.

```quote source="Google Search Central, A guide to Google Search ranking systems" href="https://developers.google.com/search/docs/appearance/ranking-systems-guide"
RankBrain is an AI system that helps us understand how words are related to concepts. It means we can better return relevant content even if it doesn't contain all the exact words used in a search, by understanding the content is related to other words and concepts.
```

The [SEO Starter Guide](https://developers.google.com/search/docs/fundamentals/seo-starter-guide) says the same thing in practical terms: "Google's language matching systems are sophisticated and can understand how your page relates to many queries, even if you don't explicitly use the exact terms in them."

TF-IDF is not dead in search technology in general: Elasticsearch's default similarity, BM25, is described in its [reference documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html) as a "TF/IDF based similarity". But that is a site-search engine, not Google. The useful conclusion is narrower: the pages that rank for a query tend to cover the same concepts, and TF-IDF is a quick way to see those concepts. Use it to read the topic, not to imitate an algorithm.

## How do you use a competitor comparison without keyword stuffing?

Turn each missing term into a question, answer the questions worth answering, and stop there. Never paste words to raise a score.

```mermaid
flowchart TD
  A[Pick the target query] --> B[Collect 3-4 pages that rank]
  B --> C[Compare each with your page]
  C --> D[Keep terms missing in several comparisons]
  D --> E[Turn each term into a reader question]
  E --> F{Does your reader need it?}
  F -- No --> G[Ignore it]
  F -- Yes --> H[Write or expand a section]
  H --> I[Check density, then re-compare]
```

1. **Pick the query and its pages.** Search the query you target and take the body text of three or four organic results that match your intent. Skip forums and marketplaces if you are writing a guide.
2. **Compare one page at a time.** Paste your draft and one competitor into the [TF-IDF tool](/resources/tools/tf-idf-analyzer/). Note the High and Medium missing terms. Repeat with each page.
3. **Keep what repeats.** A term missing against three pages out of four is a pattern; a term missing against one page is often that writer's example or brand.
4. **Translate terms into questions.** "Truncates" becomes "Why is my description cut off, and where?" "Rewrites" becomes "Why does Google show a different snippet?"
5. **Write the section, not the word.** If the question deserves an answer, write a real paragraph or an H2 with an example. The term will appear naturally, a few times, because you cannot explain the point without it.
6. **Check for overuse.** Run the [keyword density analyzer](/resources/tools/keyword-density-analyzer/) on the result to make sure you did not swing into repetition.
7. **Re-compare.** The gap list should now start with terms that are genuinely specific to the other page.

### Where the line with keyword stuffing is

Google's spam policies define keyword stuffing as "filling a web page with keywords or numbers in an attempt to manipulate rankings in Google Search results", and list among the examples:

```quote source="Google Search Central, Spam policies for Google web search" href="https://developers.google.com/search/docs/essentials/spam-policies"
Repeating the same words or phrases so often that it sounds unnatural.
```

The Starter Guide adds that "Excessively repeating the same words over and over (even in variations) is tiring for users, and keyword stuffing is against Google's spam policies." A TF-IDF gap list read as a list of words to insert leads straight there. Read as a list of questions to answer, it leads to a more complete page.

The same Starter Guide also gives the upside of writing naturally: "If you are varying the words (writing naturally to not be repetitive), you have more chances to show up in Search simply because you are using more keywords." Covering a subtopic properly brings its vocabulary with it, without forcing anything.

### Do not chase length either

Matching every competitor's subtopic can bloat a page. Google is explicit that "The length of the content alone doesn't matter for ranking purposes (there's no magical word count target, minimum or maximum, though you probably want to have at least one word)." Its helpful content page lists, among warning signs: "Are you writing to a particular word count because you've heard or read that Google has a preferred word count? (No, we don't.)" Add a section when the reader needs it, not because a competitor has one.

## A checklist for your next comparison

```checklist title="TF-IDF gap analysis"
- The competitor pages match the intent of my query (same type of page, same reader)
- I compared at least three pages, one at a time
- I kept only the terms missing in several comparisons
- I ignored brand names, product names and examples specific to one page
- Each kept term is rewritten as a question my reader would ask
- Each question I keep gets a real answer: a paragraph, an example, or a section
- I did not add any term to a sentence that does not need it
- My page still contains something the others do not (my data, my method, my examples)
- The keyword density check shows no term repeated unnaturally
- A second comparison shows the important gaps are closed
```

## Where does TF-IDF fit in a 2026 content process?

TF-IDF is a research step, done once per article before or after the first draft. It works best alongside the things it cannot do: keyword research upstream, human judgment and fact-checking downstream.

Upstream, you need to know which queries are worth a page at all: TF-IDF compares texts, it does not tell you what people search for. Downstream, you need a person to check that every new section is accurate and adds something. Google's own question applies: does the content provide "a substantial, complete, or comprehensive description of the topic?" A gap analysis helps with "complete". It cannot help with "substantial" or "original".

The same logic applies to AI assistants. Pages that answer the full set of questions around a topic, clearly and with sources, are easier to quote. The guide [GEO: get cited by ChatGPT, Gemini and Grok](/guides/generative-engine-optimization/) covers what is documented there. And if you draft with AI, [AI content and Google](/guides/ai-content-google/) explains the review Google expects before you publish.

```tonaily
A one-to-one comparison shows one competitor. Tonaily starts earlier: it **finds your competitors' keywords** and the topics they rank for, so you choose what to write before you open a blank page. From a brief, **Kira drafts the article**, the editor shows an **SEO score** while you review, and **Search Console tracking** tells you afterwards whether the page moved. The [TF-IDF tool](/resources/tools/tf-idf-analyzer/) stays free and runs in your browser.
```

## Try it on one page today

Take the article you most want to move from page two to page one. Compare it with the three best pages for its query in the [TF-IDF tool](/resources/tools/tf-idf-analyzer/), keep the terms that come back each time, and write down the questions behind them. If you end up with two or three new sections that a reader would actually miss, you have done the useful part of TF-IDF. If you end up with a list of words to insert, start again from the questions.

## FAQ

### Does Google use TF-IDF to rank pages?

Google does not say so. Its public guide to its ranking systems does not mention TF-IDF; for language, it describes AI systems such as BERT, neural matching and RankBrain, which relate words to concepts. Treat TF-IDF as an analysis method for your own content, not as a model of Google.

### What is a good TF-IDF score?

There is none in absolute terms. A TF-IDF weight only means something relative to the other terms in the same comparison, with the same set of texts. What matters is the ranking: which heavy terms in the pages that rank are absent from yours.

### How many competitor pages should I compare?

More than one. A single page reflects one writer's choices; a term that comes back in the gap list against three or four of the pages ranking for your query is far more likely to be part of the topic. Run the comparison page by page and keep what repeats.

### What is the difference between TF-IDF and keyword density?

Keyword density counts how often a term appears inside one text. TF-IDF also looks at how many texts in a set contain the term, which lets you compare pages and see what one covers and another does not. Density guards against overuse; TF-IDF points to coverage gaps.

### Should I add every missing term the tool finds?

No. Some are noise: brand names, examples specific to that page, words that happen to repeat. Keep the terms that point to a real subtopic your reader expects, write the section that answers it, and let the vocabulary come with it.


## Sources

- [tf–idf (Wikipedia)](https://en.wikipedia.org/wiki/Tf%E2%80%93idf)
- [Feature extraction: Tf–idf term weighting (scikit-learn documentation)](https://scikit-learn.org/stable/modules/feature_extraction.html)
- [Similarity module: BM25 similarity (Elasticsearch reference)](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html)
- [A guide to Google Search ranking systems (Google Search Central)](https://developers.google.com/search/docs/appearance/ranking-systems-guide)
- [Understanding searches better than ever before (Google, The Keyword, October 2019)](https://blog.google/products/search/search-language-understanding-bert/)
- [SEO Starter Guide (Google Search Central)](https://developers.google.com/search/docs/fundamentals/seo-starter-guide)
- [Spam policies for Google web search: Keyword stuffing (Google Search Central)](https://developers.google.com/search/docs/essentials/spam-policies)
- [Creating helpful, reliable, people-first content (Google Search Central)](https://developers.google.com/search/docs/fundamentals/creating-helpful-content)