Robots Meta Conflict

Detects conflicts between robots meta tags and robots.txt

Rule ID	crawl/robots-meta-conflict
Category	Crawlability
Scope	Site-wide
Severity	warning
Weight	5/10

Solution

Robots.txt and robots meta tags should work together, not conflict. If robots.txt blocks a URL, search engines won’t see the meta robots tag at all. Common conflicts: blocking a page in robots.txt while trying to noindex it (unnecessary), or allowing in robots.txt but noindexing (works but confusing). For noindex, let the page be crawled so the directive is seen. For blocked pages, robots.txt alone is sufficient.

Enable / Disable

Disable this rule

[rules]
disable = ["crawl/robots-meta-conflict"]

Disable all Crawlability rules

[rules]
disable = ["crawl/*"]

Enable only this rule

[rules]
enable = ["crawl/robots-meta-conflict"]
disable = ["*"]