# Block aggressive and marketing bots User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: Screaming Frog SEO Spider Disallow: / User-agent: MJ12bot Disallow: / User-agent: BLEXBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DotBot Disallow: / User-agent: MegaIndex Disallow: / User-agent: SiteAnalyzerBot Disallow: / User-agent: YandexBot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: LinkpadBot Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: SeznamBot Disallow: / User-agent: DataForSEO Disallow: / # Additional marketing/crawling bots User-agent: Moz Disallow: / User-agent: Majestic Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: Sogou web spider Disallow: / User-agent: spbot Disallow: / User-agent: Exabot Disallow: / User-agent: KaloogaBot Disallow: / User-agent: Baiduspider Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: barkrowler Disallow: / User-agent: PetalBot Disallow: / User-agent: GeedoProductSearch Disallow: /