User-agent: Googlebot Disallow: User-agent: Googlebot-Mobile Disallow: User-agent: Mediapartners-Google Disallow: User-agent: AdsBot-Google Disallow: User-Agent: bingbot Crawl-delay: 10 User-Agent: bingbot Disallow: User-agent: Twitterbot Disallow: User-agent: facebookexternalhit Disallow: User-agent: DuckDuckBot Disallow: User-agent: CriteoBot/0.1 Disallow: User-agent: grapeshot Disallow: User-agent: * Disallow: /