About SeoSetBot
SeoSetBot is the program SeoSet.app uses to read public web pages when it runs a free website check. If you found it in your server logs, this page explains who runs it, what it reads and how to keep it out.
Who runs it
SeoSetBot is run by SeoSet.app, the service on this website: a local SEO check for home service businesses and contractors.
Questions about the bot and requests to stop checking your website: support@seoset.app.
How it identifies itself
Every request carries this User-Agent header. The address at its end leads to the page about the bot, and SeoSetBot is the name to use in robots.txt.
Mozilla/5.0 (compatible; SeoSetBot/1.0; +https://localrate.store/en/bot)What it reads
SeoSetBot makes plain GET requests to public pages. It does not log in, fill in or submit forms, scan for vulnerabilities or change anything on a website.
A website entered in the check form
A visitor types a website address into the form on this website, and SeoSetBot reads the website at that address for the check: its home page, its robots.txt file, its sitemap and up to 20 other pages.
Competitor websites
For each check, up to 3 websites of competitors that a web search returns for the service and the city. From each of them SeoSetBot reads robots.txt, the home page, the sitemap and at most 2 child sitemaps. It opens no other page of these websites: it only looks at the page addresses listed in the sitemap and linked from the home page, to see which services have a page of their own. We do not store these addresses or the names of the websites, only our own counts.
How it behaves on competitor websites
- It reads robots.txt first and follows it. If the file has a group for SeoSetBot, that group applies; otherwise the group for all robots (*) does.
- It sends one request at a time to a website.
- If robots.txt cannot be read because the server answers with a 5xx error or with 429 (too many requests), or does not answer, it skips the website and reads nothing else from it. If there is no robots.txt file (for example, a 404 answer), everything counts as allowed.
How to keep it out
To close your whole website to SeoSetBot, add these lines to the robots.txt file at the root of the website:
User-agent: SeoSetBot
Disallow: /To close only a part of it, put its path instead of /, for example Disallow: /private/.
This file stops SeoSetBot when it reads your website as a competitor website. It does not stop a check that a visitor started by entering your website address in the form: that check reads the website because it was asked to. If you do not want your website checked, write to support@seoset.app and we will consider your request.
Where competitor websites come from
We find competitor websites through web search services (Brave Search and DataForSEO). We send them only a search phrase made of a service and a place, for example “drain cleaning Portland, OR”. We do not send the business name, the website address or any contact details.