The LocalsVoted bot

A small program that reads the public contact details on a business’s own website.

What it is

LocalsVoted sometimes checks a business’s name, address and phone number against the business’s own website, so the details on its page are right. The program that does this is the LocalsVoted bot. It identifies itself with this user agent:

LocalsVotedBot/1.0 (+https://localsvoted.com/bot; [email protected])

What it reads

  • Only the public website a business gave us or published as its own. It does not read social media, review sites, maps or directories.
  • The home page and at most one contact page, looking for the business name, street address and phone number (structured data first, then the contact page).
  • Nothing behind a login, a CAPTCHA or any other access control. If a site asks for one, the bot stops.

How it behaves

  • It reads your robots.txt first and follows it. If a page is disallowed, or the page asks not to be indexed or followed, it does not read it.
  • It makes one request at a time to a site and waits at least five seconds between requests.
  • It gives up after ten seconds, and it does not use cookies.
  • If a site refuses it (an error such as 403 or 429, a login, or a CAPTCHA), it does not visit that site again for 30 days. If a site is only down or slow, it does not retry during the same visit and may try again on a later visit.

What we keep

We keep the facts we found (name, address, phone), the address of the page they came from and the date. We do not keep a copy of the page.

Opt out

To stop the bot visiting your site, email [email protected] with your domain name (for example example.com). We will stop and delete what we kept from that site: the facts on pending submissions, the contact email we found there, and the name, address and phone we took from it for a listing (each goes back to the open-data value if we have one, or is left blank; a business name with no other source stays until a person at LocalsVoted confirms it). We keep a short log of earlier visits, which holds no page content, and a record of your opt-out so we do not read your site again.

Corrections and deletion

To ask for a correction or for the details we kept to be deleted, email [email protected] with the business name and the page address. If you own the business you can also change its details yourself once you have claimed the page.