For charity web administrators
About our crawler
If you have seen SomewhereToHelpBot in your
server logs, this page explains what it is and how to make it stop.
The 8 Houses Daily prints volunteering notices from London homelessness charities and links every one of them to the charity's own page. To keep those notices accurate, an automated reader visits each charity's public volunteering page once a week and notes what has changed.
It reads one page and, at most, six pages linked from it. It does not submit forms, follow apply links, create accounts, or look at anything behind a login. It stores what a role involves, when it runs and what you ask of volunteers — as our own summary, not as a copy of your text.
The particulars
- Identifies as
SomewhereToHelpBot/1.0- Frequency
- Once a week, early on a Sunday
- Rate
- One request per second, single-threaded — about the load of one person reading a couple of pages
- robots.txt
- Honoured. Disallow us and we stop, and the listing keeps only a link to you
- Conditional requests
- Yes — an unchanged page costs you a 304 and nothing else
Any one of these works, and none of them needs a conversation:
Email us. Write to tek@8houses.co.uk and say so. We remove the organisation entirely — not hidden, removed from the build — and stop reading your site. There is nothing to argue about and we will not ask why.
Or use robots.txt. Add this to yours and the crawler will not return:
User-agent: SomewhereToHelpBot
Disallow: /Or correct us instead. If the problem is that a notice is wrong rather than that it exists, tell us what it should say and it will be fixed within a few days.
What we are not We are not a charity, and we do not vet or rank the organisations we list. We print notices and send people to you. There is no advertising, and nothing here is paid for or sponsored.