CRAWLER POLICY
Useful data. Responsible access.
This installation identifies itself using its configured crawler user-agent, honours robots.txt by default, rate-limits requests, blocks private network targets and records crawl activity. Operators must have a lawful basis and follow website terms, privacy, copyright and database rights. Semonic does not bypass CAPTCHAs, authentication, paywalls or access controls.
Return