The error#
_check_api_response does not raise on the first bad status. It returns a failure reason:
blocked (403)
blocked (404)
blocked (429)For 429, if x-rate-limit-remaining is 0 and x-rate-limit-reset is set, the guest token is blocked until that reset, capped at 900 seconds. Every other 403, 404 or 429 blocks the token for 300 seconds. The token is then cleared and a new one is requested.
The scraper retries. The default retries on Scraper is 3. When they run out, the exception is the giving-up message, and the log line Errors: includes these blocked (...) reasons.
What triggers it#
r.status_code is 403, 404 or 429 on a Twitter API call made with the guest token. A JSON body with GraphQL errors and no instructions returns a different reason: Twitter responded with an error: {name}: {message}.
How to confirm#
The log contains blocked (403), blocked (404) or blocked (429) and Retrieving guest token again. The final exception, if the retries lose, starts with a request count and failed, giving up.
Fixes#
The code's own response is to throw away the guest token and wait 300 seconds, or until the rate-limit reset. Creating tokens faster from the same IP is the pattern that fills those statuses. snscrape 0.7.0.20230622 does not attach a user cookie in this function.
SocialScrape#
Profile is GET /v1/twitter/profile. One post is GET /v1/twitter/tweet. A profile's posts are GET /v1/twitter/profile/tweets. These read what X shows logged out.
Failed requests cost 0 credits. New accounts get 100 credits on signup, no card. Suspended accounts come back as removed, unknown ones as not_found.