The error#
{retries + 1} requests to {url} failed, giving up.retries defaults to 3, so the usual line is 4 requests to {url} failed, giving up. The constructor argument is what changes the count. The log also prints Errors: and the joined reasons. Those reasons are the callback messages (blocked (429), content type is not JSON, non-200 status code ({status}), received invalid JSON from Twitter, and the repr of a network exception).
ScraperException is the class.
What triggers it#
_request looped retries + 1 times without a successful response. Between tries it sleeps 1.0 * 2**attempt seconds and logs Waiting {seconds} seconds.
How to confirm#
The message ends with failed, giving up. The Errors: line above it is the list of why. blocked (429) means the guest-token path. content type is not JSON means Twitter returned something else, often an HTML wall.
Fixes#
Raise retries only if a later try can succeed. A block that lasts 300 seconds will still be in force across the default backoff (1, 2, then 4 seconds). The sleep is not a rate-limit reset. Fix the reason in the Errors: list. The blocked and guest-token pages are the two Twitter reasons this version produces most often.
SocialScrape#
Profile is GET /v1/twitter/profile. One post is GET /v1/twitter/tweet. A profile's posts are GET /v1/twitter/profile/tweets. These read what X shows logged out.
Failed requests cost 0 credits. New accounts get 100 credits on signup, no card. Suspended accounts come back as removed, unknown ones as not_found.