The error#
_handle_ratelimit reads the extractor config ratelimit. When it is abort:
Rate limit exceededWhen it is abort:{number} and the count reaches that number:
Rate limit exceeded ({amt}/{num})Both are AbortExtraction. Below the count, the same text is only a warning.
The wait path uses the x-rate-limit-reset header when it is present. Otherwise it waits 60 seconds. wait as a value, or wait:{seconds}, sleeps instead of raising. Any other string logs Unsupported 'ratelimit' setting '{value}'.
What triggers it#
The X API response is HTTP 429, or the pagination helper sees x-rate-limit-remaining at or below its threshold and calls _handle_ratelimit. The raise happens only when the config says abort. The default in this function, when the config is unset, falls through to wait.
How to confirm#
You see Rate limit exceeded as an AbortExtraction, and your config has ratelimit set to abort or abort:N. A long pause with no exception is the wait path, using x-rate-limit-reset or 60 seconds.
Fixes#
abort is the setting that turns the limit into an exception. wait is the setting that sleeps until the reset header. Neither gives you a higher limit. They choose between stopping and pausing. Running more parallel jobs spends the same x-rate-limit-remaining faster.
SocialScrape#
X profile and posts do not use your X app rate-limit headers. If X rate limits the fetch, the code is upstream_rate_limited and the attempt is free.
New accounts get 100 credits on signup, no card.