Caching and max_age_days

Social data that was fetched recently rarely needs fetching again. Tell us how old an answer you will accept and a fresh-enough cached copy comes back for 0 credits.

curl -G "https://api.socialscrape.io/v1/github/user" \
  -H "x-api-key: $SOCIALSCRAPE_API_KEY" \
  --data-urlencode "handle=torvalds" \
  --data-urlencode "max_age_days=7"

• max_age_days takes a number from 0 to 30. Absent or 0 means "always fetch fresh".
• The ScrapeCreators spelling works too: cache_max_age=7d. Units are d, h, m and s, so cache_max_age=12h is half a day.
• A hit is marked "cached": true with cached_at set to when the data was first fetched, and credits_charged is 0.
• A miss fetches fresh, charges normally, and stores the result for next time.

What counts as the same request#

The cache key is the endpoint plus its parameters. Presentation parameters do not matter: trim, raw and max_age_days itself read the same entry. A different cursor, sort, region or search term is a different request.

The cache is keyed by request, not by customer, so a repeat of something anyone fetched recently can be served from it.

How long results are kept#

Cached results live at most 30 days, and an endpoint may keep them for less (its page says how long). After that they are deleted. Nothing is served from the cache unless you ask for it with max_age_days.

Some endpoints are never cached, for example your own balance, usage and monitors.

Removing data#

If a person asks us to remove their data, we delete matching cached results and stop serving them: such a request answers 410 removed. See takedown and opt-out.