We have updated the rate limits in the REST API, as outlined in a recent blog post. This will ensure that we can keep the API and the data available for everyone, and is one a series of changes we’ve been looking at to improve stability of the service as the volume of metadata grows. In setting the new levels, we’ve tried to ensure that users can still access the data they are interested in, although the retrieval may take a little longer.
During implementation, we needed to make a few changes to the original plan and we aren’t yet able to make rate limits differentiate between different types of request. We’ll work on that next year. In the meantime, we’ve made the following changes:
- Updates to the rate limits and concurrency limits in the public and polite pools.
- Removing a 10 s delay in applying automatic blocking if rate limits are exceeded.
- Adding a response header containing the concurrency limit.
Rate limits now have the following values:
| Pool | Rate limit (requests per second) | Concurrency limit |
|---|---|---|
| Public | 5 | 1 |
| Polite | 10 | 5 |
The rate limit refers to the number of requests that can be made per second. The concurrency limit is the number of simultaneous requests that can be made, so in some cases you may need to wait for previous requests to finish before making a new one. The polite pool can be accessed by providing a valid email address in the mailto request parameter.