REST API pools: Which to use and when?

What are the differences between the public, polite, and plus API pools?

You may have seen that we have three different pools for accessing metadata, depending on whether you authenticate and how much information you give us about yourself. It also affects how much data you can collect at once. Importantly, though, the content of the data is exactly in each case.

Here we describe each pool and when you might want to use them.

The public pool

This is the most basic and simplest to access. You just use the API URL with any parameters and you’ll get back metadata. Easy.

You can do it anonymously, at least almost anonymously. You don’t need to give us any identifying information, but be aware that your IP address and the content of your request will appear in our logs.

There is also some risk to using the public pool. If there is a problem, for example if a script runs out of control or your request destabilises our system, we may block you without having had the chance to contact you first.

When to use the public pool

The public pool could be your first point of contact with the Crossref API. Maybe you’re making a handful of requests and don’t need the higher rate limits that other pools provide. If you have privacy concerns, it is also the best option for you.

The polite pool

Almost as easy to access as the public pool, you just need to add a mailto parameter to your request with a valid email address. In return you will get higher rate limits and concurrency limits. In the unlikely event that there is a problem with your requests, we can get in touch with you. A polite request might look like this:

https://api.crossref.org/works?mailto=yourname@domain.org

We have had great experiences contacting polite users and been able to help troubleshoot problems that have helped us and them. Note that we don’t validate email addresses, however if we try to contact you and the email bounces, we may prevent you from making future requests using that address. If you’re running requests over very long periods of time, be sure to check that the address is still valid.

We also won’t use your email address for nefarious purposes. We will use it to troubleshoot requests coming from the same user or service, contacting you to look for solutions to a problem, and very occasionally for changes that affect how you use the REST API. We won’t use it for marketing and will never pass it on to third party services.

When to use the polite pool

This is the pool we recommend for most users. It is the best option if you are making a moderate number of requests, experimenting, or running a non-critical service at relatively low request rates.

The plus pool

Some users have credentials they can use to access higher rate limits. There is a cost to accessing the Plus pool, which comes with guarantees for stability and support. One key thing to note is that the data accessed by the Plus pool is exactly the same as the other pools, only the level of service is different.

The queries and filters accessible to Plus users are the same. In addition, they have access to monthly snapshots or all Crossref DOI records in JSON and XML snapshots. Once a year we release a JSON snapshot as a public data file, which is free for anyone to access.

When to use the plus pool

If you are running a product or service for which Crossref metadata is a critical component, we would recommend Metadata Plus. It can also benefit services that run at very high volume, especially if you are passing user requests through to Crossref. By signing up to Metadata Plus, you are also contributing to the sustainability of our services and supporting the wider community.

The internal pool

This is a hidden bonus pool that we use ourselves to support many Crossref services that use our metadata.

Sorry, you can’t use the internal pool

Unless you’re a colleague of mine, unfortunately you can’t get access. You’ll have to choose one of the other options.

Summary

In practice, we see the most traffic through the public pool, followed by polite, then plus. The way they are used is quite different, though:

  • In the public pool, we see predominantly requests to look up metadata for a single DOI.
  • In the public and plus pools we see a lot more use of filters, suggesting more research tasks. Queries are also used more in the public pool.
  • In the plus pool, we see a lot of requests with very large response sets that need to be paged through, this is coupled with use of snapshots to keep a local copy of the Crossref dataset.

We hope this summary helps you to decide on the best way to use the REST API for your purposes. If you would like some more advice, please leave a comment or create a new topic.

2 Likes