Plus OAI-PMH Feed returning badArgument 500 internal server error since 3rd Sept

Hi,

There seems to be a problem with the OAI-PMH Plus feed, since September 3rd 2023. We have two daily import that makes requests for articles and books, here’s an example of the requests we make:

oai-crossref-org / oai?metadataPrefix=cr_unixml&from=2023-09-03&until=2023-09-03&set=B&verb=ListRecords

Prior to the 3rd, we were able to retrieve data from the repository. The error message we get is:

500 Internal Server Error

I’ve confirmed that the Crossref token we’ve been granted is working, can you please investigate?

Thanks,
G

I’ve had to modify the url example above since I’m not allowed to post links.

Hi @gdfry ,

Thanks for your message, and welcome to the Community Forum! We’ve heard from a few Metadata Plus subscribers about this.

It’s taken us a little longer than we would have liked to identify the problem, but our customer relationship management system underwent a routine, quarterly release on Sunday around 2000 UTC that introduced this bug. Some of our members’ internal identifiers got out of sync as a result of that release. Unfortunately, it sounds like you might be one of them. Our technical team is working on the fix now.

I’ll provide updates on our status page - Crossref Status - Member sync error causing OAI-PMH access issue and data inaccuracies - and will confirm with you on this forum post once we have this fully resolved.

Our apologies,
Isaac

Hi @gdfry ,

We deployed a fix to this bug. All of the Plus tokens that I have spot checked are providing access to their Metadata Plus subscribers. Can you let me know if you continue to experience problems on your end?

-Isaac

Hi Isaac,

Thanks for resolving the issue. We’ve been running fine until 23rd Sept, when we started seeing constant timeouts from this request:
http://oai.crossref.org/oai?metadataPrefix=cr_unixml&from=2023-09-22&until=2023-09-22&set=B&verb=ListRecords

We’re performing an import on books (set=B) only which is failing.
We also have a similar regular import on articles/journals (set=J) which has been working fine over this same period. Can you please investigate further?

Thanks

Bumping for visibility - the service is still having issues, is it related to this incident - Crossref Status - OAI-PMH cited-by and getForwardLinks queries not returning results since 2023 Sept 02

Or do I need to raise a ticket with the helpdesk for the problem I’m having?

Thanks,
G

Hi Godfrey,

While the J set has more individual DOIs, the B set has many, many more setspecs. So, a timeout might be expected when you’re asking for data for the whole set. The workaround, in that case, would be to break it down by prefix, since you can’t narrow the time window any further.

I’ll check with our technical team to see what the expected behavior is, and file a ticket if they believe that this is a bug.

Thanks,
Shayn

1 Like

Hi Shayn,

Any update on this issue? We’re still seeing a lot of timeout issues on the books endpoint, and it’s been ongoing since a few months back.
Prior to this, we’ve never really seen any issues with this endpoint, performing a daily export going at least 3-4 years back. Has something changed in recent months that have caused this?

Thanks,
Godfrey

Hi Godfrey,

I raised a ticket with our technical team, but it hasn’t been triaged yet. I’ll ask them to prioritize that for their next triage session.

-Shayn

Hi Shayn,

Just letting you know that the issue has now been resolved, we’ve not seen the timeouts in the past few days since you resolved the wider incident of the 500 timeout errors.

Thanks,
G

Hi Godfrey,

Thanks for confirming that this is still an ongoing issue for you. @ppolischuk our product manager for metadata retrieval has looked into the issue and, unfortunately, is unable to replicate. For Patrick, response times are high, between 30 and 45 seconds in his testing.

Similar requests for B sets from a single day, both with larger and smaller completeListSize values are also working fine, albeit with relatively long response times.

Given all this, we’ve asked Jon Stark on our technical team to take a look for us to see if he can determine what might be causing the 500 status code timeout errors you’re reporting.

More as we have it,
Isaac

Hello @gdfry, I’m glad the issue resolved itself from your perspective. One of our developers spent some time investigating these timeouts and found that they correspond closely with some database issues we were having during the reported period.

We’ll keep an eye out for any further performance issues. Thanks for reporting this to us.

Thank you @ppolischuk . I misread your previous message @gdfry . Sorry about that. Glad the issue is resolved from your perspective.