Finding sneaked references in the scholarly literature

As all our members are aware, by joining Crossref as a member, you agree to membership terms, which include an obligation to register accurate metadata about your content. This accuracy is important, as the metadata you provide is used by thousands of tools, systems and researchers downstream, creating a permanent scholarly record.

We were recently alerted by a downstream metadata user, and then by a group of scholarly sleuths, to a mismatch in our metadata between the references that a member had registered with us, and the actual references that were published on their articles. Our newly established Data Science team then worked in collaboration with Lonni Besançon, Guillaume Cabanac, Cyril LabbĂ©, Alexander Magazinov, Jules di Scala, and Kathryn Weber-Boer to explore methods for automatic detection of these “sneaked references” by comparing the metadata records against the PDF documents. You can read more in their preprint.

We take mistakes in the scholarly record seriously, and work with our members to ensure that they can keep their metadata records correct and high quality. If we’re alerted to a problem such as this, our first response is always to encourage the member to fix the error. However, if we determine that there has been a deliberate attempt to manipulate the scholarly record, this may, in very rare cases, lead to membership revocation.

Our membership team contacted the member mentioned in this case, and has suspended their access to register further records. We are discussing the case with our board’s Executive Committee to confirm the next steps.

The good news is that we haven’t seen any evidence of this being a large-scale problem, and there is now a programmatic way to help detect this. We would encourage anyone in the community to use the methods in the preprint and contact us via the forum if you see any further cases.

For members, this case highlights the importance of keeping your metadata record complete, accurate and up-to-date. You can find out more about how to update an existing metadata record in our documentation, and more about our (very rare) membership revocation process here.

5 Likes