{ "type": "Article", "authors": [ { "type": "Person", "familyNames": [ "Müller" ], "givenNames": [ "MarcelFuhrmannandFabian" ] } ], "identifiers": [], "references": [ { "type": "Article", "id": "bib-bib1", "authors": [], "title": "\nE. Dunne and K. Hulek, Mathematics Subject Classification 2020. Eur.\nMath. Soc. Newsl. 115, 5–6 (2020) ", "url": "https://doi.org/10.4171/news/115/2" }, { "type": "Article", "id": "bib-bib2", "authors": [], "title": "\nM. Schubotz and O. Teschke, zbMATH Open: Towards standardized machine\ninterfaces to expose bibliographic metadata. Eur. Math. Soc. Mag.\n119, 50–53 (2021) ", "url": "https://doi.org/10.4171/mag/12" } ], "title": "A REST API for zbMATH Open access", "meta": {}, "content": [ { "type": "Heading", "id": "S1", "depth": 1, "content": [ "1 Introduction" ] }, { "type": "Paragraph", "id": "S1.p1", "content": [ "For some time now, zbMATH Open has been offering its services in various formats and through different media. Biweekly\nprinted volumes for long decades in the past, distribution via CD-ROMs and DVDs, and the now standard (and only) access\noption as an online service, have been incarnations of the service over the years. Typography and design have changed\nmany times in the printed volumes as well as the online version. Furthermore, the online version has offered new ways\nof distributing the data for specific purposes. The contents of zbMATH Open are available in HTML, BibTeX, and PDF\nform, as well as via an XML-based OAI-PMH API (more on these three-letter acronyms below). In 2023 a new option was\nadded, a so-called REST API, which makes the data available in a machine-readable fashion, for automated use by\nresearchers. We will outline the features of this new API and describe new use cases made possible by this new way of\naccessing zbMATH Open." ] }, { "type": "Heading", "id": "S2", "depth": 1, "content": [ "2 Why do we need APIs?" ] }, { "type": "Paragraph", "id": "S2.p1", "content": [ "API is an acronym for “application programming interface.” It denotes any part of a software system that makes it\naccessible through programs and machines, as opposed to human users, and is an important part of technology for\nsoftware developers. The “interface” component of an API is a system that allows two or more software programs to\ninteract and understand each other without communication problems. One could think of it as a sort of intermediary that\nhelps software programs share information with each other." ] }, { "type": "Paragraph", "id": "S2.p2", "content": [ "Creating an API establishes a common language for wildly divergent systems to communicate to each other. Instead of\nhaving to learn each other’s specifications, both entities can interact in a common format, which can also be used by\nany other application or service. The so-called OpenAPI Specification defines a standard, language-agnostic interface\nto HTTP APIs. This allows both humans and computers to discover and understand the capabilities of the service without\naccess to source code. The user can understand and interact with the remote service with a minimal amount of\nimplementation effort. An OpenAPI definition can then be used by documentation generation tools to display the API,\ncode generation tools to generate servers and clients in various programming languages, testing tools, and many other\nuse cases. This provides advantages for users and data providers alike, granting access in a machine-readable,\nautomatable way for interested partners, e.g., research groups." ] }, { "type": "Heading", "id": "S3", "depth": 1, "content": [ "3 The OAI-PMH API of zbMATH Open" ] }, { "type": "Paragraph", "id": "S3.p1", "content": [ "In 2002, an informal organisation of providers of repository services called the “Open Archives Initiative,” or\n“OAI” for short, created a common protocol that allowed users to automatically harvest metadata from such a\nservice in a unified way using programmatic tools. Unsurprisingly, this was called the “Protocol for Metadata\nHarvesting,” or “PMH.” It is supported by numerous providers of scientific or other\ndatabases. In mathematics, of course, by far the largest of them is the arXiv. In September 2020, zbMATH Open started\nto support this protocol as well.", { "type": "Note", "id": "idm37", "noteType": "Footnote", "content": [ { "type": "Paragraph", "id": "footnote1", "content": [ "Available at ", { "type": "Link", "target": "https://oai.zbmath.org/", "content": [ "oai.zbmath.org" ] }, ". For\nfurther information, see [", { "type": "Cite", "target": "bib-bib2", "content": [ "2" ] }, "]." ] } ] } ] }, { "type": "Paragraph", "id": "S3.p2", "content": [ "The obvious advantages of using such a common protocol are that it is well established, easy to implement, and has a\nsubstantial positive impact on working mathematicians. Aggregators, archives, and bibliometric researchers commonly use\nthe OAI-PMH, and there are ready-made client libraries available in every major programming language. Furthermore, this\nprotocol is well suited for harvesting the entire open collection of zbMATH Open document data.", { "type": "Note", "id": "idm48", "noteType": "Footnote", "content": [ { "type": "Paragraph", "id": "footnote2", "content": [ "The data made\navailable through the OAI-PMH API is provided under a ", { "type": "Link", "target": "https://creativecommons.org/licenses/by-sa/4.0/", "content": [ "CC-BY-SA\n4.0 license" ] }, ", as are all other data offers by zbMATH Open made through APIs. Legal constraints by publishers that are\noutside the control of zbMATH Open are currently restricting the amount of data that can be distributed\nthrough APIs." ] } ] }, " Moreover, it is possible to restrict the harvesting process to well-defined subsets, and to only\nconsume updates since the last download, obliterating the need to download all the data every time." ] }, { "type": "Heading", "id": "S4", "depth": 1, "content": [ "4 Why do we need another API?" ] }, { "type": "Paragraph", "content": [ "Although the OAI-PMH API has numerous advantages – not least among them its widespread adoption – it also has various shortcomings:" ] }, { "type": "Paragraph", "content": [] }, { "type": "List", "items": [ { "type": "ListItem", "content": [ { "type": "Paragraph", "id": "S4.I1.i1.p1", "content": [ "The standard format supports only a very limited set of metadata (although additional formats can be defined)." ] } ] }, { "type": "ListItem", "content": [ { "type": "Paragraph", "id": "S4.I1.i2.p1", "content": [ "It is XML-based only." ] } ] }, { "type": "ListItem", "content": [ { "type": "Paragraph", "id": "S4.I1.i3.p1", "content": [ "It only supports filtering by time and by subset." ] } ] }, { "type": "ListItem", "content": [ { "type": "Paragraph", "id": "S4.I1.i4.p1", "content": [ "It is only designed to retrieve data, not to update or add any." ] } ] } ], "order": "Unordered", "meta": { "listType": "bullet" } }, { "type": "Paragraph", "id": "S4.p2", "content": [ "Due to this lack of flexibility, we decided to create a new REST API for zbMATH Open, with the OAI-PMH API as a\nstarting point. The acronym “REST” stands for “representational state transfer” and is a style of software\narchitecture that defines how different systems communicate, again improving reusability and interoperability. An API\nthat complies with these REST principles is known as a REST API." ] }, { "type": "Paragraph", "id": "S4.p3", "content": [ "The new zbMATH Open REST API", { "type": "Note", "id": "idm98", "noteType": "Footnote", "content": [ { "type": "Paragraph", "id": "footnote3", "content": [ "Available at ", { "type": "Link", "target": "https://api.zbmath.org", "content": [ "https://api.zbmath.org" ] }, "." ] } ] }, " now gives access to a number of\nadditional benefits. First, improving the data set offered by OAI-PMH, a complete set of bibliographic metadata for\nzbMATH documents is available to any interested party. The complex metadata structure of any document covers a vast\nvariety of information. A quick overview of the available data sets is shown in Figure ", { "type": "Cite", "target": "S4-F1", "content": [ "1" ] }, "." ] }, { "type": "Figure", "id": "S4-F1", "caption": [ { "type": "Paragraph", "content": [ "Screenshot of data structure of an example result, as seen in the browser." ] } ], "licenses": [ { "type": "CreativeWork", "url": "https://creativecommons.org/licenses/by/4.0/legalcode", "content": [ { "type": "Paragraph", "content": [] } ] } ], "content": [ { "type": "ImageObject", "contentUrl": "restapiresult_sk.jpg", "mediaType": "image/jpeg", "meta": { "inline": false } } ] }, { "type": "Paragraph", "id": "S4.p4", "content": [ "Furthermore, the API offers access to information about published authors and their work, their awards, any external\nIDs, and so on. Other data endpoints cover information about the current version of Mathematics Subject\nClassification codes for documents (see [", { "type": "Cite", "target": "bib-bib1", "content": [ "1" ] }, "]),\nmetadata information about books and journals, any published document available within the zbMATH Open database, and\nmetadata information about research software. All these available data sets are similar to what is\navailable at the zbMATH Open website. The difference is that the data is machine-readable and therefore can be easily\nused for any purpose, e.g. any bibliographic research efforts.\n" ] }, { "type": "Paragraph", "id": "S4.p5", "content": [ "Most importantly, it is now possible to search in the REST API in exactly the same way as on the website\n", { "type": "Link", "target": "https://zbmath.org/", "content": [ "zbmath.org" ] }, ", including arbitrary combinations of logical operators and filtering by\nall supported fields (in particular, arbitrary time frames)." ] }, { "type": "Paragraph", "id": "S4.p6", "content": [ "The search result is a machine-readable status report in JSON format, containing the requested data, but also including\ninformation about the search execution (for example, whether it was successful or not, and if not, why). In the future,\nother formats will be supported as well (e.g., XML)." ] }, { "type": "Paragraph", "id": "S4.p7", "content": [ "All the information is documented in the Swagger UI (see the screenshot\nof Figure ", { "type": "Cite", "target": "S4-F2", "content": [ "2" ] }, "),", { "type": "Note", "id": "idm134", "noteType": "Footnote", "content": [ { "type": "Paragraph", "id": "footnote4", "content": [ "For more information, see\n", { "type": "Link", "target": "https://swagger.io/docs/", "content": [ "https://swagger.io/docs/" ] }, "." ] } ] }, " so that, based on the information there, other developers may create their own APIs\nfor their own research projects. Like in the case of the OAI-PMH API, in some instances, parts of the information (e.g.,\nabstracts or references, based on the agreements with the respective publisher) need to be redacted for legal reasons.\nThe API takes care of that automatically." ] }, { "type": "Figure", "id": "S4-F2", "caption": [ { "type": "Paragraph", "content": [ "Screenshot of the user interface of zbMATH Open REST API." ] } ], "licenses": [ { "type": "CreativeWork", "url": "https://creativecommons.org/licenses/by/4.0/legalcode", "content": [ { "type": "Paragraph", "content": [] } ] } ], "content": [ { "type": "ImageObject", "contentUrl": "restapi_s3k.jpg", "mediaType": "image/jpeg", "meta": { "inline": false } } ] }, { "type": "Heading", "id": "S5", "depth": 1, "content": [ "5 Outlook: Pushing data via the REST API" ] }, { "type": "Paragraph", "id": "S5.p1", "content": [ "Currently, we are working on adding more features to the zbMATH Open API. So far, data can only be\nretrieved from the database. A next step is to enable users to also add new data in an automated way. We identified a\nnumber of use cases for that, which would improve the updating process of the zbMATH Open database and facilitate the\nincorporation of input from the user community." ] }, { "type": "Paragraph", "id": "S5.p2", "content": [ "The most straightforward use case would be to enable the upload of bibliographic metadata information about documents\nfrom publishers directly. Although some larger publishers have their own data delivery processes, for many (especially\nsmaller ones) it can be quite helpful to offer a well-documented way of adding their bibliographical data to\nzbMATH Open." ] }, { "type": "Paragraph", "id": "S5.p3", "content": [ "Other use cases involve uploading references or external IDs (like DOI or arXiv identifier) of existing documents to\nenrich the data contained within the zbMATH Open database. Moreover, the upload of, and therefore the access to,\nfull-text documents is planned to be possible in the near future. This requires resolving any licensing\nissues, in particular the uploader (e.g., the original author) needs to certify that they have the right to upload the\nfull text for distribution by zbMATH Open." ] }, { "type": "Paragraph", "id": "S5.p4", "content": [ "Finally, adding or correcting new information about authors and their connection to existing articles (if this\ninformation was missing before) is going to be possible via the zbMATH Open REST API." ] }, { "type": "Heading", "id": "S6", "depth": 1, "content": [ "6 Final thoughts" ] }, { "type": "Paragraph", "id": "S6.p1", "content": [ "To summarize, the new zbMATH Open REST API will be a useful tool for automated access to the data stored within the\nzbMATH Open database. It will extend the possibilities currently offered by the OAI-PMH API (without superseding\nit). With the oncoming features of adding and/or updating information to the database in an automatable way, open\naccess for zbMATH Open data will continue to expand." ] }, { "type": "Paragraph", "id": "authorinfo", "content": [ "\nMarcel Fuhrmann\nstudied physics at\nHumboldt-Universität zu\nBerlin. He completed his PhD in astrophysics and worked as a postdoctoral\nresearcher in applied mathematics at the University of Potsdam. From 2017 to 2021 he worked at the Federal Institute\nfor Risk Assessment. In 2021 he moved to FIZ Karlsruhe and works on the zbMATH Open REST API and other related\nprojects.\n", { "type": "Link", "target": "mailto:marcel.fuhrmann@fiz-karlsruhe.de", "content": [ "marcel.fuhrmann@fiz-karlsruhe.de" ] }, "\nFabian Müller\nstudied mathematics and computer science at Humboldt-Universität zu Berlin. After finishing his doctoral studies in\nalgebraic geometry in 2013, he started working at zbMATH Open, where he is team lead for development and systems.\n", { "type": "Link", "target": "mailto:fabian.mueller@fiz-karlsruhe.de", "content": [ "fabian.mueller@fiz-karlsruhe.de" ] } ] } ] }