The Quest for Authenticity: Student Develops a Program to Detect Fake Citations in Academic Books

In the era of misinformation, ensuring the credibility of scholarly works has never been more critical. After our coverage this past summer highlighted a particular book filled with fabricated references, we reached out to our readers, asking for examples of similar infractions in other books. This call to arms inspired one reader to put his coding skills to the test: compare the citations found in academic texts to identify those that are dubious.

Michał Wójcik, a Ph.D. candidate at the Free University of Berlin, came across our article about the problematic book on LinkedIn. Intrigued by the issue, he thought, "It shouldn’t be that challenging to automate the process of checking these references." This ignited a passion project, leading him to develop a functioning prototype within a few hours.

By crafting a Python script, Wójcik designed a tool that systematically scans books to validate citations against the Crossref database to verify if Digital Object Identifiers (DOIs) are legitimate. For those citations that his script is unable to identify, he rolls up his sleeves and manually conducts further research using other databases and searching through Google Scholar. This exhausting yet necessary task can take hours and sometimes even the entire day.

To gather his data, Wójcik specifically targeted books published in 2025. "With varying degrees of certainty, I examined 22 books," he mentioned to us. Among these, he flagged two particular texts as especially problematic, stating they stood out for their poor citation practices.

The first book he scrutinized focuses on urban planning strategies aimed at creating sustainable smart cities. Upon investigation, it became clear that many citations either contained significant inaccuracies or referenced entirely fictitious works. The second text discusses energy storage solutions for electrical grids. Alarmingly, it was not only filled with dubious citations, but it had also been translated to English using artificial intelligence without rigorous verification. Both pieces were published by Springer Nature, the same publisher behind the book we previously reported on.

This isn't just a minor oversight. As we highlighted previously, large language models, such as ChatGPT, are notorious for generating citations that are either fabricated or riddled with errors. Despite publishers having strict guidelines against allowing these AI systems to create text from scratch, they are increasingly leveraged in areas like copyediting or translation, albeit often with some notice provided to readers.

Springer Nature has been experimenting with AI technology for the translation of academic texts since 2023. However, it's not alone in this embrace of technological innovation; Taylor and Francis announced in March their intentions to also utilize AI for translating their academic materials into English.

It is worth noting that Springer has issued retractions on several occasions for chapters and entire books due to unverifiable references. In fact, the Retraction Watch Database shows there are over 240 books that have had one or more chapters retracted for a variety of issues including citation problems.

Wójcik’s screen found 40 entries lacking DOIs within the urban planning book titled Urban Morphology and Sustainable Smart Cities. We performed our own checks on the first 32 citations and discovered that 11 could not be authenticated. Interestingly, four of these cited documents were from Indian governmental sources that have since disappeared from online access. We reached out to the authors of the remaining seven suspicious citations, and four of them responded, stating that they either did not write the referenced works or found significant flaws in the citation itself.

Bogdan Ibanescu, a researcher at the Centre for European Studies, expressed uncertainty about whether the citation in question misattributed to him was a result of confusion with another author. Ana Lavalle, another author listed in a flawed citation, confirmed that she had no association with the paper attributed to her. Similarly, Manolya Kavakli-Thorne, a professor at Aston University, affirmed that she did not author the article that was improperly attributed to her. She noted that one of the names mentioned as a co-author was that of her former Ph.D. student who collaborated with her on various papers, which had led to this citation blunder. "What we might be witnessing could be referred to as the hallucinations created by deep learning,” she speculated.

Certain citations displayed bizarre inaccuracies. Jack Ahern, an emeritus professor at the University of Massachusetts Amherst, noticed that while his article was correctly mentioned, it improperly included two additional authors who were never part of that publication. "That article was authored solely by me. I'm puzzled as to why other authors were credited," he remarked.

Gouri Sankar Bhunia, one of the co-authors of the urban planning book and an expert in remote sensing and Geographical Information Systems, weighed in on the accusations. He suggested that the problematic citations might have arisen from unintentional oversight amid the complex process of compiling diverse sources and integrating various drafts.

When we inquired whether generative AI played a role in the creation of the text, Bhunia clarified, "The use of language models like ChatGPT has been minimal in this book, and even then, it was solely for trivial tasks like checking grammar and citation formats. We strictly limited AI’s involvement to copyediting and did not utilize it for generating substantive materials." These practices fall within Springer Nature's established AI policies, which emphasize the need for human oversight in the final content.

Bhunia insisted that the citations had been prepared and verified by qualified academic professionals, asserting that any discrepancies were both unintended and human-made.

A spokesperson for Springer Nature shared that the publisher had already initiated an investigation into both texts prior to our inquiry, indicating that many of the evaluations had been completed by late August. For the urban planning book, they informed us, "a correction is being issued in conjunction with the retraction of three chapters."

The second book, Electrical Energy Storage Technologies and Applications, originally penned in Chinese in 2020, was translated into English this year. The preface of this book openly states that its translation was conducted using AI, with subsequent revisions performed by the authors to ensure the accuracy and scientific integrity of the content.

The English-language citations refer primarily to sources in Chinese, many lacking DOIs or functional links to original documents. Wójcik pointed out the importance of citations in scholarly work is to enable readers to access source materials easily. “This accessibility tends to falter when titles are translated without providing viable links or DOIs,” he added.

We attempted to reach out to Xisheng Tang, the book's lead author, for commentary, but did not receive a response. All three authors are affiliated with the Chinese Academy of Sciences as reflected in the book's preface.

In response to our findings, the publisher affirmed that they will be correcting the citations in this second book.

Springer’s spokesperson noted that while citation errors in academic texts are uncommon, they can occur across all publishing houses. The identification of such inaccuracies typically arises through a mix of editorial review and peer input combined with scrutiny after publication. "Although some of these issues may seem straightforward from the outside, they can often be quite complicated to uncover due to various nuanced factors involved," they explained.

As for Wójcik, he continues to refine his script dedicated to spotting fraudulent references. He has even expanded its application to scholarly papers and has now reviewed approximately 100,000 entries. His findings indicate that about one in every 300 papers flagged as having suspicious references; however, he has only been able to verify about 150 of these cases manually. "Some simply have minor issues, like unresolvable DOIs or DOIs mistakenly appended with a period. Yet, a significant number present nonsensical citations," he pointed out. "Such discrepancies should never find a place in scientific literature, especially when links to Google Scholar are available in the online versions, but remain nonfunctional or irrelevant."

What are your thoughts on the use of AI in academic writing? Do you believe that the benefits outweigh the risks, or do you have concerns about citation integrity? Let us know your views in the comments below!