The Tail Tells the Story: Extreme Textual Overlap

There is now a lot of press coverage of in The Times, the Telegraph and the Mail.

From a mathematician’s or programmer’s perspective, we highlight an outstanding piece of work by Alexey Titorenko, available on GitHub here. It presents a fully reproducible, embedding-based analysis of the PhD theses by Arday (2015) and Zwozdiak-Myers (2009).

Wonderfully, the complete pipeline is included end-to-end: extraction, segmentation, embedding, control construction, Poisson testing, provenance classification, and extent estimation. The code automatically retrieves all source documents from their respective repositories. Equally remarkably, the entire process runs on a standard CPU in approximately twenty minutes.

The two PhD theses have considerable overlap.

“The signal is in the tail, and it survives controls. Against two independent education doctorates matched on discipline and partially on topic, the focal pair shows:

• 9 near-identical sentence pairs (cosine ≥ 0.999) in 11.09M comparisons, against 1 in 34.29M control comparisons – and that single control match is the standard thesis declaration (“This work has not previously been presented for an award at this, or any other, university”).
• 179 sentence pairs sharing 12 or more consecutive identical words (107 spatially independent blocks), against an expected 0.32 under the null – an exact count over all 11.09M pairs, no embedding model involved. Longest shared run: 63 words.
• Conditional on high semantic similarity, cross-document pairs share a median run of 11.5 identical consecutive words, versus 4.0 and 7.5 for sentences within each thesis – that is, among semantically matched sentences, the two documents phrase shared ideas alike more often than either thesis does when restating itself.

Under the most conservative assumptions available, the probability of observing this pattern under independent composition is 5.7 × 10⁻⁴; on the central estimate of the null rate, 8 × 10⁻¹¹. “

[From Titorekno’s GitHub page]

It is very unlikely this happened independently (specifically one in a one-in-100-billion chance of happening).

The origin of the extreme textual overlap perhaps remains unclear — but both Liverpool John Moores University and Cambridge University have got to come up with better explanations than they have done so far.

And for Cambridge University’s senior management, there are serious questions as to why this is playing out so damagingly in public, since they have known about it for a number of years.