AI (similar arch to ChatGPT) being used to read ancient scrolls for first time!

Perhaps in the future such techniques may allow scholars to restore or read ancient Buddhist scriptures? Wondering what people on the forum would think of such a thing if it did come to pass?

I wonder whether the current forum rules would even allow printing any such scriptures if AI were involved in piecing them back together :joy:

:pray:

UPDATE: Here is another article which goes into a bit more detail about the ā€œAIā€ portion that helped decipher these texts.

Farritor spent much of the past year developing and training a machine-learning model that could detect ultra-faint differences in the texture of the carbonized scrolls, which are now too delicate to unroll. Those textural differences hinted at the presence of ink ā€” and Greek letters that many thought would never be read again. Eventually, Farritorā€™s model managed to identify 10 letters in close proximity, enough to earn him the Vesuvius Challengeā€™s First Letters Prize. Experts would soon conclude that several of those letters spelled the Greek word for ā€œpurple.ā€

2ND UPDATE:

I also found this AI model code for the Vesuvius work on github: GitHub - younader/VesuviusPretraining

3RD UPDATE:

This article on arxiv [2102.05095] Is Space-Time Attention All You Need for Video Understanding? describes in precise detail the AI that is being used to read these scrolls. It is a variant of a Transformer based model - the same kind of AI architecture that powers OpenAIā€™s ChatGPT - so this would seem to be ā€œreal AIā€ as opposed to the fuzzy kind :joy:

If such a scroll or papyrus is otherwise inaccessible - how can we ever be sure that the reconstruction is correct?

Not exactly trusting the current AI models. In philosphy, they too often wind up with total gibberish.

3 Likes

I guess that would depend on whether the AI is able to explain its reasoning steps in a manner that human experts could verify them? Probably similar to how human experts are verified? Iā€™m just speculating though and I guess youā€™d have to read the paper to understand how it is verified.

Btw, Iā€™ve had no problem finding plenty of human gibberish when it comes to philosophy either :joy: :pray:

1 Like

Yes, all the steps would have to be verifiable.

Just a thought experiment: Imagine that a new found manuscript contained the sentence in Pali: ā€œThe Buddha affirmed everyone the existence of a Soul.ā€

Invevitably, by changing just two or three Pali characters, the sentence reads differently and is not at odds with 95% of the existing material in which the Buddha does not affirm the existence of a Soul.

How likely do you think it is that AI will come up with the correct reconstruction?

There is gibberish ā€¦ and then there is gibberish.

Wait a minute. So now if someone combines an AutoCAD application with an OCR application, itā€™s ā€œAIā€? The term isnā€™t even meaningful anymore. Anything that processes any kind of data is AI now. Reality check: Thatā€™s ordinary software people have been using for decades. The X-ray photography is where the magic occurs, as thatā€™s how the writing is made visible.

Hereā€™s an older story about the same process back in 2015:

Notice how the buzzword ā€œAIā€ isnā€™t mentioned?

7 Likes

Hello cdpatton, while I share your amusement about the ever widening embrace of the word AI in novel ways, in this case I think it might not be applicable. :joy:

It does seem that an actual AI was trained - a machine learning model - to improve on the techniques that you note have been in use for a long time.

Here is one of articles I found with a bit more detail:

Farritor spent much of the past year developing and training a machine-learning model that could detect ultra-faint differences in the texture of the carbonized scrolls, which are now too delicate to unroll.

You can find a nice video within where the person who came up with the ā€œAIā€ portion talks about what he did and how the it made it possible to deciphe a portion of these texts.


I also found this AI model code for the Vesuvius work on github: GitHub - younader/VesuviusPretraining

:pray:

The line for what we call ā€˜AIā€™ is fuzzy.

Technically OCR could be ā€˜machine learningā€™.
What exactly is the machine learning, with regards to these scrolls? Surely it has a familiar character set and there is no quirky human change factors.

I share Charlesā€™ amusement.

Hello Venerable, you can find the machine learning code here: VesuviusPretraining/pretraining.ipynb at main Ā· younader/VesuviusPretraining Ā· GitHub

This is using the popular pytorch framework to train a machine learning model based on the resnet architecture. It is closely analagous to the CLIP (Contrastive Languageā€“Image Pre-training) model from OpenAI you can read more about here.

Perhaps such models are not ā€œAIā€ for purposes of the community guidelines then and weā€™re able to share results from such models on the forums? Can moderators clarify if this is allowed since ā€œAIā€ is such a fuzzy term? Thanks!

@moderators question for you ^^
:pray:

I hope itā€™s okay for me to post these links?

I found some possible answers under How accurate are these pictures? and How does the unrolling work? over here.

I donā€™t understand this stuff well enough to want to form or state any confident opinions, but maybe itā€™s a good starting point for someone elseā€™s further research into how accurate these methods are.

2 Likes

Hi @Adutiya, the original article doesnā€™t go into the AI very much but if you read the other links that have been posted by myself in the UPDATE and by @Viktoria in her comment you can get more info about the AIā€™s being developed for this.

You can see transformer architectures (what ChatGPT is made out of) as well as ResNet architectures (which OpenAIā€™s CLIP basically is) given so you can see that real AI (as opposed the fuzzy fake kind Charles was referring to :joy:) is actually being used to piece together these scrolls :pray:

Thanks for this @Viktoria ! In that link you can find a reference to this paper: [2102.05095] Is Space-Time Attention All You Need for Video Understanding?

Which goes into how they are using a transformer architecture as part of the ā€œAIā€ in this work which is exactly the architecture that ChatGPT from OpenAI is made out of. Iā€™m pretty sure that ChatGPT is considered ā€œreal AIā€ for purposes of this forum as a lot of ire has been directed against the transformer architecture or the technologies that have come out of it. It is what powers nearly all the tech that is currently in the news re: AI.

From the paper:

We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named ā€œTimeSformer,ā€ adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches.

For reference, this is the paper that jumpstarted our current revolution in AI: [1706.03762] Attention Is All You Need

As you can see in the author link the future executives of the worldā€™s big foundational model makers: OpenAI, Anthropic, Googleā€™s DeepMind, etc.

:pray:

Yeah, my mistake. I thought I was on another topic and didnā€™t read above.

1 Like

Thatā€™s kinda cool - I named my daughter Tyrian - which is the shade of Purple once only worn by royalty (due to the expense of harvesting the sea snails used for it) and the dye for it was first made in Ancient Greece.

Iā€™d love to know if the actual word deciphered is my daughters name. :smiley:

ā€¦ and No, it had nothing to do with ā€˜Game of Thronesā€™ ( I get asked that all the time lol) I hadnā€™t even watched it. It was a nod to the highest academically credentialed Spiritualist on the planet who was the chairman of the education committee and first ever ā€˜fellowā€™ of the SNU UK (Spiritualist National Union), member of Mensa, wrote the courses Spiritualist world wide study for their credentials, ā€¦ and who took me under her wing and spent years educating me to understand certain little known weird abilities. Her name was Violet ā€¦ but I chose a different shade of purple :blush:

Letā€™s define our terms:

ChatGPT is a Large Language Model, a form of generative ā€œAIā€. It is trained on as much of the internet as possible.

Optical Character Recognition (OCR) recognises characters. As far as I know it is trained on characters, much like the CAPTCHA of old.

The term ā€œmachine learningā€ is very broad and would include things as simple as calculating the MSE of a data set:

For the burnt scrolls above, I believe PyTorch was mentioned? PyTorch has a wide range of applications. If you had enough burnt scrolls at hand and knew their actual contents you could theoretically train a model in this way. I just made that up of course, Iā€™m not sure how the model above was trained.

Yes, I see that now. It sounds like they are using machine-learning tool to automate the OCR process, like astronomers sorting images of galaxies with machine-learning software. Which is cool; it saves everyone a lot of time if it works well enough.

I think I was initially reacting to the claim that it was ā€œthe first timeā€ it had been done, and I remembered this being in the news years ago. I guess what they meant was it was the first time that particular scroll had been read with a machine-learning tool. Okay, then.

The Smithsonian has a good article that goes into detail about how this process was invented and the science behind it. Itā€™s a great example of thinking ā€œoutside of the boxā€ to solve a problem that seems impossible using existing technologies in new ways:

Buried by the Ash of Vesuvius, These Scrolls Are Being Read for the First Time in Millennia

It makes you wonder what other seemingly impossible things are possible if someone were to realize how to reapply existing technology. That creative spark that sets the wheel in motion is a mysterious thing; sometimes itā€™s a response to some happenstance that makes a person think, ā€œWait, what if I did this ā€¦ā€ and one thing leads to another.

1 Like

And since itā€™s such a small world (as they say), this young fellow featured in the video ā€“ Mr. Farritor ā€“ was a SpaceX intern last year. He is now one of Mr. Muskā€™s carefully selected ā€œyoung engineersā€ who is deciding how to revolutionize the US federal government with AI. This includes deciding how many current employees can be tossed to the street.

Last year, in a University of Nebraska press release video, he talked about calling his mom when he heard the news that heā€™d been selected as a winner for decoding one of the words on the scroll with AI.

Any 22-year-old would want to call their mom after achieving that. But I doubt she understands what heā€™s being tasked with now.

1 Like

Yes, I actually became aware of his work decoding the Vesuvius scrolls by first reading of his association with DOGE. I sincerely hope he returns to work on the scrolls asap. :pray:

1 Like