Statistical analysis of Early Buddhist Texts

[josephzizys]

I am quite interested in this topic, discussed a bit here and in a few scattered threads like

Sutta segments sorted by frequency?

I do not think anybody has done it yet. However, It seems like a good approach.

and

R for Sutta Science

Anything is possible. You can use rpy2 to run R code inside of python code or reticulate to do the opposite. You can also just have the R package available as well as the data files available separately. I’d really just recommend doing what’s easiest for you. Speaking for myself - honestly I just see this as a good corpus upon which to do an open-ended exploration of available techniques. I’m abstractly fascinated by NPL but I’m just not intrinsically curious about the typical use-cases show…

recently there was

Average Word Length as indicator of Pali text age

The Pali language, as every other language, has evolved over time. One of the changes that the language has undergone through the centuries is that words have become increasingly complex and words have merged together to form longer words. We would therefore expect that a statistical analysis of average word-length could be an indication of relative ”age” of the texts in question. In this research I analyse the average word-length of each text and each book in the Pali canon to find trends in t…

Inspired by this I asked:

How many words are each of the first 4 Nikayas?

I am writing up some of my quantitative arguments, focusing on the first 4 Nikayas, can someone tell me how many words in Pali, not counting implied expansions, jsut what is actually there, each of the 4 N have? or a way to easily load a file and do a word count? there where some zip files or pdfs? shared here a while back I think??

and got no on topic replies, so I thought I would bump this thread, I am going to try and use some of the info gleaned from some of @sujato 's posts in this thread to use

to work out a couple of things that I would have thought would be of quite general interest but which I can’t find any definitive information about, namely;

1. how many letters long are each of the Nikayas? this would give some frame to questions of how often a given phrase appears in say, MN vs AN. Is AN much longer than SN? if so how much longer? I was surprised that this is not more easily available as info.

2. how many different words are in each of the Nikayas? i.e what is the vocabulary of each Nikaya/Vagga/whatever, this is of similar interest it seems to me to @Vimala 's word length research.

About 20 odd years ago I was a web developer, hacking together things in asp and coldfusion, working with sql etc, but that was a long time ago, more recently I got it in my head to try and host a local copy of suttacentral and see if I could implement the new search engine idea, well, just getting docker and the site to actually run as is took so much out of me that I have avoided anything technical ever since, but I have licked my wounds enough now and will try to get back to this thread with answers to my 2 questions and a description of how I did it.

Maybe if other people have pet projects, examples or other threads of interest they could post here and we could get some mutual support going?

Metta