We can narrow down the underlying language - because the evidence limits our possible alternatives to just one language
-
Consider one core/nucleus language – In my understanding the Buddha spoke only one mainstream language (the Pali canon does not evidence the claim that he or his interlocutors changed languages based on location or audience, or needed to translate - such an evidence of code-switching is not found in any co-eval Gandhari, Ashokan or Sanskrit text either, so paradoxical as it may seem, the evidence indicates that there was only one mainstream Indo-Aryan language spoken in the Buddha’s culture). Even if he spoke other fringe/minor/secondary languages, the core underlying language of the EBTs is unlikely to be one of those fringe/minor dialects. So the idea is to identify that core dialect, not all the other ones (even assuming others did exist).
-
It was necessarily an Indo-Aryan language i.e. a linguistic tradition that originates from Old-Indo-Aryan (Vedic and derivatives). This eliminates non-Indo-Aryan (i.e. traditions that lie outside of Old-Indo-Aryan origins) - such as independent prakritic linguistic or literary traditions that could have ostensibly existed/developed side-by-side with the Old-Indo-Aryan (Vedic/sanskrit) tradition. Assuming a parallel development of a largely-unattested Indo-Aryan linguistic tradition independent of Sanskrit and Vedic would require the reconstruction of a new language or language family - which to my knowledge no philologist has so far shown interest in. Besides Sanskrit and Vedic are capable of explaining the etymology of 99.5% of Pali vocabulary, and the rest are Dravidian/Munda/Greek/Iranic loanwords. So there is no need to invent a parallel origin theory for the underlying language of early buddhism.
-
It can only be a pre-existing language. This removes from our scope languages whose first attestations (or inferred existence from co-eval sources) post-date the time of the Buddha.
-
It cannot have been a language that was limited/unique to early-Buddhism or to the EBTs but was a broader language (wider than Buddhism) of the wider lay society. This removes Pali itself from the list - which is a language restricted to the Pali canon. The same can be said of Ardha magadhi of the Jain canon. The right word to use for these languages is - ‘canonical registers’ as these languages are limited to the canonical texts. The Buddha’s language was therefore not merely a ‘Buddhist’ language.
-
It would have been a language that is sufficiently close to the language of the EBTs. Here we don’t have a choice as (except the oldest forms of Vedic), most Sanskrit and early Prākrit is very close to the language of the EBTs, so that only removes the earliest forms of Vedic and later forms of prakrit from our scope.
-
That language must be capable of being, linguistically, an etymological source for the language(s) used by the EBTs. Without this, it would be impossible to meaningfully preserve the buddhavacana word by word faithfully in a derivative register such as Pali, BHS or Gandhari.
-
There would have to be a plausible reason why such a language could not be preserved in its original form in the EBTs (and had to be changed into the language of the EBTs)
-
There are other limitations (too numerous to list here - from the internal evidences present in the EBTs, and external evidences from co-eval texts outside Buddhism, such as the language of the Ashokan edicts, the co-eval Indic words surviving in dravidian, greek & persian texts from the time of Alexander onwards etc) that further limit the possibilities of this nucleus language of proto-canonical Buddhism.
One example of internal evidence from the Pali canon - there is in the Ud 5.6 a statement that Soṇa Kuṭikaṇṇa recited (abhaṇi) the aṭṭhakavagga “with svaras” (sarena) to the Buddha, and at the end of the recital with svaras (sara-bhañña-pariyosāne), the buddha lavished praises on him for his clear and correct enunciation. But what exactly are the svaras? I take them to be the vedic svaras (the tone accents) - he recited the verses with the tone accents. The accents (svaras) exist in classical sanskrit (where they are optional) as well as vedic (where they are compulsory), so what other language other than Old-Indo-Aryan (Classical Sanskrit or Vedic) could Soṇa Kuṭikaṇṇa have recited the aṭṭhakavagga to the Buddha in? The Vedic svaras (tone accents) are inherited from the Proto-Indo-European, see Proto-Indo-European accent - Wikipedia and Vedic accent - Wikipedia
“Evaṁ, bhante”ti kho āyasmā soṇo bhagavato paṭissutvā soḷasa aṭṭhakavaggikāni sabbāneva sarena abhaṇi. Atha kho bhagavā āyasmato soṇassa sarabhaññapariyosāne abbhanumodi: “sādhu sādhu, bhikkhu, suggahitāni te, bhikkhu, soḷasa aṭṭhakavaggikāni sumanasikatāni sūpadhāritāni, kalyāṇiyāsi vācāya samannāgato vissaṭṭhāya anelagaḷāya atthassa viññāpaniyā
So in summary - it was a mainstream Indo-Aryan language belonging to the same Indo-Aryan tradition as Vedic and Sanskrit, it was pre-existing at the time of the historical Buddha’s birth, a language not limited to Buddhist texts, but a language that early-Buddhists had inherited from their culture, a language temporally and linguistically close to the languages of the EBTs, a language that can possibly claim phonemic and morphological ancestry to the attested linguistic registers of the EBTs, a language in which writing was a novelty in the time of the Buddha, a language where there was an option for Soṇa Kuṭikaṇṇa to recite the aṭṭhakavagga with svara intonations, a language that Ashoka would have understood, a language that was spoken by Brahmins in many janapadas, and a language whose word-forms are potentially attested in Indo-Aryan and non-Indo-Aryan texts of that period.
I leave it to your imagination what language could have ticked so many boxes - but you probably see that most of these postulates above are commonsensical enough to be uncontroversial.
There is no Sanskrit form of prati (or prāti) that includes a retroflexed t. The retroflexion of the t is related to the disappearance of the preceding r - they happen together when the Sanskrit prāti is prakritized (or pali-fied) into pāṭi. Sometimes the retroflexion is skipped (because the writer fails to follow the above convention), resulting in a form such as pāti, but that would have been confusing to the reader. In the first 4 nikāyas, words starting with paṭi occur 4300 times, and words starting with pati occur only 379 times - showing that the retroflexed ṭ was clearly the norm. However in the words starting with pātimokkha, it is always spelt with a dental t rather than a retroflex ṭ - I don’t think it makes much of a difference either way.