Generating examples for use in Voice and other applications with Georg’s script

Fascinating! Yes, “term frequency” is indeed closely related to “match density”:

term frequency adjusted for document length: tf(t,d) = ft,d ÷ (number of words in d)

The one significant difference with Wikipedia is the use of segments as a basis vs. words. Segments have semantic relevance, so computing by number of matched segments (v.s. number of matched phrases) is a useful refinement. I think the Python makes use of this as well since I see “line” used as a variable name and segments are stored by line in the JSON files.

Inverse document frequency is also interesting. Scv-bilara does not currently use IDF directly, but it does make sense. I look forward to your findings on how best to use IDF. Perhaps an IDF metric for the phrase (vs. the words in the phrase) can be used as filter to find phrases that occur in more than 1 but less than ~1% of suttas.

Oops. Yes. I just started using the -os parameter and mistakenly divided by 2 as I used to do for the -op parameter. That puts “to the higher mind” in a rather plump category. I tried again to find a better search term but could not find a better one. For example “belonging to the higher mind” returns a singleton, but that singleton is the first of the 44 results. Because of this, I’m inclined to think that the 44 suttas are acceptable in this case because the top five all seem relevant to the question “what does the Buddha mean by ‘the higher mind’?”

1 Like

The term ‘manifestation’, pātubhāva, is only found in 21 suttas.

By the way if I use the big search at the top of SC for the word, pātubhāva or pātubhāvo, I don’t get any suttas only the definition of the word. Is this a bug or a missing feature, or am I doing something wrong?

This seems to be an important teaching. The manifestation of anything material is suffering: ‘… and manifestation of form is the arising of suffering’ (sn26.10, and sn22.30). I just listened to a talk by Ven. Punnadhammo, which led me to search for ‘manifestation’. It is linked with the teaching of dependent origination, with the five khandhas, and with the first noble truth. In mn28, the Buddha explains the manifestation or non-manifestation of consciousness.

This formula is used in 8 suttas: “the arising, continuation, rebirth, and manifestation of …” [(1, ‘sn35.22’), (1, ‘sn35.21’), (1, ‘sn26.9’), (1, ‘sn26.2’), (1, ‘sn26.10’), (1, ‘sn26.1’), (1, ‘sn22.30’), (1, ‘sn14.36’)]

About weighting the matches: The inverse document frequency (the IDF) of ‘manifestation’ is 5.2. For comparison, the IDF of ‘suffering’ is only 1.6, meaning the latter appears much more often in the suttas.

[‘manifestation’]
number of suttas: 21
[(6, ‘mn28’), (4, ‘sn14.36’), (4, ‘an3.3’), (3, ‘sn35.22’), (3, ‘sn35.21’), (3, ‘sn26.2’), (3, ‘sn26.10’), (3, ‘sn26.1’), (3, ‘sn22.30’), (2, ‘sn26.9’), (2, ‘mn129’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’), (1, ‘sn12.2’), (1, ‘mn9’), (1, ‘mn141’), (1, ‘dn22’)]
[‘manifestation of’]
number of suttas: 19
[(6, ‘mn28’), (4, ‘sn14.36’), (3, ‘sn35.22’), (3, ‘sn35.21’), (3, ‘sn26.2’), (3, ‘sn26.10’), (3, ‘sn26.1’), (3, ‘sn22.30’), (2, ‘sn26.9’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’), (1, ‘sn12.2’), (1, ‘mn9’), (1, ‘mn141’), (1, ‘dn22’)]
[‘manifestation of the’]
number of suttas: 9
[(6, ‘mn28’), (4, ‘sn14.36’), (2, ‘sn35.21’), (2, ‘sn26.1’), (1, ‘sn26.9’), (1, ‘sn12.2’), (1, ‘mn9’), (1, ‘mn141’), (1, ‘dn22’)]
[‘manifestation of the corresponding’]
number of suttas: 1
[(6, ‘mn28’)]
[‘manifestation of the corresponding type’]
number of suttas: 1
[(6, ‘mn28’)]
[‘manifestation of the corresponding type of’]
number of suttas: 1
[(6, ‘mn28’)]
[‘manifestation of the corresponding type of consciousness’]
number of suttas: 1
[(6, ‘mn28’)]
[‘manifestation of the aggregates’]
number of suttas: 3
[(1, ‘sn12.2’), (1, ‘mn9’), (1, ‘mn141’)]
[‘manifestation of the ear’]
number of suttas: 4
[(1, ‘sn35.21’), (1, ‘sn26.9’), (1, ‘sn26.1’), (1, ‘sn14.36’)]
[‘manifestation of the earth’]
number of suttas: 2
[(1, ‘sn26.9’), (1, ‘sn14.36’)]
[‘manifestation of the earth element’]
number of suttas: 2
[(1, ‘sn26.9’), (1, ‘sn14.36’)]
[‘manifestation of the earth element is’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the earth element is the’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the earth element is the arising’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the earth element is the arising of’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the earth element is the arising of suffering’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the eye’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the eye is’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the eye is the’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the eye is the arising’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the eye is the arising of’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the eye is the arising of suffering’]
number of suttas: 2
[(1, ‘sn35.21’), (1, ‘sn26.1’)]
[‘manifestation of the air’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element is’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element is the’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element is the arising’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element is the arising of’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the air element is the arising of suffering’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the fire’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the fire element’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the sets’]
number of suttas: 1
[(1, ‘dn22’)]
[‘manifestation of the sets of’]
number of suttas: 1
[(1, ‘dn22’)]
[‘manifestation of the sets of phenomena’]
number of suttas: 1
[(1, ‘dn22’)]
[‘manifestation of the water’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of the water element’]
number of suttas: 1
[(1, ‘sn14.36’)]
[‘manifestation of old’]
number of suttas: 14
[(2, ‘sn35.22’), (2, ‘sn35.21’), (2, ‘sn26.2’), (2, ‘sn26.10’), (2, ‘sn26.1’), (2, ‘sn22.30’), (2, ‘sn14.36’), (1, ‘sn26.9’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’)]
[‘manifestation of old age’]
number of suttas: 14
[(2, ‘sn35.22’), (2, ‘sn35.21’), (2, ‘sn26.2’), (2, ‘sn26.10’), (2, ‘sn26.1’), (2, ‘sn22.30’), (2, ‘sn14.36’), (1, ‘sn26.9’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’)]
[‘manifestation of old age and’]
number of suttas: 14
[(2, ‘sn35.22’), (2, ‘sn35.21’), (2, ‘sn26.2’), (2, ‘sn26.10’), (2, ‘sn26.1’), (2, ‘sn22.30’), (2, ‘sn14.36’), (1, ‘sn26.9’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’)]
[‘manifestation of old age and death’]
number of suttas: 14
[(2, ‘sn35.22’), (2, ‘sn35.21’), (2, ‘sn26.2’), (2, ‘sn26.10’), (2, ‘sn26.1’), (2, ‘sn22.30’), (2, ‘sn14.36’), (1, ‘sn26.9’), (1, ‘sn26.8’), (1, ‘sn26.7’), (1, ‘sn26.6’), (1, ‘sn26.5’), (1, ‘sn26.4’), (1, ‘sn26.3’)]
[‘manifestation of feeling’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form is’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form is the’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form is the arising’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form is the arising of’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of form is the arising of suffering’]
number of suttas: 2
[(1, ‘sn26.10’), (1, ‘sn22.30’)]
[‘manifestation of sights’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sights is’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sights is the’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sights is the arising’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sights is the arising of’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sights is the arising of suffering’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]
[‘manifestation of sounds’]
number of suttas: 2
[(1, ‘sn35.22’), (1, ‘sn26.2’)]

1 Like

Thank you! I am adding “manifestation” | “Manifestation” to the examples.

In the list we also find AN 3.3 where “manifestation” occurs as a synonym along with “characterisitcs” and “signs”; it doesn’t render pātubhāva here (same in MN 129). These are the only instances that have “manifestations” in the plural.

AN3.3:1.1: “Tīṇimāni, bhikkhave, bālassa bālalakkhaṇāni bālanimittāni bālāpadānāni.
“There are these three characteristics, signs, and manifestations of a fool.

(This one will not be caught by the German term.)

It’s so great to have another examples enthusiast! :tada: :confetti_ball: :kite:

3 Likes

Aha! I should have written about examples on this thread! I will do that from next time. :smile:

3 Likes

Let’s just note here that we added

  • guard the sense doors | bewacht.* die Tore der Sinne
  • guaranteed | gesichert

and their Japanese counterparts … which I don’t know.

3 Likes

Today’s Q&A with my teacher, Ajahn Sona, was on right intention (samma sankappa). One of its three aspects is the intention of renunciation.

This is often discussed as ‘letting go’ (patinissagga).
Apologies if this is already on your lists.

My search found:
[‘letting go of all attachments’]
number of suttas: 21
[(5, ‘an9.36’), (2, ‘mn22’), (2, ‘an11.7’), (2, ‘an10.60’), (1, ‘sn6.1’), (1, ‘sn48.50’), (1, ‘sn22.90’), (1, ‘mn85’), (1, ‘mn64’), (1, ‘mn26’), (1, ‘mn140’), (1, ‘dn14’), (1, ‘an5.140’), (1, ‘an4.114’), (1, ‘an3.32’), (1, ‘an11.8’), (1, ‘an11.21’), (1, ‘an11.20’), (1, ‘an11.19’), (1, ‘an11.18’), (1, ‘an10.6’)]

[‘letting go of attachments’]
number of suttas: 7
[(1, ‘sn46.17’), (1, ‘sn45.34’), (1, ‘dhp89’), (1, ‘an10.170’), (1, ‘an10.169’), (1, ‘an10.118’), (1, ‘an10.117’)]

[‘letting go of negligence’]
number of suttas: 10
[(1, ‘an9.113-432’), (1, ‘an8.148-627’), (1, ‘an7.645-1124’), (1, ‘an6.170-649’), (1, ‘an5.308-1152’), (1, ‘an3.183-352’), (1, ‘an2.320-479’), (1, ‘an11.992-1151’), (1, ‘an11.983-991’), (1, ‘an10.267-746’)]

[‘letting go of greed’]
number of suttas: 6
[(1, ‘an9.95-112’), (1, ‘an7.618-644’), (1, ‘an6.143-169’), (1, ‘an5.308-1152’), (1, ‘an4.277-303’), (1, ‘an2.310-319’)]

Then perhaps also ‘identity view’, the thing entangled with greed:

[‘identity view’]
number of suttas: 15
[(11, ‘mn64’), (6, ‘an10.76’), (5, ‘sn41.3’), (5, ‘sn22.155’), (4, ‘sn35.166’), (4, ‘sn22.82’), (4, ‘mn44’), (4, ‘mn109’), (2, ‘dn33’), (1, ‘thig6.7’), (1, ‘sn2.16’), (1, ‘sn1.21’), (1, ‘mn2’), (1, ‘kp6’), (1, ‘an3.94’)]

[‘identity view comes about / does not come about’]
number of suttas: 4
[(1, ‘sn41.3’), (1, ‘sn22.82’), (1, ‘mn44’), (1, ‘mn109’)]

Then this is perhaps a singleton:
[‘Concepts of identity that emerge from the proliferation of perceptions are the source of thoughts’]
number of suttas: 1
[(1, ‘dn21’)]
https://suttacentral.net/dn21/en/sujato#2.2.23

which links identity view with the term ‘proliferation of perceptions’.

and there is a parallel with mn18:
https://suttacentral.net/mn18/en/sujato#8.1

Its translation was discussed by Ajahn Sujato here:

4 Likes

Thank you Georg for the new findings!

The stock phrase that has this term starts with “stilling of all activities” which is already in the examples.

Is interesting because all occurrences are in verse, and invariably it is about the awakening factors (7 results).

All occurrences are in abbreviation series—I am not sure how interesting these are.

The same as above.

That is definitely an important concept that needs to be given up. I find however 23 results, whereas you find only 15.

The main Suttas with identity view are not among the 4 results of this, so I think this is perhaps a bit narrow.

“Concepts of identity” is already in the examples (3 results).

Summary:

Adding

  • letting go of attachments | (no German translation yet)
  • identity view | Glaube an eine Persönlichkeit

Added from my latest translations

for German (have already English counterparts):

  • bloßem Vertrauen
  • Harfe

for German and English:

  • tuned too tight | zu straff gespannt (returns only 1 sutta, with the simile of tuning the harp)
  • ending of attachments | Auflösen der Bindungen (8 results)
  • classes of rebirth | Wiedergeburtsklassen (7 results)
  • retreat for three months | drei Monate in Klausur (3 results)
3 Likes

Thank you so much. About ‘identity view’:

This could be a bug on my end. Even after an update of bilara I still only get 15 suttas. Would you mind, please, to post the list of your 23 result so I could investigate?

I have a few more suggestions:

[‘warm-hearted quality’] or qualities
number of suttas: 7
[(6, ‘mn104’), (6, ‘dn33’), (5, ‘an6.11’), (4, ‘an6.12’), (2, ‘mn48’), (2, ‘dn34’), (2, ‘an10.50’)]

The six warm-hearted qualities that build the foundation for communal harmony.

[‘experiencing bliss’]
number of suttas: 11
[(4, ‘mn94’), (4, ‘mn51’), (3, ‘mn60’), (3, ‘an4.198’), (2, ‘mn118’), (1, ‘sn54.10’), (1, ‘sn54.1’), (1, ‘mn62’), (1, ‘dn33’), (1, ‘an3.66’), (1, ‘an10.60’)]

[‘arising and passing away’]
number of suttas: 35
[(2, ‘dn34’), (2, ‘dn33’), (2, ‘an5.136’), (2, ‘an5.135’), (1, ‘ud4.1’), (1, ‘sn55.43’), (1, ‘sn55.37’), (1, ‘sn55.33’), (1, ‘sn48.9’), (1, ‘sn48.11’), (1, ‘sn48.10’), (1, ‘mn90’), (1, ‘mn85’), (1, ‘mn53’), (1, ‘an9.3’), (1, ‘an9.1’), (1, ‘an8.76’), (1, ‘an8.55’), (1, ‘an8.54’), (1, ‘an8.50’), (1, ‘an8.49’), (1, ‘an8.30’), (1, ‘an7.67’), (1, ‘an7.6’), (1, ‘an7.4’), (1, ‘an5.53’), (1, ‘an5.47’), (1, ‘an5.2’), (1, ‘an5.14’), (1, ‘an5.134’), (1, ‘an5.122’), (1, ‘an10.50’), (1, ‘an10.18’), (1, ‘an10.17’), (1, ‘an10.11’)]

More complete, it is “the wisdom of arising and passing away”.
There is only one sutta, an5.122, which has instead:
“to understand the arising and passing away of phenomena”

3 Likes

These are the results:

23 docs:
an3.94, an6.89, an6.90, an6.91, an7.85, an7.92, an9.67, an10.13, an10.76, dn33, kp6, mn2, mn44, mn64, mn109, sn1.21, sn2.16, sn22.82, sn22.155, sn35.166, sn41.3, sn45.179, thig6.7


We have already “in public and in private” which occurs in almost all of the “warmhearted” suttas—except for AN 10.50 which has a different set of qualities altogether. Adding

  • warm-hearted quality | warmherzige Eigenschaft

This one occurs in two contexts which have the same term in English, whereas for German I need two separate ones:

  • experiencing bliss | kühl geworden, voller Wonne | beim Einatmen Wonne

Adding

  • arising and passing away | vom Entstehen und Vergehen

    Just “Entstehen und Vergehen” would be too broad and catch a lot more things; but this way it does not include AN5.122. So I am still adding

  • passing away of phenomena | Vergehen der Erscheinungen
    (which catches the same as “arising and passing away of phenomena”, but is simply shorter).

Many thanks, Georg! :pray:

2 Likes

@Georg, sorry for the delay in response. I’m having trouble with notification settings. :see_no_evil:

Thank you for the new adventure with “manifestation”. I had not bumped into that line of study before and it’s opened up new horizons! In particular, one of those suttas is MN28, which I had not read before. Indeed, after reading MN28, I found yet another example. I added “equanimity based on” to the examples. :man_cartwheeling:

I only find SC search useful for definitions. That’s actually one of the reasons why Voice has its own search. I could not find find anything useful with SC search. Blake is working on a new search algorithm for SC that uses ArangoDB. Interestingly, it returns the same results as Voice and scv-bilara. I hope he releases that new search soon so that all may use SC with greater ease.

Quite so. And the IDF for “root” is also probably quite low. However, the IDF for “root of suffering” is probably quite high. That is the magic of example phrases.

Awesome! That would explain why I saw it and had no recollection of adding it. :laughing:

Unfamiliar with Ajahn Sona, I looked him on YouTube and found his 2008 talk on “20 years as a monk”. It was hilarious to hear his insight about backpacking, “this is all one needs.”

My goodness that’s long for my head. :laughing:

Perception is a grasping aggregate, so proliferation follows perceptions like a plague.

:honey_pot:

MN18:18.1: When there is no eye, no sights, and no eye consciousness, it’s not possible to point out what’s known as ‘contact’.

There is emptiness.

2 Likes

I just added “simile of the saw” to the examples. This finds three suttas and is notable because it is an example of a use of an example in the suttas. :rofl:

See? Examples are important!

MN28:9.5: But the Buddha has said in the Simile of the Saw:

4 Likes

Thank you very much, Sabbamitta.

I found the issue. My search was still case sensitive. I corrected the error and now we get the same number of results and identical suttas.

3 Likes

New examples added from new translations ("…" means that the English example was already there):

  • … | Metteyya
  • … | beide.* Enden
  • … | steckt nicht in der Mitte
  • ox fond of crops | Ochse, der gerne die Saat frisst
  • the sweetest of the sweet | Sanfteste der Sanften
  • that’s how the Buddha declared it | so hat der Buddha es erklärt
3 Likes

As a side note to all who add examples: New examples are no longer added here, but now they go here.

2 Likes

For the curious, the ebt-site/ebt-data repository mentioned by Anagarika Sabbamitta is aiutomatically generated from the published branch of bilara-data. Ebt-data is about 1/5 the size of bilara-data, which makes it faster and easier for programs to use. It is also a good place for examples, which should not clutter up bilara-data.

2 Likes

I have added some examples from MN 140. I haven’t translated that Sutta yet, which is why the German ones sometimes are “…”:

  • the interior earth element | das innere Erdelement
  • train only for peace | …
  • heat is generated | …
  • grasp at anything in the world | ergreift man nichts in der Welt
  • feel it detached | …
  • end of the body approaching | Ende des Körpers nahen
  • an oil lamp depended on oil | …
  • become cool right here | hier auf der Stelle kühl
  • letting go of all attachments | Loslassen aller Bindungen
  • stray cow took his life | …

“train only for peace” is a singleton. I added it because I find it particularly inspiring. The whole sentence reads:

Don’t neglect wisdom, preserve truth, foster generosity, and train only for peace.

“stray cow took his life” also occurs only in MN 140. I added it because I find it important to be reminded that accidents can happen any time—this is probably what was considered a traffic accident at the time. :oncoming_automobile: :cow:

4 Likes

I was indeed surprised to learn that cows can be lethal. And I was also surprised that this happened multiple times in the suttas. :open_mouth:

3 Likes

I only found that one time. Where are the other ones?

2 Likes

ud5.3:7.2: But soon after Suppabuddha had left, a cow with a baby calf charged at him and took his life.
ud1.10:10.2: But soon after the Buddha had left, a cow with a baby calf charged at Bāhiya and took his life.
MN140:35.2: But while he was wandering in search of a bowl and robes, a stray cow took his life.

The examples that work are:

  • cow life :laughing:
  • cow took life

Which is actually pretty hilarious.

2 Likes

That’s why I couldn’t find more than this one!

I found it more frequent the other way around (“kill a cow”):

MN 119:8.3 It’s as if a deft butcher or butcher’s apprentice were to kill a cow and sit down at the crossroads with the meat cut into portions.

MN 10:12.3 It’s as if a deft butcher or butcher’s apprentice were to kill a cow and sit down at the crossroads with the meat cut into portions.

MN 146:11.1 Suppose a deft butcher or their apprentice was to kill a cow and carve it with a sharp meat cleaver. Without damaging the flesh inside or the hide outside, …

DN 22:6.3 It’s as if a deft butcher or butcher’s apprentice were to kill a cow and sit down at the crossroads with the meat cut into portions.

:cry:

2 Likes