Having a trained LLM AI model for Pali canon would be great!

There are no doubts, AI has been active in many fields and having its own fans and enemies. Regardless of our agreement to give it a voice to express itself or not, there are still other ways to use AI such as just extracting existing info directly as they are with no change, which can be hugely beneficial .

There is this general view about AI that they hallucinate which is true but it is avoidable. They are often big models which are holding lots of info, so the hallucination can happen. But there are ways to avoid it if you ā€œtrain a small model for specific tasksā€œ.

There are thousands of Suttas and we may often have difficulty to find the ones which would match the situation we are encountering. It is a common difficulty most Buddhists have with suttas. A fine-tuned ragged AI model can give you exactly what you inquire . I hope someone who is into training models and a buddhist as well, sees this someday and does something. We’d be all thankful. :relieved_face:

2 Likes

This is pretty much what A Buddhist View is doing:

4 Likes

Just a heads up. Folks around here have some pretty strong views about AI for the suttas.

https://discourse.suttacentral.net/tag/stochastic-parrots

2 Likes

[quote=ā€œPasanna, post:3, topic:40913, full:trueā€]
Just a heads up. Folks around here have some pretty strong views about AI for the suttas.

Hmm…. I won’t say too much, but people often project their own fears for that which they don’t understand yet. It reminds me of people who used to oppose Cars when they were first invented. They were calling it artificial monsters that shouldn’t replace soulful horses…

There is so much AI hatred these days and this is a period of history that has risen and would pass away as well. It is quite ironic to me as well that we buddhists all talk about Anatta, yet some of us are strictly looking for a self in AI and when we don’t find it we get mad about it. No matter we want AI or not, the technology is happening. So better to train it and master it so it would be used in more accurate and beneficial way than something else

2 Likes

That’s what their page says ā€œ This is a hybrid search system that combines semantic vector search with traditional full-text search — it’s not an LLM wrapper that generates responsesā€

Just checked it and tried it now. It is not AI, but nevertheless looks great and its responses are reasonably good! Going to use it more often. :ok_hand:

Well, it is AI that it uses Gemini AI for the vector search, but the responses are limited to direct Sutta citations. :slight_smile:

Ah I see! Makes sense. The instructions they were giving in the link about how to ask questions are just the way you treat an AI.

Great work.:flexed_biceps:

1 Like

Maybe take a read of some of the articles I linked to before deciding what people understand.

5 Likes

I did read actually and i see too much phobia and enmity in what is said. Just to say, AI is in baby-stage yet and should not be fully relied on which we would agree on, does not need such degree of demonizing towards the technology the way it is done with those posts and specifically their clickbait titles.

But to each their own.

1 Like

10 posts were merged into an existing topic: AI-2: Machine translations of suttas are the wrong solution for the wrong problem

:laughing: I would call those people prescient. Artificial Monsters sounds about right.

According to the World Health Organization (WHO), road traffic injuries caused an estimated 1.35 million deaths worldwide in 2016. That is, one person is killed every 26 seconds on average.

List of countries by traffic-related death rate - Wikipedia

Wow! 1.35 million. I knew it was large, but that figure surprised me when I looked it up. There are many other problems with cars such as local pollution, both from exhaust emissions and noise of their engines and of course we also have the cars impact on climate change.

Some people can asses the impact of new technologies weighing up the benefits and dangers, and others can’t see the dangers even when those dangers are evident.

I’m definitely going to call cars Artificial Monsters from now on :smiley:

4 Likes

This line of thinking completely overlooks other details - such as, how many people have managed to find the medical needs met just because they were able to travel to a hospital faster? How has the society improved with the speed of knowledge traveling faster with cars?

Some people can assess the impact of new technologies weighing up the benefits and dangers, and others can’t see the benefits even when those benefits are evident. :wink:

5 Likes

:laughing: Well yes, I am definitely someone who thinks that all cars should still have a man with a red flag walking ahead of them for the safety of pedestrians. But yeah, the unbearable lightness of being and all that.

There isn’t a ā€˜control’ planet Earth where we successfully discouraged the use of cars, so we’ll never know the answer to your questions I guess. :frowning:

I do own a car and I have been using it recently to ferry cancer patients to hospital, so I’m definitely aware of this potential benefit. But I also think that caring is more important than curing, and I think that maybe this aspect of humanity has waned in the advent of the need for speed.

In addition local services have been moved further away in the rush for ’efficiency’, so for some (e.g. some poor people in rural areas) I would hazard a guess that their medical needs are being less well served.

I have benefited greatly from (fairly recently) moving to an area with relatively few people, few cars and a life lived at a slower, more human pace, with more time to address people’s needs and build relationships. But yes, you are right, there is no longer a doctor in the village, so to get even the most basic medical care requires transport.

That’s a loaded question. Has society improved? Dukkha in the EBTs seems pretty much the same as dukkha today. On the whole people seem as deluded today as they were in my youth (in the 1960s) and as deluded as they were in the suttas.

4 Likes

Which overlooks even more details like the enormous amount of damage roads and motor vehicles do to wildlife every single day … The mindless slaughter is incredible. I still remember the time I was driving at night and drove by a raccoon laying maimed in the road, not dead yet mind you …

As far as LLMs go, I believe what the OP was asking about has been done already, and it didn’t work very well. The usual sort of preposterous stuff, making up sutta references, and so forth. Narrowly trained LLMs I think are for much small data sets that the Pali canon, which is a huge corpus of text in one language, much less two or more.

The search engine linked to is an interesting use of the technology to allow a more natural way to input parameters, though. It’s a way to avoid the trouble of LLMs and get some use out of them.

3 Likes

Hi Anatta,

AI right now is pretty good at this, and a lot more. I asked the latest ChatGPT to analyze the points of contention between Ven Thanissaro and Ven Brahmali’s positions on jhana, and eventually it ended up cross referencing Pali sutta references against the Chinese Agamas and doing deep dives into the Chinese and Sanskrit phrasings and grammar, and refined its own position at every step.

Directly extracting passages as you said would not be a problem, even on a free tier of your favourite AI service. Even on a locally hosted LLM that is fairly well configured. It could be set up to provide a link to the original source on SuttaCentral, so that the user doesn’t have to take its word for it.

As with any technology that advances at such a rapid rate, people have ethical concerns and that is understandable. Some things however are objectively true and not a matter of opinion:

  • AI is happening, whether you like it or not. You can reject it wholesale or adapt it to your needs.
  • AI is improving at an astonishing rate and will only continue to do so.
  • There are over 2 million AI models on huggingface.co and they can’t all be horrible. And even if they are, you can train your own.

AI is a tool, albeit a fascinating one, and it is a poor craftsman who blames his tools. We use it regularly at work to bounce off ideas, never to take its output at face value and move on to the next task. And if anybody at work were to ever do that, they would have some sharp feedback pointed their way.

1 Like

Closed for moderation.

1 Like