Sutta finder tool - vector search to find suttas with natural language

The Watercooler
1

Hello everyone. I want to share a new search tool I’ve been developing, which I hope will be a helpful resource. It allows you to find suttas by context using natural language. For example, if you can’t remember a sutta title but recall details like “The Buddha was speaking with ‘x’ person about ‘y’ topic,” you can still locate it easily. Or find sutta about a specific topic.

Basically, It uses vector embedding to map semantic “meaning” to each sutta, then your search query gets embedded to find a match.

Some of examples of search types that work well are things like:

  • “What is the sutta where the Buddha talks about de difference between a monk that lives in the forest and a monk that lives in a village?”
  • “The Buddha talks to King Pasenadi”
  • “The Buddha compares right effort to tuning a lute”

It’s still not perfect, but I plan to further break up the suttas into chunks for embedding so different themes don’t lost in longer suttas.

I’d like to thank Venerables Khemaratana and Khemarato who have been extremely helpful with feedback and suggestions so far. Any additional feedback is appreciated.

You can find a more detailed explanation of how it all works here, along with more tips for what types of search work well and what don’t: https://abuddhistview.com/posts/sutta-vector-search

And the actual search tool here: https://abuddhistview.com/suttas/sutta-search

9 Likes
2

Thanks. I tried a few things that I have had difficulty finding previously and it seems to work well!

2 Likes
3

Sounds like a clever and useful use of AI. :slight_smile: May it benefit the whole community!

Is there an API access right now or in the works?

1 Like
4

But is it AI? I guess it depends on the definition.

1 Like
5

It literally uses Voyage AI for semantic context. And the project doesn’t use Bhante Sujato’s suttas presumably because he’s against AI of any sort.

I work with the “Neural Network = AI” definition, which is what most technical people I’ve seen use the definition.

2 Likes
6

Thanks! I hadn’t thought of creating a public api, but that’s a great idea. I’ll add it to my list of things to look into.

7

I haven’t checked it fully yet, but this sort of tool is immensely powerful, and so presumably it could be tuned for things like:

“Index of all the suttas speaking about Jhana”

With/without parameters like:

  • also mentioning Arupas
  • not mentioning Arupas

Etc. And for that kind of a broad application, having a public API would be neat.

Edit: Also, Sister Medhini page seems to be 404. Not sure if it’s just not uploaded at the moment. :slight_smile:

https://abuddhistview.com/posts/suttas/translators/sister-medhini

8

Thanks for letting me know, I’ll get that updated. It should be: https://abuddhistview.com/suttas/translators/sister-medhini

I agree though. I was also thinking about adding different filters or things like required keywords to make it easier to find specific phrases. I think there are a lot different directions it could go. My goal is to make it as widely applicable as possible.

2 Likes
9

When browsing the suttas, I’ve realised that link here:

goes to:

https://suttacentral.net/api/suttas/sn1.61/bodhi

Which is broken. :slight_smile: This seems consistent with Bhikkhu Bodhi texts. I think it’s because it’s linking to API call instead of the actual live webpage.

1 Like
10

That’s a good call out. You’re right that I did intend to use show the api url since it is technically the source, but I see how that’s pretty useless in hindsite :smile:. Maybe I’ll just remove the origin urls from the individual sutta pages altogether since I’m giving credit for the sources elsewhere on the site now as well.

1 Like
11

I mean, it would be trivial to convert the link going to the live website? I mean sometimes it’s handy to have a straight link to the SC, to look for parallels, etc. :slight_smile:

12

Good point! I just updated it to use the correct url

1 Like