Search improvements

Feedback
11

I would imagine just as @Polarbear remarks above that one of the most requested features for search would be filtering out the nikayas for a search. I don’t think it has to be super complicated either. I would hazard a guess that the majority of searches that would want some kind of filtering would be wanting it by nikaya, and probably limited to the pali nikayas.

SN ▢ MN ▣ DN ▢ AN ▢ KN ▢
[ Checkboxes for SN, MN, DN, AN, KN. ]

Just those five I would think to be enough to cover most. Then more advanced searches might have to extend into a larger panel or be using search operators.

Just a guess! Maybe we could take a vote on most requested search features?

1 Like
12

Indeed. Although not everything is covered in the UI. Also, Google has thousands of engineers!

Well, isn’t that much better than a cold, heartless explainer!

That’s probably the way to go: put the most common requests in a UI and provide others in a back end. We can make a list of search operators.

You can if you want, but for now, we’re listening and starting work. A lot of it just depends on what is possible/doable/easy from an engineering point of view.

13

Interesting. I had not considered that mutually supportive continual blind discovery might actually be an architectural criterion.

I also do not think of Wikipedia as a cold, heartless explainer. Wikipedia rests on the hearts and minds of all the contributors and editors. A SuttaCentral wiki with a topic on Search would perhaps suit?

1 Like
14

I was being kinda facetious, but in fact, that was one of the reasons for setting up this forum. Documentation for software is a nightmare: It’s complex, few people read it, even if they do read them they often misunderstand it, the software changes while the documentation doesn’t, and on it goes.

Far better to make things that simply work, and if explanation is needed, give it in context.

You can make a wiki here if you like. But I would wait, it will be changing soon!

2 Likes
15

As far as search goes, I think they really nailed it. Their search interface is the best example of Alan Kay’s line: “Simple things should be simple, complex things should be possible.”

A basic input box for quick, simple searches and other ways to allow complex queries. But yes, they have armies of engineers… :slight_smile:

16

[low priority or user error]
International keyboard cannot be used to enter quoted search string:

8382 results for ¨is the root of suffering¨

Fix is to type spacebar immediately after typing double quotes
Workaround is to revert to US keyboard for typing quotes.
Theoretically possible to map %C2%A8 to double quote, but that might be too clever.

Search term for US international keyboard:
https://discourse.suttacentral.net/search/query?term=%C2%A8root%20of%20suffering%C2%A8&include_blurbs=true&_=1533317593504

Search term for US keyboard:
query?term=%22root%20of%20suffering%22&include_blurbs=true&_=1533317593501

The %C2%A8 is the UTF-8 for the diaerisis.

Resolution: added FAQ to Search page on wiki

1 Like
17

Thanks for this, we can grow the wiki over time!

1 Like
18

Sorry if this isn’t appropriate for this thread, but when I put Dhammacakkappavattana in the search, it doesn’t return (believe it or not) the Dhammacakkappavattana Sutta. Under filter I have selected all.

Any ideas? Am I doing something wrong? It is especially sad because the Dhammacakkappavattana Sutta is notoriously difficult for people to find. It’s tucked away in the very back of the SN. There isn’t even a way to find it in the indexes in the Wisdom edition.

00
00689×1080 275 KB

2 Likes
19

Have you tried Dhammacakkappavattanasutta?

I will update the wiki

1 Like
20

Well, that takes you to the Pali. Which can lead to the English. But it seems odd that if I cut the pali name from the English card the search doens’t take me back to it. (Obviously this isn’t what someone would do, I’m just saying the even the exact text dosen’t even return the result)

And it isn’t limited to this sutta. It seems that searching for the pail name doesn’t work. This seems like a big problem.

2 Likes
21

Yes it’s a bit clunky. And you need to type in the short id form to get to the English version. The wiki should provide implementation guidance since we’d like to minimize the need for such documentation.

1 Like
22

I’d say more like non-functioning. Seems like being able to search for the names of a sutta should be a basic feature. But maybe I’m the only one who would do that.

Any way, big thank you to all who are working on this.

Personally I really liked the search feature in the Digital Pali Reader.

1 Like
23

It is indeed! This is on the to-do list.

6 Likes
24

Great! This was my main complaint I was going to bring up, really strange that the most popular words like “Satipatthana” don’t have the corresponding suttas as the first result.

FWIW I think to improve the filtering setup, you have to make a major change: In the results, show the search field with the previous search text filled out, with a button to resubmit.

This is a standard practice in Google and IME most major sites with good search. It lets the user see what they typed and consider changing it if they don’t get the results they want.

Right now you have to go back up, click the icon in the header, and re-type your search. IMHO that’s a really confusing path, and when you don’t get the results you want it’s extra-discouraging to find that your search isn’t in the field and has to be copy-pasted.

Having this field at the top of results also gives you a perfect place to add documentation and/or filtering tools.

You can even replace the current plaintext output of the search term with the field version, just make it big and pretty but still obviously a field.

In terms of the nikaya filtering, I’ll +1 it as a feature.

In terms of the nikaya filtering, it could be some checkboxes (if any one of them is ticked, then only results from ticked ones are returned).

If that’s too clunky, another way would be to have text indicating how to limit a search using text query strings, which functions as guide, but then ALSO, if you click on the examples, they get inserted in the search form directly above.

So it says something like:

Filter by nikaya sn,mn,dn,an,kn:

And clicking on e.g. “sn” adds sn: to the start of the search query.

Maybe that’s too clever for it’s own good, but it would be nice to find a way that is a hybrid of “tell us what to do” and “help us do it”.

Also, it’s pretty clear that having some detailed instructions about filtering and tips for searching right on that page, but hidden behind a “more search tips” kind of .toggle button would really help.

Thanks for wanting to review this!

2 Likes
25

You’re absolutely right, this is a UI failure, and I have also noticed it. I have put it on the 2do list.

The filtering suggestion is also a good one, pretty much how I’d like to do it.

3 Likes
26

Please note that ranking search results in any way will affect which suttas will be seen the most. Anything on the first page of Google results will have an extreme boost in traffic, while the sites Google finds less relevant will experience less traffic. Modern search engines are hybrids that also use collaborative filtering techniques to get better results: instead of just looking at the contents of a file, how users interact with that certain file is even more important. The search results we receive are also filtered by the context we unknowingly provide with our search and access history (so if I search for “bus” I will most likely get results from my country on public transportation, and since I’m into IT I will also get hits about the “bus” in computer architecture). The problem with this, that it requires user data to work and users are mostly unaware that they are giving their data away during these interactions.

So in order to create an efficient search engine, first there should be a decision on how the automatic ranking should work and whether or not there is any user data that is offered for this service.

4 Likes
27

And a further consideration is that we have a didactic purpose: we are not neutral in regards to the information we serve. Primarily that means we want to highlight early texts rather than later. But it also means that we would like to broaden users’ horizons from the half-dozen suttas that are, for most, the be-all-and-end-all of of the discourses. If someone wants to find the Metta Sutta or the Dhammacakka Sutta, they should be able to. But if they want a teaching on metta or the four noble truths, they should have more choices than these.

6 Likes
28

It is also common practice to get information from outside sources to generate context (for instance Spotify’s bots -among many other things- also crawl lyrics which they use to find textual similarities for songs you enjoy and improve their models). Crawling the discussions and explanations on Suttas to improve search results could be useful - however, it also means that whatever ideas certain people had will be a basis on how users will reach these Suttas. If people’s understandings were wrong, the search results will also be off.

And yes, to broaden users’ horizons is also a challenge, as their preference biases will affect results as well. I assume that the popularity of Suttas also follow the Pareto distribution, meaning that about 20% of the Suttas will be quoted 80% of the time.

The evaluation of the results is also tricky, as it requires people with much knowledge in the domain so they can tell if the returned quotes are the ones that were indeed relevant. However, this also skews the ranking towards the preferences of the people who take part in the evaluation process.

So this is pretty much like an anti-search engine task, where the usual ranking algorithms would be partly counterproductive.

3 Likes
29

In my searches, my own preferences haven’t been relevant–engineering and rock climbing are way too specific as attributes and only indirectly related to the suttas. Instead, I rely on the diversity of translation. For example, I was led to MN1 via Thanissaro Bhikkhu, who used “delight is the root of suffering”. We also have “relishing is the root of suffering” via Bhante Sujato. The diversity of translation itself provides guidance in search.

This very diversity of translation is one of the amazing things about SuttaCentral–we can find the same suttas via many paths. I still find it remarkable that “root of suffering” comes up only about 13 times in SC search results. That’s much better than any Google search I have conducted. SuttaCentral will also support more and more languages, each with their own cultural web of meaning. When I think in German, my perspectives shift. When I think in Spanish, my perspectives shift. Each of these languages will have their own translators, and I can see that we may in the future be able to use these alternate translations to automatically inform searches. Indeed, it is interesting to note that SC returns three (3) results for “Wurzel des Leidens” specifically, we have “the will is the root of suffering”. Now isn’t that a remarkable new spin on old suttas?

Denn der Wille ist die Wurzel des Leidens

3 Likes
30

I just realized it’s possible to build an automatic synonym dictionary to improve search results, by comparing multiple translations of the same Suttas. So “delight” could return Suttas with only the word “relishing” in them.

5 Likes