Survey: What filters would you like for 🔍 search?

[Snowbird]

Filters help you narrow down the search results by adding extra criterion. There are currently quite a few already. For example you can already add in:dn to narrow it to only results from the Digha Nikaya. Or in:long to search all collections within Long (i.e. dn da da-ot). Or in:sutta to limit to everything in the Suttapitaka (i.e. all root languages but exclude Abhidhamma and Vinaya).

There are also currently three shortcuts to narrow your search to an age strata:

Narrow search to “Early Buddhist Texts” (in:ebt).

This is a shortcut and not a definitive list of what is early. Equivalent to the following.
dn, da, mn, ma, sn, sa, sa-2, sa-3, an, ea, ea-2, kp, iti, ud, snp, dhp, thig, thag, pli-tv, lzh-mg, lzh-mi, lzh-dg, lzh-sarv, lzh-mu, lzh-ka, lzh-upp, san-mg, san-lo, up, ea-ot, d, sf

Narrow search to “Early Buddhist Suttas” (in:ebs).

This is a shortcut and not a definitive list of what is early. Equivalent to the following.
dn, da, mn, ma, sn, sa, sa-2, sa-3, an, ea, ea-2, kp, iti, ud, snp, dhp, thig, thag, sf

Narrow search to “Early Buddhist Chinese Texts” (in:ebct).

This is a shortcut and not a definitive list of what is early. Equivalent to the following.
da, ma, sa, sa-2, sa-3, ea, ea-2, lzh-mg, lzh-mi, lzh-dg, lzh-sarv, lzh-mu, lzh-ka, lzh-upp, ea-ot, d

---

There is currently no way to filter by root language. So for example there could be
in:pali: dn, mn, an, sn, kp, ud, iti, snp, vv, pv, thig, thag, ja, tha-ap, thi-ap bv cp mnd cnd ps ne pe mil pli-tv-vi pli-tv-ab

My question for you: What other filter shortcuts would you like?

Some of my ideas are
in first four nikayas
in:ffn : dn mn sn an
In early KN
in:ekn : kp dhp ud iti snp thag thig
In late KN
in:lkn : vv, pv, ja, tha-ap, thi-ap bv cp mnd cnd ps ne pe mil

What ways would you like to be able to narrow down your search results?

P.S.: Hong Da has been working really hard on improving search over the last several months. If you haven’t recently, please do more searching!

Discussion on GitHub for those really interested.

[sabbamitta]

Thank you for bringing this up! I would support all your suggestions.

[Pasanna]

In theravada (not quite the right wording… I want xx language but the pali texts only)
in vinaya
are two which I can’t see in the list and would be super useful

[Snowbird]

@Pasanna

In theravada

Would that be something different from in:pali? Do we have Pali texts that are not associated with Theravada? Or will we some day? I’m honestly not sure.

I realize now that my in:pali only listed sutta texts. It needs to extend to all Pali texts.

@Pasanna

in vinaya

in:vinaya actually already works. But it includes all Vinaya texts. What doesn’t work but should is in:pli-tv-vi I’m not sure why it doesn’t.

Ven. @Pasanna I was actually hoping you would offer suggestions on Vinaya searching since I believe you work on it quite a lot. Thanks!

[bran]

Is something like in:abhidhamma or in:ab possible?

[Snowbird]

@bran

Is something like in:abhidhamma or in:ab possible?

in:abhidhamma already works. (At least it should!) As far as I can tell it searches in all Abhidhamma texts in all languages/schools. It’s at the same hierarchy level as in:sutta and in:vinaya.

Bhante Sujato in the past has mentioned not wanting to have duplicates, but I don’t know how he would feel about abbreviating Abhidhamma. On the one hand it’s long. On the other I don’t think there is a universal standard abreviation for Abhidhamma, is there? That would mean users would have to memorize the abbreviation we picked or check it each time.

What doesn’t work is the level at in:in:pli-tv-ab. Same problem with Vinaya docs.

[bran]

Ohhh I see, I thought that filter didn’t work, so I probably didn’t type it in right. Nice to know for when I’m wasting my time searching for obscure things on the abhidhamma

---

My first thought with the filtering from someone who has designed some UIs is that the text filtering could work with a panel of switches like the language filters:

Because it might make it more accessible, reduce typing, and less technical for some? It could also be tiered like

ebt
⠀ ebs
⠀⠀ DN
⠀⠀ MN
⠀ ebct
⠀⠀ da

then checking ebs could check off its children.

[Snowbird]

@bran

Because it might make it more accessible, reduce typing, and less technical for some?

For sure there would be advantages. Perhaps once search is working through the “manual” way, someone could build an interface like that on top of it. The number of texts on SC, for good or for bad, is huge. So then you start running into other issues. The Digital Pali Reader has a check box system for search, but they have way, way less texts to deal with. And now that I look at it again, I see it doesn’t allow nearly the precision that the SC search already offers.

I’m hypothesizing that 80% of people spend 80% their time searching among the same group of texts. For example, I’m almost always searching Pali suttas. So although it’s kind of a pain to always have to enter a filter, at least I know what it is and I can do it without taking my hand off my mouse.

And this is actually why I made the post; to see how other people are using search.

[bran]

I did think about how there could be so many texts, so it could be in drop downs. I fleshed this out into some subcategories for a fuller example of how the navigation could feel.

⬜ all texts

ebs
ebtc
pali
4 nikayas
shortcuts

⬜ vinaya

all analyses
all nun’s
shortcuts

⬜ Theravāda

monk’s analysis
monk’s assorted
nun’s analysis
nun’s

⬜ Mahāsaṅghika

monk’s analysis
monk’s assorted
nun’s analysis
nun’s

⬜ sutta

❎ D

like in:long
DN
DA
DA-ot

⬜ M

MN
MA

⬜ S

SN
SA

⬜ A

AN
EA
EA-2
EA-ot

⬜ K

KN
DHP
Minor Chinese

⬜ O

SAN
XCT

⬜ abhidhamma

⬜ Theravāda

Ds
Vb
Dt
Pp

Since people probably most often use just one filter, hopefully those specific filters would be easy to access from this sort of menu.

[Pasanna]

I wasn’t very clear

@Snowbird

Would that be something different from in:pali? Do we have Pali texts that are not associated with Theravada? Or will we some day? I’m honestly not sure.

What I think I meant was text from the Pali canon in (both sutta and vinaya) in:English

in:pli-tv-vi
in:sutta-tv ← a made up tag
in:english ← though if I am searching an English word this is kinda redundant

would in:pli-tv-vi be only in pali or would it be the pali texts in the language I search in?

I’m sure I had more vinaya searches, but I can’t remember what I was wanting now.

[Pasanna]

Thank you for bringing this up. I’ve had a half finished search UI mockup on my computer for a really long time! It was going to be a thing that @gillian and I just had on our desktops!

[Snowbird]

@Pasanna

in:english ← though if I am searching an English word this is kinda redundant

This was something that was really hard for me to get my head around for the entire time I’ve been volunteering on the search project.

The languages interface on the search page:

Is where you decide which documents it’s going to search in. Not the root language of the texts. The current wording is really confusing to me because it says right there root languages. What I’m proposing it say is Manuscripts.

So if you have Pali and English ticked and you search for cat, it’s going to search in all documents in both Pali and English that have that term. That means:

1. It will return results from English translations of Pali, Classic Chinese, Sanskrit, etc. texts if they have the word cat
2. If you have partial match turned on, it is going to return a bunch of Pali documents (i.e. the Pali manuscripts themselves) that have the characters cat in them. Lots of them

Does that make sense?

So except for searching for the articles written by Bhante Sujato, there are no in:english texts. in: is exclusively for ancient collections and also groups of ancient collections in the form of shortcuts, like in:ebt

Does that make sense?

So, as odd as it sounds, in:pali is not asking for Pali texts. Its asking for all the collections that are written originally in Pali. I know, it hurts my head thinking about it. If you have Pali ticked on the search settings then including it will be redundant if your search term is Pali.

So once we actually have a working in:pali filter, and once we can use operators with the same filter keyword, you could search for in:vinaya AND in:pali to get all of the Pali/Theravada texts. That would be equivalent to in:pli-tv-vi (which is ugly and doesn’t work at the moment.) All of that is why I’m proposing in:pvt for Pali Vinaya Texts.

And you are correct that there is some inherent redundancy in the whole system because if you search for feeling in all the texts on SC (so imagine every box ticked on the settings area) then I’m fairly sure it will only be returning English results.

However, if you try the same thing with cat you are going to get an avalanche of Vietnamese translations because (with diacritics, which are ignored) cat as a very common Vietnamese word. Or if you search vedana you are going to get Sanskrit manuscripts as well as Pali.

@Pasanna

in:sutta-tv ← a made up tag

We do need a tag for this. I’m proposing in:pst, In Pali Sutta Texts.

But… once we get involved with Vinaya and Abhidhamma, then there are many schools that use Sanskrit, so in:sat (In Sanskrit Abhidhamma Texts) is a thing, but is it only thing we need? And I just noticed that there is one school that has both Sanskrit and Classic Chinese root texts.

That’s kind of the reason I made the thread.

[Pasanna]

That language thing really isn’t obvious.
I just thought it was the language preferences for the site and never clicked it.

[Snowbird]

@Pasanna

I just thought it was the language preferences for the site and never clicked it.

Yeah, I have been thinking of that recently. Do you have a suggestion for what icon should be used?

There is a change to the interface on the staging site that will hopefully make it more apparent what is happening:

I’m thinking instead of "Languages searched it should say “Documents searched”. What do you think?

A previous itteration had a Change button where the globe icon is. This unifies it with the rest of the interface, but I agree it’s not good to use the same globe for the search.

[Snowbird]

For me, the only way I can keep that language option page clear in my head is if I think of it as a documents option setting. Like, what documents are you going to return to me. Then it is clearer in my mind that if Pali is ticked I’m going to get back a Pali document.

Taking a look at the Material Design icons, I see these that could be used if the “document” model is adopted:

There are a few “language” icons that mostly look something like this:

ETA: Some mockups

[sabbamitta]

@Snowbird

Taking a look at the Material Design icons, I see these that could be used if the “document” model is adopted:

I like the document icon with the magnifying glass—this makes it very clear these documents will be searched.

[Pasanna]

I feel like the language button is a form of ‘filter’ so to have it separated from the other things called ‘filter’ is a bit weird. What we are talking about is ‘advanced search’ and then various subcategories of advanced search. Mixing ‘spacing’ in with tools which are filters then is a bit confusing.

I do prefer the ‘translate’ icon, but still think this gives the impression that it is about site settings, rather than filtering the texts searched. This could be because of the following (slight diversion)…

I think I have a problem in general with the layout of SC

There is a lot of good stuff hidden on the RHS under the ‘view’ icon (for ‘normal’ pages) and now under the filter and language for search pages, which are easily missed.

I feel like the purpose of the orange bar is a bit mixed. If the breadcrumbs bar is about 'wayfinding" then the document title should be the last item in the breadcrumbs. This would allow all the handy tools to be hanging out on the LHS in their nifty orange tool-belt. (I’m not convinced the eye icon is the best for what we are calling view, but that’s another story).

Apologies that this is a bit of a deviation from your OP question but here is a visual of what I am proposing.

and it’s corresponding text page

[Snowbird]

@Pasanna

I feel like the language button is a form of ‘filter’ so to have it separated from the other things called ‘filter’ is a bit weird.

Yeah, I’ve also thought it was a little strange. It’s really more like “settings” (which is only one type of setting at the moment) and “documentation” explaining ways you can modify the search (which is really just filters at this point). They are exclusive from (to?) each other because you can’t currently add the parameters in the language check boxes into the search field.

But, I do like having those languages as something that stays selected and then sits in the background. The chances of me, for example, waking up one morning and wanting to include documents translated into something other than English is close to zero. So I don’t want to have to add in language:english every time. It’s bad enough that I have to add in:pali.

@Pasanna

What we are talking about is ‘advanced search’ and then various subcategories of advanced search.

Well, unlike Google, if we couldn’t limit by document language the search would be drastically less useful. So I don’t really see that part of the search as advanced.

Now that I think about it, Google is kind of funny… Everything you can do with “Advanced” search is available just by typing it into the search bar. When you actually go to what they call “Advanced search” It’s really just training wheels to give you access to things already available in the “main” search.

@Pasanna

I do prefer the ‘translate’ icon, but still think this gives the impression that it is about site settings, rather than filtering the texts searched.

What do you think about the page with magnifying lens? I agree with Ven. @sabbamitta that in its own way it closely resembles “documents searched in”

@Pasanna

I think I have a problem in general with the layout of SC

I agree it’s not perfect. I’m not happy with the icons being so far to the right that they almost become invisible. I’m especially not keen on the “parallels” label because most of what is under it has nothing to do with parallels.

But all that is worth a new thread.

The problem with the Advanced search drop downs is that it’s not as accessible (for the visually impaired). Although it does make those options easier to find.

I think for the sake of this thread, unless it’s proposing a very different way that filters should work (or simple things like a better icon), it’s best to concentrate on the functionality rather than the UI so much.

I really appreciate your insights Ven. @Pasanna.

[Khemarato.bhikkhu]

@Snowbird

When you actually go to what they call “Advanced search” It’s really just training wheels to give you access to things already available in the “main” search.

Although they include the filters in the top bar of the SERP:

And I’m sure this is where the majority of their filter use comes from.

@Snowbird

functionality rather than the UI

Idk. To me, the more important thing is to be able to search inexact phrases and to have the ranking algorithm work well. If search gets those things right, most people shouldn’t have to think about filters… and personally I’m pretty happy with the filters we got . It’s not the reason I still use Google to search SC, nor is it a “killer feature” that would get me to switch

[Snowbird]

@Khemarato.bhikkhu

Although they include the filters in the top bar of the SERP:

I believe there is a ui improvement in the works that will add something like this on the results page.

@Khemarato.bhikkhu

and to have the ranking algorithm work well.

Well, I guess there is the rub. I can’t believe that any ranking algo is ever going to know that I absolutely never want results for things other than Pali root texts. And I feel bad for people who want to only work with Chinese root texts.

I’m happy to see concrete suggestions for how to to improve the ranking algorithm. I notice that I’m always getting AN results first, I’m assuming because it comes first alphabetically? That’s really bad. Maybe we need a discussion thread for ranking, because I’ve seen very little in the way of ideas for that.