SuttaCentral updates, Oct 2023 edition: instant search and more

We’ve been working hard behind the scenes on some major updates for SuttaCentral, and we’re finally ready to roll them out. You should see the changes soon! What’s it all about?

First thing is, these are meant to be unobtrusive upgrades: they don’t affect the main functions on the site, just make things better.

instant search

The biggest improvement from a user point of view is the instant search. This is a turbocharged search using the awesome Oramasearch library. It took us a lot of experimenting to find a good solution, but we’re super-happy with Orama.

The idea is that it will cover a large number of people’s most common needs, and the database search will handle the rest.

Open up the search and an overlay dialogue appears, start searching and it’ll immediately populate the results from sutta titles and IDs. This gives you a new and snappy way to get to that sutta even faster and hence get enlightened even faster! Hopefully!

If you want to just search for the word, just hit enter or click the search item as usual and the normal search results appear. You also have the option to limit your search to early Buddhist suttas. (This and many other filter options are listed in the “filter” dialogue on the search result page.)

Tip: use in:ebs filter to restrict search to “early Buddhist suttas”, in:ebt for “early Buddhist texts” including the Vinaya.

I asked Oramasearch developers to support fuzzy matching for Pali/Sanskrit diacriticals, and they went ahead and did it! Open source is so awesome. (This feature is being tested, it’ll roll out soon.)

There are of course some limitations.

  • instantsearch does not search full text, just titles and IDs
  • when first opened it’ll take a few seconds to warm up: it’s downloading the data. (we do it this way out of respect for your bandwidth: you only download data when you have indicated you want to search.)
  • it is restricted to early Buddhist suttas (not Vinaya, Abhidhamma, or late parts of the Khuddaka)

We will be fine tuning this over the next few weeks, and in future will experiment with adding limited full text search.

Technical note: How instant search works is, you have a javascript library in the browser that does the searching (orama). Then you download the data into the browser. That’s why it’s instant: there’s no round call to the database as with a normal search query. The cost is, you have to download all the data up front. Currently, with just titles and IDs, it’s not too much. But if we add the texts, it’ll be a few MB. As a guess, if we add the translated text for the early suttas, it’ll be about 2 MB or so. That’s quite a lot, but bear in mind the average page size is on the internet is several megabytes. Now, most of that weight is video, images, trackers, ads, and the like. So 2 MB amortized over the whole app is not prohibitive. But then there are questions of browser performance, mobiles, and so on. Anyway, we’ll test this in coming months.

better breadcrumbs

We’ve tweaked the breadcrumbs. It’s trickier than it looks! Now:

  • the main icons at the top (Home, search, and menu) are always visible
  • if the breadcrumbs are too wide for the screen, you’ll see the last one on the right so you know where you are.
  • you can scroll back to anywhere along the line.

m3 styling

We’ve also updated the styles to fit the newer m3 version of Google’s material design. We don’t really follow material design strictly, but we do take some cues from it, like rounder corners, warmer colors, and so on. Overall, the look is softer and more consistent. But nothing really dramatic.

material web

We’ve also updated our dependencies to use the latest material web components from Google. We don’t use many dependencies, just a few things like the menu, some checkboxes, things like that. You won’t really notice these changes too much, just that the styling has been tweaked a little.

next steps

Now we want to keep on evolving the search. There are a bunch of improvements to the main search that we are looking forward to. Search is basically a bunch of tiny tiny things that add up to it working well or not. It takes time!

As mentioned above, I’d also like to see if we can include full text search in the instantseach. We might also add some abilities to customize it, but we’ll see how it’s used first.

18 Likes

:thinking: No matter what I type I get everything (starting with da1, da2…)

4 Likes

Ven. @Khemarato.bhikkhu
This is an old version of the interface and requires a hard refresh of the browser in order to load the new UI.

5 Likes

I’m finding the suttacentral site, which doesn’t really work on my old laptop, is now hard to use on my up to date phone as well.
Fortunately the legacy.suttacentral.net website is still available, maybe the newer translations can be placed there? It seems much easier to use.
Thanks.

3 Likes

Thanks for the great update! :tada:

One problem I find is that although my website language is set to German, I get only English suggestions in the instant search. (Note: the entire DN has segmented German translations.)

Is there a reason why some results include the Pali name and others don’t? And DN 1 has it even twice?

3 Likes

Instant search is fantastic!

And we need that capability for sparsely populated languages in Voice. We’ll put that in our todo list and will be eagerly stopping by for a “cup of code” to turbocharge the search experience. Fascinating to learn about orama, that would help a lot. Thank you for your hard work!

:pray:

1 Like

Much gratitude to you, @HongDa and the whole team!

5 Likes

If you want help, you’re going to have to be more specific.

The legacy site will be turned off very soon, do not rely on it.

I think it only works with English for now, we’ll add other language support soon.

2 Likes

Ok- here goes:

  1. Using my iphone I select a sutta to read in English
  2. Wanting to look at another English translation of the same sutta I select “parallels.”
  3. A new menu drops over my previous page offering options.
  4. I select another option.
  5. That’s as far as I can go. This new ‘menu’ seems to have covered up any other window, I can’t navigate around it.

Perhaps this feedback is of help.

2 Likes

This is a bug. By default, relevant data will be obtained based on the site language, but the language information is not saved when saving to the browser storage. Therefore, after the English data is saved for the first time, the program detects that the browser storage already exists data, so no new data in other languages will be requested, I’ll fix it. :pray:

2 Likes

Hi @stephen Thank you for your feedback.

Is the situation you describe roughly like this?

Normally you can close the parallels menu by clicking the parallels button or clicking on the text.

Yes- thank you.

That’s how it looks when I press “Parallels”, except there isn’t a window of sutta text that peaks out at the bottom like your screen shot.

When I press a translator name, basically ‘nothing happens’ (except an orange oval briefly lights up around what I have selected) even if I select the translation I previously had been reading. I cannot get past the screen.

Thanks with your help with this.

1 Like

Yeah, it’ll secretly swap out the page in the background. If you click “Parallels” again to close that menu you should see the new translation. Definitely a bad (confusing) experience. @HongDa - can you make it so navigating to a new sutta / translation closes the menu?

2 Likes

Aha- thanks, I didn’t realize it was a 2 step process!

One goes to the translation options page, and then needs to hide that page by pressing “parallels” again.

PS Please consider making the legacy site stick around a bit longer- for those without the latest gear and finely honed computer skills it’s vastly easier to use.

1 Like

Indeed, it should close the top sheet by default.

Not gonna happen.

Thanks @Khemarato.bhikkhu @stephen

I will make improvements :pray:

3 Likes

I’ve fixed the bug and pushed it to production.

Requires a hard refresh of the browser to take effect.

4 Likes

Thanks! It works now when you click on another translation, but it also would be nice that the menu would disappear when you click on a parallel text in the menu. :pray:

2 Likes

Thanks!
I’ve resolved this problem and pushed to Production.

requires a hard refresh of the browser to take effect. :pray:

4 Likes

Thanks so much for this fix. It has long bugged me!

4 Likes