Discuss & Discover

I’m starting this thread to discuss the new “Discuss & Discover” feature that we are rolling out soon on Suttacentral. Essentially what this does is that when you open the sidebar for any text, it shows you the relevant posts for that text here on Discourse. So if I write, say, MN56, in this thread, as soon as I save this post it will appear in the sidebar at http://staging.suttacentral.net/pi/mn56, and at any translations. I just checked, and yep, it’s there!

Let me give some background as to what I want from this feature, and then discuss the implementation.

One of the main features of SuttaCentral is that it has only the canonical texts. We’ve gone to a lot of trouble to remove footnotes, introductions, and so on. This was essential, firstly from a technical point of view, supporting them would be so difficult as to be impossible. More importantly, from a philosophical point of view, we are about what the Buddha said, not what someone said about the Buddha.

Now, from the first days people have liked to discuss the Buddha’s teaching, and there are plenty of instances of this in the Suttas themselves. This tradition evolved into what we know as the “Commentaries”. In Pali, these are called aṭṭhakathā. The commentaries are of overwhelming importance in traditional Theravada, so much so that in practice Theravada is often the teaching of the commentators.

In modern times the commentaries have come under a lot of criticism. But the main problem, I think, is not with the commentaries themselves, but in the fact that they have become virtually canonical. When we have a major respected Buddhist teacher like Pa Auk Sayadaw saying things like, “As the Buddha said in the Aṭṭhasālinī”, which is an Abhidhamma commentary written nearly a thousand years after the Buddha’s death, we have a serious problem with historical perspective.

In modern times, the commentarial tradition has been largely supplanted or adapted as footnotes, introductions, and so on. In some cases, like say Ven Bodhi’s Majjhima translation, many of the footnotes are simply summaries of what the commentary says. In other cases footnotes are independent of the commentaries, but they still play a similar role.

Footnotes and the rest of the textual apparatus involve two quite separate kinds of things, which we need to carefully distinguish. The first is the need—universal and human—to talk about things of importance. The second is the specific manner in which this discussion has been carried out.

The latter is bound and determined by technology. A footnote, or indeed an introduction, is a book thing. You read an introduction at the start, before going on to the main thing, a concept that doesn’t really apply on the web. A footnote is at the “foot” of the page, another book-derived notion. This shows up in all kinds of ways in how these things are handled. Just one specific example. Lets say we have a Buddhist technical term, say “aggregates”, which is difficult to convey in translation. So we have a note. In a book, the note appears at the first occurrence of the term, or in a prominent context. But no-one reads the suttas one after the other on the web, so where do you put the note? You can, if you want, put it every time the word occurs, something that is technically impossible in a book.

In addition to these issues of form, there is also the problem of content. I’ve seen lots of footnotes by lots of authors, and, well, maybe this is just me getting old and cranky, lots of them seem of limited use, or dubious, or highly technical, or just plain wrong. Yet when they are included in a book, they assume a quasi-canonical status. They are there, right with the text, and they worm their way into your mind as the “right” way of reading a text, even if you consciously know this is not the case.

On the other hand, it is obvious that the Suttas do have lots of things that need some explanation, things that are deep or with context lacking and so on. So we need something, but something that goes beyond the static possibilities of books.

I’d like to get back to the original idea, found in the Suttas, and the origin of the commentarial tradition, of a “discussion of the meaning”. Rather than a single, authoritative voice telling us what the Sutta “means”, we can learn from, and contribute to, a culture of progressively informed opinion, experience, and knowledge. And that is what this Discourse platform is for.

So, we can create a rich environment for learning the suttas, but we need to connect that with the texts themselves. The connection shouldn’t be too close, so that the text and the comment is confused, but neither should it be too distant, so that someone reading the sutta doesn’t know where to find help.

This concept of distance is one of the features of Discourse. It is built so as to host comments, but especially comments on a separate site, rather than at the foot of the page (although it is possible to do that as well.)

So what we do is we pull Topics where suttas are discussed, via the Discourse API, into the sidebar. If @blake gets a moment, maybe he’ll share the technical details of how this is done. Currently we are an initial implementation, and I’m sure much improvement will happen. But the basic function is working fine already.

I’ll leave this discussion here as the main topic, and below I’ll raise some questions of details.

2 Likes

The first thing that strikes me about the current implementation is that it’s small. @blake made the font size smaller to fit it in, and to me that’s a red flag: too much info, too little space. And on my mobile it’s basically unreadable.

So we should either increase the space or decrease the information.

I’d opt for the former; I think the amount of info is good. So how do we create more space?

There could be many approaches to this, but a simple one would be to give the “Discuss & Discover” 100% of the sidebar when it is activated. In other words, it replaces the other info in the tab (much like the expanding search bar in mobile view).

If it’s not clear I did that inspired by the Discourse search, basically I tried to reproduce the contents and style of the dropdown you get when performing a search on Discourse.
On Discourse, when you do a search on mobile, the blurb/snippet disappears and the title text is full size.

Okay, good. That means we can be pretty consistent. We should handle tags, though; perhaps keep on eye on whether Discourse includes these.

Sounds like a good idea.

Still need more room!

i see that the links are to topics, whereas the sutta may only be referenced in a single post and therefore is hard to find in lengthy discussions such as this

of course linking to each and every individual post a sutta is mentioned in is a bad idea

i’m aware that one can use browser search to hop about these posts within a topic

but maybe for convenience sake it makes sense to implement one more step between a link to the topic and the topic page itself, specifically a page where all posts which reference the sutta in a particular topic are aggregated, wherefrom one can access these posts selectively instead of scanning the entire discussion for them

That’s a perennial issue in search engines. What does google do if you search something? It shows you a snippet, but when you follow a link, it just takes you to the top of the page. It’s entirely up to you to find the part of the page which caused it to show in the search results. It seems google has concluded the best place to drop a person is at the start of a page. Who knows why. But I’m sure they performed thorough user-testing before settling on this.

However I believe I actually intended it to go directly to the referenced post, there might be something wrong with the links I’m constructing causing it to go elsewhere in the thread.

edit: And yup there is. So for the next iteration it’ll link directly to the appropriate post.

1 Like

at least for this topic Kamma of unintentional actions the link is erroneous, because the address for some reason refers to post 513

http://discourse.suttacentral.net/t/kamma-of-unintentional-actions/232/513

while the topic only contains 80 of them and that’s why this particular link doesn’t make the user land at a particular post but rather drops him off at its end

in fact all links have a way larger posts integer than the number of posts the topics actually contain and thus a user every time ends up at their bottom

another thing i spotted is that the last 80th post URL generated by the ‘share a link’ tool actually gives a link to a 81th post, which is non-existent

Kamma of unintentional actions

as if numbering got wrong somewhere along the posts count, because a few first posts share links are correct

1 Like

Okay I’ve pushed a new iteration onto staging. You now have more room! It now takes over the whole sidebar and the other controls are removed while discuss is open.

@LXNDR it now links to the correct post

@sujato I’ll look into categories. Unfortunately the JSON data for a ajax search request does not include information about categories, it only includes a category ID. So I need to ajax the category data separately just to build a few little icons. Worth it I guess though because it gives an indication about the kind of things we talk about.

Is it an idea to have a target="_blank" in there for the links so you don’t loose the tab with the sutta you were reading?

Category badges are now implemented - for reasons of complexity it doesn’t show subcategories. Also I made the links _blank as suggested by @vimala as it’s an external link.

does it take time to come into effect?

in the latest Chrome the pages open in the same window

@blake, this is amazing work, we are building something wonderful here! This morning I’ll add a bunch of content.

One detail: we should have an arrow on this button that points down when it is opened, like the one for “languages”.

@LXNDR, it opens in the same window but a new tab. If it’s not doing this for you, probably it’s a cache thing.

1 Like

We need to pay attention to sorting. I’ve added a bunch of resources for DN1 to test this out. We’re getting a real mix of things, with the actual DN1 resources sometimes missing, or below things for other suttas. One problem is the matching isn’t whole word, so “DN1” matches “DN11”, etc.

I would suggest to use exact whole word matching and weight in this order:

  1. Sutta IDs in tags
  2. Sutta IDs in titles
  3. Sutta IDs in posts.

And then other stuff. I know this may take some time to implement, just recording it here.

Unfortunately at the moment we are somewhat constrained by the behavior of discourse search which is very much inclined towards search-as-you-type.

The two possible approaches to getting better results, is either performing custom search queries against the PostgreSQL database using a Discourse ruby plugin, or indexing the posts into (say) elasticsearch on our server. PostgreSQL text search is perfectly respectable and the synchronization will always be perfect (or at least as perfect as the discourse devs have made it). With elasticsearch there is easy communication between the server and search, but the difficulty is communication between discourse and elasticsearch - keeping it synchronized. Both are doable but non-trivial.

No problems, for now I think we just focus on getting it rolled out, we can keep our eye on it and worry about tweaking the results as we move forward.

Actually, this may be a worse problem than I thought. After adding more resources, on say DN2 we now have a bunch of things showing up for DN20 onwards, and nothing for DN2.

I believe the Abbidhamma is not the words of the Buddha.

3 Likes

Okay, busted! And there are plenty of things in the Vinaya and suttas that are not the Buddha’s words, too. Let me rephrase: SuttaCentral contains only the canonical texts themselves.

Incidentally, the inclusion of the Abhidhamma serves a number of purposes, but one of them is that there are many sutta parallels in the abhidhamma.

2 Likes

Thanks Bhante. In my opinion it would be great to have the full Canonical Abhidhamma readily available. Actually, that’s almost possible now that PTS has put a lot of their material into the public domain.

There is a great deal of confusion between the Canonical Abhidhamma, which hardly anyone has read, and the “summaries” in the Visuddhimagga and the Abhidhammatthasangaha, which were written about many centuries after the canonical Abhidhamma. See:
A Comprehensive Manual of the Abhidhamma:The Abhidhammatthasangaha of Acariya Anuruddha, Buddhist Publication Society, Edited by Bhikkhu Bodhi and others.
http://store.pariyatti.org/Comprehensive-Manual-of-Abhidhamma-A--PDF-eBook_p_4362.html

The Canonical Abhidhamma doesn’t contain billions of mind moments arising every second. It is much more along the lines of analysis and systematisation of the Suttas as @sujato mentions.

Take a look at, for example: http://www.ancient-buddhist-texts.net/English-Texts/Short-Pieces-in-English/Conditions.htm The Enumeration of the Conditions:

3 Likes

SuttaCentral coverage of Abhidhamma is not as extensive as for the Suttas and Vinaya, but we still have more than pretty much any other site that I’m aware of.

We have the full Pali text of the seven Abhidhamma books. These are very extensive, especially compared to the PTS editions. The Patthana in particular is very detailed; I spent several weeks last year sorting out the incredibly complex internal structure of this mind-blowing text, which has over 10,000 structural headings. It is perhaps the only text produced by humanity that genuinely requires using an <h6> tag!

We also have full English translations of three of these texts, the Dhammasangani, Vibhanga, and Kathavatthu. These are mostly from the PTS editions, with some improvements. I have added the Mātikā translation, which was missing from the PTS. And Ven Anandajoti has retranslated the Satipaṭṭhāna Vibhanga.

The Vibhanga and Kathavatthu were digitized by us, while the Dhammasangani was digitized and available on the Wisdom Library and adapted for SuttaCentral.

We are still missing the text and translations for the Abhidhamma texts in Chinese and Sanskrit. These are on our 2do list. Particularly interesting is the Sanskrit text of the Dharmaskandha, which quotes from and discusses a number of interesting suttas. I’d like to prepare the translation of the Puggalapaññatti also, as this is little more than a collection of passages from the suttas. Full sets of parallels for all Abhidhamma materials is also on the 2do list. (It’s a long 2do list!)

1 Like