SuttaCentral API/Apps?

Budu Saranai. I’m new here and this is my first post. First I want to congratulate the team on a job well done. You have certainly pushed the state of the art of Buddhist websites. I would like to push it further.

I’d like to know more about the internal structure of the site. Do you have an API? Or is there a granular link structure that could be used by external or related apps?

Here’s what I have in mind. I’d like to create a site that allows students of the Suttas to drill down from a high-level abstraction of the Buddha’s teaching, through summary explanations to original sources. One view of this might be a self-paced online course on Theravāda. Another might be a comprehensive web ontology of the Suttas, allowing users to submit natural-language queries.

What kind of hooks exist for these types of apps? Do you encourage outside devs to link to SC? Any plans to change the architecture of the site? Can you point me toward any public documentation of internal architecture, API or features? Thanks in advance for any help.

Hi Dhammasar,

That sounds fantastic, and we would certainly like to support this kind of extension or development of our content. Because SC begins from a certain perspective—namely, the parallels between suttas in different collections—there are certain approaches that suit us, and others that are best developed on another platform. We are well aware that one of our biggest weaknesses is that it is not easy to discover new material, especially for beginners, and this Discourse platform is one step towards that. What you’re proposing is another way to expose the suttas, especially for newcomers. There is more than enough room for multiple different takes.

May I ask, do you have any sites that you’ve developed previously, or anything that you are looking at as a model for what you want to do?

If you check our Github issues, you’ll find that I have already made a proposal such as you suggest:

However, as you can see, this is just a bare suggestion. I have no idea how such a thing is to be done, or what would be useful.

What I would suggest is that you have a poke around our Github; the wiki is quite extensive, and all the code is open. In the Issues you’ll find all the things we plan to do.

If you’d like to pursue matters further, contact @blake on this forum, and outline your proposal. He’ll help you, obviously depending on whether he has free time. We can also give you permissions for the Github issues, and you can develop the issue further there.

Generally speaking our approach is to be both pragmatic and standards-compliant. So if you had a specific idea for what you want, we could develop something based on that; at the same time, we’d keep in mind best practices for development, so that other applications can be built later.

One thing to remember: our site is still under rapid development. In particular, we will be focussing on launching localized versions, with all the text in the language of translation. However, the basic structure is dependent on the URLs of the individual files, and these are very consistent.

Thanks for your reply. My proposal is still in the concept stage, playing around with ideas. A couple of years ago I started a book about the Buddha’s teaching based on the Four Noble Truths. I stopped working on it to concentrate on intensive practice. Looking back, it seems that the approach is ontologically valid, but would be better as a dynamic web app.

Some of the most valuable insights come not from the content of the Buddha’s teaching, but from its structure. It’s fascinating that the Buddha’s teaching is ontologically coherent at every scale. It’s fractal, in that the whole (symbolized by nibbāna) is present in each of its parts. And any of those parts, if followed to its root, can open up on nibbāna.

From this view came the idea that the Four Noble Truths pervade the Buddha’s teaching structurally, and each is revealed in hundreds if not thousands of examples in the Suttas. The book idea didn’t work because going through the Four Noble Truths in linear sequence seems too narrow. OTOH, there should be some structure to guide the reader’s exploration of the Suttas.

The piṭakas don’t serve the purposes of the newcomer very well. They are organized as perhaps a librarian might classify them. So I hit upon the idea that the reader should be loosely guided by the structure of the Four Noble Truths, and others like the Eightfold Noble Path, but still free to explore within them, and drill down to the original texts to get the context.

The concept would be much easier to communicate with a simple demo. I will check out your github and try to understand the link structure, which seems quite regular down to the paragraph level.

Oh, one more thing. Is there a way to implement the low-level links without using Flash? I’m sure you’re aware of the security issues…

Sounds amazing, I look forward to it. I’m not sure if you’re familiar with my work in A History of Mindfulness, especially the first part, where I develop the same idea, and argue that the 4 noble truths in fact formed the literal, not merely metaphorical, framework for the earliest texts, a structure that is still evident today in the Samyutta.

Re Flash, yes I agree, I want to vineyya it ASAP:

Note that this feature will be rejigged as part of my translation project, as we will then have much more robust hard-coded IDs at a segment level, which will be consistent across languages.

Be careful of this. The structure is very rigorous at the text level, but paragraphs are handled differently in different editions. We inherited a vast ranges of texts produced from different sources, and have not attempted to reconcile them, with a few exceptions.

Several of the texts, for example in DN and MN, use similar paragraph numbers, ultimately related to the PTS numbers for DN and Ven Bodhi’s numbers for MN. We recently introduced these in the Pali text as well, so you can coordinate these.

However, it may be worth bearing in mind that the next generation of translations, which I am commencing very soon, will have the text matched segment by segment between the Pali and all translated languages. A segment is usually a sentence or a line of verse, or some other major break in the text. So you can work semantically using natural language analysis on the English text (since English has the best developed tools in this area), for example, and supply the results for any language. If you want to work with these next generation texts, we can share with you a few samples so you can see what is going on, but the full thing will be a while, maybe 18 months or so.

does this imply that the breakup into segments (say sentences) of the translation text must follow the same of the source text?

Generally speaking, yes. Of course there is always some flexibility; it’s not a disaster if some things don’t match up 100%. But on the whole, I will be translating segment by segment. The underlying idea is that a sentence (or sentence-like syntactical structure, such as a sub-clause) is the smallest linguistic unit with a clearly articulated meaning. In the vast majority of cases we can translate segment by segment, and express the meaning clearly. Occasionally this doesn’t work, notably in verses, where we sometimes have to completely rejig the order of the lines, but this is only a tiny percentage of the text.

1 Like

SuttaCentral has some API functions. For example you can get a version of the page sans header and footer, by adding ?embed=1 to the url, which is perfect for embedding in an iframe (this works for absolutely any page, including search result pages and such).

It is also possible to make certain queries of the sutta database via a data endpoint, for example:
SuttaCentral - get JSON data about DN 1 and DN 2 (includes url of english translations and such)

http://localhost/data?parallels=mn10&ll_lang=en - get JSON data consisting of english translations of parallels of mn10

At the moment those endpoints are very hacked on!

Our data is exposed in a more full-featured way via our elasticsearch server which can be accessed via the URL:
https://elastic.suttacentral.net

For example the english translation texts can be searched under:
https://elastic.suttacentral.net/en/text/_search

For security reasons, naturally it is not exposed directly, but instead is proxied through Nginx which restricts access to read only operations. Also for reasons of paranoia it is rate-limited if you make too many requests too quickly you’ll be rebuffed.

Our complete texts in all languages are indexed in Elasticsearch, and so is (most of) the sutta database.
For many purposes, the Elasticsearch API would probably be adequate and it is of course utterly awesome and requires zero work on our part. Someone who is familiar with elasticsearch can explore the indexes via endpoints such as _mappings

Over time we generally plan to use elasticsearch more, and to make more of our data accessible via it. Anyone who wants to interface with our site directly would be well advised to do so via the elasticsearch API - although I am also generally open to adding new URL endpoints for specific functions, returning either HTML or JSON data.

2 Likes

Thanks Blake. I’ll keep all this in mind. Right now I see an important opportunity. I’ve always wanted to do a deep ontological analysis of the Buddha’s teaching. Bhante Sujato’s translation project will create a semantically regularized text base that can be referenced as instances of categories defined in a logically and terminologically consistent ontology. That could provide a codebase for not only automated semantic and linguistic analysis, but also for natural-language interaction with the Suttas. I’m sure you can see where this is headed: apps like a personal digital assistant for learning and realizing the Suttas, or even interactive educational games.

Namo Buddhaya!
First of all, I want to say a very big thank to the webmaster, who has created a wonderful Tipitaka page for Buddhist.
I’m now practice to make a small site about Theravada Buddhism, and in my posts need to quote some discourses. I’m not specialist about coding, and I want to asking a question: How to get only content of a discourse from sutta central and embed to my website, thanks !

Hi,

Looks like I can’t get JSON endpoint, because if I go to https://suttacentral.net/data?suttas=dn1,dn2 , I get 404 error code. I want to build some easy android app using Suttacentral JSON endpoint. Is there any docs about how to using consumable JSON endpoint on Suttacentral? Sorry for the irrelevant question.

1 Like

It’s been too long, sorry about that! Looks like we missed your comment the first time, but if you’re still around, we’re here.

There is a RESTful API, but I can’t really tell you much meaningful about it. Check the documentation here:

And hopefully @vimala or @blake can be of more use.

But what I can say is that we absolutely want to support such reusing of our material, and would love to help you if we can!

3 Likes

Hi,

Thank you for the reply. I’ve visited the Suttacentral Github pages, but I don’t really understand about what written in there. I’ve tried run command “make run dev” (maybe I’ll find some clue about API documentation /api/docs after I run “make run dev”) but the process consumes about 5 GB of my hard disk, and finished with error in the end :slight_smile: . But it’s okay because I uninstall the docker after that, and my 5 GB hard disk back to normal :slight_smile: .

The point is, I try to find RESTful API docs in Suttacentral. I believe that many people try to find the same Buddhist API like me. If Suttacentral actually already has RESTful API, could someone teach me how to use it? :slight_smile: Am I must clone the Suttacentral Github page to find the RESTful API?
Pardon me I’m beginner software developer. Sorry for my bad English.

1 Like

I just discussed with with our developers. They set the API up so that it did only work with a local environment. So if you want to set that up you can, maybe we can talk you through it. However I asked that the API be made available via the web interface as well, so he’s setting that up. I think that should be better for you, you can simply call the API on the web and get any updates automatically. That should be working in a day or two, I’ll let you know.

3 Likes

Whoaaah, thank you very much :grinning:. Suttacentral is very cool. I hope many people use Suttacentral API if the developer team open it to public via web interface in the future.

Thank you very much.

3 Likes

Hi everyone. I just wanted to check if there’s been any further progress in this domain? I’m tried the elastisearch links above as well as the JSON request links (what I’m looking for), but neither seem to be working. I see there’s an NPM package for an API, but I’m just looking to experiment and there doesn’t appear to be any docs for it yet.

Thanks!

Here are three wikis with the beginnings of documentation:

https://discourse.suttacentral.net/tag/api-documentation

If you find errors please feel free to update. And of course if you have questions, please post.

2 Likes