Upgrade to Bilara v2 together with bilara-data upgrade

Work on the v2 of Bilara is proceeding well, and we hope to have a working version by, say end of January.

Meanwhile, the work exposed a weakness of the current bilara-data specification, namely that we permit omitted segment IDs. This makes it harder to keep the different files in sync.

So the new requirement is that:

  • All versions of the same text must have the full set of segment IDs.

This is currently permitted but not required. The change will make it required.

Since this is simply a subset of what is already allowable, we do not expect any problems upgrading data. Nonetheless, we should do so cautiously and give developers time to test downstream applications, including:

  • SC
  • Bilara
  • Github actions
  • Publishing
  • Voice
  • anything else?

After discussion with the STXnext team, I propose we make the change like this.

stxnext

  1. Bilara v2 is developed using the new specification.
  2. STXnext devs are responsible for scripting the adaptation of the new spec.
    • The data in Bilara v2 dev is forked and therefore stale. STXnext devs will update any data before pushing to bilara-data.

SC devs testing

  1. Meanwhile, we will push one file in the next spec to bilara-data/published
    • say, dn1
  2. Developers test applications.
  3. Push a collection to bilara-data/published.
    • say, mn-mulapariyayavagga
  4. Developers test applications.

rollout

This will bring us to say mid-Jan. Let us accept Bilara v2 as ready for production when it is stable and has feature parity with old Bilara, i.e. it does not need to be feature complete, just usable.

  1. When ready for rollout, we inform all devs and bilara users. sujato
  2. Roll out Bilara v2 on a Monday. stxnext
  3. Pause all use of Bilara for at least 48 hours. sujato
  4. Upgrade bilara-data both unpublished and published to new spec. This pushes the new data spec to production. stxnext
  5. Launch Bilara v2 to bilara.suttacentral.net, test that everything works. stxnext
  6. Devs test applications one more time. hongda karl sabbamitta sujato anyone else!
  7. If all checks out, rescind pause and let people start using new Bilara. sujato
  8. Gambol with newborn lambs among the daisies :dancer: :sheep: :sunflower:

@hongda @karl_lew @sabbamitta @Snowbird

9 Likes

So for the internationalization files currently they can include just some of the key value pairs. But after this they will require all key/values with blank values for all yet untranslated segments? And do I understand that step 4 is going to take care of doing that?

Congratulations on all the work!

5 Likes

That’s correct, every version of the same file—whether it is different translations, markup, refer4ences, anything at all—will have the exact same set of keys, and null values for empty segments.

Indeed, our fine friends at STXnext will do all this, we just have to test it.

I’m guessing this will affect your apps too?

5 Likes

I doubt it. As it is I’m always expecting to have empty segments. And I think I often go off of the keys of the English rather than the array of ordered segment names. Which is probably a bad idea.

In any case most of my apps reuse the same basic code so it shouldn’t be too hard to fix. (I know, famous last words) I might even get motivated to actually make it actual reused code in case something like this comes up again. It’s good for me to revisit the code from time to time anyway.

Most of the missing segments are probably in the variation and other offset files, eh? So far I don’t think I’m using them.

5 Likes

I think Voice apps will be fine. We have code that merges non-aligned segments. After the upgrade, we could delete that code for performance improvements.

4 Likes

That’s correct, but there are a non-trivial number in other places, over a thousand inserted headings in the Vinaya being one example.

Good to know.

3 Likes

If it wasn’t too difficult, it would be great to have a few examples of texts where there were changes so they could be specifically tested.

3 Likes

Maybe we’ll add a Vinaya text too, that’s where the extra headings are.

4 Likes

That’s why we needed the merge code. Thank you for normalizing this.

1 Like

Bhante @Sujato, I’m wondering if in Bilara v2 translators will be notified if a root segment they have translated changes.

We recently had the case where there was a typo in an English interface segment. I had to go back and manually notify translators of the change. Of course if the work was done outside of Bilara there’s nothing to be done. However it would be great for them to be notified, since editing root segments is one of the new features (as I understand it).

Let me know if you want me to write up an issue. I also realize it may be too late for this.

2 Likes

There is an issue for notification about changes in an existing translation on which a translator relies, but I am not sure about changes in the root text. These happen less often obviously, but they do happen. (Entire roadmap here.)

2 Likes

Perfect. I have mentioned it there. Thanks!

2 Likes