Bilara-data folder structure proposal

Alright, so checking the folders only, not the file content.

  • Seem good: root, translation, reference, comment, variant
  • HTML: kv is duplicated in both sutta and abhidhamma (it belongs in abhidhamma)

I’m pushing a couple of minor changes to the metadata.

3 Likes

Thank you, Bhante. This is a great relief.

I deleted sutta/kv since it was older.

I will now start updating Voice using the sutta-vinaya-abhidhamma branch.

NOTE: my head is exploding slightly and to simplify matters, I’ve merged in Anagarika @Sabbamitta’s German translations as well as @Kaz’s Japanese translations into the sutta-vinaya-abhidhamma branch. The Voice automated tests rely on this content. The merged content is in the new folder format.

3 Likes

Okay, great. Now, I will check things more thoroughly, but we need some testing and so on for the next steps.

I don’t want to explode your head further, but on the 2-do, once all the Pali texts are present and accounted for:

  • run text-content diff of bilara pali texts vs. original mahasangiti files, to make sure we haven’t accidentally lost any texts. When to do this? It should definitely be done as a final thing, but maybe do interim tests as well?
  • Redo segment numbering of previously-segmented texts to ensure consistency across the whole corpus. This will mainly affect things like headings.
  • Merge in reference data from all_concordance.

Shall we have a skype to chat about these?

3 Likes

We should have issues to track each of these until I can tackle them. Right now Voice is on the operating table and I’m performing a bit of brain surgery, moving slowly and carefully. After that, I’ll need to sit down with Anagarika Sabbamitta to address our delayed release. We’ve delayed all our releases substantially to accommodate the bilara-data imports and restructuring. If we have one issue for each of the above then we can slowly integrate progress across all the issues. Each one of the issues will require a chat to understand what needs to be done in detail.

BTW. Just chatted with Anagarika Sabbamitta. We both recommend strongly that these three issues should be done AFTER the merge of sutta-vinaya-abhidhamma into master. The bilara-data folder structure divergence should be minimized and the merge should completed as soon as @Blake and I are ready. I plan on being done with Voice software changes next week. Blake, let us know if you can’t complete your own software changes by next week. There’s a fair amount of testing that will need to be done in staging as well beyond that software change.

3 Likes

Update 2/16: scv-bilara v1.2.0 is now available. The scv-bilara library is the search engine used by Voice. The v1.2.0 version automatically detects bilara-data folder structure and is happy with both new and old folder structures. Starting tomorrow I’ll integrate the latest scv-bilara into Voice itself for testing in our staging environment next week. Once that testing is completed Voice will be ready for merging sutta-vinaya-abhidhamma branch into bilara-data master.

@Blake, please advise if you need any more time to complete your own transition for the sutta-vinaya-abhidhamma folder structure. We might be able to merge the new folder structure into bilara-data as early as next week.

@Sabbamitta, large earthquake merge approaching! Date is not fixed but we are discussing merge date. Please advise if you need more time to merge all your own translation branches into master.

@Kaz, this won’t affect you since you’re essentially working sutta by sutta offline.

@Sujato, the import-pali branch is actually the basis of sutta-vinaya-abhidhamma branch. Please make any changes to sutta-vinaya-abhidhamma branch instead of import-pali branch, which will eventually be deleted.

4 Likes

2 posts were merged into an existing topic: There’s blank pages where there weren’t any before

Thanks for the heads up! No, I don’t need extra time to merge translation branches. I am not working on more than one branch at a time right now, and the suttas in this section are all fairly short, so there is no problem.

2 Likes

Update 2/17: Voice v2.1.21 is ready for staging. This release of Voice works with either bilara-data folder structure.

At this point I would like to propose that we merge sutta-vinaya-abhidhamma branch of bilara-data into master on Wednesday 2/19. Please advise if this merge would disrupt anything. The merge will change all existing content in root, translation and other folders. ATTN: @blake, @sujato, @Brahmali, @sabbamitta

4 Likes

Fine with me!

1 Like

Thanks, Anagarika! I’m now looking into bringing over all new content from the old folders into the new folders…

1 Like

That means I will stop editing existing content or adding new content from now on until we’re done, I guess.

1 Like

Ah yes! Thanks for the reminder. Please also close out any branches that you may have open.

Ajahn @Brahmali, I am less familiar with your authoring paradigm, but similar concerns will apply. The destination for all your edits is changing. To avoid content loss, please keep your changes offline until we have completed the merge. Also make a record of recent changes so that we can ensure that the merge is successful. Any new content of yours must appear in sutta-vinaya-abhidhamma branch or might be lost.

3 Likes

There’s one open. I guess I’ll be done with that in about 30 min. I’ll still publish that one and let you know when done.

1 Like

I am currently merging master content into sutta-vinaya-abhidhamma. Please review that these are correctly named and placed. We should all make necessary corrections in the sutta-vinaya-abhidhamma branch:

@sujato @brahamali I noticed a lot of changes in root files:

924 pli-tv-bi-vb-pc56_root-pli-ms.json differ
925 pli-tv-bi-vb-pc93_root-pli-ms.json differ
926 pli-tv-bi-vb-pj6_root-pli-ms.json differ
927 pli-tv-bi-vb-pj7_root-pli-ms.json differ
928 pli-tv-bi-vb-sk1-75_root-pli-ms.json differ
929 pli-tv-bi-vb-ss1_root-pli-ms.json differ
930 pli-tv-bi-vb-ss4_root-pli-ms.json differ
931 pli-tv-bu-vb-np16_root-pli-ms.json differ
932 pli-tv-bu-vb-np22_root-pli-ms.json differ
933 pli-tv-bu-vb-np24_root-pli-ms.json differ
934 pli-tv-bu-vb-np29_root-pli-ms.json differ
935 pli-tv-bu-vb-np3_root-pli-ms.json differ
936 pli-tv-bu-vb-np5_root-pli-ms.json differ
937 pli-tv-bu-vb-pc10_root-pli-ms.json differ
938 pli-tv-bu-vb-pc12_root-pli-ms.json differ
939 pli-tv-bu-vb-pc21_root-pli-ms.json differ
940 pli-tv-bu-vb-pc22_root-pli-ms.json differ
941 pli-tv-bu-vb-pc2_root-pli-ms.json differ
942 pli-tv-bu-vb-pc34_root-pli-ms.json differ
943 pli-tv-bu-vb-pc39_root-pli-ms.json differ
944 pli-tv-bu-vb-pc3_root-pli-ms.json differ
945 pli-tv-bu-vb-pc43_root-pli-ms.json differ
946 pli-tv-bu-vb-pc44_root-pli-ms.json differ
947 pli-tv-bu-vb-pc48_root-pli-ms.json differ
948 pli-tv-bu-vb-pc58_root-pli-ms.json differ
949 pli-tv-bu-vb-pc5_root-pli-ms.json differ
950 pli-tv-bu-vb-pc64_root-pli-ms.json differ
951 pli-tv-bu-vb-pc68_root-pli-ms.json differ
952 pli-tv-bu-vb-pc6_root-pli-ms.json differ
953 pli-tv-bu-vb-pc7_root-pli-ms.json differ
954 pli-tv-bu-vb-pc81_root-pli-ms.json differ
955 pli-tv-bu-vb-pc84_root-pli-ms.json differ
956 pli-tv-bu-vb-pc9_root-pli-ms.json differ
957 pli-tv-bu-vb-pj1_root-pli-ms.json differ
958 pli-tv-bu-vb-pj2_root-pli-ms.json differ
959 pli-tv-bu-vb-pj4_root-pli-ms.json differ
960 pli-tv-bu-vb-sk14_root-pli-ms.json differ
961 pli-tv-bu-vb-sk15_root-pli-ms.json differ
962 pli-tv-bu-vb-sk17_root-pli-ms.json differ
963 pli-tv-bu-vb-sk19_root-pli-ms.json differ
964 pli-tv-bu-vb-sk21_root-pli-ms.json differ
965 pli-tv-bu-vb-sk23_root-pli-ms.json differ
966 pli-tv-bu-vb-sk25_root-pli-ms.json differ
967 pli-tv-bu-vb-sk27_root-pli-ms.json differ
968 pli-tv-bu-vb-sk28_root-pli-ms.json differ
969 pli-tv-bu-vb-sk30_root-pli-ms.json differ
970 pli-tv-bu-vb-sk32_root-pli-ms.json differ
971 pli-tv-bu-vb-sk38_root-pli-ms.json differ
972 pli-tv-bu-vb-sk41_root-pli-ms.json differ
973 pli-tv-bu-vb-sk42_root-pli-ms.json differ
974 pli-tv-bu-vb-sk43_root-pli-ms.json differ
975 pli-tv-bu-vb-sk49_root-pli-ms.json differ
976 pli-tv-bu-vb-sk50_root-pli-ms.json differ
977 pli-tv-bu-vb-sk51_root-pli-ms.json differ
978 pli-tv-bu-vb-sk52_root-pli-ms.json differ
979 pli-tv-bu-vb-sk54_root-pli-ms.json differ
980 pli-tv-bu-vb-sk57_root-pli-ms.json differ
981 pli-tv-bu-vb-sk58_root-pli-ms.json differ
982 pli-tv-bu-vb-sk59_root-pli-ms.json differ
983 pli-tv-bu-vb-sk60_root-pli-ms.json differ
984 pli-tv-bu-vb-sk61_root-pli-ms.json differ
985 pli-tv-bu-vb-sk62_root-pli-ms.json differ
986 pli-tv-bu-vb-sk63_root-pli-ms.json differ
988 pli-tv-bu-vb-sk65_root-pli-ms.json differ
989 pli-tv-bu-vb-sk68_root-pli-ms.json differ
990 pli-tv-bu-vb-sk69_root-pli-ms.json differ
991 pli-tv-bu-vb-sk70_root-pli-ms.json differ
992 pli-tv-bu-vb-sk71_root-pli-ms.json differ
993 pli-tv-bu-vb-sk72_root-pli-ms.json differ
994 pli-tv-bu-vb-sk73_root-pli-ms.json differ
995 pli-tv-bu-vb-ss1_root-pli-ms.json differ
996 pli-tv-bu-vb-ss6_root-pli-ms.json differ
997 pli-tv-bu-vb-ss7_root-pli-ms.json differ
998 pli-tv-kd11_root-pli-ms.json differ
999 pli-tv-kd9_root-pli-ms.json differ

These surprised me because they are root files from which Ajahn Brahmali may already have made translations. Examples of differences are:

2 27,28c27,28
3 < “pli-tv-bi-vb-pc56:2.1.5”: “Abhikkhuko nāma āvāso”,
4 < “pli-tv-bi-vb-pc56:2.1.6”: “na sakkā hoti ovādāya vā saṃvāsāya vā gantuṃ.”,
5 —
6 > “pli-tv-bi-vb-pc56:2.1.5”: “Abhikkhuko nāma”,
7 > “pli-tv-bi-vb-pc56:2.1.6”: “āvāso na sakkā hoti ovādāya vā saṃvāsāya vā gantuṃ.”,
8 37c37
9 < }

What action is required?
sva-diff.zip (8.1 KB)

3 Likes

AN3.101 done, merged and branch deleted. I’ll now keep my fingers away from your files.

1 Like

With the Vinaya files, keep the latest version used by Brahmali.

The changes are mostly to accommodate headings and the like. I have been coordinating these with Ven Brahmali, and we hope that they will not be too disruptive. The root of the problem is simply that when he began this translation, several years ago, we did it very differently and we have had to adapt across multiple systems. We won’t be able to really settle the texts down until the revision is finished. But anyway, this text is not published yet, so it will not affect the apps.

3 Likes

Hokay! Does this mean I can continue to make changes until Wednesday morning US west coast time?

So am I! My paradigm is to do stuff whenever I have the time. I will work around your schedule. It’s no problem for me.

Are you suggesting changes might get lost? I am not sure how I would make a record of all changes. And the question then is since when.

I did not translate from the segmented text, but from the original Mahāsaṅgīti document. Once the text was segmented, there was the occasional bad correspondence between my translations and the Pali segments. To fix this, I have sometimes had to adjust the Pali segments. Usually this has involved moving one or more Pali word from one segment to another. That’s all really. Entire segments have not been deleted. In fact, I am still occasionally making such changes.

4 Likes

I don’t think so, Venerable.

Karl has created the bilara-data branch called sutta-vinaya-abhidhamma which has the new folder structure, and copied all existing content from the old folders into this new branch. Since that time more edits have been done to the old folders, and in order to do the merge the new folders need to be updated by these edits.

If you keep making changes the updating process will never be finished. :smile:

I guess after the merge is done you will be able to make changes again. Until that time I too am restricting my work to my local working environment without touching the files in the GitHub repositories.

Probably this concerns all changes that have been made after the sutta-vinaya-abhidhamma branch has been created—I guess.

2 Likes

No problem, I can wait. I haven’t done any changes for quite a while anyway. Bilara has been down.

When did this happen?

Hmm, yes. I believe the Buddha warned against this.

3 Likes

Not entirely sure. A few days or a week ago, maximum two weeks I’d say.

2 Likes