My idea is to create document embeddings from OpenAI’s API, then upload these embeddings to Pinecone. The purpose of this little app is to search for suttas. I’m not sure what the exactness of this search is, I invite people to test it. I created vectors for the suttas manually.
I wrote my code using GPT-4. I manually indexed the sutta, so I only uploaded a small portion of it.
Update: I don’t know why my app can’t be accessed by clicking directly on the link. If you want to try, copy the link and paste it into the browser.
Or just click Reload.
I’ve just tried it. Gives too many suggestions.
Search box is too small for detailed input.
Yes, it’s indeed returning too many results. Actually, this is a list of all the sutras I’ve indexed. You should only care about the results with the highest score. In the future, I may limit the number of results. Anyway, this doesn’t affect the search.
My prompt: buddha tells someone that his idea of practice means people who are deaf and blind would be doing well spiritually
Expectation: App suggests MN152
Result: 26 suggestions, none of which is MN152
I just index MN152. Now test your search again to see the result
You already know that it works now.
What have you indexed so far?
Btw, I suggest you rename the title of this thread as Sutta searching AI app. That should attract more attention to this worthy project.
Yes, as someone unfamiliar with the technologies you are using it’s not clear really what makes this something different than a regular search. And I really don’t understand the “create your own” bit.
Here is my code if someone wants to know how I made this app:
Not much. I’m thinking of a way to index all the sutta faster. OpenAI only supports about an 8k token maximum, so some sutta will need to be split.
Now I’m automatically indexing all the sutta. The script is in my repository for reference.
I admire the complexity of your code. Admittedly, I understand very little apart from (if / else) basic rules of logic.
I know how difficult it can be. I essentially taught myself some basic C# to pursue a gaming project on Unity. After getting the physics of the player the way I wanted it. I struggled with the game logic.
Thankfully I met an extremely talented Unity game dev whom I’ve been collaborating with for the last six months.
I look at some of his work and it reminds me of yours - extremely complex and beautiful in that it asks the computer to perform unique tasks it likely has never encountered … and it’s all YOUR brain child!
Incidentally, I couldn’t access the search engine. Have you taken it down?
No, I have not. Just copy paste the link to the browser. Now I’m indexing all the sutta. Code was written using GPT-4, I don’t think too much
1071 files left. I downloaded all these files from suttacentral github.
I think I can finish indexing tomorrow.
Aha! I almost forgot that AI can help us code now.
Lol! Well if anything it makes me feel a million times better about my very minimal C# knowledge!
Lol again. Good work anyways. I’ll copy pasta the url when I get a chance
It’s not working anymore. My attempts produce an error message that begins like this:
pinecone.core.exceptions.PineconeProtocolError: This app has encountered an error.