internals

Warning! Doom! Gloom! This page explains why you should not copy implementing the chaining API, it may cause your life to become a dumpster fire of pain, misery, and suffering despite it being so seductively interesting. This page was copied from a chat and is not properly formatted:

Generally speaking I think it is a bad idea if people who come to use GUN get stuck in "how to recreate/build GUN" or "how GUN internals work" as that would mean GUN is failing to create a "separation of concern" - the classic trope: People don't want to know how a car engine works before they can drive it.

However when people are 🤯 by such amazing new possibilities that a car could unlock, if they are only used to horses, then they I think it is naturally good and curious they want to know how it works. And it would be good if that information was available upon them looking for it (not because it was thrown in their face, or even advertised heavily in the docs).

The way I've handled this, is I've historically put more time (of what I could) into docs on showing off what you can do with GUN and how. And left up the "well now how does it work?" to being very active in the community & chat over the last 6 years and answering all/any questions thoroughly.... even tho that information often doesn't get copied back to docs (actually, I just mentioned this in the main thread too, about people wanting to do Notion clones with GUN). It would be good if that got copied back to docs, however from a priority standpoint, the then "I need to clean this up, reformat, and make it perfect" is too much of an overhead that I personally don't do it myself (unless it is around a usability issue, versus a how it works), even if it means I repeat myself 42 times answering the same question over 6 years. Because actually, I believe, each time the same question is asked, it has a slightly different nuance or context.

That said there is 1 very important edge case: Porting GUN!

But this results in a similar trick - it would be natural for somebody, without my direction, to start porting GUN by starting with the API not the protocol. As this, as a developer, is... how we use a project! We don't use the protocol. BUT if they were to start with porting the API then 3 problems occur:

• (A) GUN actually isn't the API.
• (B) Lol, even I barely know how to implement the API so I don't even know how to explain it other than "this is how it should work" which is why we've discussed the "if we only had an AI that we could ML train it on the unit tests as a constraint, it could solve for the all the complexity inside." (tho just soul/prop chains are decently straightforward enough, but this gets into the next 3rd point...)
• (C) It probably takes 10X the time to port a simple/limited understandable version of the chaining API (like what go1dfish did, no .map and no recursive). This already is too much of an overhead "commitment" issue that will reduce the number of GUN protocol-only ports to other languages, and that is actually much more important at first.

Which as you can see from the docs, quite a few people have implemented the protocol, pretty fast. My mantra there is "Understand it all in 30min & have it working. Then optionally make it better in 3 hours if you want. And finally, take 3 days to make it so it could actually be usable with a real app."... 30min -> 3 hours -> 3 days.

only after all of that should somebody who is trying to Port GUN be like "Hmm, I now want to add an more user friendly API for this so I don't speak wire protocol directly." and this also has a bunch of caveats:

1. No, they should go and implement fully things like DAM, SEA, RAD etc.
2. No, they should decide to make a thin wrapper around the wire protocol as their 1st API instead.
3. No, even now, they should implement a language specific API that matches the norms of the language they are in, that uses (2) underneath.
4. OK, finally, if you also want to add a chaining API to it, start with the basic understandable version like go1dfish did, with extremely limited functionality.
5. Do you really need to go past this? Probably not. Fine, ruin your life trying to implement a recursive mappable chaining API. Oh, BTW, this is possible after thousands of hours of pain, misery, and fire - that was the easy part, now you're going to discover it might finally work mostly correctly but the performance is terrible!!! Now rewrite it again and spend 100X time ripping your hair out getting stuck in unit test cases that alternate working/failing or failing/working after you think you understand what is going on after the millionth time grasping it - but wait, isn't the API simple & so elegant? So shouldn't it be easy to grasp after millions of thoughts on it?

So the neat thing, is if we actually perfectly grasp the chaining APi this time (the 7th time I've rewritten it) and it somehow magically happens to be bug free and performant, then I feel like wow maybe it will be at the point where it is worthwhile to explain for our sake & sanity. But (see above), despite feeling like we've conquered Mt Everest, and feeling enlightened, and wanting to brag about how cool this is... there is still very little value in explaining it publicly since it would only distract from protocol & alternative API implementations (who knows, maybe SQL is actually a superior API query language, but we became so obsessed with this "challenge" of the chaining API, we never bothered to try anything else), since GUN is GUN, not the chaining system. Tho yes, since I believe in Open Source, if creating those docs is worth it for ourselves privately after we know it is correct & the new version is out & it doesn't take significant time distracting away from the next most important items, like the DHT... then might as well make it public but still keep it "buried" so people have to ask or search for it first.