Re: [GNUnet-developers] Proposal: Make GNUnet Great Again?

[Schanzenbach, Martin]

> On 9. Feb 2019, at 20:32, Amirouche Boubekki <address@hidden> wrote:
> 
> I think splitting the codebase will be a pain for gnunet.
> 
> The only good reasons for manyrepos are social or ego politics "this is my 
> lawn" or legal. The only one that applies to gnunet is legal because one 
> needs to fill a gnu form to be able to contribute.
> 
> I am biased toward monorepo by experience dealing with big project (100k+ 
> SLOC) and the only time it made sens to split the project into many 
> repositories because it was different teams / workflow (social) and different 
> legal terms for the various services/daemons, at previous $WORK, they had to 
> fork gentoo to make it work.
> 
> Otherwise, each time I saw another repository it was a source of pain:
> 
> - Need to manage several versions

Why? Are you talking about packaging?

> - git submodule workflow is not good enough, it doesn't track branch, I 
> personally I never remember how to know the branch of a commit, plus it 
> requires some more git-fu to bump the submodule.

What? If you have a separate repo for e.g. the File-Sharing component of 
GNUnet. Then if you test it, you would pull the current, tested version of the 
GNUnet base (e.g. through the use of a docker image in gitlab-ci) and test your 
code.
In master/HEAD you do not care about versions and we certainly should not use 
git submodules for that.

> - refactoring anyone?

Refactoring what? Why would you refactor (often) between something like 
filesharing and the base gnunet components?

> - generally speaking manyrepos at small scale is more work
> 
> And again, it requires somehow to track down every versions (what works with 
> what) and you end up with another repository (or distribution) with another 
> build system that puts everything together. Continuous Integration can do 
> that? Where is the code of the CI? Another repo? More versions, more git 
> clone more grep across repositories / directories not even in sync.

I think you (and to some degree also grothoff) fundamentally misunderstand my 
argument. I am not proposing to put every component in
gnunet/src/ into a separate repo. That would be crazy.
I am proposing to separate components which have clearly disjunct use cases 
from the basic functionality and other use case components.
The arguments why this is reasonable are laid out by me in this thread from 
both a development and packaging/user perspective.

> 
> Popularity arguments:
> 
> a) Ok, everybody know GAFAM love monorepos and that is a also a source of 
> pain (dedicated team and software). That said, gnunet is not the size of any 
> GAFAM, hence it will not suffer from monorepo pain points.
> 

This actually reflects a core problem that hasn't even been discussed, yet: 
What is "gnunet"?
Is gnunet everything from file sharing to a social network to a digital 
identity management system? Or is gnunet a framework; a vehicle which allows us 
to build such applications. If it is the latter, then it should be 
compartmentalised as such as soon as enough applications are built. _I_ think 
this point has been reached.

> b) Github and Javascript made the manyrepos popular for various ego reasons 
> and because JavaScript is not good. I won't take inspiration from that part 
> of the JavaScript noosphere. gnunet-leftpad anyone?
> 
> c) Now, there is GNOME. GNOME is famous for its bazaar model of development 
> and also famous for the adoption of meson (maybe even its inception) or its 
> previous incarnation jhbuild. Anyway, even if GNOME and GNU (which is also a 
> bazaar) success is appealing, gnunet is not GNU or GNOME. From my point of 
> view the bazaar development model scales better / more easily in a socially 
> distributed setting. Also why Linux is still a single repository?
> 

We should not mix umbrella projects with actual products. Linux is a kernel. 
GNOME loosely is a software collection (project) and so is GNU.
I recently commented that some time ago, the gnunet.org homepage stated that 
gnunet was a "peer-to-peer framework" (It doesn't anymore).
But that is how I see gnunet. It is a framework. A platform. The base platform 
needs generic functionality (where we draw the line on what "basic" is is 
obviously a point of discussion). I am arguing that fs, social, reclaim are not 
basic functions such a platform needs (or devs need to build applications).
GNUnet can become an umbrella project as well if we can agree on that. Under 
this umbrella will exist: The core platform and any app/service that wishes to 
share the umbrella project resources (so atm all of them).

> Le sam. 9 févr. 2019 à 18:16, Schanzenbach, Martin <address@hidden> a écrit :
> 
> 
> > On 9. Feb 2019, at 17:13, Christian Grothoff <address@hidden> wrote:
> >
> > On 2/9/19 5:04 PM, Schanzenbach, Martin wrote:
> >> I have some inline comments as well below, but let us bring this 
> >> discussion down to a more practical consensus maybe.
> >> I think we are arguing too much in the extremes and that is not helpful. I 
> >> am not saying we should compartmentalise
> >> GNUnet into the tiniest possible components.
> >> It's just that I think it is becoming a bit bloated.
> >>
> >> That being said, _most_ of what is in GNUnet today is perfectly fine in a 
> >> single repo and package.
> >> For now, at least let us not add another one (gtk) as well?
> >>
> >> Then, we remain with
> >>
> >> - reclaim (+the things reclaim needs wrt libraries)
> >> - conversation (+X)
> >> - secureshare (+X)
> >> - fs (+X)
> >>
> >> as components/services on my personal "list".
> >> I suggest that _if_ I find the time, I could extract reclaim into a 
> >> separate repo as soon as we have a CI and I can
> >> test how it works and we can learn from the experience.
> >> Then, we can discuss if we want to do the same with other components, one 
> >> at a time, if there is consensus and a person that
> >> would be willing to take ownership (I am pretty sure we talked about this 
> >> concept last summer as well).
> >
> > Maybe you could start with extracting the SecuShare components? That
> > should do for a first "experience", and be a bit more effective at
> > reducing bloat as well ;-).
> 
> Well, I could, but our secushare people are quite active so maybe there are 
> volunteers (if they agree with the proposal at all).
> Regarding "bloat". If we want to effectively eliminate bloat than let's look 
> at numbers:
> 
> File Sharing:
> src/fs: 36918 (!) LOC in .c files
> src/datastore/cache: ~15k LOC in .c files
> 
> Conversation:
> src/conversation: 10538 LOC in .c files
> 
> SecuShare:
> src/psyc* : ~17000 LOC in .c files (altough I am not sure about this because 
> theoretically psyc is a general use protocol, no?)
> src/social: 9447 LOC in .c files
> src/multicast: 5633 LOC in .c files
> 
> Reclaim:
> src/reclaim* : ~6500 LOC in .c files
> 
> Now, considering that fs is practically always built for everybody and 
> SecuShare and reclaim are experimental, it hurts the most for devs that 
> actually compile from source.
> Everything combined are 110000+ LOC which is 22% of the codebase (~500k, oO). 
> Considering that there is a significant redundancy in transport/ (75k) at the 
> moment, this number is probably closer to 25%.
> Granted, this is a lot less than I expected ;), but maybe illustrates the 
> dimensions.
> 
> 
> >
> > That said, splitting of reclaim seems also much less problematic than
> > fs/conversation, and if you then integrate reclaim with the libgabe
> > tree, the overall number of downloads/installation for reclaim wouldn't
> > go up, so that would certainly kill my argument of making the
> > installation more complex (might indeed simplify it, as one doesn't have
> > to remember to install libgabe before GNUnet to get reclaim).
> 
> Could do, but libgabe has some nasty additional deps (libpbc and gmp) which 
> we _might_ eventually get rid of completely by implementing GNS-based 
> encryption.
> 
> 
> _______________________________________________
> GNUnet-developers mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/gnunet-developers

signature.asc
Description: Message signed with OpenPGP