Checkpoint

[Amirouche Boubekki]

Hello schemers,

tl;dr. Finally I accept that I am a megalomaniac

It's been a long time since I did not start a topic about the status
of my projects and plans.

Here is the two last topic I've started:

- Smarter cooperative scheduler (maybe fiber related)
  http://lists.gnu.org/archive/html/guile-user/2018-03/msg00024.html

- neon: git for structured data [Was: Functional database]
  http://lists.gnu.org/archive/html/guile-user/2018-02/msg00011.html

First, there is no more projects and plans. There will be one single
project and one single plan, at the very least it will appear as a
single project because there will be a single repository. The purpose
of the project is explained at the very end of the mail.

Basically, all my ideas converged and doing small experiments each on
their own repository doesn't deliver. I need to build and learn to 
refactor.

monorepo for the win like others have said [0] and used. It has been I
believe easing the progress of guix itself.  Also, my attempts at
building guile-wiredtiger as its own repository [1] did not help
enough people in my mind.  Let's write it down.  We agree that we
disagree that guile-wiredtiger is the way forward.  I won't try to
convince people again [1].  I agree It's not the conventional wisdom
aka. it is not mainstream until now in free software.  And only a
secret weapon, but it's not a secret anymore.  I won't try to argue
anymore against no feedback.  Because I agree with what I read in
between silent IRC channel full of join and leave ``Show me quality
code'', ``Build something useful'' and ``Scratch your own
itch''. That's what I have been doing, even if seemed erratic and
unpredictable.

[0] https://danluu.com/monorepo/
[1] search for wiredtiger @ http://hyperdev.fr/

My last project, neon is also known as zeheyu93600 was a moon shot.  Me,
Myself and I took upon the task of building versioned in a
Direct-Acyclic-Graph quad store!  WAT!  This was for me like an
``Eureka!'' kind of moment. The best all things seemed too obvious and
almost trivial.  I had picked a more difficult task that the defacto
standard functional database (defacto because there is only one).  But
with a dire consequence: the ability to apply git workflow to
virtually any work of knowledge. Long story short, I was catching up
on Audit Trail, Event Source and CQRS. And still, I see how versioned
database can be useful.

That being said, I don't think one should ``version all the things'', I
still, think multi-paradigm disk persistence (``multi-paradigm
database'' is the term that is catching up) is the best choice for the
foreseeable future. Any, all this poor data management. I will not add
a paragraph about the disaster of the king-god Model class as in
Model-View-Controller architecture, because the correct solution is
blurry to me, but for sure it doesn't involve a spagetti ORM code.

So I changed the topic. And looked into the serverless dubbed
revolution. That's how I came up with 'Smarter cooperative scheduler'
topic which then leads to fill the following issue against Celery,
a distributed background task queue scheduler, for help designing a
distributed single task based on priority and age [0].

[2] https://github.com/celery/celery/issues/4901

You already prolly agree with me but serverless is just a way to be
locked in to a specific platform, it's only useful when you don't know
better.  It's only a problem when you applied the "micro services"
dogma blindlessly and need an escape hatch too shy to admit the old
way of the monolith can work. It creates more dev and ops issues. How
do you think you test and develop those serverless based information
systems?

(By the way docker is a neat way to monitor and easily kill the
business of those who not do pay the bill. It's not necessary for
reproducible builds that is just a side effect of building an image
that embeds all the things. Again the winner is the big corp selling
software nobody wanted but is forced to use because ``industry
standards'', you know, catch the train before the train is too late.)

Then Apple free FoundationDB. Their promise is simple as distributed
consistent fault tolerant ACID multi-paradigm database thanks to an 
ordered
key-value store. Simply said, it's wiredtiger distributed. There's more
to say about the subject but I will stop here.

Anyway, how does this related to me and my project. What is that 
project.

Euraka! I am at the intersection of many ideas that look promising,
useful, liberating for the first billion of Internet users and the next.

The name of the project is 'Earth Software System'.

Because I don't know which application will be the revolution I will 
just
copy everything that is useful. I will start with (micro) blogging,
wiki, forum and real-time discussion.  To keep thing simple, I will not
support interop with other Internet protocols in the first versions. And
since the goal is to create a civilization bootstrap kit (code +
data), I don't plan to interop with legacy protocols. I want it to be
used today to build micro-communities hence I will rely on web
technologies supported by IceCat or any other ethical browser. I will
say that web browsers are too costly to bootstrap. GTK is a much
better path. The step after that is to rely on GTK.

I think a small set of powerful primitives are the way to go. I want
to prove two things:

a) ordered key-value stores are such a primitive
b) priority and age based distributed task queue is another

To my mind, the architecture is innovative, the applications are not.


NB: I don't have a solution for global warming


Note for future me: write about Korg paper and expert systems.