Re: [GMG-Devel] Deprecating sqlite

[Boris Bobrov]

Hi,

On 05/08/2017 06:06 AM, Christopher Allan Webber wrote:
> Hello!
> 
> So, I've begun looking and planning at my jumping back into MediaGoblin
> development, and one thing that's been on my TODO list is raising
> deprecating sqlite and going postgres-only.  This is something that
> Jessica Tallon and I talked about quite a bit, and I said I'd send an
> email, but oops.
> 
> There are two main reasons to do it:
> 
>  - sqlite is *the worst* when it comes to migrations.
>    For more info:
>      https://dustycloud.org/blog/sqlite-alter-pain/
>    Probably more time has been sunk into fixing broken sqlite migrations
>    than anything else in MediaGoblin's development.
>    It's a huge timesink and painpoint.  Nobody likes working on these.
> 
>  - There are some big features we could really take advantage of if we
>    went postgres-only.  Number one being jsonb support probably.
>    This could end up improving our ability to adopt ActivityPub
>    considerably (it's not a blocker on it maybe, but it could mean a
>    dramatically cleaner implementation).
> 
> As to why not to do it:
> 
>  - Some people are already using it.  We could add scripts to help them
>    move away though... we already did this once before with MongoDB ->
>    SQL.
>  - It's easy to administer as a *user* (even if it's the very opposite
>    for developers).
>  - It's easy to set up and uses very few resources.
>  - It especially makes unit tests and etc very easy to do.
> 
> The last one is probably the most compelling of all of them to me
> personally.  We need a decent way to start up / tear down databases for
> testing.  (It may be possible to write some scripts that can start up /
> tear down a temporary postgres cluster though...)  It's possible that we
> could keep sqlite for a "development mode only", at least in some
> interim, though it means still not having access to postres-only
> features.  My feeling is, we should probably do this first.  If we do
> that, we don't have to write migrations for sqlite anymore, at least.

Testing with sqlite is a mess. There are 2 types of sqlite -- on-file
and in-memory, and they are very very different.

> Anyway, feedback welcome.  Personally, I think it would be a good move
> towards this.
> 
> But, it's a pretty big change, so here's opening up conversation...

Yes, lets drop sqlite.

I suggest the following plan:
1. Drop support for in-file sqlite. Just don't advertise it anywhere.
2. Write something for bootstrapping and tearing down databases. I
think we could wrap py.test with something that would, using username
and password from env variables create a temporary database and user,
run tests with it, tear down the database and user. Temporary database
name should be unique and auto-generated.
3. Switch to postgres-only testing