Re: [GMG-Devel] Plugins plans

[Christopher Allan Webber]

Heya Tumulte,

Thanks very much for this email.  I've been wanting to extend things to
properly handle plugins, but really we honestly needed a good list of
use cases to properly think it through.  So this is helpful!  I'll make
some comments inline on how I think we can do things and what needs to
be done still.

Tumulte Dogma writes:

> Hey !
>
> I'm currently working on some plugins, and it seems that I keep asking
> for features that doesn't exist or that are not documented. Therefore
> paroneayea asked me to make a list of what I'm planning to do so he can
> have an idea of what to add to plugins (while I can have an idea of how
> to make them the MG way !)
>
> So here you go :
>
> - Adding informations to medias. Namely, genre, composer, musicians...
> But it doesn't really matter. It's easy to add fields to the submit
> form... but there's no views to treat the collected data of the new
> forms (should be possible to extend/override views in the plugins)
>
>         - Important : I want to be able to add fields using several
>         plugins. For instance, the "Genre" plugin can be useful for
>         others, so I'll make a specific plugin for it. On the other
>         hand, we have specific fields that are for us only... I think
>         template hooks are good when you want to extend a feature (e.g.
>         the submit form) instead of overriding them. We should have
>         hooks cleverly placed and a list of them.

So yeah, this is a good point.  You've got two requests here: Adding new
fields to a form, and handling processing them and doing things with
them.  Let's handle these both in turn.

Adding fields to a form
-----------------------

So, there are two ways to add fields to a form: you can just tack them
in the template underneath the form rendering manually as actual html
<input /> elements, or you can add them to the form object via WTForms.

Adding to WTForms is the ideal case.  If we could somehow attach fields
to the form, then that would be useful both for display as well as
validation and cleaning of submitted input.  (If validation is taken
care of here, it also allows us to skip another step...)

The problem is that if I remember correctly, this isn't easy.  The class
style API WTForms uses means that there's something complicated about
it, like you have to use metaclassing or etc.  But it would be good to
provide a generic method of updating forms...?

Other downside: you can't control the order of things tacked onto the
form, though that's true for any of the template_hook stuff we have
also.  (You can control it a little by affecting the order of which
plugin is loaded when.)  I don't know any clean solution to this, though
there might be one.  In the meanwhile I'm not horribly worried about it.
(The "controlling ordering" issue will come up multiple times in this
thread, and my attitude for now is mostly going to be the same thing.)

There might also be a question of "well what if you want to remove a
field entirely?"  That seems really tricky, so I don't know how to deal
with that.  I think in such a case you might have to override a view
entirely.

Doing stuff with form data
--------------------------

Okay, so you have your form data.  Now you want to do something extra
with it!

I think the callback hooks we have in the pluginapi should actually work
here.  If we iterate through a list of functions and pass in the request
and processed form, that should probably be good.

One question: what to do if a plugin runs into something they need to
inform the above function is a serious error?  The one thing I can think
of is maybe to throw an exception.  Other than that, not sure.

> - Multi upload : (in progress) not much to say, except  for the need to
> add several javascripts. Now, we need to override the head... And I
> think it's rather tedious. I opted for a manual list in mediagoblin.ini.

So you're right that overriding the head is not great.  However, I think
maybe the way that the new template hook approach works is good.  In
fact, this is now used by the new geolocation plugin, and I think it's
great.

There's a new hook, so in the "image" media type template, you see that
the head is extended:

>From mediagoblin/templates/mediagoblin/media_displays/image.html:

===============
{% extends 'mediagoblin/user_pages/media.html' %}

{% block mediagoblin_head %}
  {{ super() }}
  {% template_hook("image_extrahead") %}
{% endblock mediagoblin_head %}
===============

Note that if you look at base.html, you see this is putting things in
the head.

Then, in the plugin setup sets up a page that allows for setting up
extra javascript:

>From mediagoblin/plugins/geolocation/__init__.py:

===============
def setup_plugin():
    # Register the template path.
    pluginapi.register_template_path(os.path.join(PLUGIN_DIR, 'templates'))

    pluginapi.register_template_hooks(
        {"image_sideinfo": "mediagoblin/plugins/geolocation/map.html",
         "image_extrahead": "mediagoblin/plugins/geolocation/map_js_head.html"})
===============

Note the image_extrahead template hook being registered; this says to
lead map_js_head, which puts things in the head.

So let's look at that template:

>From 
>mediagoblin/plugins/geolocation/templates/mediagoblin/plugins/geolocation/map_js_head.html
(Yes, I know, owch!  That's a long path.)

===============
<link rel="stylesheet"
      href="{{ request.staticdirect('/extlib/leaflet/leaflet.css') }}" />

<script type="text/javascript"
        src="{{ request.staticdirect('/extlib/leaflet/leaflet.js') }}"></script>
<script type="text/javascript"
        src="{{ request.staticdirect('/js/geolocation-map.js') }}"></script>
=====================

And so, the plugin's stuff is put at the top on here.

So this is maybe a bit of work, but the reason I like this better than
putting it in the config file is several things:
 - It makes use of the mechanisms we have for templating, and I really
   want to advocate using template things for template things.
 - It makes it so that this shows up *only* on the image page!  The
   leaflet css and js files don't need to show up on every page... by
   using the template mechanisms we have, we can take advantage of this
   kind of granularity without a lot of work.

I don't really like the config approach as much because it doesn't allow
for this granularity (at least not without reproducing a lot of the
above) and it means that you have to ask users to add lines manually to
a section of a config file every time they install a plugin.. that's not
helpful, and means they're stuck if the plugin gets upgraded.

> - Album view : It uses the colection system but it's called "Album". The
> main difference is that you can play files in a playlist rather than one
> after the other.

Sounds super useful!  I think this should already be possible since
you're able to add new views mapped to urls.

> -Play all : as an extension of the previous line : I'll add the ability
> to play all audio files of a user from the user's page. 

Cool!  Likewise I assume this is possible with what we have?

> - Users group (?) : Don't know how to call it, but basically, it comes
> from the need to have labels that'd manage several artists (users).
> Basically it means that a user can create "sub-users". His page will
> display all sub-users' collections and medias.  The owner of the user
> group will be optionally allowed to add/edit/remove medias for users of
> his group.
> Aside from my personal need, I think that'd be awesome for people who
> have multiple activities and want to order them. For instance, Bobby
> would be allowed to create : Bobby-photos, Bobby-Music,
> Bobby-patato-soup-modeling. 

Ah okay... so that's interesting.  I assume this is possible, though it
does raise a lot of complicated questions about user management.
I'd like to understand better what "sub-users" means.

I assume you still need some extensions to the way logins/authentication
work.... that's the one thing I think is clear, and that we want to
handle already.

> And there's more : 
> For now, having the DB set right is my priority (cause it's the most
> tedious thing to change afterward), the first point is the most 
> important.
> However, soon enough I'll have to build a page to summarize and organize
> user's medias (genres, tags, location...) so I'm gonna have to access
> user medias and display them. After that I'll have to build an API so
> people can embed albums and such elsewhere...

I still think such pages are possible under the systems we have now.

> Annnnd, that's all for today. 
>
> Looking forward to your feedbacks,
>
> cheers,
>
> E.:.T 

Great!  And I'm going to try to transform this and other bits into a
document tomorrow related to plugins plans.

 - Chris