Re: [GMG-Devel] Plugin for page view counter

[Christopher Allan Webber]

That would probably be possible... it would require allowing plugins to
set up other connections like that (I guess would probably mean
registering some things in mg_globals if we were going to do things the
"right" way?).  We could probably also use celerybeat to occasionally
dump it to disk... I'm not sure how much work that is.

Related to the celery thing, I assume if we end up queueing up a number
of tasks in celery to increment, it's still the same number of writes to
do so, it's just that the writes are not happening at page load time...?
That does seem like it would avoid holding up a page from loading,
though if it's a "more writes at a time than the database can handle"
thing, I suppose that issue isn't going away in this solution.  But it
probably would at least not throw a bottleneck on page loading...

That's as far as I can think of this at the moment :)

will kahn-greene writes:

> Or adding something that does fast atomic counting. I've seen Redis used 
> for that in a few places. Then you dump the results of that to the db 
> periodically so it's persisted. Not sure if there's anything in the GNU 
> MediaGoblin infrastructure already that can do this.
>
> /will
>
>
> On 01/14/2013 12:06 PM, Christopher Allan Webber wrote:
>> Yes, you're right, celery is a better solution to processing these
>> things!
>>
>> Nathan Yergler writes:
>>
>>> I'd like to advocate for *not* doing a database write on a page view,
>>> particularly not one that's going to be subject to a high degree of
>>> lock contention on a higher traffic site. Write-on-GET should be a
>>> signal to think about things carefully, as it imposes a limit on how
>>> you can scale the software. If I recall correctly the stack still has
>>> Celery in it; tossing a job into the queue and accepting some latency
>>> with the stats isn't free, but it would reduce the amount of
>>> contention.
>>>
>>> Also keep in mind that either proxying the media file to trigger a
>>> stats increment, or embedding it in the HTML surrounding the media,
>>> mean you can't do any caching in front of the application: every
>>> request needs to hit the application server.
>>>
>>> Best,
>>>
>>> NRY
>>>
>>> On Mon, Jan 14, 2013 at 7:36 AM, Christopher Allan Webber
>>> <address@hidden> wrote:
>>>> Tiberiu C. Turbureanu writes:
>>>>
>>>>> Thanks for the pointer, Chris!
>>>>>
>>>>> I need a way to increment the view counter only when the media file is
>>>>> loaded. How can I achieve this?
>>>>
>>>> I'm not sure what you mean.  Do you mean:
>>>>   - only when the page is loaded
>>>>   - or something more complex: only if someone clicks "play" on a song or
>>>>     video?
>>>>
>>>> The solution I was talking about involves number one.  As for number
>>>> two, that would be a lot more complex, but I imagine it could be
>>>> achieved via one of two things:
>>>>   - Easier solution: javascript callback on play that notifies the
>>>>     browser of playing starting.
>>>>   - Much, much trickier solution: you might be able to set up a proxy
>>>>     view before serving the actual video that first increments the file,
>>>>     but then tells the web server to send the video via X-Sendfile.  That
>>>>     sounds too complex to me though, and might be especially hard if a
>>>>     browser "seeks" the media to read just a few frames to get a preview
>>>>     image or do some preloading, which it might do in some
>>>>     configurations.
>>>>
>>>>> Also, Sebastian Spaeth told me that at the moment there is no way for a
>>>>> plugin to alter the template of the media page, but a hook mechanism is
>>>>> being discussed. That why until the implementation is ready, I think I
>>>>> will go for a new page under a route like this
>>>>> /u/<string:user>/m/<string:media>/viewcount.
>>>>
>>>> Spaetz is right, there's no present solution to this.  After I go
>>>> through my merge-a-thon this week though, I'll be switching onto some
>>>> plugin development, and will be looking into a solution to allow plugins
>>>> to extend pages more easily.
>>>>
>>>>   - Chris
>>>> _______________________________________________
>>>> devel mailing list
>>>> address@hidden
>>>> http://lists.mediagoblin.org/listinfo/devel
>>
>> _______________________________________________
>> devel mailing list
>> address@hidden
>> http://lists.mediagoblin.org/listinfo/devel
>>
> _______________________________________________
> devel mailing list
> address@hidden
> http://lists.mediagoblin.org/listinfo/devel