[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [GMG-Devel] Plugin for page view counter

From: will kahn-greene
Subject: Re: [GMG-Devel] Plugin for page view counter
Date: Mon, 14 Jan 2013 15:26:44 -0500
User-agent: Mozilla/5.0 (X11; Linux i686; rv:18.0) Gecko/20121128 Thunderbird/18.0

I think there's a misunderstanding here. I wasn't suggesting we use Redis (or some equivalent) to keep tack of each +1 and then periodically go through all those +1 and individually write them to the db--that's dumb. Instead, I was suggesting we use Redis (or some equivalent) to keep track of the current total and periodically persisting the total to the database.

On 01/14/2013 03:26 PM, Christopher Allan Webber wrote:
That would probably be possible... it would require allowing plugins to
set up other connections like that (I guess would probably mean
registering some things in mg_globals if we were going to do things the
"right" way?).  We could probably also use celerybeat to occasionally
dump it to disk... I'm not sure how much work that is.

Related to the celery thing, I assume if we end up queueing up a number
of tasks in celery to increment, it's still the same number of writes to
do so, it's just that the writes are not happening at page load time...?
That does seem like it would avoid holding up a page from loading,
though if it's a "more writes at a time than the database can handle"
thing, I suppose that issue isn't going away in this solution.  But it
probably would at least not throw a bottleneck on page loading...

That's as far as I can think of this at the moment :)

will kahn-greene writes:

Or adding something that does fast atomic counting. I've seen Redis used
for that in a few places. Then you dump the results of that to the db
periodically so it's persisted. Not sure if there's anything in the GNU
MediaGoblin infrastructure already that can do this.


On 01/14/2013 12:06 PM, Christopher Allan Webber wrote:
Yes, you're right, celery is a better solution to processing these

Nathan Yergler writes:

I'd like to advocate for *not* doing a database write on a page view,
particularly not one that's going to be subject to a high degree of
lock contention on a higher traffic site. Write-on-GET should be a
signal to think about things carefully, as it imposes a limit on how
you can scale the software. If I recall correctly the stack still has
Celery in it; tossing a job into the queue and accepting some latency
with the stats isn't free, but it would reduce the amount of

Also keep in mind that either proxying the media file to trigger a
stats increment, or embedding it in the HTML surrounding the media,
mean you can't do any caching in front of the application: every
request needs to hit the application server.



On Mon, Jan 14, 2013 at 7:36 AM, Christopher Allan Webber
<address@hidden> wrote:
Tiberiu C. Turbureanu writes:

Thanks for the pointer, Chris!

I need a way to increment the view counter only when the media file is
loaded. How can I achieve this?

I'm not sure what you mean.  Do you mean:
   - only when the page is loaded
   - or something more complex: only if someone clicks "play" on a song or

The solution I was talking about involves number one.  As for number
two, that would be a lot more complex, but I imagine it could be
achieved via one of two things:
   - Easier solution: javascript callback on play that notifies the
     browser of playing starting.
   - Much, much trickier solution: you might be able to set up a proxy
     view before serving the actual video that first increments the file,
     but then tells the web server to send the video via X-Sendfile.  That
     sounds too complex to me though, and might be especially hard if a
     browser "seeks" the media to read just a few frames to get a preview
     image or do some preloading, which it might do in some

Also, Sebastian Spaeth told me that at the moment there is no way for a
plugin to alter the template of the media page, but a hook mechanism is
being discussed. That why until the implementation is ready, I think I
will go for a new page under a route like this

Spaetz is right, there's no present solution to this.  After I go
through my merge-a-thon this week though, I'll be switching onto some
plugin development, and will be looking into a solution to allow plugins
to extend pages more easily.

   - Chris
devel mailing list

devel mailing list

devel mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]