bug#25045: [Website] Packages page takes too long to load

[ng0]

Ludovic Courtès <address@hidden> writes:

> Luis Felipe López Acevedo <address@hidden> skribis:
>
>> On 2016-11-28 12:00, Alex Sassmannshausen wrote:
>>> Hi Luis,
>>>
>>> Indeed, I had a first bash at solving this problem by providing a
>>> set of
>>> static html pages paginated by the first letter of the package name.
>>>
>>> I'm not particularly wedded to this solution, so if you feel strongly
>>> about going another way, I'd be keen to hear/see about it.
>>>
>>> In the meantime, I'm open to testing/feedback.  I will unfortunately
>>> not
>>> be able to put work into this until at least Saturday/Sunday, as some
>>> Perl work has higher priority at the moment.
>>>
>>> But let me know if you have questions or feedback!
>>>
>>> Best wishes,
>>
>> Hi, Alex :)
>>
>> The solution I had in mind includes an alphabetic index, and
>> pagination as well. However, it includes a few more things, and would
>> take some time to design and implement. So I think we should use your
>> patch to fix the size issue as soon as possible.
>
> I agree!
>
> Alex, was there anything left to address?  If not, feel free to push.
> :-)
>
>> For what is worth, this is what I had in mind:
>>
>> /packages/
>> First page of the list of packages. Packages listed here only provide
>> a summary: package logo (if any), name, version, description, and an
>> indicator if it has issues. This page also has filters to find
>> packages (for now, alphabetic filter. In the future, category filter,
>> and search box).
>>
>> /packages/page/N/
>> Page N of the list of packages.
>>
>> /packages/a/
>> First page of the list of packages whose name starts with A. Packages
>> are presented the same way as in /packages/.
>>
>> /packages/a/page/N/
>> Page N of the list of packages starting with letter A.
>>
>> /packages/icecat/X.Y.Z/
>> Page with details about IceCat version X.Y.Z. It includes all the
>> information about this package, including its issues (which are
>> currently listed in a separate page along with the issues of other
>> packages: /packages/issues.html). Including the issues of a package in
>> its detail page could avoid having the current issues page grow too
>> much, like the current Packages page.
>>
>> This static pagination and filtering would generate A LOT of pages,
>> but of reasonable size for web browsers to load.
>
> Sounds like a good plan as well, though that’s indeed a lot of web pages
> for that rusty CVS repo to handle…

If I remember correctly, Gnu projects do not have to host their
websites on gnu.org, if the CVS should be the blocker to move
forwards and stay static.
But with every package and every version of them we add, this
would add a lot of generated pages (right now around 4550
packages and rising), so it might get difficult to manage that
unless management for this in place somehow (gets cleared on
reboot, make clean, or something similar).

> Medium-term, I think we should consider a solution involving pages
> generated on the fly server-side, with a caching proxy (nginx!) in front
> of it.  We’ll have to seek assistance from the gnu.org web masters, but
> ISTR they were not against that idea.
>
> Thanks!
>
> Ludo’.
>
>
>
>

-- 
♥Ⓐ  ng0  | ng0.chaosnet.org