[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

PSPP-BUG: [bug #17241] Performance optimizations for procedures

From: Ben Pfaff
Subject: PSPP-BUG: [bug #17241] Performance optimizations for procedures
Date: Sun, 30 Jul 2006 19:21:42 -0700
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv: Gecko/20060406 Firefox/ (Debian-1.5.dfsg+


                 Summary: Performance optimizations for procedures
                 Project: PSPP
            Submitted by: blp
            Submitted on: Sunday 07/30/06 at 19:21
                Category: Other
                Severity: 1 - Wishlist
                  Status: None
             Assigned to: blp
             Open/Closed: Open
                 Release: None
                  Effort: 0.00



These are the realistic part of a more ambitious idea for performance
optimizations that I posted earlier to the pspp-dev mailing list.

1. Eliminate the need to cache procedure output.  Currently,
   every procedure reads the entire active file and writes out a
   new copy of it.  For big data sets that's often a huge waste
   of time and space.

   This was almost impossible to get right before I refactored
   the code for implementing procedures.  Now, it should only be
   a few days of work.

2. Implement an optimizer for transformations.  SELECT IF can
   often be moved earlier without changing any semantics.
   Creation of variables whose values are never used can be
   dropped entirely.  I suspect that this is especially valuable
   after #1 is implemented because #1 will retain any
   non-temporary transformations from one procedure to the next
   for re-execution, whereas currently that does not happen.

   Doing a basic job of this should be a few days of work.  It's
   probably not worth putting more into it than that (and maybe
   not that much) unless it is demonstrably useful.

2a. Implement caching of data between procedures.


Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]