Re: [lmi] testing actuarial_table performance

[Vaclav Slavik]

Hi,

On 2012-05-27 19:40, Greg Chicares wrote:
> Here's a very direct way: do File | New | Illustration | OK and look
> at the statusbar. The "Calculate:" time is about thirty percent 
> longer with xml instead of binary.

Surprisingly, it's only ~8 % for me (using debug build with Visual C++;
average of 2nd and 3rd run because the 1st loads the database;
LMI_USE_XML_TABLES set and actuarial_table_rates*() functions modified
to load just one version,
i.e. use the "SOA !! Ultimately, there will be only one class" portion).

With the (testing, not production) patch below, I get the slowdown to ~2
%. What is its effect on your system?

> I guess (a) doesn't work, so we'll either have to do (b), or invent a
> (c) or (d) that mitigates the slowdown. I haven't looked into it, so
> I don't know where the penalty arises--e.g., do numeric conversions
> like this: 
> data.push_back(value_cast<double>(xml_lmi::get_content(*i))); take so
> much time that the libxml2 overhead is unimportant?

I ran a test program under Instruments and value_cast<> is irrelevant,
most of the time is spent in xmlwrapp/libxml2:

33.1 %  in dom_parser ctor & dtor (about half each)
26.2 %  in xml::node ctor & dtor (about half each)
22.7 %  in xml_lmi::get_content()
 4.7 %  in xml_lmi::get_attr() (some more free() calls here)
 3.9 %  in value_cast<>

A surprisingly large amount of time is spent deallocating libxml2 nodes
(the "dtor" portions above). Granted, this is not on Windows, but on OS
X, because I don't have any profiling tool comparable to Valgrind or
Instruments for Windows, but I think the results won't be that different.

> And if we retain many MiB of xml trees, then we might gain speed but
> risk taking too much space.

xml_actuarial_table doesn't keep XML tree in memory (that would indeed
be wasteful w.r.t. memory, as well as terribly inefficient). It's memory
requirements are roughly same as soa_actuarial_table's -- just two
vectors of doubles with the data.

Regards,
Vaclav


diff --git a/actuarial_table.cpp b/actuarial_table.cpp
index da81d53..bbf2824 100644
--- a/actuarial_table.cpp
+++ b/actuarial_table.cpp
@@ -928,6 +928,34 @@ namespace
     }
 } // Unnamed namespace.
 
+
+#define TEST_CACHING
+
+#ifdef TEST_CACHING
+
+actuarial_table const& find_atable
+    (std::string const& filename
+    ,int                number
+    )
+{
+    typedef std::pair<std::string, int> Key;
+    typedef std::map<Key, actuarial_table*> Map;
+    static Map s_cache;
+
+    Key key(filename, number);
+    if(s_cache.find(key) == s_cache.end())
+    {
+        actuarial_table *t = new actuarial_table(filename, number);
+        s_cache[key] = t;
+        return *t;
+    }
+
+    return *s_cache[key];
+}
+
+#endif // TEST_CACHING
+
+
 std::vector<double> actuarial_table_rates
     (std::string const& table_filename
     ,int                table_number
@@ -935,7 +963,7 @@ std::vector<double> actuarial_table_rates
     ,int                length
     )
 {
-#if defined LMI_USE_XML_TABLES
+#if 0 //defined LMI_USE_XML_TABLES
     xml_actuarial_table     z(table_filename, table_number);
     soa_actuarial_table z_soa(table_filename, table_number);
 
@@ -948,7 +976,11 @@ std::vector<double> actuarial_table_rates
     return values;
 #else  // !defined LMI_USE_XML_TABLES
 // SOA !! Ultimately, there will be only one class:
+#ifdef TEST_CACHING
+    const actuarial_table& z = find_atable(table_filename, table_number);
+#else
     actuarial_table z(table_filename, table_number);
+#endif
     return z.values(issue_age, length);
 #endif // !defined LMI_USE_XML_TABLES
 }
@@ -963,7 +995,7 @@ std::vector<double> actuarial_table_rates_elaborated
     ,int                      reset_duration
     )
 {
-#if defined LMI_USE_XML_TABLES
+#if 0 //defined LMI_USE_XML_TABLES
     xml_actuarial_table     z(table_filename, table_number);
     soa_actuarial_table z_soa(table_filename, table_number);
 
@@ -988,7 +1020,11 @@ std::vector<double> actuarial_table_rates_elaborated
     return values;
 #else  // !defined LMI_USE_XML_TABLES
 // SOA !! Ultimately, there will be only one class:
+#ifdef TEST_CACHING
+    const actuarial_table& z = find_atable(table_filename, table_number);
+#else
     actuarial_table z(table_filename, table_number);
+#endif
     return z.values_elaborated
         (issue_age
         ,length