Re: [O] Painfully Slow Export

[Berry, Charles]

> On Jun 12, 2018, at 10:32 PM, Ken Mankoff <address@hidden> wrote:
> 
> 
> 
> On Wed, Jun 13, 2018 at 12:12 AM Berry, Charles <address@hidden> wrote:
> > On Jun 12, 2018, at 2:30 PM, Ken Mankoff <address@hidden> wrote:
> > 
> > Based on the profiling, I think the reason it takes 1 minute is because the 
> > code blocks use <<noweb>> blocks?
> 
> I do not see this.
> 
> It takes a bit longer to run the noweb example you gave with 200 `baz' src 
> blocks, but nothing crazy - like 10 seconds versus 5 for your other example. 
> I used `C-c C-e l L' for each.
> 
> And the profiler gives `org-babel-expand-noweb-references' about 2% of the 
> total.
> 
> You are using a recent version of org, right?
>  
> Yes: 9.1.13-elpa or org-20180604.
> Emacs-mac 26.1 port on OSX from Macports.
> Or Emacs 25.3 downloaded from https://emacsformacosx.com/builds
> 
> Are you sure you hd 200 baz blocks? I'm not actually able to test with that 
> many. 10 takes 1 second. 20 takes 10 seconds. 30 takes 30 seconds. It seems 
> to grow exponentially. I don't have the patience to debug how long 100 takes.

So here it is again - using your ECM. 

Use this block to record the timing and count the number of named src blocks:

#+begin_src emacs-lisp :eval never-export
  (let ((sb (length (org-babel-src-block-names)))
        (st-time (time-to-seconds (get-internal-run-time))))
    (org-latex-export-as-latex)
    (format
     "# of src block names: %d\n Number of Seconds: %10.4f"
     sb  (-
          (time-to-seconds (get-internal-run-time))
          st-time)))
#+end_src

Now run it, add more src blocks after each run, and paste the results elsewhere 
(argh! I should have used `:results append'):

#+RESULTS:
: # of src block names: 3
:  Number of Seconds:     0.0498


#+RESULTS:
: # of src block names: 68
:  Number of Seconds:     2.1021

#+RESULTS:
: # of src block names: 200
:  Number of Seconds:    15.4018


#+RESULTS:
: # of src block names: 266
:  Number of Seconds:    26.6689

#+RESULTS:
: # of src block names: 332
:  Number of Seconds:    41.2211


So my mental count of 10 seconds for 200 blocks was a bit off.

And the time is very nearly quadratic in the number of named src blocks:

#+begin_src R :results output

  x <- c(3, 68, 200, 266, 332)
  y <- c(0.0498, 2.10, 15.40, 26.67, 41.22)
  fit <- lm(y ~ x + I( x^2 ))
  summary(fit)

#+end_src

#+RESULTS:
#+begin_example

Call:
lm(formula = y ~ x + I(x^2))

Residuals:
       1        2        3        4        5 
-0.02650  0.04187  0.01194 -0.05929  0.03197 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept) 5.783e-02  5.584e-02   1.036   0.4092    
x           5.083e-03  8.755e-04   5.805   0.0284 *  
I(x^2)      3.578e-04  2.579e-06 138.762 5.19e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.05973 on 2 degrees of freedom
Multiple R-squared:      1,     Adjusted R-squared:      1 
F-statistic: 1.67e+05 on 2 and 2 DF,  p-value: 5.986e-06

#+end_example

This is on my late-2012 MacBook Pro. MacOS 10.13.5, Emacs 25.3.1, org 9.1.13. 
So this sounds almost exactly like your setup.

Maybe time to try a minimal init file to see if something you have customized 
is bogging things down. 

HTH,

Chuck