[gnuastro-commits] master bfed8f15: Book: more practical example for Make extension ast-text-prev-in-list

[Mohammad Akhlaghi]

branch: master
commit bfed8f15a921d9ad04df47289f62eb65192c9866
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>

    Book: more practical example for Make extension ast-text-prev-in-list
    
    Until now, the example of this Make extension was too basic to show its
    usefulness!
    
    With this commit, a more practical scenario has been shown in the
    documentation to help readers better appreciate it.
---
 doc/gnuastro.texi | 50 ++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 40 insertions(+), 10 deletions(-)

diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index a2e0e216..45307026 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -35918,25 +35918,55 @@ all:
 Returns the word in @code{LIST} that is previous to @code{TARGET}.
 If @code{TARGET} is the first word of the list, or is not within it at all, 
this function will return an empty string (nothing).
 
-One scenario when this function can be useful is when you want a list of 
targets to always be executed in sequence (even when Make is run in parallel).
-The @code{sleep} command is used in the recipe to show the serialization more 
clearly.
-To see the effect of this function put the example below in a @file{Makefile} 
and run @code{make -j4} (to simultaneously exectute 4 jobs); then 
comment/remove this function (so there is no prerequisite) and re-run 
@code{make -j4}.
+One scenario when this function can be useful is when you want a list of 
higher-level targets to always be executed in sequence (even when Make is run 
in parallel).
+But you want their lower-level prerequisites to be executed in parallel.
+
+The fully working example below shows this in practice: the ``final'' target 
depends on the sub-components @file{a.fits}, @file{b.fits}, @file{c.fits} and 
@file{d.fits}.
+But each one of these has seven dependencies (for example @file{a.fits} 
depends on the sub-sub-components @file{a-1.fits}, @file{a-2.fits}, 
@file{a-3.fits}, ...).
+Without this function, Make will first build all the sub-sub-components first, 
then the sub-components and ultimately the final target.
+
+When the files are small and there aren't too many of them, this is not a 
problem.
+But when you hundreds/thousands of sub-sub-components, your computer may not 
have the capacity to hold them all in storage or RAM (during processing).
+In such cases, you want to build the sub-components to built in series, but 
the sub-sub-components of each sub-component to be built in parallel.
+This function allows just this in an easy manner as below: the 
sub-sub-components of each sub-component depend on the previous sub-component.
+
+To see the effect of this function put the example below in a @file{Makefile} 
and run @code{make -j12} (to simultaneously exectute 12 jobs); then 
comment/remove this function (so there is no prerequisite in @code{$(subsubs)}) 
and re-run @code{make -j12}.
 
 @example
+# Basic settings
 all: final
 .SECONDEXPANSION:
 load /usr/local/lib/libgnuastro_make.so
 
-list=$(foreach i,a b c d,$(i).txt)
-$(list): %.txt: $$(ast-text-prev-in-list $$@@, $(list))
-         @@sleep 0.3; echo "$@@: $^"
+# 4 sub-components (alphabetic), each with 7
+# sub-sub-components (numeric).
+subids = a b c d
+subsubids = 1 2 3 4 5 6 7
+subs=$(foreach s, $(subids), $(s).fits)
+subsubs=$(foreach s, $(subids), \
+          $(foreach ss, $(subsubids), \
+            $(s)-$(ss).fits))
 
-final: $(list)
+# Build the sub-components:
+$(subsubs): %.fits: $$(ast-text-prev-in-list \
+                       $$(word 1, $$(subst -, ,%)).fits, \
+                       $(subs))
+        @@echo "$@@: $^"
+
+# Build the final components
+$(subs): %.fits: $$(foreach s, $(subsubids), %-$$(s).fits)
+        @@echo "$@@: $^"
+
+# Final
+final: $(subs)
+        @@echo "$@@: $^"
 @end example
 
-As you see, when this function is in the prerequisite list, the recipes are 
run one by one; but without it, all four are printed after an initial 0.3 
seconds of delay.
-Unfortunately the @code{.NOTPARALLEL} target of GNU Make, will also serialize 
the prerequisites of the targets given to it.
-But through this function, the prerequisites of the serialized targets will 
still be executed in parallel.
+As you see, when this function is present, the sub-sub-components of each 
sub-component are executed in parallel, while at each moment, only a single 
sub-component's prerequisites are being made.
+Without this function, make first builds all the sub-sub-components, then goes 
to the sub-components.
+There can be any level of components between these, allowing this operation to 
be as complex as necessary in your data analysis pipeline.
+Unfortunately the @code{.NOTPARALLEL} target of GNU Make doesn't allow this 
level of customization.
+
 
 @item $(ast-text-to-upper STRING)
 Returns the input string but with all characters in UPPER-CASE.