[gnuastro-commits] master cf56e2e: Tmpfs configure and build explanation moved inside book

[Mohammad Akhlaghi]

branch: master
commit cf56e2e6e40a28a783443009976a29c6c542ad70
Author: Mohammad Akhlaghi <address@hidden>
Commit: Mohammad Akhlaghi <address@hidden>

    Tmpfs configure and build explanation moved inside book
    
    A sub-sub-section was added to the book explaining the `tmpfs-config-make'
    script for users. The explanation is actually taken from the comments at
    the start of the script. So that long initial explanation was removed and
    after a one-line explanation, the comments refer the curious user to the
    proper part of the book to read more comfortably.
---
 doc/gnuastro.texi |   86 +++++++++++++++++++++++++++++++++++++++++++++++++-
 tmpfs-config-make |   90 ++++++++++++++++++++---------------------------------
 2 files changed, 118 insertions(+), 58 deletions(-)

diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 830d409..ba9727d 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -267,6 +267,7 @@ Configuring
 * Gnuastro configure options::  Configure options particular to Gnuastro.
 * Installation directory::      Specify the directory to install.
 * Executable names::            Changing executable names.
+* Configure and build in RAM::  For minimal use of HDD or SSD, and clean 
source.
 
 Common behavior
 
@@ -3001,6 +3002,7 @@ are discussed.
 * Gnuastro configure options::  Configure options particular to Gnuastro.
 * Installation directory::      Specify the directory to install.
 * Executable names::            Changing executable names.
+* Configure and build in RAM::  For minimal use of HDD or SSD, and clean 
source.
 @end menu
 
 @node Gnuastro configure options, Installation directory, Configuring, 
Configuring
@@ -3365,7 +3367,7 @@ names as important system programs or libraries.
 
 
 
address@hidden Executable names,  , Installation directory, Configuring
address@hidden Executable names, Configure and build in RAM, Installation 
directory, Configuring
 @subsubsection Executable names
 
 @cindex Executable names
@@ -3463,6 +3465,88 @@ $ ./configure --program-transform-name='s/ast/ /'
 
 
 
address@hidden Configure and build in RAM,  , Executable names, Configuring
address@hidden Configure and build in RAM
+
address@hidden File I/O
address@hidden Input/Output, file
+The configure and build process involves the creation, reading, and
+modification of a large number of files (input/output, or I/O). Therefore
+file I/O issues can directly affect the work of developers who need to
+configure and build Gnuastro numerous times. Some of these issues are
+listed below:
+
address@hidden
address@hidden HDD
address@hidden SSD
address@hidden
+I/O will cause wear and tear on both the HDDs (mechanical failures) and
+SSDs (decreasing the lifetime).
+
address@hidden Backup
address@hidden
+Having the built files mixed with the source files can greatly affect
+backing up (synchronization) of source files (since it involves the
+management of a large number of small files that are regularly
+changed. Backup software can ofcourse be configured to ignore the built
+files and directories. However, since the built files are mixed with the
+source files and can have a large variety, this will require a high level
+of customization.
address@hidden itemize
+
address@hidden tmpfs file system
address@hidden file systems, tmpfs
+One solution to address both these problems is to use the
address@hidden://en.wikipedia.org/wiki/Tmpfs, tmpfs file system}. Any file in
+tmpfs is actually stored in the RAM (and possibly SAWP), not on HDDs or
+SSDs. The RAM is built for extensive and fast I/O. Therefore the large
+number of file I/Os associated with configuring and building will not harm
+the HDDs or SSDs. Due to the volatile nature of RAM, files in the tmpfs
+filesystem will be permanently lost after a power-off. Since all configured
+and built files are derivative files (not files that have been directly
+written by hand) there is no problem in this and this feature can be
+considered as an automatic cleanup.
+
address@hidden Linux kernel
address@hidden GNU C Library
address@hidden GNU Build System
+The modern GNU C Library (and thus the Linux kernel) define the
address@hidden/dev/shm} directory for this purpose (POSIX shared memory). So 
using
+GNU Build System's ability to build in a separate directory (not
+necessarily in the source directory), we can configure and build the
+programs in @file{/dev/shm} to benefit from the RAM. To simplify the
+process, Gnuastro comes with a @file{tmpfs-config-make} script. This script
+will create a directory in the shared memory, and put a symbolic link to it
+(called @file{build}) in the top source directory (the backup/sync software
+therefore only needs to ignore this single link/file). The script will then
+internally change to that directory and configure and build (@command{make
+-kjN}, where @command{N} is the number of threads for a parallel build)
+Gnuastro in there. To benefit from this script, simply run the following
+command instead of @command{./configure} and @command{make}:
+
address@hidden
+$ ./tmpfs-config-make
address@hidden example
+
address@hidden GNU Emacs
+After this script is finished, you can address@hidden build}' and run other
+Make commands (for example, address@hidden check}', address@hidden
+install}', or address@hidden pdf}') from there. In Emacs, the command to be
+run with the @command{M-x compile} command (by default: @command{make -k})
+can be changed to address@hidden build; make -kjN}', or address@hidden -C
+build -kjN}' (@command{N} is the number of threads; an integer
address@hidden). For subsequent builds (during development) the
address@hidden recompile} command will also do all the building in the RAM
+while you modify the clean, and backed-up source files and make
+minimal/efficient use of your non-volatile HDD or SSD.
+
+This script can be used in any software which is configured and built using
+the GNU Build System. Just copy it in, and run it from, the top source
+directory of that software. The default number of threads and location of
+the shared memory (@file{/dev/shm}) are currently hard-coded within the
+script. If you need to change them, please open the script with a text
+editor and set their values manually.
+
 @node Tests, A4 print book, Configuring, Build and install
 @subsection Tests
 
diff --git a/tmpfs-config-make b/tmpfs-config-make
index 4e5cf14..6661d0d 100755
--- a/tmpfs-config-make
+++ b/tmpfs-config-make
@@ -1,63 +1,9 @@
 #! /bin/sh
 
 # This script will configure and build Gnuastro in parallel within a
-# temporary tmpfs directory in the RAM. Then it will create a symbolic link
-# to access that temporary directory which is called 'build'. Ideally this
-# script should only be run the first time when you want to run
-# './configure' and 'make'. For future builds 'make' should be run in the
-# 'build' directory, either run 'cd build' first, or 'make -C build'. If
-# the configure script dependencies didn't change, you can also run this
-# script again to run make in the build directory.
-#
-# MOTIVATION:
-#
-# The configure and build process involves the creation, reading, and
-# modification of a large number of files (input/output, or I/O). Therefore
-# file I/O issues can directly affect the work of developers who need to
-# configure and build Gnuastro numerous times. Some such issues are listed
-# below:
-#
-#   - I/O will cause wear and tear on both the HDDs (mechanical failures)
-#     and SSDs (decreasing the lifetime).
-#
-#   - Having the built files mixed with the source files can greatly affect
-#     backing up (synchronization) of source files (since it involves the
-#     management of a large number of small files that are changed a
-#     lot. Such systems can ofcourse be configured to ignore some files and
-#     directories but that will require a high level of customization.
-#
-# One solution to these problems is the tmpfs file system (see
-# https://en.wikipedia.org/wiki/Tmpfs). Any file in tmpfs is actually
-# stored in the RAM (and possibly SAWP), not on HDDs or SSDs. The RAM is
-# built for a large number of fast file I/O. Therefore the large number of
-# file I/O will not harm the HDDs or SSDs. However, due to the volatile
-# nature of RAM, files in the tmpfs filesystem will be permanently lost
-# after a power-off. Since all configured and built files are derivative
-# files (not files that have been directly written by hand) there is no
-# problem in this and this feature can be considered and automatic cleanup.
-#
-# The modern GNU C library (and thus the Linux kernel) define the
-# '/dev/shm' directory for this purpose (POSIX shared memory). Therefore
-# this script takes '/dev/shm' as the default temporary directory (the
-# value to TMPDIR). This script will make a 'gnuastro' directory in TMPDIR,
-# and a symbolic link to it called "build" in the top source directory.  It
-# will then internally change to that directory and configure and build
-# (run 'make -kjN') Gnuastro in there.
-#
-# After this script is run, you can 'cd build' and run other 'make'
-# commands (for example 'make pdf', 'make check', or 'make install') from
-# there. In Emacs, the command to be run with the 'M-x compile' command can
-# be changed to 'cd build; make -kjN' (N is an integer >=1) instead of the
-# default 'make -k'. In this manner, the 'M-x recompile' command to be run
-# during the development will also build in the RAM while you modify the
-# source in your non-volatile HDD or SSD.
-#
-# To further speed up the build process, this script will build ('make')
-# Gnuastro on multiple threads (value to the 'NUM_THREADS' variable
-# bellow). This script can also be used in any other source file that uses
-# the GNU build system.
-#
-#
+# temporary tmpfs directory in the RAM. Please see the "Configure and build
+# in RAM" section of the Gnuastro book/documentation for a full
+# explanation.
 #
 # Original author: Mohammad Akhlaghi <address@hidden>
 # Contributing author(s):
@@ -78,11 +24,17 @@
 # with Gnuastro. If not, see <http://www.gnu.org/licenses/>.
 
 
+
+
+
 # Set the variables:
 NUM_THREADS=8
 TMPDIR=/dev/shm
 
 
+
+
+
 # Check if TMPDIR exists
 if [ ! -d $TMPDIR ]; then
     echo "$TMPDIR doesn't exist. Aborted."
@@ -90,11 +42,17 @@ if [ ! -d $TMPDIR ]; then
 fi
 
 
+
+
+
 # Keep the address of this source directory (where this script is being run
 # from) which we will need later.
 srcdir=$(pwd)
 
 
+
+
+
 # Set the build directory name in tmpfs. If the .version file exists, use
 # it to allow multiple version builds there (which might happen during
 # development).
@@ -106,12 +64,18 @@ else
 fi
 
 
+
+
+
 # Make the build directory in tmpfs (if it doesn't already exist).
 if [ ! -d $build_dir ]; then
     mkdir $build_dir
 fi
 
 
+
+
+
 # Make a symbolic link to the tmpfs build directory for users to easily
 # access the built files and also follow the progress. We are first
 # deleting any existing symbolic link and remaking it since the possible
@@ -124,6 +88,9 @@ if [ -h $build_sym ]; then
 fi
 
 
+
+
+
 # Create the link only if the symbolic link doesn't exist.
 if [ ! -e $build_sym ]; then
     ln -s $build_dir $build_sym
@@ -134,15 +101,24 @@ else
 fi
 
 
+
+
+
 # Go into the build directory to start the configure and/or build:
 cd $build_dir
 
 
+
+
+
 # If a 'Makefile' doesn't exist, then configure Gnuastro:
 if [ ! -f Makefile ]; then
     $srcdir/configure --srcdir=$srcdir
 fi
 
 
+
+
+
 # Build Gnuastro in that directory with the specified number of threads
 make -kj$NUM_THREADS