[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Gnuastro 0.17.81 released

From: peter teuben
Subject: Re: Gnuastro 0.17.81 released
Date: Wed, 6 Jul 2022 00:34:45 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1

hi   Mohammad

I grabbed this latest "lz" file and also skimmed over the new manual as well, which looks fantastic.

I ran this on Ubuntu22.04, and it passed all 74 tests in the "make check"

Here are some comments I jottted down while waiting for the compilation to finish:

I'm partial to advertising PGPLOT the way you do. On debian and redhat you get two completely different implementations, mostly because redhat (grrrr) doesn't provide PNG, arguably the only useful format for modern environments. Maybe add a caveat, and advice people to use a native package manager before torturing them with the manual method (which don't have the 64bit patched so I suspect they will fail)

Bravo on package manager examples. Could you add pgplot to that list,  ds9 is :-) It's pgplot5 for ubuntu, and it's pgplot for redhat, but one may need to enable the non-free repo?

The manual talks about using "gz" files for the install, yet, your message tells us to grab an "lz" file. Even in your email you need to remind the audience how to untar it.  I had to install lzip, after which this worked for me (after some grumpy language)

tar --lzip -xf gnuastro-0.17.81-1a40.tar.lz

so I was sad this took me a few mins to figure out, whereas arguably it should be 2 seconds. I am getting too old? Does really everybody now know what an lz file is? 



On 7/2/22 13:52, Mohammad Akhlaghi wrote:
Dear all,

The 18th official release of Gnuastro is almost ready. To prepare for
it, Gnuastro 0.17.81 is announced (81 commits/changes have been made
since version 0.17, see [1] for the noteworthy changes).

It would be great if you can try building, installing and using this
alpha release to report any build-warnings, crashes or generally
unexpected behavior. This way, we can be sure that Gnuastro 0.18 will
be as smooth as possible for everyone.

Here is the compressed source and the GPG detached signature for
this release. To uncompress Lzip tarballs, see [2]. To check the
validity of the tarballs using the GPG detached signature (*.sig)
see [3]:


Here are the SHA1 and SHA256 checksums (other ways to check if the
tarball you download is what we distributed). Just note that the
SHA256 checksum is base64 encoded, instead of the hexadecimal encoding
that most checksum tools default to (see [4] on how to generate it).

ca7aa46e998073c804a4a936251bf739b87297cc  gnuastro-0.17.81-1a40.tar.lz
lwZrqdQk5mAgfzhebWImFEr2DgWwW+PkzckuetEMOUw  gnuastro-0.17.81-1a40.tar.lz

As you see in [1] below, many new features have been added in this
release, and many bugs have been fixed. Besides those, I am happy to
announce that Gnuastro finally has a logo:


See [5] For a full description of this logo (taken from the newly
added "Logo of Gnuastro" section of the book). This logo is now used
on the top Gnuastro webpage and also in the top page of the PDF manual
(the bleeding-edge PDF manual: https://akhlaghi.org/gnuastro.pdf).

For their direct contribution to this version's source code, I am very
grateful to Raul Infante-Sainz (with 12 commits), Sepideh Eskandarlou
(6 commits), Jash Shah (1 commit), Elham Saremi (1 commit), Marjan
Akbari (1 commit), Pedram Ashofteh Ardakani (1 commit) and S. Zahra
Hosseini Shahisavandi (1 commit). I am also grateful to Marjan Akbari,
Faezeh Bijarchian, Sepideh Eskandarlou, Sílvia Farras, Raul
Infante-Sainz, Teet Kuutma, Jeremy Lim, Juan Miro, Irene Pintos
Castro, Ignacio Ruiz Cejudo and Elham Saremi, for their good
suggestions or reported bugs that have been implemented in this

If any of Gnuastro's programs or libraries are useful in your work,
please cite _and_ acknowledge them. For citation and acknowledgment
guidelines, run the relevant programs with a `--cite' option (it can
be different for different programs, so run it for all the programs
you use). Citations _and_ acknowledgments are vital for the continued
work on Gnuastro, so please don't forget to support us by doing so.

This tarball was bootstrapped (created) with the tools below. Note
that you don't need these to build Gnuastro from the tarball, these
are the tools that were used to make the tarball itself. They are only
mentioned here to be able to reproduce/recreate this tarball later.
  Texinfo 6.8
  Autoconf 2.71
  Automake 1.16.4
  Help2man 1.49.2
  ImageMagick 7.1.0-39
  Gnulib v0.1-5265-gfc555c0a4b
  Autoconf archives v2022.02.11-15-g066994d

The dependencies to build Gnuastro from this tarball on your system
are described here:

Best wishes,

Staff Researcher
Centro de Estudios de Física del Cosmos de Aragón (CEFCA),
Plaza San Juan 1, Planta 2, Teruel 44001, Spain

[1] Noteworthy changes since Gnuastro 0.17:

   - New "Measuring the dataset limits" section has been added in second
     tutorial. Using the catalogs produced during the analysis, it shows
     how to derive a dataset's magnitude limit, completeness limit
     (crudely: without simulations), and surface brightness limit. All are
     important measures in any scientific analysis! This section was
     written with the help of S. Zahra Hosseini Shahisavandi and Sepideh

   - New operators (also available in Table's column arithmetic):
     - stitch: connect any number of input images along the given dimension
       (for example to stitch the images of separate amplifiers of a CCD
       into one image).
     - noblank: Remove blank elements from the input and return a 1D output
       (irrespective of how many dimensions the input had).
     - jy-to-counts: convert Janskys to counts using an AB-magnitude-based
       Zeropoint (the inverse operator 'counts-to-jy' already existed).
     - mag-to-jy: convert AB magnitudes to Janskys.
     - jy-to-mag: convert Janskys to AB magnitudes.
     - load-col-%-from-%-hdu-%: Load a certain column into the operands
       stack. The name or number of the column is the first '%'. The file
       name hosting that column is the second '%', and HDU (if the file is
       a FITS file), is identified by third '%'. This can be used to load
       columns outside the main table in Table's column arithmetic or by
       operators that need columns as input parameters (like the new
       'random-from-hist' operator).
     - random-from-hist-raw: return a set of random points based on a
       custom distribution defined by its histogram. See the description
       and complete examples in the book for more.
     - random-from-hist: Similar to 'random-from-hist-raw', but within each
       bin, the final returned value is itself the result of a uniform
       sampling (to make the final distribution less descrete). See the
       example in the book for selecting random star magnitudes based on
       Gaia's observed magnitude distribution.
   - New options to add metadata (standard FITS keywords) to your output
     for easy understanding of the data in the future, or by your
     colleagues if you share the output). Until now, to have this feature,
     you needed to call Fits on the output of Arithmetic (for setting the
     HDU name to "my-sum" for example):
         astarithmetic img-a.fits img-b.fits + --output=sum.fits
         astfits sum.fits --write=EXTNAME,"my-sum"
     But with the following commands, you can add important metadata to the
     output of Arithmetic in the same command and simplify your scripts:
       --metaname: Metadata name of the output (FITS keyword: 'EXTNAME').
       --metaunit: Unit of the output's pixels (FITS keyword: 'BUNIT').
       --metacomment: Description of the data (FITS keyword: 'COMMENT').
     Like the following for the example above:
         astarithmetic img-a.fits img-b.fits + --metaname="my-sum" \

   --oneelemstdout: when a crop has a single pixel and this option is
     called, the single pixel's value will be printed on the standard
     output instead of creating a FITS file. This option can be useful in
     labeled images like detection maps (when you want to see what label a
     certain coordiante/pixel corresponds to). This option was suggested by
     Raul Infante-Sainz.

   - When no option is specified (thus printing the HDU metadata in the
     given file), the units of the HDUs are also printed as a new
     column. For tables, it will be blank ('n/a'); because tables have
     separate units on each column (use 'asttable table.fits -i').
   --copykeys: can now take any number of keyword names also. For example
     '--copykeys=KEYNAME1,KEYNAME2,KEYNAME3'. Until now, it was only
     possible to give this option a range of keyword positions (for example
     '--copykeys=10:13' to copy the 10th to 13th keywords (inclusive). But
     now, you can also select non-contiguous set of keywords to copy from
     one HDU into another HDU (possibly in another file). This was
     suggested by Juan Antonio Fernández Ontiveros and implemented with the
     help of Jash Shah.

   - It is now possible to insert a custom image as a new (10th)
     'custom-img' profile. This is useful when you have no-noise FITS
     images from numerical simulations for example, and you want to insert
     those at a certain location within a larger image (while optionally
     adding existing Sersic, point or other profiles in the same
     command). The magnitude of the inserted image can be set in the
     magnitude column of the input catalog. This can be disabled (to use
     the original image pixel values) using the '--mcolnocustimg' option.
  --mcolnocustprof: new option to disable scaling of custom radial profiles
    to required magnitude when building a 2D image from them.

   - Short name for Gaia's DR3 dataset has been added for both ESA's Gaia
     database and VizieR, for example with the following commands:
         astquery gaia   --dataset=dr3     --center=1.234,5.678 --radius=0.01
         astquery vizier --dataset=gaiadr3 --center=1.234,5.678 --radius=0.01

   --zeroisnotblank: new option to account for zero-valued pixels in the
     profile measurement. By default, during the internal cropping phase,
     zero-valued pixels are set to blank (and thus not considered during
     the profile measurement). When this option is called, it is directly
     passed to the internal cropping command ('astcrop' has an option with
     the same name). This was suggested by Ignacio Ruiz Cejudo and
     implemented by Sepideh Eskandarlou.

   - GAL_ARITHMETIC_OP_UNIQUE: to find unique elements in dataset.
   - GAL_ARITHMETIC_OP_NOBLANK: to remove blank elements from dataset.
   - GAL_ARITHMETIC_OP_STITCH: to stitch multiple datasets.
   - GAL_ARITHMETIC_OP_RANDOM_FROM_HIST: the new 'random-from-hist' operator.
   - GAL_ARITHMETIC_OP_RANDOM_FROM_HIST_RAW: the new 'random-from-hist-raw' op.
   - gal_arithmetic_load_col: Low-level function for parsing the string of
     the newly added 'load-col' operator, mentioned in the Arithmetic
     section above.
   - gal_statistics_std_from_sums: return the standard deviation using the
     sum, sum of squares and number of a distribution.

** Removed features

  Nothing has been removed in this alpha release.

** Changed features

   --std: measures the standard deviation of the object/clump pixels of the
          values image. Until now, there was no column to calculate this,
          and the '--std' column returned the _sky_ standard deviation over
          the object/clump labels (from the standard deviation image,
          ignoring the values image). But that was confusing, especially
          since we have columns like '--mean' or '--median' which only use
          the values image (see bug #62685).
   --skystd: new name for old '--std', see description of '--std' above.
   --sfmagnsigma: default value changed to 3 (more commonly used).
   --sfmagarea: default value set to 100 arcsec^2 (more commonly used).

   - The string identifier for custom radial profiles is now called
     'custom-prof' (until now, it was called 'custom'). This was necessary
     because of the new custom image feature that has been added. The code
     for this custom radial profile function (8) hasn't changed.
   - After constructing a custom radial profile (the 'custom-prof'
     profile), MakeProfiles will scale it to have the desired total
     magnitude in the input catalog of all profiles. Until now, the 2D
     images created from a custom radial profile wouldn't be touched and
     the pixel values would exactly come from the custom radial profile
     (ignoring the magnitude column of the input catalog). To have the old
     behavior, use the new '--mcolnocustprof' option. This new
     functionality was proposed by Elham Saremi.

   --catcolumnfile: the '-N' suffix (where 'N' is a counter for the files
     with concatenated, or appended, column) is only added if a column with
     a identical column name (not case sensitive) existed in the
     table. Until now, this suffix would be added to all the newly
     concatenated columns (recall that the suffix addition can be disabled
     in general with the '--catcolumnrawname' option).
   --rowrange: new name for the old '--rowlimit'. This option takes the
     range of desired rows based on their position (for example
     '--rowrange=2,5' will only print rows 2, 3, 4 and 5 of the table). But
     the old name was very confusing and would not directly convey this

** Bugs fixed
  bug #62216: MakeProfiles crash when a 3D cube is requested and input
              catalog is from a pipe. Found by Irene Pintos Castro.
  bug #62220: astscript-psf-stamp crashes when coordinate not in image.
              Found and fixed by Sepideh Eskandarlou.
  bug #62250: Convolve crashes when doing PSF matching. Found with help of
              Raul Infante-Sainz.
  bug #62252: Radial profile giving one larger radius in high axis-ratio
              profile. Found and fixed by Sepideh Eskandarlou.
  bug #62253: Arithmetic: segmentation fault when invalid input given for
              'ra-to-degree' or 'deg-to-degree' operators. Found by Pedram
              Ashofte Ardakani.
  bug #62305: Not reading the 'MemFree' keyword in '/proc/meminfo' to find
              the available RAM (and thus printing an annoying
              warning). Reported by Juan Miro.
  bug #62309: MakeCatalog segmentation fault when clumps image given as
              main input (objects should be given). Reported by Sepideh
  bug #62429: Segment prints one line status-progress line when called with
              '--quiet'. Reported by Teet Kuutma.
  bug #62548: Table doesn't recognize plain-text string column when defined
              width is larger and it is the last column. Found with the
              help of Sepideh Eskandarlou.
  bug #62563: Some checks of installed scripts get skipped on macOS. Found
              and fixed by Raul Infante-Sainz.
  bug #62564: Arithmetic not parsing numbers in scientific notation (for
              example '1e5'). Found by Elham Saremi.
  bug #62590: Failure to build on macOS when building in debugging mode
              ('./configure --enable-debug'). Found by Raul Infante-Sainz.
  bug #62597: Arithmetic not writing single-valued output into a file when
              called with '--output'. Reported by Raul Infante-Sainz.
  bug #62636: Table ignoring '--range' when '--head' is also
              called. Reported by Sepideh Eskandarlou.
  bug #62662: Table crashes when column arithmetic is applied after adding
              rows from another file (with '--catrowfile').
  bug #62674: Column arithmetic can't use columns from --catcolumnfile.
  bug #62678: Radial profile script using values image instead of STD image
              when '--instd' not used. Found and fixed by Raul Infante-Sainz.
  bug #62679: MakeProfiles kernel only normalized when zeropoint is
              zero. Found by Raul Infante-Sainz.
  bug #62680: MakeProfiles using WCS-options when --background is given and
              the background doesn't have WCS. Found with the help of Raul
  bug #62683: Arithmetic's 'set-' operator not working after the 'makenew'
              operator. Found by Raul Infante-Sainz.
  bug #62685: MakeCatalog's '--std' option is wrongly interpreted as the
              standard deviation of the values image, while it is actually
              the root mean square of the Sky standard deviation
              pixels. Found by Raul Infante-Sainz.
  bug #62694: Radial profile script overestimates the surface brightness
              error. Found and fixed with the help of Raul Infante-Sainz.

[2] Lzip has better compression ratio and archival features compared
to the `.gz' or `.xz' formats. Therefore Gnuastro's alpha/test
releases are only in this format, but for historical reasons we also
include `.gz' tarballs in the official releases. If you don't have
Lzip (you can check with `lzip --version' command), download and
install it from its webpage:


If Lzip is present and you use GNU Tar, then the single command below
should uncompress and un-pack the tarball:

  $ tar xf gnuastro-0.17.81-1a40.tar.lz

If the command above doesn't work, you have to un-compress and un-pack
it with two separate commands (or use a pipe to feed the output of the
first into the second: `lzip -cd gnuastro-0.17.81-1a40.tar.lz | tar -xf -'):

  $ lzip -d gnuastro-0.17.81-1a40.tar.lz
  $ tar xf gnuastro-0.17.81-1a40.tar

[3] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify gnuastro-0.17.81-1a40.tar.lz.sig

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to update
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --keyserver hkp://pgp.rediris.es --recv-keys 71E899012D174B66

[4] To get the base64 SHA256 checksum, you can use the command below
(assuming you already have the 'openssl' command-line program).

  cat gnuastro-0.17.81-1a40.tar.lz \
      | openssl dgst -binary -sha256 | openssl base64 -A

[5] Gnuastro’s logo is an abstract image of a barred spiral galaxy
(https://en.wikipedia.  org/wiki/Barred_spiral_galaxy). The galaxy is
vertically cut in half: on the left side, the beauty of a contiguous
galaxy image is visible. But on the right, the image gets pixe- lated,
and we only see the parts that are within the pixels. The pixels that
are more near to the center of the galaxy (which is brighter) are also
larger. But as we follow the spiral arms (and get more distant from
the center), the pixels get smaller (signifying less signal).

This sharp distinction between the contiguous and pixelated view of
the galaxy signifies the main struggle in science: in the “real”
world, objects aren’t pixelated or discrete and have no
noise. However, when we observe nature, we are confined and
constrained by the resolution of our data collection (CCD imager in
this case).

On the other hand, we read English text from the left and progress
towards the right.  This defins the positioning of the “real” and
observed halfs of the galaxy: the no-noised and contiguous half (on
the left) passes through our observing tools and becomes pixelated and
noisy half (on the right). It is the job of scientific software like
Gnuastro to help interpret the underlying mechanisms of the “real”
universe from the pixelated and noisy data.

Gnuastro’s logo was designed by Marjan Akbari. The concept behind it
was created after several design iterations with Mohammad Akhlaghi.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]