[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Gnuastro 0.14 released
Gnuastro 0.14 released
Mon, 25 Jan 2021 12:19:30 +0000
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1
I am happy to announce the availability of Gnuastro 0.14. For the full
list of added and changed/improved features, see the excerpt of the
NEWS file for this release in  below.
Gnuastro is an official GNU package, consisting of various
command-line programs and library functions for the manipulation and
analysis of (astronomical) data. All the programs share the same basic
command-line user interface (modeled on GNU Coreutils). For the full
list of Gnuastro's library, programs, and a comprehensive general
tutorial (recommended place to start using Gnuastro), please see the
links below respectively:
The most prominent new feature may be the new Query program (called
with 'astquery'). It allows you to directly query many large
astronomical data centers (currently VizieR, NED, ESA and ASTRON) and
only download your selected columns/rows. For example with the command
below you can download the RA, Dec and Parallax of all stars in the
Gaia eDR3 dataset (from VizieR) that overlap with your
'image.fits'. You just have to change '--dataset' to access any of the
+20,000 datasets within VizieR for example! You can also search in the
dataset metadata from the command-line, and much more.
astquery vizier --dataset=gaiaedr3 --overlapwith=image.fits \
See the new "Query" section in the Gnuastro book for more:
Here is the compressed source and the GPG detached signature for this
release. To uncompress Lzip tarballs, see . To check the validity
of the tarballs using the GPG detached signature (*.sig) see :
Here are the MD5 and SHA1 checksums:
I am very grateful to (in alphabetic order) Pedram Ashofteh Ardakani,
Thérèse Godefroy, Raúl Infante-Sainz, Sachin Kumar Singh, Samane Raji
and Zahra Sharbaf for directly contributing to the source of Gnuastro
since the last alpha-release. It is great that in this release we have
an equal gender balance in the contributors. I sincerely hope this can
continue in the next release :-).
I am also very grateful to (in alphabetic order) Antonio Diaz Diaz,
Paul Eggert, Andrés García-Serra Romero, Thérèse Godefroy, Bruno
Haible, Martin Kuemmel, Javier Licandro, Alireza Molaeinezhad, Javier
Moldon, Sebastian Luna Valero, Samane Raji, Alberto Madrigal, Carlos
Morales Socorro, Francois Ochsenbein, Joanna Sakowska, Zahra Sharbaf,
Sachin Kumar Singh, Ignacio Trujillo and Xiuqin Wu for their very
useful comments, suggestions and bug fixes that have now been
implemented in Gnuastro since the last alpha-release.
If any of Gnuastro's programs or libraries are useful in your work,
please cite _and_ acknowledge them. For citation and acknowledgment
guidelines, run the relevant programs with a `--cite' option (it can
be different for different programs, so run it for all the programs
you use). Citations _and_ acknowledgments are vital for the continued
work on Gnuastro, so please don't forget to support us by doing so.
This tarball was bootstrapped (created) with the tools below. Note
that you don't need these to build Gnuastro from the tarball, these
are the tools that were used to make the tarball itself. They are only
mentioned here to be able to reproduce/recreate this tarball later.
Autoconf archives v2019.01.06-98-gefa6f20
The dependencies to build Gnuastro from this tarball on your system
are described here:
Postdoctoral research fellow,
Instituto de Astrofísica de Canarias (IAC),
Calle Vía Láctea, s/n, E38205,
San Cristóbal de La Laguna, Tenerife, Spain.
 Noteworthy changes in release 0.14 (library 12.0.0)
** New features
- Tutorials improved with more useful usage examples, for example
the "General program usage tutorial" tutorial now includes the
merging of table columns into one table and the creation of a
color-magnitude diagram. Also the "Detecting large extended
targets" now accounts for the recent improvements in NoiseChisel.
- New sub-section on "Memory management" in the "Common program
behavior" chapter. It fully describes how to optimally deal with large
datasets that may exceed your system's RAM.
- Examples and better description added to many operators in the
"Arithmetic operators" subsection of the Arithmetic program's section.
- Query ('astquery') is a new program to query to external datasets
and retrieve the resulting datasets from the command-line. It is
possible to get list of datasets or column names from the
databases. The basic spatial query (finding objects in the
vicinity of a certain point) can be managed without knowing the
query language of the database: with basic options (like
'--center' with '--radius' or '--width'), or an input image that
has WCS. Currently Query supports VizieR (containing +20500
datasets, making it the largest database in astronomy), NED, as
ESA's Gaia database and ASTRON. See the new "Query" section of
the book (under the "Data containers" chapter) for more. Adding
any new data base that supports the Table Access Protocol (TAP)
of the International Virtual Observatory Alliance (IVOA,
https://ivoa.net), is trivial.
- Plain text table inputs can have floating point columns that are in
sexagesimal format of '_h_m_s' or '_d_m_s' (where '_' is a
number). These are the classical format to respectively represent
Right Ascension (RA) and Declination (Dec). They will be directly read
as a single floating point number (in units of degrees) into
memory. Therefore the same column of a plain-text table, can be
degrees in some rows and sexagesimal in others. Besides large tables,
with this feature, conversion to sexagesimal coordinates to degrees
becomes very easy, for example:
echo "7h34m35.5498 31d53m14.352s" | asttable
Recall that the inverse can also be done with the more general column
echo "113.64812416667 31.88732" \
| asttable -c'arith $1 degree-to-ra $2 degree-to-dec'
- If input is a HEALpix grid (1D table column that represents the 2D
spherical representation of datasets), the programs will print a
warning, suggesting to use the 'HPXcvt' utility of WCSLIB.
- New operators:
- 'interpolate-maxofregion': interpolate connected blank regions with
the maximum value that is immediately touching it. This can be used
to fill the blank centers of saturated stars for example.
- 'interpolate-minofregion': similar to 'interpolate-maxofregion', but
for the minimum.
- 'makenew': new operator to create an empty (zero-valued) new dataset
with given dimension and size (given as operands).
--primaryimghdu: Write the final cropped image into the primary (or
0-th) extension of the output FITS file, so the output only has
- New '--skycoverage' option will report the area of the input
image in RA/Dec, both in units of center/width, and box
--maximum: maximum value of labeled regions pixels (clump/object).
--areaarcsec2: area of labeled region (clump/object) in arcsec^2.
--surfacebrightness: the surface brightness of the labeled region.
--fwhm: observed FWHM in pixels (non-parametric), along the major axis.
--halfmaxarea: number of pixels with a value larger than half the
--halfmaxradius: radius of region that is larger than half the maximum.
--halfmaxsum: sum of pixels with a value larger than half the maximum.
--halfmaxsb: surf. brightness within half of the maximum.
--fracmax: fractions to use in '--fracmaxarea1' or '--fracmaxarea2'.
--fracmaxsum1: sum of pixels brighter than first given fraction of max.
--fracmaxsum2: sum of pixels brighter than second given fraction of max.
--fracmaxarea1: number of pixels brighter than first given fraction
--fracmaxarea2: number of pixels brighter than second given fraction
--fracmaxradius1: radius derived from '--fracmaxarea1'.
--fracmaxradius2: radius derived from '--fracmaxarea2'.
--halfsumsb: Surface brightness within area reported by '--halfsumarea'.
--halfsumarea: area containing half of the summed object or clump
--halfsumradius: radius derived from '--halfsumarea', underestimates
--areaminv: the number of pixels that are equal to the minimum value.
--areamaxv: the number of pixels that are equal to the maximum value.
- New columns to return the position of pixel with minimum or maximum
value: '--minvx', '--maxvx', '--minvy', '--maxvy', '--minvz',
--bgisbrightness: new option to say that the value of '--background'
(used to simulate Poisson noise) should be interpreted as brightness,
- It is now possible to make any custom radial profile with the 'custom'
profile (with code '8'). A table should be given to the new
'--customtable' option which will define each radial interval and the
value to use for that radial interval. See the description of
'--customtable' for more.
- 2D histograms can now be built as a FITS image with a linear WCS that
contains axis values and box size. This allows using the power of FITS
viewers for plotting/inspecting distributions of two columns in a
table relative to each other (for example color-magnitude plots). You
can also convert these 2D histogram images to PDF or JPEG with
Gnuastro's ConvertType to directly use in your papers/reports (see the
newly added "2D histogram as an image" section of the book for more).
- New '--noblank' option will remove all rows in output table that have
at least one blank value in the specified columns. For example if
'table.fits' has blank values (NaN in floating point types) in the
'magnitude' and 'sn' columns, with '--noblank=magnitude,sn', you can
be sure that all rows with blank values in these columns have been
- New trigonometric operators for column-arithmetic (inputs in units of
degrees): 'sin', 'cos' and 'tan'. Their inverse trigonometric (outputs
in units of degrees) have also been added: 'asin', 'acos' and 'atan'.
The 'atan2' operator (inverse tangent that preserves the quadrant, see
its description in the book) is also now available.
- New hyperbolic operators for column-arithmetic: 'sinh', 'cosh' and
'tanh'. Their inverse has also been added: 'asinh', 'acosh' and
- GAL_ARITHMETIC_OP_MAKENEW: new 'makenew' operator.
- gal_binary_connected_adjacency_list: finding connected components
without a square adjacency matrix (which can consume major RAM).
- gal_blank_flag_remove: Remove all flagged elements in a dataset.
- gal_blank_remove_rows: remove all rows that have at least one blank.
- gal_dimension_dist_elliptical: Elliptical dist. of a point from
- gal_fits_hdu_is_healpix: Return 1 if HDU is a HEALpix grid.
- gal_fits_hdu_datasum: calculate DATASUM of given HDU in given FITS
- gal_fits_hdu_datasum_ptr: calculate DATASUM of opened FITS file
- gal_fits_key_list_comment_add: add a COMMENT keyword to the
- gal_fits_key_list_comment_add_end: add a COMMENT keyword to end of
- gal_pointer_allocate_ram_or_mmap: allocate in RAM or memory-mapped
- gal_pointer_mmap_free: "free" (actually delete) the memory-mapped
- gal_wcs_create: create WCSLIB-compatible WCS from raw values.
- gal_wcs_coverage: Return the sky coverage of given image HDU.
- gal_wcs_dimension_name: return the name of the requested WCS dim.
** Removed features
- Nothing has been removed from Gnuastro in this release.
** Changed features
- Memory management: Until now, an internal array was only allocated in
the RAM when its size was smaller (in bytes) than the value given to
the '--minmapsize' option. But this was annoying/buggy when the system
has enough RAM to keep large files. From this version, all Gnuastro
programs will first attempt to write the array in RAM, only when it
fails (there is no more RAM left), will they use a memory-mapped file
(which can dramatically slow down the program). Please see the newly
added "Memory management" section of the book for a complete
explanation of Gnuastro's new memory management strategy.
- When an array needs to be memory-mapped (read into a file on HDD/SSD,
not RAM, usually due to RAM limitations), it is written in a
'gnuastro_mmap' directory of the running directory, not the hidden
'.gnuastro_mmap' directory. Since the files in this directory are
usually large, it is better that the user sees them by default (in
case the program crashes and they aren't deleted).
--interpnumngb: the default value has been increased to 15 (from 9). The
reason for this is that we now have a more robust outlier removal
algorithm (see description under "NoiseChisel & Statistics").
- When cropping a single image in WCS mode, there is no longer any
limitation on the WCS. Until now for all WCS mode crops, it was
necessary for the WCS to be aligned to the celestial coordinates. But
from this version, this is only necessary when cropping from many
files (and stitching them together where necessary). For WCS-mode
crops of a single image, any WCS that is recognized by WCSLIB is fine.
- The '--pixelscale' option also prints the pixel area (for 2D inputs,
or 2D slices of 3D inputs) and the voxel volume (for 3D inputs). Until
now, it would only print the pixel scale along each dimension.
- When printing FITS file HDU information (no options given), a new
"Comments" column may be printed for each HDU in the end of the line.
It will be printed if special situations are found (for example a 2D
HEALPix grid, that is usually stored as a 1D array/column).
NoiseChisel & Statistics:
- New algorithm used to reject outlying tiles. In NoiseChisel this is
done when estimating the quantile threshold, the pseudo-detection
threshold and the final Sky value. In Statistics, its just the Sky
value. Unlike the previous method that used the global distribution of
tile values to identify outliers, the new algorithm uses a relative
measure. See the book for more. Since outliers are now rejected more
robustly, the default value of '--meanmedqdiff' has been increased to
0.01 (was 0.005) and '--outliersigma' has been decreased to '5' (was
10). Also 'smoothwidth' has been increased from '3' to '5' to have
smoother tessellation values.
- The '--histogram2d' now takes a string argument: either 'image' or
'table'. For the old behavior please use '--histogram2d=table'. See
the new features above for the 'image' mode.
- Column arithmetic operators 'degree-to-ra' and 'degree-to-dec' will
return the sexagesimal format of '_h_m_s' and '_d_m_s'
respectively. Until this version, they would both use colons as
- gal_pointer_mmap_allocate: new name for 'gal_pointer_allocate_mmap'.
- gal_threads_dist_in_threads: now accounts for billions of threads,
thus includes memory management options.
- gal_threads_spin_off: now accounts for memory management.
- gal_units_degree_to_ra: new 'usecolon' argument to optionally format
output string with colons as delimiters ('_:_:_'). When this option is
zero, the string will be in the '_h_m_s' format.
- gal_units_degree_to_dec: similar to 'gal_units_degree_to_ra', but when
'usecolon' is zero, the string will be in the '_d_m_s' format.
** Bugs fixed
bug #59017: Segment's object IDs are not thread-safe (i.e.,
bug #59105: Column arithmetic operator degree-to-ra, returning to dec.
bug #59136: Makeprofiles with --replace is not thread-safe.
bug #59155: Match cannot find the proper row when coordinates have NaN.
bug #59371: MakeCatalog crash with clumps on non-contiguous object
bug #59400: CosmicCalculator fails --printparams when redshift isn't
bug #59459: Unclear WCS when both PC and CD exist in input, but conflict.
bug #59625: MakeProfiles uses last --kernel, if it is called more
bug #59700: Segment's excessive RAM usage when many clumps over a
bug #59765: MakeCatalog crash when to-be-subtracted Sky is a
per tile tessellation (e.g., NoiseChisel's
 Lzip has better compression ratio and archival features compared
to the `.gz' or `.xz' formats. Therefore Gnuastro's alpha/test
releases are only in this format, but for historical reasons we also
include `.gz' tarballs in the official releases. If you don't have
Lzip (you can check with `lzip --version' command), download and
install it from its webpage:
If Lzip is present and you use GNU Tar, then the single command below
should uncompress and un-pack the tarball:
$ tar xf gnuastro-0.14.tar.lz
If the command above doesn't work, you have to un-compress and un-pack
it with two separate commands (or use a pipe to feed the output of the
first into the second: `lzip -cd gnuastro-0.14.tar.lz | tar -xf -'):
$ lzip -d gnuastro-0.14.tar.lz
$ tar xf gnuastro-0.14.tar
 Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify gnuastro-0.14.tar.gz.sig
If that command fails because you don't have the required public key,
then run this command to import it:
gpg --keyserver keys.gnupg.net --recv-keys 71E899012D174B66
and rerun the 'gpg --verify' command.
|[Prev in Thread]
||[Next in Thread]|
- Gnuastro 0.14 released,
Mohammad Akhlaghi <=