Re: xmemdup0, attribute malloc

[Bruno Haible]

Eric Blake wrote:
> |   (M) memory with arbitrary contents, where the start address and the length
> |       are given,
> |   (S) memory that is terminated with a NUL byte and otherwise does not
> contain
> |       NULs; here only the start address is usually given.
> 
> Actually, I consider there to be a third type:
> 
> ~    (M0) memory with arbitrary contents, where the start address and
> length are given, and where one past the length is guaranteed to be NUL
> ...
> | xmemdup0 is a function to convert from (M) to (S) in certain situations.
> 
> Yes, but it is more than that.  xmemdup0 is also a function to create a
> substring of type (M0) from type (M) or (S); I have several use cases for
> this in m4, as part of my series of patches converting m4 to pass embedded
> NUL transparently from input to output.  For example, 
> m4_format(address@hidden,
> 1) expands to three characters, 'a', '\0', '1'.  Now consider
> m4_format(address@hidden): it turns out that checking for valid % sequences
> is somewhat faster (fewer conditionals per iteration of the outer loop) if
> I can always blindly read the bytes after % until determining that one of
> the bytes created an invalid specifier (the ^@, guaranteed by (M0), is
> always an invalid specifier), rather than checking on encountering each %
> and successive byte whether it falls in the last byte of the string (which
> I would have to do if I only had (M)), and different than stopping at the
> first NUL (which it used to do, when m4 used (S) instead of (M0)).  Only
> when encountering ^@ do I then decide if I saw an invalid sequence %^@ in
> the middle, or an unterminated % at the end (and even then, only so that
> the error message is more informative).  Another instance of faster
> processing due to (M0) is escape processing in the replacement string of
> regexp and patsubst builtins; GNU regex can transparently handle NUL, and
> identifying dangling \ is faster when a read of \^@ is guaranteed.

Very interesting. These insights should be documented.

> It seems like both xsubstring (blind copy of (S) to (S), without worry
> about embedded NUL) and xmemtostr (checking copy, and abort() on embedded
> NUL) are useful, but neither has the same semantics as xmemdup0.

Also very interesting.

> Meanwhile, should we be using __attribute__((__malloc__)) on xmemdup0 and
> the various xalloc.h functions, to aid in gcc optimization?

It cannot hurt. Although it will often not help. The situation where
__attribute__((__malloc__)) can help is when assignments are done into
freshly allocated memory, like this:

    int *px, *py, *pz;
    int *new_vector = (int *) xmalloc (3 * sizeof (int));
    new_vector[0] = *px;
    new_vector[1] = *py;
    new_vector[2] = *pz;

Here, if xmalloc was declared as __attribute__((__malloc__)), the compiler
could reorder the memory fetches for better pipelining:

    int *px, *py, *pz;
    int *new_vector = (int *) xmalloc (3 * sizeof (int));
    int x = *px;
    int y = *py;
    int z = *pz;
    new_vector[0] = x;
    new_vector[1] = y;
    new_vector[2] = z;

(To convince yourself, try compiling the attached source file with gcc for
SPARC.)

So this declaration is more effective with xmalloc that with xmemdup0.

__attribute__((__malloc__)) was introduced in GCC 3.0 (or possibly 2.96).
Therefore I'm applying this:


2008-05-14  Bruno Haible  <address@hidden>

        Help GCC to do better code generation.
        * lib/eealloc.h (eemalloc) [GCC >= 3]: Declare with attribute 'malloc'.
        * lib/pagealign_alloc.h (pagealign_alloc, pagealign_xalloc): Likewise.
        * lib/xalloc.h (ATTRIBUTE_MALLOC): New macro.
        (xmalloc, xzalloc, xcalloc, xmemdup, xstrdup, xnmalloc, xcharalloc):
        Declare with attribute 'malloc' if supported.

*** lib/eealloc.h.orig  2008-05-15 01:55:05.000000000 +0200
--- lib/eealloc.h       2008-05-15 01:45:33.000000000 +0200
***************
*** 1,5 ****
  /* Memory allocation with expensive empty allocations.
!    Copyright (C) 2003 Free Software Foundation, Inc.
     Written by Bruno Haible <address@hidden>, 2003,
     based on prior work by Jim Meyering.
  
--- 1,5 ----
  /* Memory allocation with expensive empty allocations.
!    Copyright (C) 2003, 2008 Free Software Foundation, Inc.
     Written by Bruno Haible <address@hidden>, 2003,
     based on prior work by Jim Meyering.
  
***************
*** 34,39 ****
--- 34,42 ----
  #if MALLOC_0_IS_NONNULL
  # define eemalloc malloc
  #else
+ # if __GNUC__ >= 3
+ static inline void *eemalloc (size_t n) __attribute__ ((__malloc__));
+ # endif
  static inline void *
  eemalloc (size_t n)
  {
*** lib/pagealign_alloc.h.orig  2008-05-15 01:55:05.000000000 +0200
--- lib/pagealign_alloc.h       2008-05-15 01:38:41.000000000 +0200
***************
*** 1,6 ****
  /* Memory allocation aligned to system page boundaries.
  
!    Copyright (C) 2005 Free Software Foundation, Inc.
  
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
--- 1,6 ----
  /* Memory allocation aligned to system page boundaries.
  
!    Copyright (C) 2005, 2008 Free Software Foundation, Inc.
  
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
***************
*** 26,36 ****
     to the next multiple.
     Return a pointer to the start of the memory block. Upon allocation failure,
     return NULL and set errno.  */
! extern void *pagealign_alloc (size_t size);
  
  /* Like pagealign_alloc, except it exits the program if the allocation
     fails.  */
! extern void *pagealign_xalloc (size_t size);
  
  /* Free a memory block.
     PTR must be a non-NULL pointer returned by pagealign_alloc or
--- 26,44 ----
     to the next multiple.
     Return a pointer to the start of the memory block. Upon allocation failure,
     return NULL and set errno.  */
! extern void *pagealign_alloc (size_t size)
! # if __GNUC__ >= 3
!      __attribute__ ((__malloc__))
! # endif
!      ;
  
  /* Like pagealign_alloc, except it exits the program if the allocation
     fails.  */
! extern void *pagealign_xalloc (size_t size)
! # if __GNUC__ >= 3
!      __attribute__ ((__malloc__))
! # endif
!      ;
  
  /* Free a memory block.
     PTR must be a non-NULL pointer returned by pagealign_alloc or
*** lib/xalloc.h.orig   2008-05-15 01:55:05.000000000 +0200
--- lib/xalloc.h        2008-05-15 01:45:20.000000000 +0200
***************
*** 37,42 ****
--- 37,50 ----
  #  define ATTRIBUTE_NORETURN __attribute__ ((__noreturn__))
  # endif
  
+ # ifndef ATTRIBUTE_MALLOC
+ #  if __GNUC__ >= 3
+ #   define ATTRIBUTE_MALLOC __attribute__ ((__malloc__))
+ #  else
+ #   define ATTRIBUTE_MALLOC
+ #  endif
+ # endif
+ 
  /* This function is always triggered when memory is exhausted.
     It must be defined by the application, either explicitly
     or by using gnulib's xalloc-die module.  This is the
***************
*** 44,56 ****
     memory allocation failure.  */
  extern void xalloc_die (void) ATTRIBUTE_NORETURN;
  
! void *xmalloc (size_t s);
! void *xzalloc (size_t s);
! void *xcalloc (size_t n, size_t s);
  void *xrealloc (void *p, size_t s);
  void *x2realloc (void *p, size_t *pn);
! void *xmemdup (void const *p, size_t s);
! char *xstrdup (char const *str);
  
  /* Return 1 if an array of N objects, each of size S, cannot exist due
     to size arithmetic overflow.  S must be positive and N must be
--- 52,64 ----
     memory allocation failure.  */
  extern void xalloc_die (void) ATTRIBUTE_NORETURN;
  
! void *xmalloc (size_t s) ATTRIBUTE_MALLOC;
! void *xzalloc (size_t s) ATTRIBUTE_MALLOC;
! void *xcalloc (size_t n, size_t s) ATTRIBUTE_MALLOC;
  void *xrealloc (void *p, size_t s);
  void *x2realloc (void *p, size_t *pn);
! void *xmemdup (void const *p, size_t s) ATTRIBUTE_MALLOC;
! char *xstrdup (char const *str) ATTRIBUTE_MALLOC;
  
  /* Return 1 if an array of N objects, each of size S, cannot exist due
     to size arithmetic overflow.  S must be positive and N must be
***************
*** 97,106 ****
  # if HAVE_INLINE
  #  define static_inline static inline
  # else
!    void *xnmalloc (size_t n, size_t s);
     void *xnrealloc (void *p, size_t n, size_t s);
     void *x2nrealloc (void *p, size_t *pn, size_t s);
!    char *xcharalloc (size_t n);
  # endif
  
  # ifdef static_inline
--- 105,114 ----
  # if HAVE_INLINE
  #  define static_inline static inline
  # else
!    void *xnmalloc (size_t n, size_t s) ATTRIBUTE_MALLOC;
     void *xnrealloc (void *p, size_t n, size_t s);
     void *x2nrealloc (void *p, size_t *pn, size_t s);
!    char *xcharalloc (size_t n) ATTRIBUTE_MALLOC;;
  # endif
  
  # ifdef static_inline
***************
*** 108,113 ****
--- 116,122 ----
  /* Allocate an array of N objects, each with S bytes of memory,
     dynamically, with error checking.  S must be nonzero.  */
  
+ static_inline void *xnmalloc (size_t n, size_t s) ATTRIBUTE_MALLOC;
  static_inline void *
  xnmalloc (size_t n, size_t s)
  {
***************
*** 219,224 ****
--- 228,234 ----
  /* Return a pointer to a new buffer of N bytes.  This is like xmalloc,
     except it returns char *.  */
  
+ static_inline char *xcharalloc (size_t n) ATTRIBUTE_MALLOC;
  static_inline char *
  xcharalloc (size_t n)
  {

foo.c
Description: Text Data