bug#56469: 29.0.50; Unibyte dir in directory_files_internal

[Stefan Monnier]

Package: Emacs
Version: 29.0.50


If you have a directory named "/tmp/\303a" with a file named "fée"
inside, then (directory-files "/tmp/\303a" 'full) is likely to return
a funny string which is multibyte but contains an invalid
utf-8 sequence (its bytes spell "/tmp/\303a/f\303\251e").
That strings seems to be printed as "/tmp/¡/fée" which corresponds
to "/tmp/\303\241/f\303\251e".

Such a string with an invalid UTF-8 sequence is handled quite graciously
by Emacs, so I wasn't able to get an actual crash out of it, but it's
still something we should avoid.

I suggest the patch below.  In a comment I suggest we don't try to use
unibyte strings when a multibyte string would work as well.  This is
because for those ASCII-only strings, it's cheaper to test bytes==chars
to (re)discover that they are ASCII-only (when they're multibyte) than
having to loop through the bytes (when they're unibyte).


        Stefan


diff --git a/src/dired.c b/src/dired.c
index 6bb8c2fcb9f..33ddfafd8e7 100644
--- a/src/dired.c
+++ b/src/dired.c
@@ -219,6 +219,13 @@ directory_files_internal (Lisp_Object directory, 
Lisp_Object full,
     }
 #endif
 
+  if (!NILP (full) && !STRING_MULTIBYTE (directory))
+    { /* We will be concatenating 'directory' with local file name.
+         We always decode local file names, so in order to safely concatenate
+         them we need 'directory' to be multibyte.  */
+      directory = Fstring_to_multibyte (directory);
+    }
+
   ptrdiff_t directory_nbytes = SBYTES (directory);
   re_match_object = Qt;
 
@@ -263,9 +270,10 @@ directory_files_internal (Lisp_Object directory, 
Lisp_Object full,
          ptrdiff_t name_nbytes = SBYTES (name);
          ptrdiff_t nbytes = directory_nbytes + needsep + name_nbytes;
          ptrdiff_t nchars = SCHARS (directory) + needsep + SCHARS (name);
-         finalname = make_uninit_multibyte_string (nchars, nbytes);
-         if (nchars == nbytes)
-           STRING_SET_UNIBYTE (finalname);
+         /* FIXME: Why not make them all multibyte?  */
+         finalname = (nchars == nbytes)
+                     ? make_uninit_string (nchars, nbytes)
+                     : make_uninit_multibyte_string (nchars, nbytes);
          memcpy (SDATA (finalname), SDATA (directory), directory_nbytes);
          if (needsep)
            SSET (finalname, directory_nbytes, DIRECTORY_SEP);