gnulib-tool.py: Fix a bug in removed / added files handling

[Bruno Haible]

Trying the GNULIB_TOOL_IMPL=sh+py procedure on my 'diffutils' checkout,
I see a failure:

/GNULIB/gnulib-git/gnulib-tool: *** gnulib-tool.py produced different files 
than gnulib-tool.sh! Compare /DIFFUTILS/diffutils-1 and /DIFFUTILS/glpyEnEmgU.
/GNULIB/gnulib-git/gnulib-tool: *** Stop.
./bootstrap: gnulib-tool failed

$ diff -r -q /DIFFUTILS/diffutils-1 /DIFFUTILS/glpyEnEmgU
Files /DIFFUTILS/diffutils-1/gnulib-tests/.gitignore and 
/DIFFUTILS/glpyEnEmgU/gnulib-tests/.gitignore differ
$ diff -u /DIFFUTILS/diffutils-1/gnulib-tests/.gitignore 
/DIFFUTILS/glpyEnEmgU/gnulib-tests/.gitignore
--- /DIFFUTILS/diffutils-1/gnulib-tests/.gitignore        2024-04-20 
17:46:52.402418151 +0200
+++ /DIFFUTILS/glpyEnEmgU/gnulib-tests/.gitignore 2024-04-20 17:46:13.439333961 
+0200
@@ -29,7 +29,6 @@
 /locale.c
 /macros.h
 /mmap-anon-util.h
-/nan.h
 /nanosleep.c
 /nap.h
 /netinet_in.in.h

The particular circumstances are that
  - my last gnulib-tool invocation in that directory was from July 2023,
  - in October 2023, the file tests/nan.h was moved to lib/nan.h in Gnulib,
  - now I'm invoking gnulib-tool again.

I now added this scenario to the unit tests. gnulib-tool.py not only modifies
a .gitignore file when it shouldn't. It also produces this different output:

$ diff -u test-diffutils-1.out tmp524739-out 
--- test-diffutils-1.out        2024-04-20 19:26:38.053084605 +0200
+++ tmp524739-out       2024-04-20 19:28:26.145970250 +0200
@@ -1787,6 +1787,7 @@
   top/GNUmakefile
   top/README-release
   top/maint.mk
+Removing file gnulib-tests/nan.h (backup in gnulib-tests/nan.h~)
 Copying file gnulib-tests/at-func.c
 Copying file gnulib-tests/infinity.h
 Copying file gnulib-tests/isnan.c
@@ -1799,6 +1800,7 @@
 Copying file gnulib-tests/math.c
 Copying file gnulib-tests/math.in.h
 Copying file gnulib-tests/minus-zero.h
+Copying file gnulib-tests/nan.h
 Copying file gnulib-tests/priv-set.c
 Copying file gnulib-tests/priv-set.h
 Copying file gnulib-tests/rmdir.c
@@ -2067,7 +2069,6 @@
 Copying file gnulib-tests/locale.c
 Copying file gnulib-tests/macros.h
 Copying file gnulib-tests/mmap-anon-util.h
-Updating file gnulib-tests/nan.h (backup in gnulib-tests/nan.h~)
 Copying file gnulib-tests/nanosleep.c
 Copying file gnulib-tests/nap.h
 Copying file gnulib-tests/netinet_in.in.h

In the debugger, I see that
  filetable['old'] contains ('gnulib-tests/nan.h', 'tests/nan.h')
  filetable['new'] contains ('gnulib-tests/nan.h', 'tests=lib/nan.h')

While gnulib-tool.sh recognizes that it's the same file name on both 'old'
and 'new' — due to this line
  for g in `LC_ALL=C $JOIN -t"$delimiter" -v1 "$tmp"/old-files "$tmp"/new-files 
| sed -e "$sed_take_first_column"`; do
—, gnulib-tool.py doesn't. The problem is an 'in' operation on the entire
pairs. This patch fixes it.


2024-04-20  Bruno Haible  <bruno@clisp.org>

        gnulib-tool.py: Fix a bug in removed / added files handling.
        * pygnulib/GLImport.py (GLImport.execute): When looking for files that
        are in both filetable['old'] and filetable['new'], consider only the
        first element of each tuple, not the entire tuple.

diff --git a/pygnulib/GLImport.py b/pygnulib/GLImport.py
index e65e3e1235..59c7295cbc 100644
--- a/pygnulib/GLImport.py
+++ b/pygnulib/GLImport.py
@@ -1048,11 +1048,18 @@ def execute(self, filetable: dict[str, list[str]], 
transformers: dict[str, str])
         # Create GLFileAssistant instance to process files.
         assistant = GLFileAssistant(self.config, transformers)
 
+        # Set of rewritten-file-names from filetable['old'].
+        old_rewritten_files = { pair[0]
+                                for pair in filetable['old'] }
+        # Set of rewritten-file-names from filetable['new'].
+        new_rewritten_files = { pair[0]
+                                for pair in filetable['new'] }
+
         # Files which are in filetable['old'] and not in filetable['new'].
         # They will be removed and added to filetable['removed'] list.
-        pairs = [ f
-                  for f in filetable['old']
-                  if f not in filetable['new'] ]
+        pairs = [ pair
+                  for pair in filetable['old']
+                  if pair[0] not in new_rewritten_files ]
         pairs = sorted(set(pairs), key=lambda pair: pair[0])
         files = sorted(set(pair[0] for pair in pairs))
         for file in files:
@@ -1074,9 +1081,9 @@ def execute(self, filetable: dict[str, list[str]], 
transformers: dict[str, str])
         # Files which are in filetable['new'] and not in filetable['old'].
         # They will be added/updated and added to filetable['added'] list.
         already_present = False
-        pairs = [ f
-                  for f in filetable['new']
-                  if f not in filetable['old'] ]
+        pairs = [ pair
+                  for pair in filetable['new']
+                  if pair[0] not in old_rewritten_files ]
         pairs = sorted(set(pairs))
         for pair in pairs:
             original = pair[1]
@@ -1088,9 +1095,9 @@ def execute(self, filetable: dict[str, list[str]], 
transformers: dict[str, str])
         # Files which are in filetable['new'] and in filetable['old'].
         # They will be added/updated and added to filetable['added'] list.
         already_present = True
-        pairs = [ f
-                  for f in filetable['new']
-                  if f in filetable['old'] ]
+        pairs = [ pair
+                  for pair in filetable['new']
+                  if pair[0] in old_rewritten_files ]
         pairs = sorted(set(pairs))
         for pair in pairs:
             original = pair[1]