[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[INSTALLED 5/5] sort: fix unlikely int overflow with -r
From: |
Paul Eggert |
Subject: |
[INSTALLED 5/5] sort: fix unlikely int overflow with -r |
Date: |
Tue, 17 May 2022 19:32:25 -0700 |
* src/sort.c (keycompare, compare): Don’t overflow if -r is
specified and a comparison function returns INT_MIN, as this
causes the comparison to have undefined behavior (typically the
reverse of correct). glibc memcmp on s390x reportedly returns
INT_MIN in some cases, so this is not a purely academic issue.
---
src/sort.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/src/sort.c b/src/sort.c
index 29c9f39f3..81045b1ab 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -2794,7 +2794,9 @@ keycompare (struct line const *a, struct line const *b)
}
}
- return key->reverse ? -diff : diff;
+ if (key->reverse)
+ diff = diff < 0 ? -1 : -diff;
+ return diff;
}
/* Compare two lines A and B, returning negative, zero, or positive
@@ -2839,7 +2841,9 @@ compare (struct line const *a, struct line const *b)
diff = (alen > blen) - (alen < blen);
}
- return reverse ? -diff : diff;
+ if (reverse)
+ diff = diff < 0 ? 1 : -diff;
+ return diff;
}
/* Write LINE to output stream FP; the output file's name is
--
2.36.1