Well, it's now the default behavior. It wasn't previously. :-)
That is, I haven't changed my completion settings since the mid-90s (except the icomplete-mode, which I set up a decade ago) , and they all worked fine until emacs 27.
They look like this:
;;; Shows globs of possible completions in minibuffer
(setq-default completion-auto-help t)
;;; Case-insensitive file and buff
(setq read-file-name-completion-ignore-case t)
(setq read-buffer-completion-ignore-case t)
;;; Show completions in M-x
`M-: completion-styles` shows the same thing in Emacs 26 and 27: '(basic partial-completion emacs22)
In Emacs 27, I tried `(setq completion-styles '(basic emacs22))` and `(setq completion-styles '(emacs22))`, but the completion behavior remained broken.
(Perhaps interestingly, when I tried `(setq completion-styles '(emacs22))` in Emacs 26, while it still got stuck on the first Gemfile, the minibuffer started showing me that the collision was caused by Gemfile.lock, which it never did before.)
In any case, it seems like I'll need to stay with Emacs 26 for a while, as C-Tab is by far my most-used minibuffer command.