Re: Python list micro-benchmarks

[Bruno Haible]

Collin Funk wrote:
> That reminds me, is there a reason why in many places there is
> something like this:
> 
>      newtail = []
>      for item in tail:
>          newtail += [item]
> 
> instead of this?
> 
>      newtail = []
>      for item in tail:
>          newtail.append(item)
> 
> I like the second more, but I can't tell if it is another overly
> strong personal preference of mine...

I like the first one a little more. So I asked the prior knowledge
summarization engine (ChatGPT):

--------------------------------------------------------------------------------
User
For appending an item to a list in Python, is it better to write `l += [item];` 
or `l.append(item);` ?

ChatGPT
Both l += [item] and l.append(item) achieve the goal of appending an item to a 
list l, but they have slightly different implications and use cases.

 1. l.append(item): This is the standard method for appending an item to a list 
in Python. It is clear, concise, and specifically designed for this purpose. It 
directly modifies the list in place.

 2. l += [item]: This uses the augmented assignment operator (+=) with a list. 
It is essentially shorthand for l = l + [item]. While it achieves the same 
result, it involves creating a new list (by concatenating the existing list l 
with [item]) and then assigning it back to l. Depending on the size of the 
list, this can be less efficient in terms of memory usage and potentially 
slower, especially for large lists.

In general, if you simply want to append an item to a list, l.append(item) is 
preferred because it is more explicit and efficient. However, there might be 
scenarios where using += with a list might be appropriate, such as when you 
want to concatenate multiple lists together.
--------------------------------------------------------------------------------

So, ChatGPT sides with your preference.

> Not that we will notice this, but an interesting implementation
> detail:
> 
> >>> import timeit
> >>> timeit.timeit(stmt='var += ["a"]', setup='var = []', number=100000000)
> 6.761068544998125
> >>> timeit.timeit(stmt='var.append("a")', setup='var = []', number=100000000)
> 1.7367604600003688

Since you show me how to do benchmarks, and since ChatGPT warns about large
lists, I ran the same benchmark with a list of length 100 or 1000. See
attachment.

The result is mysterious: Appending to a list of length 100 or 1000 takes
about the same time either way. In particular, the 'var += ["a"]' is *not*
particularly slow when the list is long. It's impossible to clone a list of
length 1000 and add an item to it, in 41 nanoseconds; that makes no sense.

There's something we must have overlooked. Is this being particularly optimized
by the Python compiler? Or is a reference-counting trick providing the nearly-
constant execution time?

Bruno

list-append-experiment.txt
Description: Text document