Re: global variable to avoid call-by-value

[David Bateman]

Stefano Ghirlanda wrote:

> Dear all,
>
> I am developing a small library for some neural network stuff. Since a
> network is potentially a large structure (node layers, weight
> matrices, etc) I figured that call-by-value could be a problem.  After
> reading a related thread on the list I tried to avoid call-by-value by
> creating a global cell array that holds network structures, and
> passing indices into this array to functions. It made no difference
> (details below).
>
>  
Call by reference will only make speed difference for values that are 
altered. The reason is that when octave creates a copies of the variable 
it copies for example the container of type Array<T>, copies the pointer 
rep that is a pointer to the actual data and increments the reference 
counter rep->count. This is a relatively low cost operation. If you try 
to alter the passed variable in the function, the first time you try 
octave sees that the rep->count is greater than 1, and at this point it 
will copy the data.


> One example. Suppose I want to update the activation state of nodes in
> the network. In a language with pass-by-reference I could write
>
>  update_network( &net )
>
> (adopting the C++ & syntax for references), while with call-by-value I
> have to write something like
>
>  net = update_network( net )
>
> where update_network uses the following simple-minded strategy:
>
>  function net_out = update_network( net_in )
>    net_out = net_in;
>    # operate only on net_out from now on
>  endfunction
>  
You don't gain anything with the copy of net_out = net_in, except an 
extra copy of the data container, which isn't that costly. So keep if if 
it makes your code more readable, but

function net = update_network (net)
 # operate on net
endfunction

works just as well.

> Note that update_network does modify the network rather heavily
> updating the activation states of all nodes, possibly the nodes'
> internal states and also some other maintenance variables.
>
> I have tried use the global variable approach in more or less this
> way:
>
>  global Networks = {}; # global structure holding all networks
>
>  function update_network( net_index )
>    global Networks;
>    # operate directly on Networks{ net_index }
>  endfunction
>
>  

Ok, so you have a case where it should make a difference. But only makes 
a single copy at each call of the function update_networks

> I have used exaclty this strategy for every function call that would
> need to modify its argument. I have then run a test teaching a
> two-layer network to respond with given outputs to four inputs. I have
> used a network with 25, 250, 2500 and 25000 input nodes with these
> results (times in seconds):
>
> Global variable approach: 13, 13, 15, 41  
> Simple-mindes call-by value: 10, 10, 13, 39
>
>  
The differences are in the noise... I suspect the cost of the copy is 
low relative to your operations on the data.

> So the latter is, if anything slightly better. Should I conclude that
> Octave is very very smart in deciding when to actually copy data and
> when to just count and shuffle pointers, or am I missing something?
>
> For instance, does Octave notive that the "net" on the lhs and rhs of
>
>  net = update_network( net )
>
> are the same object and thus avoids any duplication of data
> whatsoever?
>  
Octave definitely doesn't do this, though it has been discussed in the past.

Regards
David


--
David Bateman                                address@hidden
Motorola Labs - Paris                        +33 1 69 35 48 04 (Ph) 
Parc Les Algorithmes, Commune de St Aubin    +33 1 69 35 77 01 (Fax) 
91193 Gif-Sur-Yvette FRANCE

The information contained in this communication has been classified as: 

[x] General Business Information 
[ ] Motorola Internal Use Only 
[ ] Motorola Confidential Proprietary



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------