Re: [Bug-gnubg] Confidence intervals from rollouts

[Nis Jorgensen]

--On 04  Sep 2002  08:22 -0700 David Montgomery <address@hidden> wrote:

> Consider a single position, for which we have 3 rollout
> samples A, B, and C.  The idea of rotating the first ply
> or two is that the variance of the *difference* between
> two plays should be reduced, since one aspect of the
> luck has been eliminated.

Permission to disagree, sir. I lack the understanding of exactly what you 
mean above, but I do understand that I disagree on several levels.

The idea of rotating the first ply or two should be to reduce the 
difference between the "true" value of a position and the results of 
rollouts.
The variance (or rather standard error) is just a _measure_ of how much we 
trust the result, and reducing the value is not a goal in itself.

This is very important to stress, especially in cases like this, where we 
should expect the standard error to go _up_ even though the actual 
trustworthyness of the rollout should improve

Also, I don't understand why you bring "different plays" into this. This is 
of course relevant for the "duplicate dice" evaluation, but not for the 
rotation (for which I would reserve the word "stratification").

> So, for example, we would
> expect that the standard error of abs(A - B) would be
> less than sqrt(2)*[true standard error of rollout of
> that size of that position].

I am not sure what you mean by the "standard error of abs(A-B)". I assume 
you just mean Abs(A-B)?

> So I think what I did was to consider that from C
> I could get an unbiased estimate of this true standard
> error, and then I compared this (* sqrt(2)) with abs(A-B).
> I repeated with C-A vs B and C-B vs A, and repeated the
> whole thing for hundreds or thousands of positions.

Hmmm - I think I start to see what you are doing here. I am not convinced 
that this is a good way to estimate the value of stratification.

I think I will do some simple coin experiments, and see if this brings me 
any insights.

--
Nis Jorgensen
Greenpeace
Amsterdam