
From:  Nis Jorgensen 
Subject:  Re: [Buggnubg] Confidence intervals from rollouts 
Date:  Thu, 05 Sep 2002 11:00:56 +0200 
On 04 Sep 2002 08:22 0700 David Montgomery <address@hidden> wrote:
Consider a single position, for which we have 3 rollout samples A, B, and C. The idea of rotating the first ply or two is that the variance of the *difference* between two plays should be reduced, since one aspect of the luck has been eliminated.
Permission to disagree, sir. I lack the understanding of exactly what you mean above, but I do understand that I disagree on several levels.
The idea of rotating the first ply or two should be to reduce the difference between the "true" value of a position and the results of rollouts. The variance (or rather standard error) is just a _measure_ of how much we trust the result, and reducing the value is not a goal in itself.
This is very important to stress, especially in cases like this, where we should expect the standard error to go _up_ even though the actual trustworthyness of the rollout should improve
Also, I don't understand why you bring "different plays" into this. This is of course relevant for the "duplicate dice" evaluation, but not for the rotation (for which I would reserve the word "stratification").
So, for example, we would expect that the standard error of abs(A  B) would be less than sqrt(2)*[true standard error of rollout of that size of that position].
I am not sure what you mean by the "standard error of abs(AB)". I assume you just mean Abs(AB)?
So I think what I did was to consider that from C I could get an unbiased estimate of this true standard error, and then I compared this (* sqrt(2)) with abs(AB). I repeated with CA vs B and CB vs A, and repeated the whole thing for hundreds or thousands of positions.
Hmmm  I think I start to see what you are doing here. I am not convinced that this is a good way to estimate the value of stratification.
I think I will do some simple coin experiments, and see if this brings me any insights.
 Nis Jorgensen Greenpeace Amsterdam
[Prev in Thread]  Current Thread  [Next in Thread] 