bug-gnubg
[Top][All Lists]

## RE: [Bug-gnubg] Luck analysis question

 From: Nis Subject: RE: [Bug-gnubg] Luck analysis question Date: Wed, 11 Jun 2003 17:59:26 +0200

```

--On 11 June 2003 11:28 -0300 Albert Silver <address@hidden> wrote:

```
```Perhaps one interesting way to examine this would be to run GNU 0.12,
the version that played sgainst Snowie 3.2, and have it analyze the luck
factor and see what the results are.

The reason I was a bit concerned is this: the luck is NOT independent of
the evaluation of the moves and positions as I see it.
```
```
```
Of course the measured luck is not independent of the measured skill. This follows from the equation
```
luck + skill = final - initial

(found in Zare's article).

```
```It is entirely
dependent in fact. Suppose in a given position, I say the equity is
0.400, move A is the best giving that 0.400 equity, and move B is a
blunder leading to only 0.250. Mind you, this is because I do not
realize that mave B is in fact stronger and leads to 0.450.
```
```
I'm with you so far.

To summarize:

Move               A       B
Gnubg Equity    0.400   0.250
True Equity(TM) 0.400   0.450

```
```You
correctly play move B, I play a move that does nothing still thinking it
is only 0.250, and you play a move that improves the position by a mere
0.050. When I analyze this new resulting position, I now see the truth
of it and see that the current equity is 0.500, however having misjudged
it previously will declare you're a lucky guy whose move led to a 0.250
gain.
```
```
```
I lost you here. I will try to draft the sequence of events (P is a position, R is a roll, M is a move. M' and P' are the moves and resulting positions used by gnubg for luck analysis:
```
P0 (starting position)
R1
M1 | M'1
P1 | P'1
R2
M2 | M'2
P2 | P'2
...

Rn
Mn | M'n
Pn = P'n (Clear win or loss)

Luck = sum from i= 1 to n of P'i - P(i-1)

Skill = sum from i = 1 to n of Pi - P'i

Thus Luck + Skill = Pn - P0

```
The nice thing about the luck calculation is that it is mathematically guaranteed to be unbiased. This is obtained by having Pi evaluated one ply deeper than P'i. This is not to say that it is always correct - only that the errors average to zero. Thus the same goes for the measurement of skill calculated like this - it is unbiased, but imprecise.
```
```
Note that the calculation of skill given above is different from the one normally used for analysing single moves - since P'i is on a lower ply than Pi.
```
Assume we have reached position Pn.
```
Assume that the value of P'(n+1) when rolling 4-3 is misevaluated by -0.200 (possibly because gnubg chooses the wrong move). Assume that the other rolls are evaluated correct. This affects the value of P(n-1) by -0.200 * 1/18 ALWAYS, no matter if we roll 4-3 (since this is a 1-ply evaluation. It affects the value of P'(n+1) by 0.200 IF we roll a 4-3, for an average value of 0.200 * 1/18. The average influence on luck is thus 0.
```
```
```My point being only that the luck factor IS dependent on GNU's
evaluation as far as I can see. As to the variance reduction in
rollouts, it is different. Variance reduction doesn't guarantee the
rollouts will be correct, it just reduces the number of trials needed to
reach the same result of a larger number of trials without variance
reduction. Thus I can get the right or wrong result with only 1000
```
```
```
Well, luck analysis is exactly the same as variance reduction. It increases the accuracy of the measurement of skill provided by every game. What we are doing is (in the simplest case) equivalent to a one-trial rollout of the value of the starting position (given the two players).
```
--
Nis Jorgensen
Greenpeace
Amsterdam

```