Re: [Bug-gnubg] Analysis of race rollouts

[Joseph Heled]

I am still working on analyzing and thinking about the races results. 

Ian Shaw wrote:
> 
> I've just had  a look at Joseph's analysis of race rollouts. I've got a bunch 
> of questions, but I'm short on answers.
> If I understand the numbers correctly, 2-ply evaluation produces less errors 
> than the one sided bearoff database (OSRDB). How can this be? One would 
> instinctively think that the OSRDB would be most accurate.

I am not sure yet. It might be the OSR noise (it is a rollout of sorts). It
might be (Like I said in the page) that the two perform the same, and it is the
noise from the race rollout itself. It might be because the "sub-optimal" play
of the OSR is significant in more positions than I thought. I hope I can find
the answer, as this is not trivial.


> As Joseph notes, the rollout might be wrong. With 97507 positions, we can 
> expect some of them to produce results outside the 95% CI. Would comparing 
> the rollout to Sconyers' or Trice's databases prove anything?
> The OSRDB might be wrong in positions where two sided database might get it 
> right. E.g. desperate situations where you need to play to take advantage of 
> specific doubles. I guess it's not surprising if 2-ply gets some of these but 
> OSRDB can't. Is it possible to find the error rate for OSRDB supplemented by 
> the two sided database when the position falls within the scope of the latter?
> Jørn's been slaving away at extending the OSRDB, but are we getting anything 
> extra from it if 2-ply is just as good/better?

It is much faster, and it works for any position, but I think the point is
valid. That is why making this "unbiased benchmark" is so important. Now we will
be able to make informed decisions.

-Joseph

> Is there any value to comparing 3 or 4-ply results with the rollouts? I'd be 
> prepared to give some PC time to this if it would help.
> 
> ---
> Ian Shaw
> 
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/bug-gnubg