Re: [Bug-gnubg] Neural Net to play Trictrac

[Øystein O Johansen]

Hi David,

The TD training code is hidden in gnubg.c line 4635 ff. As you can see, the
code call a function called TrainPosition, which basically takes the
evaluation of the new position and adjusts the weights (by backpropagation)
for the previous position. Suttons TD(lambda) training with lambda = 0.

A value for the learning rate? I have no idea, but I think I would try 0.1
or even larger a value if I started with random weights. Gary and Joseph
know more about this.

-Øystein


----8<-----

extern void CommandTrainTD( char *sz ) {

    int c = 0, n;
    int anBoardTrain[ 2 ][ 25 ], anBoardOld[ 2 ][ 25 ];
    int anDiceTrain[ 2 ];
    float ar[ NUM_OUTPUTS ];

    if( sz && *sz ) {
     if( ( n = ParseNumber( &sz ) ) < 1 ) {
         outputl( _("If you specify a parameter to `train td', it\n"
               "must be a number of positions to train on.") );
         return;
     }
    } else
     n = 0;

    ProgressStart( _("Training...") );

    while( ( !n || c <= n ) && !fInterrupt ) {
     InitBoard( anBoardTrain );

     do {
         if( !( ++c % 100 ) )
          Progress();

         RollDice( anDiceTrain, rngCurrent );

         if( fInterrupt )
          break;

         memcpy( anBoardOld, anBoardTrain, sizeof( anBoardOld ) );

         FindBestMove( NULL, anDiceTrain[ 0 ], anDiceTrain[ 1 ],
                 anBoardTrain, &ciCubeless, &ecTD );

         if( fAction )
          fnAction();

         if( fInterrupt )
          break;

         SwapSides( anBoardTrain );

         EvaluatePosition( anBoardTrain, ar, &ciCubeless, &ecTD );

         InvertEvaluation( ar );
         if( TrainPosition( anBoardOld, ar, rAlpha, rAnneal ) )
          break;

         /* FIXME can stop as soon as perfect */
     } while( ( !n || c <= n ) && !fInterrupt &&
           !GameStatus( anBoardTrain ) );
    }

    ProgressEnd();
}



-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you.