The "success" response sent from server to client in step three of DIGEST-MD5 contains the "rspauth" key. There seem to be two ways this final message can be sent, according to [1].
A: The rspauth message can be sent as the body of the success message.
S (challenge): realm="example.com",nonce="...",... C (response): nonce="...",response=...
S (success): rspauth=...
B: The server can send the rspauth message as a challenge, expect an empty response, and then send success with no message.
S (challenge): realm="example.com",nonce="...",...
C (response): nonce="...",response=...
S (challenge): rspauth=...
C (response): "" S (success): (null)
The DIGEST-MD5 implementation in gsasl seems to do the latter. This wastes a round-trip, so I looked into eliminating it. It seemed trivial: at the end of step 1 in digest-md5/server.c, return GSASL_OK instead of GSASL_NEEDS_MORE.
However, when gsasl_step() returns GSASL_OK, how does it distinguish whether the output is the empty string or no response at all? In B above, the success response is not the empty string, but null (DIGEST-MD5's third and final step concluded with the second challenge). rfc6120 [2] explicitly distinguishes these: if success does not contain data at all it sends <success/>; if it contains zero-length data it sends <success>=</success>.
Currently, my code (a minor patch to jabberd2) assumes that if gsasl_step() == GSASL_OK and strlen(out) == 0, it means to send no response rather than an empty response, but that's a brittle assumption. The obvious thing would be for gsasl_step's "out" parameter to be set to NULL, but it doesn't do that.