So, just feed the audio as if it's a baseband signal right into the
USRP? Or do I need to put it through an SSB modulator
first?
You'll want to modulate it. WSJT gives real output. Of course "modulating" here means not much more than complexifying and filtering, since WSJT will do the tuning on its own.
BTW if sample rates do turn out to be an issue with JACK, you can always use the dummy driver simply for connecting your GR program and WSJT. Dummy will take whatever sample rate you tell it, within reason. You can't monitor the signal by ear that way, but in this case, big deal.
Frank