I agree with both of these guys on this one. Mostly, anyways.
You are going to want to be careful on how harshly you scrub/treat the seatbelt. YOu don't want to weaken it in any way. I would try pulling the seatbelt all the way out, treating with something like Folex (maybe even dilute it 1:1 with distilled water), agitate lightly with a toothbrush, and then have at it with the pressure washer.
Folex can be found at walmart, by the brooms/mops. It comes in a 1L white spray bottle with purple writing. This is what I'm using right now for interior carpets.
You could also use a citrus degreaser. You can buy the Zep one at home depot.