I have openser configured with asterisk as following:
If a sip REGISTER packet arrives,
     fix_nated_register()
     save("location")
     exit;

If a sip packet arrives at openser with a source ip != asterisk ip,
      t_relay(to asterisk)
If a sip packet arrives from asterisk,
      lookup("location");
      force_rport()
      fix_nated_contact()
t_relay();

The asterisk dial plan says the following:
if inbound traffic arrives from openser's ip address, Dial(SIP/dstuser@dstdomain).

phones a and b are on separate networks behind a firewall, openser and asterisk are on public ip addresses.

The sip traffc seems to work just fine. I.e. all the handshakes seem to be happening as they should.
However, rtp traffic does not.

Whether audio traffic will transmit or not is a crap shoot.
Sometimes, if a calls b, b hears a but not vice versa, while if b calls a, audio is two way.
In this case, one audio stream is going through asterisk, the other is being directed to go point to point.
Sometimes, a calls b and b hears a, and a hears b for a second but a second INVITE comes to phone B that causes it to redirect rtp to be point to point.
Sometimes there is no audio.
Sometimes, everything works fine.

At one point, rtp from a was going to asterisk, but asterisk was not sending the rtp on to b, and b was trying to send traffic point to point.