Also my first post - actually 2nd but first was just me complaining about registration process for this site
I also use Voip behind my endian and dont have any issues. From what you are describing, it sounds more like bandwidth issues rather than port forwarding / firewall blocking. If it was the firewall it probably wouldnt work at all or you would get things like one way speech or ring connect and no speech at all. Are you using Asterisk or something based on Asterisk, if so check codecs etc. Thats my 50 cents anyway.