[Serusers] newbie question

I did a test last week: setup: 2 computers on separate locations, on both installed: Skype and XTen's free sip phone: x-lite, 1 SERr server on a third location Linksys WRT54g router on one location, Cisco SOHO78 on the others. No QoS. test: voice connection with skype between the two computers: perfect quality test2 voice connection with x-lite between the two computers: After authorizing both computers on the third server, the established connection showed terrible quality...long silence during phonecalls. First I thought that the actual x-klite/sip voice traffic is routed through the server and since the server was heavily loaded and the router didn't have QoS, the bad quality was explainable. Later I heard that the actual voice traffic is not routed through the ser server, but goes directly from one computer through the other. Now if the latter is true, how can one explain the quality difference between skype and x-lite/sip ? Thanks in advance for your answers.