QOS (quality of service) is a critical and necessary component of any good phone system. Even great internet needs QOS since operating systems run updates and phones search for firmware, this can impact the audio quality on your voice network.
If your firewall or network gateway appliance does not have this function....REPLACE IT, OR SUFFER THE CONSEQUENCES!
commonly you would like to have 2 networks one for VOIP and the other for your data network. If you are as to be so lucky to be running both services on one network, it's not impossible to get good call quality but it will require more rules.
Step one) Determine your actual internet speed not the speed your ISP states you have but the speed report from a service like http://speedtest.net
Step Two) Set your download and upload speed to the network interface distributing internet to phones and your PBX
calculations: 1MB is equal to 1024KB
Step Three) Prioritize UDP network traffic to your PBX as a rule of thumb, then prioritize by port and since VOIP does not commonly use TCP
it is safe to assume this network traffic to be of lower priority so set accordingly.
In the image provided is a sample configuration made in a ClearOS firewall