Welcome to the forums!
Budgies bite as a response when they are feeling scared/threatened, when they want to defend their personal space and boundaries and as a result show displeasure and we get a bite in return.
How long have you had your budgie? Most budgies do not like to be petted, when it comes to that particular form of affection, they would much rather be preened by a same species companion.
Before having more of a hands on approach with your budgie, you need to gain your budgie's trust in you. To build the foundations of trust, you have to go back to the very beginning as far as training goes. Sit close to your budgie's cage and talk/read to him in a calm, positive and encouraging way. By doing so, your budgie will increasingly get used to your presence and your voice and soon he will realize you are not there to hurt him.
It's important to watch your budgie's behaviour and body language, when you see him getting more curious towards you and moving closer to you while still in the cage, then you can slowly introduce your hand to him, always respecting his personal space (don't try to reach him, let him come to your hand), you can offer him a tasty treat and encourage him to step onto your hand/finger.
Be sure to check the Training and Bonding section of the forums for all the detailed info and tips on how to form a lasting bond and on how to tame your budgie.