Hi there and
to the forums!
Congratulation on your new baby budgie--they certainly are fascinating little creatures, aren't they?
I'm glad he's starting to be more comfortable in his new home, that's good to know. Budgies are terrified when they first are moved to a new environment, you're right. If he still seems agitated sometimes, playing some soft music in the background will help him to relax. Budgies perceive silence as danger, so in general you should leave music on when there aren't people in the house or in his room for long periods of time.
Even though he seems comfortable with your finger, I would lay off for a little bit and let him slowly get used to you. He very well may be still adjusting and may be acting submissive simply because he's so scared of you!
It's important to go at his pace and slowly introduce your hand into the cage. If he really does seem comfortable with your hand and doesn't seem scared at all, you can try feeding him some seed or millet in the palm of your hand and letting him step up on his own.
I'm glad he likes being out of his cage sometimes, he will get more used to the home environment in time. So that he isn't overwhelmed, focusing on just one room of the house that is completely budgie proof (no open liquids, poisonous plants, dark crevices, open stoves/ovens, fireplaces, etc) will help him to feel more comfortable out of the cage.
As for his toys, you're right that he needs more time. Many budgies don't touch their toys for more than three weeks after coming home! Once he starts to see his cage as his "home", then he will be more open to exploring his toys
Keep trying with the veggies! The key is patience and consistency, as well as variety. Offer him several different veggies per week served in different ways, such as shredded carrot one day, chopped parsley another day, etc. Eventually he'll get curious and take a nibble, I'm sure
Finally, budgies often go through long periods of not needing calcium from a cuttlebone or a mineral block in their systems, so they seem to ignore it, until one day when they decide it's time for a calcium boost and they can go through a whole cuttlebone in a day!
Because of this, it's important to keep both in the cage, as you have done, so that one day when he decides he'd like some calcium, he knows where to find it
Be sure to check out the Budgie Articles as well as the "stickies" (important threads "stuck" to the top of each subforum for easy reference) for tips and information on everything from budgie behaviour to a budgie's diet and everything in between. The links Deborah provided will be a great place to start!
If you have any questions after reading through everything, please be sure to ask as we'd love to help!
Hope to see you around and your little budgie, as well