If he's calmed down now, then leave them right where they are. It will take him a few days to get used to the new items in his cage.
If the budgie is still going "crazy", then take the swing and perch back out of the cage and put them outside the cage where the budgie can see them.
Leave them outside the cage for a few days and then try again. Be sure you move slowly and carefully at all times and talk gently and reassuring to the budgie whenever doing anything in or near the cage.
In the future, always place new items outside the cage for a few days so the budgie can see the items and become used to them before introducing them into his cage.
It's also best to introduce new items in the morning so the budgie will have the full day to get used to having them in the cage before he needs to settle for the night.