I was in a really neat craft shop the other day and saw some little coloured cupcake papers (the kind that you pour your cupcake mix into when you bake them) as well as some paper straws. I assume the ink in these would be safe for my budgies, since they are safe for us to use with food, but I thought I would double check before I bought any. I tried googling but couldn't find a clear answer.
As long as the dye is food safe, then it should be safe for them! I'd make sure you buy them from a reputable place to ensure that they're actually safe. Just make sure you don't use the metal ones as I believe those could cause issues if ingested.