Congrats on the eggs, hopefully we can help you out
First, if you haven't already removed the egg that was broken, you should do that now so it doesn't affect the development of the other eggs.
Since you mentioned that you are in a very hot and very humid location, it's likely that your hen is exiting the nest to cool off. Nest boxes can get pretty hot and sticky, especially if the male is in there with her too.
Since it's pretty hot, the eggs likely won't cool down too much before she heads back into the box again. Remember, it's totally natural for hens to come out to eat, drink, poop and stretch their wings a bit, before heading back to her eggs.
You can definitely cover the cage a little earlier than normal to encourage her to head back to the nest. You still want her to get enough daylight hours though.
Yo me it doesn't sound like she's given up on her nest, so trust in her natural instincts and I'm sure it will all be fine.