First off, what type of box, and construction method of board is it?
Generally speaking you only need to remove any fiberglass that is delamed or is no longer serving a purpose…‘dead glass.’ This can be done by taking a flat razor blade and cutting around the area to be removed then sliding the blade sideways and around.
If the box is loose it needs to be glued back in with some resin. Depending on how loose the box actually is, i.e. you can move it to one side and pour some resin in, you may need to remove it to be able to get the resin in there.
Mix up your resin, add some cabosil, or milled fiberglass, and if any foam is gashed out you may want to use some white pigment to fill in the cracks. set the fin back to the proper angle and tape until the resin dries. Ready the area around the box for some fiberglass by sanding, tape off the fin slot and add a couple football patches over the box. Cut out the slot once its almost cured, sand and blend in the fiberglass, hotcoat and sand and your done.