As in remove the catalytic converters?
You sure can, but before you do make sure you don't have to pass any emissions tests where you live.
The easiest would be just to have an exhaust shop cut them out & replace them with pipe (but be sure to ask them to save them for you - they are worth some $)
Im not too familiar on the mustang since I just upgraded from a different car, but you will (pretty sure) throw a CEL without the cat on as well. There are ways to try to trick the car to not show this though.
Also, realize the cat is there for a good reason (if you care about pollution at all),