Welcome to the world of Mustangs! There will be alot of mixed opinions in the answers to your question, however, it will be alot easier to narrow things down if you tell us what your budget is.
Free - Bump timing a couple degrees (12* a good start)
- Pull the air silencer off (if not going with cold air intake setup)
50-100$ - Ebay cold air kit/filter
- Quality tune up plugs/wires/cap/rotor
- Synthetic fluids
100-500$ - Gears (probably the number one choice among enthusiasts!!)
- Intake manifold such as Cobra/GT40/Edelbrock/Trick Flow
- Underdrive pulleys. Mixed reviews on these, but they do free up power!
- Nitrous Oxide kit
- New seats! While not a performance mod, your body will thank you for this, and new seats will jazz up the interior quite a bit!
500-1k $ - Cylinder heads. These will probably give you the biggest power increase of anything else you can upgrade short of forced induction.
- Camshaft. The reason why I put a cam in this price bracket is because of the other supporting hardware you'll have to buy to go along with the new cam.
1k - 3k $ -Forced Induction! Supercharger/Entry lvl turbo kit
-Complete engine top end kit such as the TFS kits
The sky really is the limit with these cars. You just have to set your goals of what you want to do with the car ultimately then prioritize the steps to make it happen. Having realized my own goal with my last 'Stang, I can tell you it's fun and frustrating and expensive all at the same time.
Good luck and enjoy your new ride!