You're right that 3.8L is a fairly large engine. The reason that this engine doesn't make all that much power (or torque) is that it's old-school pushrod technology, and not a lot of work was done on the induction by Ford. It's intended to be one of Ford's corporate engines for mid-size cars, not a performance engine.
You can pick up some HP by doing the usual stuff like a pulley and CAI. You can also make the car feel faster by switching the rear end to 3.73 gears. Any of those could be done for $500 or less.
If you are looking for power you can definitely notice, you're probably looking at either a blower or a head and cam swap (
Super Six Motorsports). Either of those are going to take some handyness and a day or two to put in, but they'll net you 70-100 extra horsepower (at a cost of about $2k).
There are others on this board who have done the engine swap. It's a lot of work, and as has been said before, it's probably just easier to buy a GT and go from there. Even though there are other engines in the 3.8L range that make 300+ horsepower unmodified, they're a lot newer technology. With our 3.8, we just don't have a lot to work with!