Funny you say this. I've driven a 97GT, an 04GT, and an 06 GT. They all 'felt' the same to me, but I knew the 06 was faster because it is happy to rev higher. What really makes a car fun to drive is TORQUE. All these engines have fairly similar torque numbers, being the same displacement. I know the 06 has a bit more torque, but it also has a couple hundred lbs more to drag around.
I would argue the 4.6 isn't 'fast' without forced induction. Yeah, you can do all these mods to rev higher, but why? You are only extending the torque you have to higher rpms. Then, you are fighting to get the rpms up like a honda, while the LS1 next to you makes an extra 100 ft*lbs from idle to 5000rpm. The 4.6 is a bit too small in my opinion, and don't count on the 5.4; the rod ratio goes to hell.
I guess I became disenchanted with the NA 4.6 when I race my 2.3 vs an 00 GT with nearly every NA mod you can imagine... ported heads, cams, long tubes with full exhaust, ect, ect ect. Literally 5000+$ in mods and he was happy with the 280+rwhp he made on the dyno. It honestly wasn't even funny the way I ran away from him! And I'm running basically junk yard parts, stock turbo, stock intake/head/exhaust manifold, and not reving over 5500 (yes, the 1/4 mile time on the 2.3 is out of date).
I think to be competitive with what's stock and on the streets now, the 4.6 needs forced induction because it is a bit on the small side. True fun comes from big torque, and the NA 4.6 doesn't have it.
I have had 2 96 gt's , an 03 gt and an 03 mach 1. The npi engines just suck. plain and simple. Anyone who says they can be made fast is full of it. they probably just havnt had a FAST car. Do a pi swap. I found an 04 engine with 27k for 900 bucks. still had paint on the pullies. You will never get better than 13.9- 14.2 with all the bolt-ons and drag radials. If you really want your car fast, put a pi motor in and be content. if you get too radical with heads on these motors, you lose torque. and you cannot make up with revs unless your bottom end is built for it. just my two cents.......