I had the same reservations as you for all the same reasons. My kids 9,6,4. It took losing someone close to us for me to really decide to go for it.
I found the car I wanted 100 miles away in PA, it was pre-owned and had 600 miles on it. The previous owner won it at a casino, drove it home and traded it in for F-series. At one point I talked myself out of buying it and my Wife actually got pissed at me, she then said she was going to buy it for me for our 10 year anniversary and she would not try to negotiate the price. She knew that would be too much for me, so I called the Dealer and they accepted my offer, to my surprise. When I hung up the phone and told her I bought it, a funny thing happened, she got really horny.
It kind of sucks not to have 5 seats, but it keeps the mileage down. My wife has a SUV and my daily is a 98 VW jetta TDI, which is pretty much a turd on wheels, so that makes the Stang feel even more special when I drive it. As for the kids, they love it, the 9 yo is the official shifter and he thinks the ambient lighting button is the rocket boost. I caught him actually blowing a kiss to the car in the garage and I have heard him protecting it from his little brother and sister. I told him I will share it with him when he gets his license.
It has been said already, life is short and you can't take the $$ with you. We only have a few years left until the kids find out the truth about us, and they will not want to talk to us anymore. A cool car might make them stick around a little longer. IMO the Vert is the way to go, It gives you the same feeling as riding a bike, but with the safety of 4 wheels and a cage.
Good Luck