The A/C pulley is no different than your tensioner pulley riding on bearings when it is not in use.
Hacking the A/C out of a good car is rediculous. If you want a light-track car, build a stripped out Notch with crank windows. The v8 version weights in around 2800 lbs.
AC dehumidifies the air and extracts the heat out of it as well. My blown coupe has functioning AC.
Spartan was cool in the 60's, but it will severely hurt resale when you become tired of your car.