Short Answer is YES.
You are loosing all your MPG due to the O2 not being hooked up.
The computer needs the input from the O2 sensor because it's tells the computer how the mix is going. As soon as the computer does not receive the 0 to one 1 volt signal, it throws the CEL and goes into limp mode - it's best safe guess for what fuel it needs based upon the throttle position sensor and Mass Air (90 to 93) or MAP sensor (87 to 89). The limp mode is set rich to keep the engine from burning up in a lean condition. Much cheaper to replace spark plugs than an engine.
The O2 fine tunes the mixture to balance out the pulse of the injectors. When the computer sees less than 1/2 volt it pulses the injectors more (the lean side) and when it shows over 1/2 volt it leans out the mixture - this happens several times a second.
I have a 93 and an 88 Mustang - i drive the 93 daily - most highway but some city. I get an average of 27 to 28 with the ethynol crap they use here. I do have a 5 speed and i run an 8.8 axle with 2.71 gears - i get my best MPG in 4th doing 75 MPH at 2500 RPM. The 88 is a convertible and still sports the original 7.5 axle, and on my last road trip it got 30 MPG.
In short, you need to hook up the O2 sensor - it should remove the CEL. The O2 is before the cat originally, and having no cats is actually a benefit - it runs better and more power. I gutted both converters on both cars and it gained from getting 24 MPG to the 27 .
Get a 93 or 94 Ford Ranger exhaust manifold for a 2.3 - it's a factory header with the O2 bung in it. (95 to 97 Rangers are similiar but no O2 bung) The Ranger header will bolt in without having to move your current exhaust pipe.
Also, disable the EGR. The tube that runs from the exhaust up to intake isn't doing you any favors either. It's worth the effort to plug it up at both ends (but leave the electric hooked up). You can plug it up cheaply (or free) by undoing the pipe, cutting it in half, take a hammer and flatten the cut part of the pipe for about 3-4 inches, thentake some vise grips fold about an inch or so over and then hammer the bent part back to the flatten out part. do this for both areas and bolt them back into the manifold and intake.
If you still have the stock air delivery tube from the air filter to throttle body, there is a major restriction in the bulge area. you can remove the inner plastic parts of this by undoing the hose clamps and you will see an inner plastic 2 piece that can be pried out of both ends with a screwdriver, it takes a little effort to get it started, but once you get going, it pops out. It necks down the airflow to a little over a size of a quarter.
After you do any of the above, disconnect your battery for 10 minutes and let the computer reset itself and learn it's new found power.