You might not be getting the square center part of the cable mated up straight with the speedo, itself. When installing the speedo cable in her Capri, I had to have Frankenstang turn the other end of the cable slowly while I pushed the gauge cluster end into the back of the speedo until it finally clicked into place. (Doing this by myself while installing the speedo cable on my Notch was a major PITA.)
Likewise, when installing the VSS (Vehicle Speed Sensor) end of the cable, I've found that it's easier to remove the VSS first, plug the cable into it (allows you to rotate the gear and such to get it to mate up), and THEN plug the assembly into the tranny. Might save you a bit of a headache there, hopefully.