He never says he was pushed out, just that he thought he would be a distraction to his colleague's re-election. Meaning, if you stay in the race Joe, you will hurt your colleague's reelection chances.
For this reason, only he did the selfishness of falling on the sword.