I figured it out! On the layshaft the gears after the spacer were pressed on the wrong way around. So I pressed them out flipped them the other way, and pressed them on again. This gave me the clearance for the uprated bearing and the bolt I needed. The bolt is tightened at the end and the upgraded end case is fitted without any problems.