Triple checked everything, ran the car in 6th gear without shift linkage and it still popped out, at this point it's narrowed down to either faulty parts or 5th and 6th gear decided to leave the chat while the car was on jackstands. I've got a replacement assembly coming in tomorrow so we'll see.
I just replaced the shifter assembly (bearing, seal, and breather) all pre assembled off ebay. I've taken the assembly in and out of the car like 8 times and verified that I'm putting everything back together correctly. From what I can tell it's not linkage, everything's put in correctly...