How are you attempting to shift? Remote control? If so, remove cable at engine and see if engine will shift on it's own. If so, control box shifting arm worn out and binding.
If it does it with remote disconnected you have something binding internally or you didn't get your balls and spring installed properly, that control the position of the clutch dog inside the prop shaft. I assume the engine runs ok in F and in gear with no grinding or funny noises.
The shift rod from above spline connects to the shifter splined in the LU. The other end of that rod splines into a piece of metal that has 3 groves machined into it and each is a different distance from the pivot point of the piece of metal which is called the shifter cam. The front of the LU casting has a cutout for the cam and a hole at the top and bottom, similar to your doing this......stick your thumb between your index and middle fingers and bend those two fingers back into your palm. The thumb is the cam.
On the front of the prop shaft is a pointed cylinder that rides in the groves in the cam and it's name is the "cam follower". There are 2 springs associated with it, one going around the clutch dog to hold a locating pin in place which is the pin that slides through the cam follower pellet locking it to the clutch dog, and the other spring is in the canal with the cam follower along with 3 ball bearings (3 in every LU I dissected, never did a 50) whose purpose is to keep the follower jammed against the cam detent so that you will remain in the gear selected.
As the shift rod is varied through the 3 positions (FNR) the cam follower turns and as it turns, since all 3 grooves are at different distances from it's pivot point, it forces the clutch dog to move back and forth.
F gear is where the dog is all the way forward contacting the rear of the spinning F gear. Both F and R gears are constantly spinning as the drive shaft turns (them), but the dog position determines whether or not they are connected to the prop shaft and which one, aka F, N, or R .
N is the mid position where the dog is positioned between both gears allowing the prop shaft to free wheel.
R is where the cam is exerting max pressure on the cam follower and pushes it (and the clutch dog) back against the R gear cogs.
In walking through this process with you I am looking for a failure mechanism to support what you said about going into gear but not wanting to come out. I don't see one other that binding on the prop shaft between the dog, pellet positioning the dog, and the prop shaft...maybe groves worn in it that cause the dog to bind.
I assume that when shifting you are at idle rpm.....600-700 rpms. If you aren't that could explain shift binding......just like the manual transmission in a tractor or a car/truck where you are driving along in 1st or 2nd gear and with it in gear, no clutch, you let off the gas. The pressure of the rear tires grabbing the road puts pressure on the drive line and as a result the meshed gears don't want to separate....unless you release the pressure....push in the clutch, or give the engine some gas getting the input shaft of the tranny in sync with the output shaft, relieving the pressure and allowing the gears to disconnect.
Think about what I said and talk more tomorrow.
Mark
