The topic of Hydro-Electric shift has been discussed here so many times, that I guess we get tired of typing it over again. Do a search and if you are still in the dark, we can be of help.
To sum up your problem, you either have a electrical problem like a bad shift switch or wires, or you have a hydraulic problem in the lower unit. Basic electrical troubleshooting methods apply to the first. Unfortunately, there isn't much you can do about the hydraulic part without going into the lower unit. So.......obviously the first thing to do is electrical checks. If it is all good, then it's on to the next part.
However a clue here is that RPM seems to have an effect. That suggests a hydraulic problem. But don't assume. It could also be receiving higher voltage from the alternator at high RPM, which translates to more voltage being applied to the shift circuit.
Now that I have you thoroughly confused, start with voltage checks at the blue & green wires leading down to the lower unit, and resistance checks from there down. If you don't know how to do voltage and resistance checks, that is another lesson.
I'm betting on a bad push button shift switch because they are notorious for that. But don't assume----TEST.
EDIT: See, there I go assuming just because of the shift switch reputation. But thinking about the RPM factor, I'll go out on a limb and bet the blocking diode is open or missing. The blocking diode is between the ignition switch and shift switch. If the diode is open or missing, the shift supply comes entirely from the alternator, which is affected by RPM.
