It might just be normal. Due to how impellers work, they are more efficient at lower speeds (see attached photo, credit: Evinrude 1966) some times the metal bushing that holds the impeller to the shaft is glued on to the impeller. Make shure that it cant turn or slip.