You are barking up the wrong tree, thinking starter or solenoid. You have bad key switch. That is, assuming everything is wired correctly and correct parts.
Refer to the diagram I posted. When you turn the key on, the BAT terminal is connected to the IGN terminal, which sends juice to the ignition system so it can run. When you turn the key to the start position, the BAT terminal is connected to both the IGN and ST terminals, so the ignition is still on and the starter will run. Let the key off to the run position and now only the IGN is connected to the BAT again so it keeps running without the starter.
Trouble is, the switch is not connecting the BAT and IGN when in the on position---only when it is turned to the start position.
EDIT: On second reading, I see that you replaced the ignition switch. OK, you wired it wrong or have the wrong switch, or----?