Yup, most likely a fuel pump diaphragm problem. But then if the fuel pump is not working properly, then the priming bulb should not be emptying. In short, the most likely cause is there is leak between the suction side of the fuel pump and the suction line from the tank. Or it could just be as easy as clogged or closed vent from the fuel tank or faulty check valve from the bulb itself.
If priming bulb collapses, it's most likely a clogged or closed vent in the fuel tank or faulty check valve in the priming bulb.
As for the starter, it would not be surprising to see that starter has not been opened and cleaned in at least 2 decades!