There are check valves in the fuel pump so when not running it would not pull fuel through the pump.
I would suggest you to check the pump diaphragm. If there are holes or a rip in it, fuel will go direct into the cylinders and not through the carb.
Changing fuel pump diaphragm is just regular maintenance and they are cheap.
I wonder if anyone has bypassed their fuel pump with an electric fuel pump. I have had some trouble with my diaphragm water pump. They seem to wear out quickly. Some have bypassed their diaphragm fuel pumps on mowers. Seems like it could also be used instead of the primer bulb. Just wondering if it has been tried. Might solve the diaphragm pump problems.