OK so I hooked up a relay for the switched 12V and noise was still there, I then ran the constant from the deck to the amps 12V input, which comes from battery, and boom noise is gone. Have not checked with engine running but the blower noise is gone.
So yeah, combination of a relay and constant directly to battery did the trick. There's still some minor hiss coming from either the deck or amp, not exactly high end stuff here, but its too low to hear in normal situation and does not go up with volume.