16ft Big Fisherman with a 20hp 4Stroke, Need to know if a 20hp is big enough?

[Michigan Friend]

I have an older 16ft Big Fisherman. Currently use a 25 Evinrude 2 stroke. Boat runs 28mph with one person and gear and 22mph with 2 people and gear. Looking to got to 4stroke and find the 25 too heavy at like 180lbs. A 20hp 4stroke is like 130lbs. Some say the 20hp 4stroke is too small, any info appreciated. Ps- The boat is 420dry weight.
Thanks- Michigan Friend