Home Cookin's statements here are spot on. The rest of his post is also good stuff. Ocean knowledge, like swell and wave behavior recognition on a split second basis can make the difference between a hair ball scary trip and a routine trip on a snotty ocean. So the captain's skill and experience is most important for sure.
I am at home on the ocean. I have been surfing since I was 6, and I am now 60. Surfing and ocean fishing have been my life, first in Southern California and the last 25 years in Humboldt County near the Oregon border. 100 percent of my trips are out of an Oregon like harbor bar, Humboldt Bay, and it is identical to all of the bar entrances up into Washington. I go out into the ocean here every weekend I can during the window of 01 May to 31 October. And I do it in a 15 foot Chrysler outboard powered tri hull built in 1977. This boat was never intended to do what I do with it but I've been using it here like this for 6 years. It's actually very well suited for ocean work and it's been extensively modified to be better than with the generic stock aluminum frame windshield most of these things come with. Gutted and rebuilt innards too including transom, stringers and deck.
The most extreme offshore operators I can think of are the tow in jet ski guys towing surfers into gargantuan waves at Teahupuu, Tahiti. In this video as you watch, after about 8 rides into it, you'll see that every one of these guys has been towed into these monsters by a jet ski, the point being that the skill of these boat operators allows them to navigate in what seems impossibly dangerous conditions but they do it every surfing day that they are there and they do it safely. It's the skill of the operator, not the size of the boat. A bigger boat simply can be a cushion of safety for a less experienced operator. And some operators are so bad that a boat of any size would be dangerous in their hands.
So it's no big deal, you pick the right conditions that you are comfortable with, observe all safety protocols, wear a kill switch lanyard if you are alone (I go solo most of the time). And my furthest offshore? In my tri hull, 37 miles off Humboldt Bay. 5 out to 20 miles is a normal distance for me, but up here in the Pacific Northwest, by necessity you must become sort of a local weather expert to be the safest you can be. Like he said, you can push yourself being as safe as possible in order to learn your limits. But learn things like entering a harbor or river bar by looking all that stuff on YouTube and other sites for best advice and technique because it's those weird situations like currents rushing out of bar entrances into oncoming swells that can get you. And again, the kill switch lanyard is an awesome device. So is a life jacket. NEVER boat alone without both of them on your body and connected to the switch. We had a fishing boat out here on the west coast found with nobody in it recently, the guy was never found. Think of what it would be like watching your boat motoring away from you.