Condition over brand ... if ya' find a clean well kept and runnin' Bayliner vs. a Riva that's been neglected in a driveway uncovered for 5 years, go with the Bay. You're gonna' have to be patient to meet your price point, but you can do it. Like Thumpar, I bought my boat well below $3k.
The Bayliner you mention with the 85HP, I'd bet has an 85HP Force outboard. For a newbie, I may steer clear of that. They aren't necessarily bad outboards, but they've been outta' business for a while and you probably need to know what you're doing with 'em. The 85 would pull a skier, but it would work at it with the load you'd have on board.
Got a link to the Sea Ray?
Welcome aboard.