There are a couple of ways of doing this. One is to measure at various locations and transfer those measurements onto the plywood you will use for the new transom. Second way is to get a large sheet of cardboard, and cut and trim it until it slots in perfectly. The transom should be very close to the sides.
When I prepared to rebuild the transom for my 16' Naden, I discovered that the transom that had been in there was not original, and it had been poorly sized, so I had to work how to size the new one largely from scratch. I measured several times, before transcribing those measurements onto the laminated plywood I was using, and cutting. i had to trim it once I cut it to get it to fit properly.
If you check out the Naden Big Fisherman thread in my signature, you'll see a couple of posts on the topic of sizing the transom, and the advice I got.
if you dug this thing out with a putty knife, check the transom carefully, as there may well be a number of bolts or screws that will need to be removed. If you have any solid pieces of transom left, you can try to use those to measure the transom thickness. Beware, however, as if the wood was swollen with water, the old transom may be thicker when you measure it, than it was originally.