Bruce, I just went through that same argument with a friend ... they told me they were saving $160 per month if they refied. I told them I doubted it and went to my amortization table to show them how much they would be paying OVER THE COURSE OF THE LOAN for the refi verses if they just kept paying at the higher current rate. They found my argument hard to swallow at first, but when seeing it in black and white and understanding we were talking about 30 more years instead of the existing 25 years ... then I showed them what a difference it would make if they paid an additional $50 a month in principle each month, or even just and additional $25 in principle, I even went to $10 a month additional principle, and they were shocked at how much they would "Save" (not pay the bank). They decided to "suck it up" and pay the additional principle when they could ...
Not saying you SHOULD NOT refi ... just make sure you are comparing apples to apples when you do your figuring, and make sure you know the TOTAL cost of the transaction (points, fees, and total interest to be paid over the course of the loan). The total cost is what you want to look at with the refinance ...