You can swim at both Houghton's Pond and Ponkapoag in the Blue Hills.
Wendell State Forest has a great swimming pond.
Walden Pond State Reservation is not mountain bike friendly (other than there's a bike rack at the main beach). Probably my favorite place to swim in the Boston area though. The conservation land around Walden Pond (other than the park) is mostly open to mountain bikes.
I forgot about Houghton's Pond. That's where they used to have the Blue Hills Mountain Bike Day (which I hope will restart sometime).