I am close to your age and expecting out first child anyday now. Last thing I need to do is get hurt.

For the last couple years I have been doing the EFTA series races. In my mind it is no different then going on a ride with your buds; riding hard and telling hero stories afterwards over lunch. I have never been run off the course or put in any danger by another rider/racer. The risks you take while riding or racing are your decisions. You can always walk anything that you're not comfortable with.

Like any challange you have to weigh the risks and decide if it is worth it.
But be careful with the racing bug, it's easy to get hooked!

Good luck!