You can use audacity, a free audio editor:
http://www.audacityteam.org/
That looks like the official site, but when you download check the links carefully to make sure it’s something that doesn’t have malware attached.
Once you get the hang of it, it’s as easy as editing text in a text editor. many of the standard windows short cuts work, like control-c for copy, control-x for cut, control-v for paste, etc.
When I record, I typically average around 2 mistakes every 30 minutes. When I do make a mistake, I leave a long pause, say 5 seconds, then say, “error, repeat sentence (or whatever the necessary correction is)”, and then it’s easy to find it in the editor the delete the offending part.