I gotta go the rather unpopular vote here and say Timothy Dalton (The Living Daylights, Licence to Kill) was the best, even though he only did two movies. I've been reading some of the original novels, and Bond was quite the * (even in the official sequels to the Fleming books, written by John Gardner and Raymond Benson in the '80s and '90s).
Wall of text that used to be here omitted. Instead, some links to the more * fight scenes of the series.
James Bond fights Red Grant, From Russia With Love:
http://www.youtube.com/watch?v=_jEubPVLPqA
James Bond fights Hans, You Only Live Twice:
http://www.youtube.com/watch?v=xrg9Va3U ... re=related (I love the Fist-Cam effect going into Sean Connery's face in this one.)
James Bond fights Ross Kananga, Live and Let Die:
http://www.youtube.com/watch?v=CBncb56Z ... re=related (Famous for the extremely awful not-so-special effect of Kananga's death.)
James Bond fights Necros, The Living Daylights:
http://www.youtube.com/watch?v=17SP5sAM ... re=related
James Bond fights Alec Trevelyan, GoldenEye:
http://www.youtube.com/watch?v=fsyacUJz ... re=related (This, along with the fight from A Clockwork Orange, is probably one of the most brutal fistfights I've ever seen in a movie. Apologies for the rather short length.)
The entire first four minutes of Casino Royale:
http://www.youtube.com/watch?v=Gvr6uRvE ... re=related
If you're able to, I recommend most of the books (Not The Spy Who Loved Me, because that isn't really a Bond book.) because they're full of awesome.