Going down memory lane, he said he was an ‘accidental politician’, with none in the family being in politics