Abhishek Banerjee was allegedly set upon by locals and beaten up when he tried to visit the family of a party leader who allegedly died in poll violence.