"To compete against all the stars around him would have been very, very tough," Rayudu says