The decision to sack Ndume was contained in a letter to the Senate by the APC caucus on Tuesday.
But Ndume, who was not at plenary when the decision was announced, expressed shock at the development.
He said he was not consulted.
He told newsmen after plenary: “I don’t know whether I have been removed as the Senate Leader.
“This announcement came to me as a surprise.”