Onetime confidant has accused the prime minister of destroying the party; colleagues say they were blindsided by the move