Workers describe former Citizen No. 1 as insulting, dismissive, reducing some to tears; Rivlin apologizes, but notes claims are anonymous; former spokesperson backs him