Following uproar, minister comes to defense of party colleague Rafi Peretz, says critics are 'mainly thought dictators'