Ex-PM denies that saying his predecessor and family have mental health problems is defamation, calls for them to undergo psychiatric tests