Opposition head defends weekend in Paris, saying he and his wife endured 'craziest year and a half of our lives' when he was FM and PM, but he'll be back Sunday to join fight