PM's decision is 'responsible' as he meets thousands each day, aides say; defense minister's adviser excluded from meeting with PM after daughter returns from Thailand with cold