'Everyone was asked to leave the premises,' former US president says; FBI declines to provide extra information