A fire broke out in the demolished UNRWA headquarters in Jerusalem, the Israeli Fire and Rescue Authority announced early Sunday morning.