Buckingham Palace says Prince Harry will not remain at the palace during his visit to London this week, contradicting an earlier statement from his spokesperson that he had accepted an invitation to do so.

Royal sources say the duke did not respond quickly enough to the offer of accommodation at a royal residence, and that he was informed on Saturday evening that he would not be able to stay there. The palace says there was initially an offer, but that the Sussex team either failed to confirm acceptance in time or formally declined it—before later changing its position.

The Sussex team’s version is different: they say the palace withdrew an offer that had been accepted, describing the outcome as disappointing.

The row unfolds as Prince Harry carries out charity-related engagements connected to the Invictus Games. The palace also reportedly weighed concerns about the optics of the duke staying at the symbolic HQ of the monarchy during the week a court result in his legal case involving Associated Newspapers was expected.