Olympics chief Thomas Bach agreed that 2021 was the "last option" for holding the delayed Tokyo Games, stressing that postponement cannot go on forever.
Bach said he backed Japan's stance that the Games will have to be cancelled if the coronavirus pandemic isn't under control by next year.
In March, the Tokyo 2020 Olympics were postponed to July 23, 2021 over the coronavirus, which has killed hundreds of thousands around the world and halted international sport and travel.
"Quite frankly, I have some understanding for (Japan's position) because you cannot forever employ 3,000, or 5,000, people in an organising committee," Bach, president of the International Olympic Committee, told the BBC.
"You cannot every year change the entire sports schedule worldwide for all the major federations.
READ MORE
Kenyans gear up for World Development Cup in Kilifi County
FKF keen to see Harambee Starlets qualify for the 2028 Olympics
Lofty goals as NOC-K set in motion Kenya's journey to LA2028
The IOC leader said it was a "mammoth task" to reorganise the Olympics, which have never been cancelled outside of the world wars.
However, Japanese officials have been clear that they have no intention of postponing the Games again beyond next year.
Bach warned that "nobody knows" how the situation will play out, but said the IOC will act on advice from the World Health Organisation.
"We have to organise these Games in a safe environment for all," he said.