The data is stored on a supercomputer at the headquarters of the DGSE intelligence service, the paper says.
The operation is "outside the law, and beyond any proper supervision", Le Monde says.
Other French intelligence agencies allegedly access the data secretly.
It is not clear however whether the DGSE surveillance goes as far as Prism. So far French officials have not commented on Le Monde's allegations.
The DGSE allegedly analyses the "metadata" - not the contents of e-mails and other communications, but the data revealing who is speaking to whom, when and where.
The paper alleges the data is being stored on three basement floors of the DGSE building in Paris. The secret service is the French equivalent of Britain's MI6.
The operation is designed, say experts, to uncover terrorist cells. But the scale of it means that "anyone can be spied on, any time", Le Monde says.
There is a continuing international furore over revelations that the US has been systematically seizing vast amounts of phone and web data.
The French government has sharply criticised the US spying, which allegedly included eavesdropping on official EU communications.
The scale of surveillance by America's National Security Agency (NSA) emerged from classified intelligence documents leaked by whistleblower Edward Snowden.