Tanzania's Deputy Interior Minister Pereira Silima said they had come from Ethiopia and died from asphyxiation.
Local officials said more than 120 people had been packed into the truck, which was headed for Malawi.
Tanzanian and Malawi have become key staging posts for people fleeing drought and conflict in Somalia and Ethiopia, and trying to reach South Africa.
'Dumped the dead'
Mr Silima said that according to survivor accounts, the 127 migrants had started off from Ethiopia in two trucks.
They passed through Kenya's capital, Nairobi, and onto the Tanzania town of Arusha where they were transferred into one lorry, he told the BBC's Network Africa programme.
The driver noticed there were problems in the central Dodoma region and stopped the vehicle.
The container truck then drove off and the authorities are trying to trace it and the driver, he said.
Nearby villagers helped the survivors, one of whom later died.
Mr Silima said when the migrants were well enough, they would be taken to court and their embassy would be contacted.