According to eyewitness accounts, Mr Elalem was in his car with a colleague when two vehicles carrying armed men forced him to stop in the capital.
His colleague was left in the car.
Mr Elalem's whereabouts remain unknown, and there has been no information yet about the identity of the alleged kidnappers.
He added that Mr Elalem was taken away by the men when he was putting his laptop in the back of his car.
Libya's Sports and Youth Minister Fethi Tarbel told Reuters news agency: "We condemn this kind of action. Whoever did this are criminals.
"This goes against the revolution, this brings us back to the Gaddafi culture... I don't know if he [Mr Elalem] had any issues with anyone."
The President of the International Olympic Committee, Jacques Rogge, expressed "grave concern" over the incident, adding that his organisation "offered any possible help we can give".
Before the fall of Muammar Gaddafi in 2011, the Libyan Olympic committee was headed by his son Mohammed, who has since sought refuge in Algeria.