Thaddeus Ma Daqin, Shanghai's auxiliary bishop, announced his resignation from the Catholic Patriotic Association (CPA) at his ordination mass last week.
Catholic media and other religious sources say he has been confined in a seminary near Shanghai.
There has been longstanding tension between Beijing and the Vatican.
The Vatican, which appointed Mr Ma, does not recognise the CPA. The Chinese church in turn rejects the Pope's authority.
China's estimated 10 million Catholics are split between followers of the Pope and the Chinese Patriotic Catholic Association.
Reports said Mr Ma told a 1,000-strong congregation that he was stepping down from the governing body to focus on his new responsibilities. This drew loud applause, said a report in the Hong Kong-based South China Morning Post newspaper.
But analysts say the move was seen as a challenge to Chinese state control over Catholic churches and clergy.
Mr Ma did not return for mass on Sunday and is said to have been taken away to a seminary in Shanghai and forbidden contact with others, according to an Associated Press report, citing Catholic researcher Anthony Lam and the AsiaNews and UCAnews websites.
The vice chairman of the CPA, Liu Bainian, has also said that he is awaiting the results of an investigation into Mr Ma, according to Hong Kong's Ming Pao newspaper.
China broke off diplomatic relations with the Vatican in 1951, but in recent years tensions between Beijing and Rome had eased somewhat, with the occasional visit by a senior Vatican cardinal.
Relations suffered a setback in 2010 with the consecration of the first Chinese bishop for almost five years without the approval of Rome.