No reason was given for Li's removal and no replacement was immediately announced.
Li had not appeared in public since August 29, when he gave a speech at a Chinese-African peace forum.
He is the second senior leader to be ousted under mysterious circumstances after Foreign Minister Qin Gang was abruptly removed from office without explanation in July.
According to state broadcaster CCTV, both were also stripped of positions as state councillors.
Li's removal means China is without a defense minister as it prepares to host foreign defense ministers at the Beijing Xiangshan Forum, which takes place next week.
Li reportedly investigated for corruption
Li, 65, had only been on the job since March.
The Reuters news agency reported last month that he was being investigated for suspected corruption related to equipment procurement and development.
Experts have said President Xi Jinping handpicked both Li and Qin, making their removal less than a year on the job particularly notable while raising questions about the stability of the leadership team around Xi.
