Doctors say "hundreds" of people have been taken to hospital - many complaining of a burning sensation in the eyes and difficulties breathing.
The incident, which took place around 03:00 local time (21:30 GMT), may have been due to negligence, officials say.
The leak occurred when the plant was being re-opened for the first time since 24 March when India went into lockdown to curb the spread of coronavirus.
The state Industries Minister Goutam Reddy told BBC Telugu that it looked as though proper procedures and guidelines were not followed when the plant was being re-opened.
As the gas spread, residents ran out of their homes in panic. Distressing visuals of people fainting and dropping unconscious on the streets are being shared on social media.
Some factory employees are believed to have been inside when the leak occurred, but officials say they have no information about them.
It is feared that the fumes have spread over a radius of about 3km (2 miles) and officials have been evacuating people from surrounding areas.
A senior district official said that initial attempts to control the gas leak were unsuccessful. However, local news agencies have reported that the situation is now under control.
