The children went to his spaza shop on Sunday in Zola, Warrant Officer Kay Makhubela said.
The man was arrested on Tuesday night with 68 other men during a raid in the area.
The men faced charges ranging from murder to rape, sexual assault, fraud, theft, and business burglary, Makhubela said.
Another among those arrested was a 35-year-old man who allegedly raped a woman, 22, on 1 March.
She was raped in an open veld after attending a night church vigil.
All the men were expected to appear in the Protea Magistrate's Court on Thursday.