Musannaf Ibn Abi Shayba 33:872

Abu Khalid narrated to us, from Yahya ibn Sa'id, from Isma'il ibn Abi Hakim, from Umar ibn Abd al-Aziz, who said: "Indeed, Allah does not punish the general public for the deeds of the elite. But when sins appear and are not denounced, they all become deserving of punishment."