Musannaf Ibn Abi Shayba 37:454

Abu Bakr narrated to us, he said: Waki’ narrated to us, from al-A’mash, from Shamir, from Abu Yahya, from Ka’b: He said: The Hour will not be established until a single man is the guardian of fifty women.