Musannaf Ibn Abi Shayba 9:1594

Yazid ibn Harun narrated to us, he said: Sa'id ibn Abi 'Aruba narrated to us, from Qatada, from Raja' ibn Haywa, from Qabisa ibn Dhu'ayb, he said: Umar said: "Whether you keep them chaste or do not keep them chaste, no woman among them shall give birth on the bed of one of you except that he is joined to him," meaning the concubines.