Musannaf Ibn Abi Shayba 9:424

Waki’ narrated to us, from Ma’mar, from Abu Ja’far, who said: The child of zina (adultery/fornication) may take concubines, and the child of the woman [with whom he had the concubine] is not sought.