Musannaf Ibn Abi Shayba 7:403

Ibn Fudayl narrated to us, from Mujahid, who said: Al-Abbas emancipated some of his slaves during his illness. Ibn Abbas returned two of them, as they (the people) used to consider them to be children of adultery.