Musannaf Ibn Abi Shayba 9:1020

Ghundar narrated to us, from Shu'bah, from Ya'la ibn Muslim, from Sa'id ibn Jubayr, who said: "No one commits adultery except with an adulteress or a polytheist woman."