Musannaf Ibn Abi Shayba 9:260

Abdullah ibn Idris narrated to us, from al-Salt ibn Bahram, from Shaqiq, who said: Hudhayfah married a Jewish woman. Umar wrote to him: "Release her." He wrote back to him: "If she were forbidden, I would have released her." He wrote back to him: "I do not claim that she is forbidden, but I fear that you might associate with the harlots among them."