Musannaf Ibn Abi Shayba 7:347

Al-Thaqafi narrated to us, from Khalid, from Muhammad ibn Sirin, who said: If a man swears an oath (ila') to abstain from his wife, then he returns (to her), then upon him is the expiation.