Musnad Ahmad 8:75

Hajjaj narrated to us, Layth informed us, 'Aqil narrated to me, from Ibn Shihab, from Sa'd ibn Abi Waqqas, that he said: "The Messenger of Allah (peace and blessings of Allah be upon him) forbade a man from coming to his family (at night) after the night prayer (Isha)."