Musnad Ahmad 551:5

Muhammad ibn Ja‘far narrated to us, he said: Shu‘bah narrated to us, from al-Hakam, he said: I heard Dhakwan narrating from a freed slave of ‘Amr ibn al-‘As, that he [‘Amr] sent him to ‘Ali to ask his permission regarding Asma bint ‘Umays, and he [‘Ali] gave him permission. When he had finished his need, the freed slave asked ‘Amr about that. He said: "The Messenger of Allah (may Allah bless him and grant him peace) forbade us, or forbade that we enter upon women without the permission of their husbands."