Musannaf Ibn Abi Shayba 9:1701

Jarir narrated to us, from Mansur, from Mujahid, who said: 'Ata' and some sisters of his were in a house in Makkah. The people of Makkah would go back and forth to their families at night many times. He [Ata'] would come to them at night, so he asked Ibn Abbas, "Should I seek permission from them every time I enter?" He said, "Yes." And he did not grant him a concession to enter upon them without permission.