Musannaf Ibn Abi Shayba 13:368

Yazid ibn Harun narrated to us, he said: Sa'id informed us, from Qatadah, from Ikrimah, from Ibn Abbas, he said: He would not gather the fruit if it did not have a wall (enclosure), except with the permission of its owner.