Musannaf Ibn Abi Shayba 13:345

Waki’ narrated to us, from Jabir, from Abu Ja’far, that the Prophet, peace and blessings of Allah be upon him, used to: When he went out, he would command Ali to sweep the walls.