Musannaf Ibn Abi Shayba 26:507

Mu'tamir narrated to us, from Hajjaj, from al-Sha'bi, from al-Harith, from Ali, who said: "The *wala'* (patronage rights) returns to the paternal relatives if he (the father) manumits." And he narrated that Umar and Uthman judged by it, and that Shurayh did not judge by it, then he judged by it.