Musannaf Ibn Abi Shayba 26:506

Hafs narrated to us, from Ash'ath, from al-Sha'bi, that Umar, Ali, Abdullah, and Zayd used to say: "If manumission reaches him while he has children from a free woman, he draws their wala' (patronage rights)." I said to al-Sha'bi: "What about the grandfather?" He said: "The grandfather draws [it] just as the father draws [it]."