Musannaf Ibn Abi Shayba 2:217

Abu Bakr narrated to us, he said: Ibn Fudayl narrated to us, from Yaman, from Qays, who said: Umar said: "If I were able to perform the Adhan along with the Caliphate, I would have performed the Adhan."