Musannaf Ibn Abi Shayba 3:4493

Waki’ narrated to us, he said: Sufyan narrated to us, from Mansur, from Ibrahim, he said: It used to be said: "The Witr is only for the people of the Qur'an."