Musnad Ahmad 30:1980

Husayn narrated to us, Ibn Abi Dhi’b narrated to us, from al-Zuhri, from Ibn al-Musayyib, from Abu Hurayrah, that the Prophet, peace and blessings of Allah be upon him, said: "If you say to your companion while the Imam is delivering the sermon, 'Listen silently,' then you have engaged in idle talk."