Musnad Ahmad 30:3589

Uthman ibn Umar narrated to us, Yunus informed us, from al-Zuhri, from Sa'id ibn al-Musayyib, from Abu Hurairah, that the Messenger of Allah (peace and blessings of Allah be upon him) said: "If you say to your companion while the Imam is delivering the sermon on Friday, 'Listen silently,' then you have engaged in idle talk."