Musannaf Ibn Abi Shayba 35:379

Hafs narrated to us, from al-A'mash, from Abu Sufyan, from Jabir, who said: Sulayk al-Ghatafani came while the Prophet, peace and blessings of Allah be upon him, was delivering the sermon on Friday. He said to him, "Have you prayed?" He said, "No." He said, "Pray two rak'ahs and make them brief." And it was mentioned that Abu Hanifah said: The Imam should not speak to anyone during his sermon.