Musannaf Ibn Abi Shayba 3:2904

Abu Usama narrated to us, from Hisham, from al-Hasan, that he saw no harm in a man, on the day of Friday, remembering Allah within himself while the Imam is delivering the sermon.