Musannaf Ibn Abi Shayba 3:5478

Yazid ibn Harun narrated to us, from Hisham, from al-Hasan, that he used to dislike that a man should fidget with something during his prayer.