Musannaf Ibn Abi Shayba 16:448

Abu Bakr narrated to us, he said: Ibn Idris narrated to us, from Husayn, from al-Sha’bi, from Suwayd ibn Ghafala, who said: We witnessed Yarmouk, and Umar met us while we were wearing brocade and silk. He commanded [that we be pelted], so we were pelted with stones. He said: We said, "What has reached you about us?" He said: "We removed it, and we said, 'He disliked our attire.'" When he met us [again], he welcomed us and said: "You came to me in the attire of the people of polytheism. Indeed, Allah was not pleased with brocade and silk for those who were before you."