Musannaf Ibn Abi Shayba 16:435

Abu Bakr narrated to us, he said: Hafs narrated to us, from Layth, from 'Ata', from Tawus, that he used to dislike wearing silk.