Musnad Ahmad 32:38

Isma'il ibn Ibrahim narrated to us, 'Abd al-'Aziz ibn Suhayb narrated to us, from Anas ibn Malik, who said: "The Prophet of Allah, may Allah bless him and grant him peace, forbade a man from using saffron."