Musannaf Ibn Abi Shayba 23:19

'Affan narrated to us, he said: Hammad ibn Salamah narrated to us, from Ja'far al-Khatmi, from his father, from his grandfather, 'Umayr ibn Habib ibn Khumashah, that he said: "Faith increases and decreases." It was said to him: "And what is its increase and what is its decrease?" He said: "If we remember Him and fear Him, then that is its increase; and if we are heedless, forget, and neglect, then that is its decrease."