Musnad Ahmad 26:694

Yahya narrated to us, from Sufyan, Mansur narrated to me, from Khaythama, from the one who heard Ibn Mas’ud, who said: The Messenger of Allah (peace and blessings of Allah be upon him) said: "There is no late-night conversation except for one who is praying or one who is a traveler."