Musnad Ahmad 27:238

Yahya narrated to us, from al-A’mash, from Abu Salih, from Ibn Umar, from the Prophet (peace and blessings of Allah be upon him), who said: "If you are three, then two should not whisper privately to the exclusion of their companion." He said: We asked: "What if they are four?" He said: "It does not matter."