Musnad Ahmad 30:2277

He said: "Whoever attaches himself to a people without the permission of his masters, upon him is the curse of Allah, the angels, and all of mankind. Allah will not accept from him any obligatory deed nor any supererogatory deed."