Tafsir of As-Saffat 37:29

(They said, "Nay, you yourselves were not believers.")

(They said) - an initiation of discourse following the previous style. That is, the leaders or the companions said in their response, by way of refutation of what they [the followers] had said: "(Nay, you yourselves were not believers.)"

This is a denial of their having led them astray. That is: You led yourselves astray through disbelief, and you were not believers in your own essence, not that we were the ones who led you astray.