Musannaf Ibn Abi Shayba 3:2237

Ibn Idris narrated to us, from Husayn, from Amir ibn Dhuwayb, who said: I asked Ibn Umar about praying behind the Hijr. He said: "They fear that they might kill them."