Musnad Ahmad 27:1144

Muhammad ibn Ja‘far narrated to us, Shu‘bah narrated to us, from Mansur, from Sa‘d ibn ‘Ubaydah, who said: I was with Ibn ‘Umar, then I stood up and left a man from Kindah with him. I then came to Sa‘id ibn al-Musayyib. He said: The Kindi came in a state of alarm and said: A man came to Ibn ‘Umar and said, "I swear by the Ka‘bah." He said, "No, rather swear by the Lord of the Ka‘bah, for ‘Umar used to swear by his father, and the Messenger of Allah (ﷺ) said: 'Do not swear by your father, for whoever swears by other than Allah has committed shirk (polytheism).'"