Musnad Ahmad 25:1322

Muhammad ibn Ja'far narrated to us, Shu'bah narrated to us, from Qatadah, who said: I heard Abu Hassan al-A'raj say: A man from Banu al-Hujaym said to Ibn Abbas: "What is this fatwa that has captivated—or branched out among—the people: 'That whoever circumambulates the House [the Kaaba] has become permitted [to exit the state of Ihram]?'" He replied: "It is the Sunnah of your Prophet, may Allah bless him and grant him peace, even if you are reluctant."