Musannaf Ibn Abi Shayba 3:2947

Hushaym narrated to us, he said: Hisham ibn Abi Abd Allah informed us, from Yahya ibn Abi Kathir, who said: It was narrated to me from Umar ibn al-Khattab that he said: "The khutbah (sermon) was only made as a replacement for the two rak'ahs. So, if one does not catch the khutbah, let him pray four."