Musannaf Ibn Abi Shayba 3:6344

Waki’ narrated to us, from Rabi’ ibn Sabih, from Abu Ma’shar, from Ibrahim, that he disliked for a man to adjust anything in the prayer, even the button of the shirt. He said: And Ibrahim did not see any harm if his lower garment (izar) became loose during the prayer, for him to lift it up.