Ibn Ḥazm records this in al-Muḥallā, and al-Ḥāfiẓ (Ibn Ḥajar) endorses it in Fatḥ al-Bārī.

As for the Sunnah, among its proofs are the Prophet’s words, peace and blessings be upon him: “Safeguard your private part except from your spouse or from those whom your right hand possesses,” and his words: “Do not walk about naked.”

The first ḥadīth has already been quoted in full under the section on the etiquette of bathing.
The second ḥadīth is narrated by al-Miswar ibn Makhramah (may Allah be pleased with him). He said, “I came carrying a heavy stone while wearing a light izār (waist-wrapper). My izār came undone, and because the stone was in my hands I could not set it down until I had reached the place where it belonged. The Messenger of Allah, peace and blessings be upon him, said, ‘Return to your garment, secure it, and do not walk about naked.’”

Muslim recorded this report. Ibn Ḥazm, Abū Dāwūd, and al-Bayhaqī also transmitted it through the chain of Yaḥyā ibn Saʿīd al-Umawī, from ʿUthmān ibn Ḥakīm, from Abū Umāmah ibn Sahl, from al-Miswar.

On the same subject, Jābir ibn ʿAbdullāh relates that the Messenger of Allah, peace and blessings be upon him, was helping them carry stones for the rebuilding of the Kaʿbah while wearing his izār. His uncle al-ʿAbbās said, “My nephew, if you would loosen your izār and place it over your shoulders beneath the stones…” So he loosened it and placed it on his shoulders, whereupon he collapsed unconscious. After that he was never again seen naked, peace and blessings be upon him.

Al-Bukhārī and Muslim both recorded this ḥadīth.