He served as vizier for seven years and seven months, after which he was dismissed by order of Sultan Malikshah at the caliph’s behest because of a grievance. Abu Shujaʿ thereupon recited:
He assumed it when he had no foe …
and left it when he had no friend. (1)

He then went out to the Friday prayer, where the common folk crowded round him, offering supplications and shaking his hand. Because of this public display he was ordered not to leave his house, so he built a small mosque in the vestibule of his residence. That same year he performed the Hajj; on his return he was barred from entering Baghdad and sent to Rūdhrawar, where he stayed for two years. After the deaths of Nizam al-Mulk, the sultan, and the caliph, he made the pilgrimage again, settled in Medina, and adopted an ascetic life. When one of the servants died, he gave the remaining attendants gold until he was appointed to the dead man’s place, whereupon he himself would sweep the floors and light the lamps. (2) He donned plain khām (coarse, undyed cloth), memorised the Quran there, and when Abu ʿAli al-ʿIjli asked to read his Dīwān with him, he refused and recited only a portion of it. (3)

Abu al-Hasan al-Hamadhani states: He was buried in al-Baqiʿ on the fifteenth of Jumada II, 488 AH, at the age of fifty-one years—may Allah Most High have mercy on him.

Among his children was al-Ṣāḥib Nizam al-Din, who died in Isfahan in 513 AH. Nizam al-Din was the father of the distinguished grand vizier Zahir al-Din Muhammad ibn Abi Mansur Husayn, grandson of the vizier Abu Shujaʿ.