Yes, we must believe in those mentioned in the Holy Quran or authentic Sunnah.
كُلَّمَا دَخَلَ عَلَيْهَا زَكَرِيَّا الْمِحْرَابَ وَجَدَ عِنْدَهَا رِزْقًا قَالَ يَامَرْيَمُ أَنَّى لَكِ هَذَا قَالَتْ هُوَ مِنْ عِنْدِ اللَّهِ إِنَّ اللَّهَ يَرْزُقُ مَنْ يَشَاءُ بِغَيْرِ حِسَابٍ
Every time Zakariyah entered upon her at the Mihrab (her prayer area), he found sustenance with her. He said O Maryam, where did you get this from. She said, it is from Allah (Most High), indeed Allah grants sustenance to whomsoever He wishes without limit. (Surah Ali Imraan, 37).
The scholars of Ahl al-Sunnah are in agreement regarding the concept of Karamaat, which are miracles that occur for pious people who are not prophets. This is due to the multiple evidences supporting this concept found in the Holy Quran and Sunnah. The Ayah above mentions one of many miracles related to Sayidah Maryam .
However, there are many misconceptions regarding this issue and people must study this thoroughly from reliable, classical works to avoid falling into misguidance. Many people do not know that there are significant differences between a Karaamah and a Mujizah, miracle of a Prophet, and therefore hold problematic beliefs.
من الفرق بين المعجزات والكرامات أن الأنبياء عليهم الصلاة والسلام مأمورن بإظهارها والولي يجب عليه سترها وإخفاؤها والنبي يدعي ويقطع القول به والولي لا يدعيها ولا يقطع بكرامته لجواز أن يكون ذلك مكرا
Amongst the differences between (prophetic) miracles and Karamaat, is the Prophets, peace and blessings be upon them, have been commanded to make them apparent. Whereas it is obligatory for the Wali to veil and hide them. The Prophet calls to it and it is a matter of certainty. Whereas a Wali does not call to it, and they are not confirmed as Karamaat because of the possibility that it could be deception. (Imam al-Nawawi, Bustaan al-Arifeen, p63).
It is also important understand that we are obliged to believe in Karamaat confirmed in the Holy Quran and authentic Hadith. We are not obliged to believe in every Karaamah people claim for their own Shaykhs. Many ignorant people do not understand this, and unless a person unequivocally accepts every miracle they claim for their Shaykh, they will consider him to be outside of Ahl Sunnah. This is absurd and clearly misguided, but unfortunately it is rife in some circles and leads to great corruption and even abuse.
And Allah (Most High) Knows Best.
– Answered by Shaykh Noorud-deen (21.06.2021)
Do Awliyaa know the unseen?
Must Awliyaa hide their Miracles?
Is there a difference between Karaamah and Mu’jizah?
See also (video):