Comparing aunt and nephew should be able to give you evidence as to whether or not your Mom and her sister are full siblings or half siblings.
If it suggests half siblings, then to confirm you would need a close relative related to your grandmother to test who is not a descendant of your grandmother. If that person does not match your mother or matches distantly, then your grandmother is likely not your Mom's mother.
You may already have such relatives tested. Look for cousins in your Mom's and her nephew's match lists. If you have any, see if both your Mom and nephew match them. Check your Mom and her nephew's common match list.
Also check if your Mom has any close matches, e.g. 2nd cousin or closer that you don't recognize.
With DNA, you can never prove anything 100%, but you can get good evidence pointing one way or the other.