If grandparents are pulling into a smart group list that is set up for parents, this is because it's a type of parent relationship and they have "List as Parent" checked off in the relationship.
- Navigate to Core > User/Access > Profile
- Search for the user name and select the appropriate user profile
- Select Relationship Settings
- Select Edit
- Unmark the checkbox "List as Parent"
This will remove the grandparents from the Smart Group list and only keep those who are checked as "List as Parent".