If grandparents are pulling into a smart group list that is set up for parents, this is because it's a type of parent relationship and they have "List as Parent" checked off in the relationship.
 

To remove this setting:
  1. Navigate to Core > User/Access > Profile
  2. Search for the user name and select the appropriate user profile
  3. Select Relationship Settings
  4. Select Edit 
  5. Unmark the checkbox "List as Parent"
  6. Save

This will remove the grandparents from the Smart Group list and only keep those who are checked as "List as Parent".