Sometimes it's not unusual to be out and about with less than your total number of children. Some are in school. Sometimes it's a weekend and we just send one along with a parent on an errand to divide and conquer. People get that. What I have found throws people off is being without the youngest child, and since our foster children have always been the youngest children in our family, this happens regularly when foster children are at visits.
At a play group, the foster child is with me one week because a visit was canceled, then gone the next because the visit is happening. It's the middle of the day, so it's not common in many families that he or she would be at home for some reason with the other parent.
Or there's an overnight visit and suddenly we have fewer children at church. Did he move? Where is she? A bit hard to explain without prompting more questions about the case. I usually explain anyway, and just try to avoid the other questions the best I can.
I need a t-shirt that says "Number of Kids May Vary, No Biggie."