I expect that hiking in a mixed group would elicit fewer negative comments or at least be less alarming to people, although that is something I have not done, not counting being with my wife. But be aware that opinions and reactions may vary over the decades, since people seem emboldened by whatever the current political climate happens to be. Right now the political climate is stormy. Naturally, the climate isn't the same everywhere and it may not be what you think it is, if you happen to be somewhere away from home.
All the same, people can be surprisingly conscious of "intruding" upon someone else's presence. Just this morning I had taken a hike (this was in the neighborhood woods and fully clothed) to one of my favorite nearby sites (and sights), where I was sitting on the shore of a lake wondering what a certain blue heron perched high in a tree was going to do. Ten minutes later a family showed up, father, mother and three little girls. Both parents apologized for intruding. It seems that the unspoken and unwritten rule is first one there gets the spot, at least where fishing is concerned. And sometimes in the backcountry, the first one there sets the norms.