One thing I really appreciate about Jesus is that he was never concerned about being guilty by association. If anyone could be accused of having an interesting set of relationships and people whom he hung around, it would be Jesus. There were tax collectors, sinners, prostitutes. His associations ran the gamut, but he never worried about how he would be perceived by others if he spent his time around these people of question. I think that's one reason why Jesus was so well liked by those who were far from God. He loved them even though his lifestyle was vastly different than theirs.
If you wonder why Christians are perceived as judgmental in our world today, it's because they don't act like Jesus. They're concerned about guilt by association and therefore aren't willing to be around people who might tarnish their Pharisaical record. But to live in such a way isn't to walk in Jesus' footsteps. To the contrary, it is the exact opposite of how Jesus lived.
Are you concerned about guilt by association? Don't be. Jesus paved the way for his followers to confidently care for people from all backgrounds without fear of what others might think.