Personally, prejudice wouldn’t arise if people refrained from doing wrong. You can’t stop society from looking down on people who have committed grave mistakes; it comes with the territory. It’s a bitter fact that most people are judgmental, and they have good reason to be.
Admit it, we frown on people who do wrong. And we become more disappointed in them if we know them personally and were convinced that they couldn’t have possibly commit grave mistakes.