Judgement.
There are a lot of reasons that people don't speak up. Obviously, it's scary to think that you won't be believed. Speaking the truth only to have it skewed, mocked, or flat out disbelieved is horrible- dehumanizing and invalidating. It's painful.
But there's worse things, too. What if you are believed, and it still just doesn't matter. Maybe it doesn't matter because you aren't on trial, and you don't have anything to prove to anyone, but maybe it doesn't matter because you don't matter; the truth is irrelevant, only important to you.
How much do the opinions of others matter? How much history should a person clear up, and what does it matter?
You shouldn't care what other people think. Yeah, obviously, this is easier said than done, and in reality it can be excruciating to know that other people believe things about you that aren't true. Worse though, if they believe things about you that are true- but without proper context, without knowing the complete story, the whole truth. It sucks if someone thinks the worst of you, and the truth is, there's nothing you can do about it. Nothing.
The truth is, we hear things about people, and we remember them. We treat them differently, we decide that they're risky based on whatever we've heard. But people say things for a lot of reasons. Sometimes they've been given an incomplete picture, and they're filling it in with their takes, or just passing on part of the history. Other times, it aligns with something else they already believe, and it's always nice to be proven right- and tell someone about it. Some people are just bored, or out of other things to say. Talking about people is a bad habit, especially if someone's not used to really saying much about themselves in a particular relationship- it fills up air time, allows for shared opinions, bonding- without ever having any real vulnerability or having to share uncomfortable situations about oneself.
In some cases, people lie about people to cover their own asses, literally or emotionally or even vengefully. People slant the truth to cast themselves in a better light. Because even though we all know we shouldn't care, the truth is- we have to care; the opinions of others have a weight that we wear.
It keeps some people quiet.
Being accused of something sticks with someone whether or not it's true- we think there must be a reason they're accused of it without questioning the motivations of the source or seeking out the whole truth. It's embarrassing to talk about things with someone directly, easier to convict or acquit in the court of our own opinions, and just wait to see what happens.
Some people stay quiet because some things are unforgivable. Or they think that gratitude is more important than honesty. If someone did this good thing, then they deserve a pass from any bad things that might have happened.
We say innocent until proven guilty, but really, we don't think bad things happen to good people.
Sometimes we have so harshly judged ourselves that there is no amount of truth that gets us off the hook, so we stay quiet, so that everyone doesn't judge us for whatever it is we hate ourselves for. We might have a truth to tell that we don't because we have deemed ourselves incredible, not worthy of a truth that might cast someone else in a bad light.
If we think we are guilty of something, we don't defend ourselves from anything.
Let them say what they will, right? We aren't supposed to care what people think anyway. If you know the truth, isn't that all that matters? Why should we care?