This actually hurts veganism as a movement and you as a person. If you put this much pressure on yourself for "personal purity" you may decide you hate being vegan, not because being vegan is actually bad but because you've put so much stress on yourself you're going to be like "**** it I'm going to eat whatever." SO don't do that.
Also, it sends a message to other people that veganism is hard, or is about being perfect or holier-than-thou, which isn't going to help animals or the earth. In addition to that, it doesn't kill or harm any new animals to consume a tiny amount of whey or honey accidentally, or to be given an item you thought was vegan and then realize it had cheese in it after you bit into it.
If something had meat in it, I'd spit it out and probably make myself vomit, but other than that, don't stress yourself out so much. There's definitely a balance: you don't want to be OVERLY lazy like "hey I'm a vegan and I actually eat eggs all the time" but that's a damn site different than "oh no the sugar in this Pepsi might have been processed by bone char."