I sympathise with your sentiment, which is why I'm a vegan. I believe back in the day members of the Vegetarian Society had a disagreement about the exact meaning of "vegetarian", and this resulted in some members leaving to form their own group and coined the term "vegan". "Vegetarian food" had come to mean "non-flesh foods", so eggs and dairy were still considered vegetarian.
In India, which is the likely origin of western vegetarianism, "vegetarian" is synonymous with lacto-vegetarian. Although of course back in the day they didn't speak English, so I suppose they used a different word which today is translated as "vegetarian". In any case, the concept originated in India, and made its way to the West several times. The first proponent in the West was probably Pythagoras, the famous Greek mathematician.
Back in the day the vegetarian concept was often intertwined with religion. Religion was perhaps more or less the only source of ethical standards in those days. More recently the concept of animal rights and the ethical considerations in our interactions with animals have been developed independently of religion, and the problems of animal by-products and exploitation beyond just the flesh have become more clear.
Back in the day they were also limited by less knowledge about nutritional requirements, and no non-animal sources of B12. So it would have been impossible for veganism to develop at that time.
I hope that answers some of your questions, sorry if it was a bit incoherent! 
