When you say "witch", the first thing that comes to mind is a senile old lady, living by herself in the woods and distributing suspicious potions to shady characters.

It's prejudices like that that made it so hard for female magi to get any sort of respect in the field.