Why Are Blacks Judged More Harshly for their Accents than Other Ethnic Groups?

Black people live everywhere, just about. So the world isn't ready to hear blacks say "y'all" or openly speak with a Southern slang or accent, but it's okay for Paula Dean to use it. I wish I could answer without coming off racist or biased towards whites (I'm not trying to purposely) but there's a double standard for everything.