On the mental disability front, it's more likely to be the potential unpredictability of the person's reactions that make them 'scary' - children are experimenters and build up a secure world when surrounded by things that are predictable (routines, consistency from parents, comforters) - if they get reactions that are not what they're expecting, this may scare them. Of course, this massively depends on the disability and how it manifests of course.