Inferring is largely an automatic and unconscious process in adults; we have to operate most of the time using higher level abstraction in order to process the huge amounts of information we gather through our senses.
Our assumptions or attributions about other people are extrapolations from perceived data at various levels of abstraction. Chris Argyris describes this as a ladder of inference: the higher the rung in the ladder, the more abstract and less reliable the inference.