How to understand the basic emotions which represent the interlocutor's intentions in Besѣda®
Content
Design
As I’m developing the minimal viable secure product (MVSP) based on the communication system V.I.C.T.O.R.I.A., I felt there is too much text and not enough visual stimulation. This is an issue because:
- The healthy human mind relies on the sense of vision primarily.
- Body language is a large part of communication. Maybe not as large as the popular belief (Albert Mehrabian’s 7-38-55 Communication model) claims but that is a topic for another time.
(1) + (2) => There was a gap in the application.
Requirements
The facial expressions must be universal because Besѣda® itself is a web application which can be used by anyone on the international network. Furthermore, because there are different temperaments - four structures of the skull, I’m not going to include the nose. Strange, I know!
The expressions should be subtle because the intentions and speech delivery of the communicator can vary significantly and the animations should serve for orientation and not be a source of confusion. Many animators exaggerate the expressions to convey the internal state for clarity and emotional attachment to the avatar. For this system it should be subtle.
The technical requirement is that it needs to be animated programatically depending on the choices of the communicator.
For this initial stage, showing a 2 dimentional face is sufficient. It is sufficient to animate only the eyebrows and mouth. More features and more animations can be added iteratively.
Limitations
Besѣda® is intentionally limtied to only voice communication for two main reasons:
- lower bandwidth as a result of not using video means it is accessible in various conditions and geographies,
- visuals can be distracting - people are influenced by the outer appearance. It has been proven that the more physical attractive the person, the more likely they are to have better results in the labour market, politics, criminal justice and other domains. For no other reason than because of the outer appearance. The design of Besѣda® is such that the parties are equalised as far as possible. Therefore it doesn’t support visual communication.
Research
What are emotions? Usually they are defined as the involuntary reaction of the human vehicle to stimuli. Whereas a feeling is the voluntary reaction to that reaction. In a philosophical sense, the voluntary reaction is the only behaviour which we have control over - not what happens to us, not the results be they success or failure. We have complete control only over our conscious reactions to the human vehicle’s reactions.
Traditional Chinese medicine claims that emotions can be both the symptoms and causes of physical reactions. The following five emotions are defined:
- happiness (heart)
- anger (liver)
- saddness (lungs)
- fear (kidneys)
- worry (spleen)
The famous facial acting coding system (FACS) by Paul Eckman is similar to this traditional knowledge. In Eckman’s model the emotion worry is absent, the emotion disgust is present and there is a sixth emotion - surprise. FACS is focused on the outer expression of the emotions.
Researchers at the Institute of Neuroscience and Psychology at the University of Glasgow have found out that:
- surprise and fear share a common signal – the wide open eyes
- anger and disgust share the wrinkled nose
They explain it from the point of view of evolution:
“widened eyes increases intake of visual information useful for escape”
“the wrinkled nose prevents inspiration of potentially harmful particles”
So they claim that the basic emotions are four - happiness, saddness, surprise/fear, anger/disgust.
Analysis
Fear and worry are very similar - negative anticipation about the future. Fear is specifically associated with a stimuli, worry is non specific.
The six emotions seem to be 3 pairs of opposites. Naturally, the opposites are happiness <-> saddness, fear (appear small) <-> anger (appear large) and … surprise <-> disgust? Hmmm, something is not right - there are more negative emotions than there are positive! Who else noticed this?! This seems to be a manifestation of the problem solving paradox - in order to solve a problem, you need to make it bigger first!
If anger and disgust are similar I decide to group them together. Because the requirement is for subtle emotions I remove disgust which includes lowered eyebrows and mouth morphed downwards.
After reflection I chose to provide the emotion of hope to the system. Hope being the positive, future oriented emotion which is opposite to fear. If fear is universal why haven’t researches found that hope is unversal as well? Maybe because the expression is not as clear as the rest… It may seem like a variation of happiness.
In conclusion, the application includes the following emotions: happiness <-> anger, surprise <-> saddness, hope <-> fear.
Technical implementation
I chose Scalable Vector Graphics (SVG) because each element is defined separately (eyebrows, eyes, mouth) which makes it easy to manipulate them individually as well as change their characteristics. Furthermore, SVG is interactive which is potentially going to be a future requirement.
Pictograph
I have no experience with SVG so I searched for an existing image which matches my requirements. Emojis (from Japanese 絵, ‘e’, picture + 文字, ‘moji’, character) are public and their use is usually permissive because “there is insufficient expression to create authorship”.
Usually, the neutral face is depicted with eyes and mouth only. But because the animations I envisaged include the modevement of eyebrows, there needs to be a point of reference for them - the eyes should also be present.
I found what I was looking for in an older emoji by Samsung.
I converted the image to SVG and verified that the facial features are separate components.
Animation
Initially I was searching for JavaScript libraries but then I found a (paid) web application - SVGator, which includes a graphical user interface and very precise editor. I decided to use that and it was quick and easy to do the animations using rotation, positioning and morphing.
Based on the research by the Institute of Neuroscience and Psychology at the University of Glasgow, the animations for surprise, fear and hope begin with raising eyebrows.
Personally, what I appreciate the most from the result is that now whever the communicator reconsiders a choice, that change of mind is elegantly visible to the interlocutor. This small change has a large effect on surfacing, exposing subtleties of the communication and bringing to it more life.
Roadmap
This initiative is about basic emotions. However, the beauty is in the complex emotions. I plan to expand the animations to 23 based on the points of view/motivations. The total number is odd (not 24) because Abstain does not necessarily involve the expression of an emotion.
The Chinese connect the organs to the five elements (fire, metal, wood, water and earth respectively to the above list). It is interesting because there is a model for how the energies of each element interact with each other. It would be interesting to experiment with this in the context of emotions: how does transitioning from one emotion to another affect the energy levels?
This also touches upon a different topic - dominance (the Pleasure Arousal Dominance (PAD) emotional state model by Mehrabian and Russel) or fatigue “Les États d’âme : Un apprentissage de la sérénité”: how much control over the emotions the communicator has. Another possible research area.
Finally, while researching child development I noticed that Envy, Jealousy and Fear (of someone specific) are oriented towards the interlocutor. Pride, Guilt, Shame, Worry (about the future) are towards the self. Therefore it is possible that the emotions+motivations may be 23x2 = 46.