How attention is simulated and how positioning works in Besѣda®
Content
Design
Issue
Let’s take the following interaction:
- 01 : may I have that beer?
- 02 : are you eighteen?
- 03 : no
- 04 : no
04 is the response to 01 even though it is at the end of the sequence. 02 - a condition to satisfy or deny the request for alcohol depending on the legal age seems to be unrelated to 01. How to represent these in Besѣda®?
Further still, there are occasions where there are retrospections (“flashbacks”) whereby the chronological order contradicts the sequential order (an utterance which was delivered later in the conversation is about an earlier chronology).
Because Besѣda® is a simulation of the complex process of talking, more generally, what have been concerning are the following two questions:
- Why change the topic and not talk inside one container during the whole meeting?
- Why use different actions and not deliver all the speech using the most convenient button?
Requirements
The feature should allow the communicator to determine which adjacency position s/he is intending to deliver.
The feature should allow the communicator to select which utterance they respond to.
The feature should allow the communicator to create a new topic which begins a new turn structure. The communicator should be able to create as many topics as needed.
Limitations
The system does not record and interpret the actual utterances and therefore some of the more common linguistic techniques (see the section about turn taking) to satisfy these requirements are not part of Beseda. Alternatives should be implemented.
Research
I’m going to present my research not how it chronologically happened but how it should logically flow. Attention is fundamental to any action and interaction so let’s begin with that.
Attention
The human brain is such that it can only keep focused on one thing at a time. There’s the myth known as multitasking (performing different tasks simultaneously) but that is fundamentally switching rapidly the attention between tasks.
Because language is a very complex system, the attention is dynamically shifting between the topic, the method of delivery of the content, the interlocutor etc.
Intention
An interesting paper from 2002 which logically divides the structure of the conversation in three components. Specifically, one of the components is the structure of purpose (also called intentional structure). There is one or more main purposes for the entire conversation and there are purposes for the different segments of the conversation which have a relation to the main purpose as well as between each other.
Actions
I found validation for my beliefs and design so far simply by searching the internet for the query “pragmatic or epistemic”. A side note - epistemic is alternatively known as gnostic but that term has more religious connotations.
I found this paper “On Distinguishing Epistemic from Pragmatic Action” in which amusingly two scientists perform rigorous examination on people playing the video game Tetris. They observed that some actions cannot be explained with the traditional conceptual model of action so they introduced a new concept - epistemic action: “actions performed to uncover information that is hidden or hard to compute mentally”.
Additionally, researching this distinction lead to this paper about emotions as epistemic actions. The introduction contains a wonderfully rich summary about research on emotions. I recommend the paper. It also provoked me to rethink my previous work on emotions.
Turn taking
Turn taking is universal accross the entire planet. This research work was produced by many scientists and contains a clear structure of the adjacency pair and the three expansions: before, between and after.
a turn’s talk will be heard as directed to a prior turn’s talk, unless special techniques are used to locate some other talk to which it is directed - John Heritage
Some of these special techniques are mentioned at figure 12 in the paper which is referred to in the intention section of this blog post. For example - cue phrases such as “now”, “but”, “anyways”, “this reminds me of”. Other techniques are the intonation, tense and aspect.
Analysis
The formula mimics the behaviour: attend to, intend to do and act upon.
One communicator who used Besѣda® argued that he doesn’t know what beforehand what the intention is for a specific message. I appreciate this author’s eloquent explanation about action:
does not necessarily require a premeditated hypothesis that is deliberately tested, but can be a very simple behavioral mechanism - Wendy Wilutzky
Implementation
Even though I was aware of some of this research as far back as 12 months ago, I did not implement the concepts at the time because I was concerned that they would make the system more complex without adding value. Now it all seems to fit together and result in a more intuitive experience.
In terms of attention - Besѣda® tracks if the listener is adjusting their intention while the interlocutor is speaking because that likely means that s/he is not paying attention to the content of the delivery.
In terms of the linguistic structure - the actual utterances in that particular conversation episode, I implemented tabs which contain the HTML canvas to which the moves are added.
We call the collection of focus spaces available at any one time the focusing structure and the process of manipulating spaces focusing - “On Distinguishing Epistemic from Pragmatic Action”
Focus spaces in Besѣda® are tabs. I considered a few options about how to display the linguistic structure:
- as a fractal tree. However that is not applicable as the structure is not symmetric and it may be incomplete.
- as the Vitruvian Man because the structure comprises of two pairs and three expansions. But it does not make visual sense.
Finally I created a layout similar to a mind map. You can see it in the image in the user manual section.
Besѣda® is based on the V.I.C.T.O.R.I.A. communication framework and intentions are central to the use of the system - each of the 48 pragmatic postures is an intention. At a later time I’m going to share the framework publically.
For the epistemic actions there is shared text editing which allows the knowing party to provide exact information to the other side or for the unknowing party to write down their questions and/or notes of the answers.
Finally, Besѣda® features both:
- spontaneous turn taking - where the parties take turns at their own discretion but without the possibility to interrupt ongoing deliveries of speech
- strict turn taking - a strict mode in which the parties have only 10 second moves and they must listen to the interlocutor after their single move. This mode is appropriate when one or both of the parties is/are abusing the system. It is also appropriate when there is low trust.
User manual
Attention
Chose which utterance to respond to or create a new focus space. The space is organised in the following way: the center is a square which represents the interlocutor, above the root square, the pre expansion moves are drawn. The first pair part is drawn to the left side and slightly below the root node. The second pair part is drawn to the right side slightly below the root node. The insert expansion is drawn between them. And the post expansion is further to the right after the second pair part.
The chosen move is highlighted with a dark blue border. By default the newest move is highlighted - you can see on the image above that when the interlocutor starts delivering their message, a new node is created, the basic emotion is displayed inside along with a button to interject (turn the microphone on for 1 second). After the interlocutor completes the delivery there are additional steps such as the option to reciprocate. See the earlier blog post about reciprocity.
Position
Chose the adjacency position: the plus signs are expansions and the puzzle like elements are the first and second pair parts respectively. Note that if the interlocutor delivers a first pair part and you reciprocate (select one of the suggested reciprocal speech action) that would make your move the second pair part.
You can hover over the elements in the image below to see an update. All rights are reserved on this image.
There are two limitations to positioning - you cannot insert without a first pair part; you cannot expand without a second pair part.
Intention
The direction of action is inside -> out (pragmatic action) OR outside -> in (orientation/epistemic action). Click on the image below to reveal the arrows.
All rights are reserved on this image.
The current design of Beseda is such that the semantics of the conversation are freeflow - in other words, epistemic actions cannot be a first or second pair part. If a conversation consists only of epistemic actions then the linguistic structure would consist only of the pre expansion.
Implications
The author makes the following hypothesis:
- Given that the first pair part (FPP) can be repeated and/or modified, any previous versions of the part are expired. By extention this also means that the act of modification, repetition and update simultaneously dismisses the current version of the second pair part (SPP). This rule applies asymmetrically to the SPP - repetition or modification of the SPP does expire the previous versions but it does not dismiss the FPP.
- Pragmatic actions create parameters of the conversation which epistemic actions can refer to and modify.
- (An) Epistemic action(s) can be embedded within a pragmatic action. The reverse is not possible. Specifically, the emotive and referential functions can be embedded individually or combined within the conative function (per Roman Jackobson’s definitions).
To exemplify these hypothesis, let us examine an actual conversation from Chapter 12 of the book Action Ascription (with modifications for illustration purposes):
- Parent: Tell me about your day.
- Parent: What did you learn?
- Child: We went on an adventure in the park.=
- Parent: Yiea:h, an’- an’- tell me about it. (1.5s)
- Child: Uh:m_ .h It was fu:n?, (0.5s)
- Parent: No. (.)
- Parent: You’re gonna h(h)afta do uh lot better than that.
At line 1 the parent implements the pragmatic action Instruct - this can be claimed with high confidence because of the last utterance in the excerpt (line 7) where the parent refers explicitly to the expectation. An instruction includes a claim to examine and hold the interlocutor accountable for their fulfillment of an expectation. If the parent did not have any expecation that utterance could be interpreted to be the pragmatic action Invite, because both the child and the parent would be engaged in a free flowing conversation. According to the second hypothesis, the pragmatic action creates a parameter for the conversation - an expectation.
At line 2 the parent makes an epistemic action which defines the scope of the expectation. This is an expansion which is done after FPP (a post FPP insert expansion). According to the second hypothesis, the pragmatic action created a parameter for the conversation and this epistemic action refers to it and modifies it. The epistemic action thereby expires the first version of the FPP. The new version is the following combination: “Tell me what you learned today.”
At line 3 the child makes an epistemic action - to share the story about the most important and unusual event of the day. The current design of Beseda is such that an epistemic action in and of itself cannot serve as an FPP or as a SPP because it does not serve a conative function. Considering this design, the epistemic action of line 3 is embedded inside the pragmatic action Appear - the personality has been present in that location at that past time period. The pragmatic action serves as the SPP. Note that the reasoning for the 3rd hypothesis is not based solely on this design choice for the application - the epistemic action is the vehicle for the message whereas the pragmatic action is the change agent.
At line 4 something interesting happens - at this point of the conversation it becomes clear that the parent had already been aware of what happended during the day. The parent’s higher order goal for the conversation is to teach the child to explain clearly. The tactical choices which the parent makes are to take the epistemic stance of knowing (K+) by delivering “Yiea:h”. Then the parent implements the pragmatic action Instruct again. The parent repeats the first part. According to the hypothesis this means that the previous versions of the FPP are expired and the current second part is dismissed/cancelled.
At line 5 the child makes the epistemic action - to share her subjective experience. A subjective experience can neither be true nor false - it is. Therefore the question mark at the end can be confusing. Actually, what the child is doing with the elevated pitch in the turn-final position is implementing the pragmatic action Ask for confirmation - does this version of the SPP satisfy the parent’s expectation?
At line 6 the parent makes a post SPP expansion (also called the third position), which is often where the initiating communicator can accept, decline or modify the interpretation by the interlocutor of their first pair part. The parent’s rejection is the pragmatic action Dismiss. The dismissal is more direct and explicit than the repetition of the FPP.
At line 7 the parent makes the pragmatic action Demand - challenging the child to do “better” in whatever sense of the word is intended in that particular context. There speech act includes the evaluative qualifier “a lot” which signals that the performance of the child is below the expectation. If the parent had been more supportive and encouraging other techniques would have been implemented.
Roadmap
The present development opens the possibility to include designing the turn - recipient design, dis/preference and other functions.
Changelog
- October 9 2024: Changed the title of this blog post from “How turns are organised and structured” to “How attention is simulated and how positioning works”. Moved the
- September 11 2024: Added the section Implications