Download Understanding Rapid Decision Making: Filtering vs. Framing and more Exercises Decision Making in PDF only on Docsity!
International Conference on Naturalistic Decision Making 2015, McLean, VA
(Very) Rapid Decision Making: framing or filtering?
Chris BABER a1^ , Xiuli CHEN b^ and Andrew HOWES b
aSchool of Electronic, Electrical and Systems Engineering, University of Birmingham
bSchool of Computer Science, University of Birmingham
ABSTRACT
This paper addresses two issues that arise from the challenge of studying the rapid decision making that characterises the domains of Naturalistic Decision Making. First, we are interested in how it is possible to make decisions in fractions of seconds. Second, we are interested in how such rapid decision making can be modelled. As a corollary to the first issue, we are also interested in exploring decision making which eschews the need to appeal to a concept of schema. Taking a cybernetic approach to decision making, we describe a model in which expertise is defined by the ability to filter salient features from the environment rather than in terms of the complexity of schema that is applied.
KEYWORDS Recognition-Primed Decision Making; Schema; Mental Models; Cybernetics.
INTRODUCTION
When expert decision makers respond to a situation, they rapidly determine the most appropriate course of action. The speed of response is such that it seems unlikely that experts engage in the sort of reasoning process which form the basis of traditional decision making techniques, and thus the theories of Naturalistic Decision Making (NDM) have developed to explain how such decision making is possible. The core question is, how can someone make (very) rapid decisions? In this paper, we propose that there is conceptual weakness in some of the dominant theories of NDM and that there is an alternative form of explanation which reflects the underlying intent of these theories while overcoming this weakness. The aim is not to overturn the NDM theories because these have proven themselves to be very useful in explaining behaviour, particularly in terms of the post-hoc accounts provided by expert decision makers, but to suggest that the initial stages of decision making might involve processes which have, to date, been under-represented in NDM theories. In short, the question is whether very rapid decision making is a matter of framing (cognizing a situation in terms of the schemata that experts develop and apply) or filtering (perceiving the situation through rapid extraction of salient information).
In many NDM models, features in the environment correspond to features in schemata held by the expert decision maker which, in turn, correspond to action (Klein et al., 1986; Lipshitz and Ben Shaul, 1997; Zsambok and Klein 1997). This is a similar process to that assumed in the Norman and Shallice’s (1986) Supervisory Attentional Control system, and can be seen in Cognitive Architectures such as Anderson and Lebiere’s (1989) Atomic Components of Thought (ACT). The implication of such approaches is that experts use a schema-driven control of action. In high tempo, high stress environments (such as incident response or, indeed, many sports), the time available for a decision to be made can be defined by milliseconds (even accounting for the ability of experts to anticipate environmental states). Information is extracted from the environment and then compared to a store of schemata and then, on the basis of weighted matching, an action selected, feels as if it might involve too much cognitive activity. We argue that this high level of cognitive activity need not arise from the decision making itself but from the focus on schema (and the declarative knowledge entailed in this approach). Consequently, the question is whether it is possible to define decision making in terms of procedural knowledge? In other words, to focus less on the structure of schemata and more on the manner in which perception is tuned to the environment.
Given the manner in which NDM case studies are often (but not always) constructed through post-hoc interviews, it is not surprising that a schema-based approach could prove conceivable. Gathering these verbal reports allows concept maps to be built and it is not too difficult to imagine that the concept maps, rather than representing the information provided by the experts can actually represent the knowledge held by the experts. From this, it is an easy step to assume that, as the concept map is the expert’s knowledge space, dealing with an
(^1) Corresponding Author: c.baber@bham.ac.uk; 0044 121 414 3965 ; School of Electronic, Electrical and
Systems Engineering, University of Birmingham, Birmingham. UK B15 2TT
incident involves activation and enactment of this concept map. Thus, we propose that many approaches to NDM not only assume that the expert decision maker approaches the situation in terms of declarative knowledge but also that decision making itself becomes a matter of negotiating the space of declarative knowledge. This further lends itself to phenomenological approaches in which expert accounts not only define the type of information that the expert is using but also define the type of decision making in which they engage. In other words, there is an assumption that the account provided by the expert after the event somehow becomes the contemporaneous account of doing decision making, rather than retrospective explanation of the consequences of these decisions. A consequence of this approach (and we propose a potential weakness) is the implication that decision making thus involves only declarative knowledge (either in terms of the repertoire of patterns held by the expert or in terms of the schema-driven search for information).
DECISION MAKING AND DECLARATIVE KNOWLEDGE
In their study of Authorised Firearms Officers (AFOs) in the UK, Mitchell and Flin (2007) suggest that the decisions to shoot or not shoot are “ …likely to be influenced by the experience [and] also by their expectations from prior information” (p. 377). In this study, AFOs, in a Firearms Training System, were asked to respond to the appearance of targets when they had received a neutral (no threat) or threat briefing (indicating that the target was armed and dangerous). The briefing did not appear to have an effect on either response time or decision to shoot. Either this suggests that the decisions were not made on the basis of this prior information, or the prior information was not presented in a manner which could influence decision making. The authors did note that it was possible that the participants responded to cues in the scenario which influenced their decisions. In a simpler task, Luini and Marucci (2013) asked participants to respond (using key presses) to images on a screen. Comparing trained and untrained participants, they showed that response to an ‘armed target’ was significantly faster than to an ‘unarmed target’ (i.e., images with and without a gun in their hand), and that trained participants showed significantly higher correct response and significantly lower false alarms than untrained participants. Taking these studies together, we propose that the shoot-no shoot decision depends on the appropriate definition of features in the environment and we further claim that this need not involve recruitment of schema. Indeed, it might be the case that Mitchell and Flin (2007) could have (through its use of briefing to stimulate differences in performance) have implicitly assumed that the participants would be responding using a more elaborate and detailed schema in the threat condition. For example, the ‘threat’ briefing of Mitchell and Flin (2007) could be represented in the form of a concept map (figure 1).
Figure 1 : Concept Map of ‘Threat’ scenario from Mitchell and Flimn (2007)
mental model which ultimately drives the decision process because it is the mental model which contains situation-specific information structured in a way that allows coherent and consistent decisions to be made; if the mental model is incomplete, erroneous or ambiguous then decision making will be less successful. In terms of their distinction between schemata and mental models, Lipshitz and Ben Shaul (1997) seem to want their cake and eat it; if mental models are specific representations of a situation then one would expect these to involve organisation and retrieval of information from memory, which are also characteristics of schema. The only logical step (we believe) that can be taken is to claim that there is a process by which information is acquired and a separate process by which information is stored. Such a distinction is beneficial because it provides a way of separating level 1 (which is primarily perceptual) from levels 2 and 3, which become increasingly cognitive (in terms of involving more detailed and elaborate activity using mental models). The question then arises as to where action selection occurs. For Lipshitz and Ben Shaul (1997) action selection is a response to the mental model, which implies that action selection arises at level 3 (or possibly level 2). However, might it be the case that there are, to use Gibson’s (1977) term, perception-action couplings, which would support action selection at level 1? Our proposal in this paper is that, rather than viewing action selection in level 1 in terms of schema, there is a simpler, more elegant description to account for the perceptual cycle that lies at the heart of expert decision making.
DECISION MAKING AND PROCEDURAL KNOWLEDGE
The approach we adopt shifts focus from declarative to procedural knowledge. To enable such a shift we adopt a cybernetic approach to human decision making, inspired by the work of Baron and Kleinman (1969). In their work, they applied concepts from control theory to model the operator of a complex dynamical system, such as an airline cockpit. In their model of cockpit instrument scanning, visual sampling is considered to occur in parallel with action selection. Sampling depends on the control task being performed.
Chen et al. (2015a) demonstrate how this approach can be used to model visual search in applied and laboratory tasks. Specifically, an optimal control model embedded with the assumptions of human visual mechanisms (e.g., visual acuity degradation away from fovea, saccadic duration, and fixation duration) offers explanations for the observed human behaviours in these visual search tasks (e.g., the gaze distribution, the search time, the saccadic selectivity across colour and shape). Decision-making, skills and rules are an emergent consequence of rational adaptation to (1) the ecological structure of interaction, (2) cognitive, perceptual and motor limits (e.g., visual and/or motor constraints), and (3) the goal to maximize the reward signal. This requires a theory which allows us to predict behaviour on the basis of utility, environment and information processing mechanisms. To do this, the model uses a state representation and an optimal controller approach.
The optimal control approach predicts behaviour from a model of the temporal costs of eye and head movements, a model of how visual acuity degrades with eccentricity from the fovea, a model of cue validity, and the assumption that operators optimise speed/accuracy trade-offs. A given feature in the environment is fixated and the result of this fixation (a percept) is encoded, in terms of specific attributes. The percept updates a state vector, which is used to compare current with previous state. Thus, for example, assume that we are facing a person who might be about to use a gun. Movement of the hand could constitute a change in state. However, depending on our decision policy, movement of the hand might not be sufficient to determine whether there is a threat or not. This means that we might require further information, such as what is the person holding in that hand, before we make the decision.
Through feedback, and experience, the behaviour of the control policy comes to resemble recognition-primed decisions. The model aims to predict the operators’ behaviours given theoretical assumptions about utility (e.g., a measure of the goal), psychological mechanisms (e.g., human eye-head coordination mechanism) and environment (e.g., the interaction between the operator and the interface). To achieve this goal, a state estimation and optimal control approach is used, as shown in Figure 3. In the task environment (bottom left), the operator moves head and eyes to acquire information. The state estimator (the bottom right) encodes a percept from the environment, which is then integrated with the previous state to generate a new state representation. Subsequently, the optimal controller chooses an action on the basis of the available state estimate and the current policy (which determines a state-action value function). State-action values are updated incrementally (learned) as reward and cost feedback is received from the interaction.
The control policy is a probabilistic mapping from states to action, depending on the constraints of the task and environment. This notion of a context-dependent mapping between the expert and the state of the environment feels analogous to Level 1 in figure 1. In this model, the control policy is a rule that allows the agent to select an action in terms of an action-value function. The optimal policy can be constructed by selecting the action with the highest value in each state. Using Q-learning, a form of model-free reinforcement machine learning, it is possible to define control policy as a utility function which is adjusted and tuned to the feedback from action to task performance (where task performance results in a ‘reward’, i.e., change in value between environment and
action). It is important to note that the state representation does not rely on a priori assumptions about the details of specific features or their relations. In other words, the state representation makes no assumptions about the content of declarative knowledge of the person, but is focused on selecting those features which best fit the policy.
In terms of the State Representation, a state will consist of decision relevant cues. For the shooting task, the cues could involve the nature of the object in the person’s hand, the posture of the person, line of sight etc. Each of these cues would have a different indication of the threat level the person presents. The state is then represented as all or some of these cues. To obtain information for these decision related cues, the model selects both eye movements and head movements (actions). The different sources of information result in different time costs and reliabilities of the information. During this process, the operators/model need to decide which cue to access, and when to stop information searching and make a decision.
In terms of Action, the output of the decision process would be to either ‘shoot’ or ‘search for more information’. The ‘search for information’ would involve finding and checking an information source, which costs time to acquire and the validity of information from each source varies. This task has been studied extensively in cognitive psychology (Newell and Shanks, 2003). In terms of probabilistic inference the observations which an operator makes are noisy, and sometimes using multiple sources, and the reliability of these sources differ. In a probabilistic inference problem, the key questions concern how people integrate noisy observations, and how people weigh different sources of information. These cues can vary in the reliability of the information provided. The more cues examined, the more information gathered thus more likely to make a correct decision. However, extra time cost and/or financial cost would be required. The probabilistic inference task has been used in cognitive science in efforts to discover the decision-making heuristics used by people (Gigerenzer & Goldstein, 1996; Bröder & Schiffer, 2006; Rieskamp & Hoffrage, 2008).
Chen et al. (2015b) use data from Wickens et al.’s (2003) account of visual scanning in a simulated aircraft pilot task. In this task, the pilot monitors the state of the aircraft, using several displays, during approach and landing in a flight simulator. Details of the study can be found in Chen et al. (2015b). For the purposes of this paper, figure 3 illustrates how the pilot’s attention switches between different information sources, on the basis of the model, and figure 4 shows how variation in scanning activity differentiates good and poor performance. In Wickens et al. (2003) original study, participants needed to detect misalignment across the displays. Participant 5 performed this task well, while participant 3 performed this task poorly. Figure 4 compares the observed monitoring behaviour of these participants with the predictions from the model, and shows that participant 5 (on the left) has data which fit the model well (r = 0.75), and participant 3 (on the right) has less of a fit (r = 0.33).
Figure 3: An overview of the optimal control model
ACKNOWLEDGEMENTS
The work in this paper is partly supported by European Union project FP6-619435 SPEEDD “Scalable ProactivE Event-Driven Decision-making”
REFERENCES
Anderson, J.R. and Lebiere, C. (1998) The Atomic Components of Thought , Mahwah, NJ:
Erlbaum
Bartlett, F.C. (1932) Remembering : a study of experimental and social psychology,
Cambridge : Cambridge University Press.
Baumeister, R. F. & Masicampo, E. J. (2010) Conscious thought is for facilitating social and
cultural interactions: How mental simulations serve the animal–culture
interface, Psychological Review 117 , 945–71.
Baumeister, R.F., Vohs, K.D. and Masicampo, E.J. (2014) Maybe it helps to be conscious,
after all, Behavioral and Brain Sciences, 37, 20-21.
Bröder, A., & Schiffer, S. (2006). Adaptive flexibility and maladaptive routines in selecting
fast and frugal decision strategies. Journal of Experimental Psychology: Learning,
Memory, and Cognition , 32 (4), 904.
Chen, X., Bailly, G., Oulasvirta, A., Brumby, D. & Howes, A. (2015a) The Emergence of
Interactive Behavior: A Model of Rational Menu Search, CHI 2015 , New York: ACM.
Chen, X., Howes, A., Lewis, R.L., Baber, C. & Feary, M. (2015b) A Model-based Approach
to Assessing Pilot Attention Allocation, University of Birmingham, School of Computer
Science, Internal report (available from first author).
Gigerenzer, G. & Goldstein, D. G. (1996) Reasoning the fast and frugal way: models of
bounded rationality. Psychological review , 103 , 650-669.
Klein, G. A., Calderwood, R. & Clinton-Cirocco, A. (1986) Rapid decision making on the fire
ground, Proceedings of the Human Factors and Ergonomics Society ??nd Annual Meeting ,
Santa Monica, CA: HFES, 576-580.
Klein, G.A., (1993) Recognition-primed decision making (RPD) model of rapid decision
making, In G.A. Klein, J. Orasanu, R. Calderwood and C. Zsambok (eds.) Decision-
making in Action: models and methods, Norwood, NJ: Aplex, 138-147.
Lipshitz, R., & Ben Shaul, O. (1997). Schemata and mental models in recognition-primed
decision making. Naturalistic decision making , 293-303.
Luini, L.P. & Marucci, F.S. (2013) Effect of emotiuons on cognitive processes : are proficient
decision-makers inoculated against the influence of affective states? International
Conference on Naturalistic Decision Making 2013,
Mitchell, L. and Flin, R. (2007) Shooting decisions by Police Firearms Officers, Journal of
Cognitive Engineering and Decision Making, 1, 375-390.
Neisser, U. (1976) Cognition and Reality , San Francisco, CA : W.H. Freeman
Newell, B. R. & Shanks, D. R. (2003) Take the best or look at the rest? Factors influencing
"one-reason" decision making, Journal of Experimental Psychology: Learning, Memory,
and Cognition , 29 , 53-65.
Norman, D.A. & Shallice, T. (1986) Attention to action : willed and automatic control of
behaviour, In R.J. Davidson, G.E. Schartz and D. Shapiro (eds.) Consciousness and Self-
Regulation : advances in research, New York : Plenum Press, 1-18.
Plant, K. L., & Stanton, N. A. (2013). The explanatory power of Schema Theory: theoretical
foundations and future applications in Ergonomics, Ergonomics , 56 (1), 1-15.
Rieskamp, J., & Hoffrage, U. (2008) Inferences under time pressure: How opportunity costs
affect strategy selection, Acta psychologica , 127 , 258-276.
Wickens, C., McCarley, J. S., Alexander, A. L., Thomas, L. C., Ambinder, M., & Zheng, S.
(2008) Attention-Situation Awareness (A-SA) model of pilot error, In D.C. Foyle and B.L.
Hooey (eds) Human performance modeling in aviation , Boca Raton, FL: CRC, 213-239.
Zsambok, C.E. and Klein, G. (1997) Naturalistic Decision Making, New York: LEA.
