Using the Safety Culture Discussion Cards to help understand textual data

‘What we call our data are really our own constructions of other people’s constructions of what they and their compatriots are up to’ (Geertz, 1973)

Probably the most common approach to trying to understand safety culture is via safety climate questionnaires, usually comprising a set of items with a Likert-scale to indicate the level of agreement with each item. Unfortunately, such questionnaires alone do little, if anything, to help understand the meanings that people ascribe to their values, beliefs and behaviour, and so do not explain why we do things, why we do things in the way that we do them, or the conflicts between what we say and what we do. To gain a deeper understanding, a qualitative, interpretive approach is more fruitful, not necessarily to supplant questionnaires, but at least to supplement them. Prior to interactive methods such as focus groups and interviews, one source of data from the questionnaire itself can be a useful starting point to an interpretive approach - the free-text comments written by the respondents.


I recently used the Safety Culture Discussion Cards to help analyse several hundred typed/written unstructured comments from a safety culture questionnaire - a fairly large amount of textual data. Many of the comments were several paragraphs long and referred to a variety of issues, and were mostly very interesting, well thought out and well-written. Making sense of rich textual data is never easy. But a common approach to understanding is via 'content analysis' (Krippendorff, 2004), or textual analysis. This often involves reading the text and applying a set of codes or categories to try to understand the data.

In this case, I decided to try to use the Safety Culture Discussion Cards to help code the data. The aim was to get a detailed understanding of the issues that questionnaire respondents were motivated to comment on - the specific issues, the way the writers related issues to each other, and the number of times that each issue was mentioned. An assumption was that issues mentioned more often by respondents reflect concerns that are important to them.

The cards cover most relevant aspects of safety culture but are (deliberately) not mutually exclusive, so this had to be kept in mind during the analysis. Prior to and during the coding, it was necessary to remove or combine cards as appropriate in order to achieve some satisfactory level of mutual exclusivity.



I started the analysis by reading all of the comments very carefully, and coding pieces of text within each comment using the eight elements of safety culture covered by the cards (Management Commitment; Resourcing; Just Culture, Reporting & Learning; Risk Awareness & Management; Teamwork; Communication; Responsibility; Involvement). Because a person's comment could cover all sorts of issues, it is not possible to apply just one element code to each comment. Even a particular sentence within a comment could cover two or more issues, such as 'Management Commitment' and 'Resourcing'. So at this stage, a sentence or paragraph could be coded using one or more elements.

The next stage was to re-read the comments and now apply more specific codes to the various pieces of text. The specific codes relate to the codes on the safety culture discussion cards, from 1a to 8e, noting also where the text was positive/favourable or negative/unfavourable in nature, or sometimes both. Since some of the cards overlap, where a piece of text could be coded using more than one card (and the cards could not reasonably be mutually exclusive) the codes were combined.

The final stage involved rechecking the use of the codes for each comment to ensure consistency and calculating the usage of each code. (An even more rigorous application of this method would involve having independent coders repeat the exercise with all or some of the text, as I and Amy Chung did when analysing comments relating to HF/Ergonomics practitioners' views on barriers to research application; see Chung and Shorrock, 2010.) This allowed the relative frequency of each issue to be determined, and gave an impression of the perceived pertinence of the various issues.

The frequency of each element as well as the top 20 issues were calculated. The quantitative data, combined with discussion of the actual content of the comments, added substantially to the data received from the Likert-scale standard questionnaire items.

A final interesting output from this exercise is the ability to the the cards to visualise the narratives in the comments by mapping the relationships between issues and the possible meanings emerging. This will be the subject of a different blog entry. The exercise also revealed a few issues that are not covered by the existing cards, as well as the issues that are covered by the cards but that were not mentioned at all by the commenters. This is useful feedback for the further development of the cards.

References

Geertz, C. (1973). The interpretation of cultures: Selected essays. Basic Books.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.
Chung, A.Z.Q. and Shorrock, S.T. (2011). The research-practice relationship in ergonomics and human factors - surveying and bridging the gap. Ergonomics, 54(5), 413-429.

Five questions about boredom, fatigue and vigilance

Below are five questions posed by a safety colleague, and the brief responses.

1. How different are boredom and fatigue?
Both affect our ability to pay attention - to notice something that may need attention - but they are different in terms of their causes and can occur completely independently. A person can be bored during a period of low activity, but not fatigued. Prolonged boredom, tends to result in fatigue, but so can high workload, lack of sleep or disruption to sleep patterns, or stress. Other than sleep or rest, there is little that you can do to manage fatigue effectively while on position, while more can be done to tackle boredom and stay in the loop. So preventing and managing fatigue is a key priority to ensure that people remain able to deal with unusual events.

2. Is low workload more dangerous then high workload?
Attention is stretched by both 'overload' and 'underload'. Both require hard work and can be stressful, particularly if there are safety consequences when something is missed. Which is more dangerous will depend on the situation and the person (for instance personality, experience and levels of stress and fatigue), but skilled professionals tend to cope better with higher workload up to the point of overload, when performance degrades more dramatically.

3. How to remain aware and vigilant for unusual situations?
Ask colleagues - people develop different visual and mental strategies that may not be obvious from the outside. But applied research using eye movement tracking gives some tips in terms of scanning. So-called "active scanning" can help to counteract degraded vigilance under low workload situations. With active scanning, people scan displays proactively in sequences or cycles depending on the traffic situation, linking specific information from different information sources. The scanning is more strategic, and helps to anticipate developing situations.

4. When are we most and least vigilant?
In a non-shiftwork environment we could highlight some times of day when we are least alert, especially during the very early morning hours, but shiftwork is a fact of life for many workers working a 24-hour operation. What we can say is that we are most vigilant when well rested, engaged and interested in the activity, not distracted (e.g. TV, radio, visitors) or preoccupied with other thoughts, well hydrated, and well supported by colleagues and supervisors.

5. Is the theoretical human performance knowledge adding value?
Yes, but not nearly as much as it should. So much is known about human performance that it seems that policy and practice are decades behind. But so much that is published is irrelevant to complex systems and activities, does not offer solutions, and technology and practices change fast and do not wait for research to catch up. Much theoretical knowledge in human factors comes from sterile experimental environments, normally focusing on one issue (e.g. vigilance) while 'controlling' (or ignoring) some of the most relevant real-life issues that interact to shape performance in the real world (e.g. motivation, risk, teamwork, supervision, background shift-fatigue). The hard part for practitioners is evaluating what aspects of the research are relevant, piecing them together and drawing out practical implications. With this in mind, the most directly useful human performance knowledge is gained by spending time with end users, listening to and observing them at work, and working with end users and other stakeholders to find solutions to human performance issues.

Should the Institute of Ergonomics and Human Factors be more of a campaigning organisation? Yes.

Published in 'The Ergonomist', Newsletter of the Institute of Ergonomics and Human Factors, October 2012, p. 4

In September's The Ergonomist, the President of the Institute of Ergonomics and Human Factors asked whether the IEHF should become more of a campaigning organisation. Assuming that we want to be a relevant organisation, then the answer must be 'yes'. While we have many interesting research findings and effective applications, we rarely seem to communicate our impact in the world.

It is a sad state of affairs that the 'impact' of publicly-funded research is judged primarily by the citation of journal articles by one's fellow researchers and oneself. It is equally sad that we have so few press releases, white papers, blogs or videos of our impactful theories, findings or applications. We seem to put most effort into forms of communication that are least visible to policy makers, decision makers and the public. Perhaps this is why we are still too anonymous to the wider world.

We cannot be content with only writing to each other via Old Media or speaking to each other in closed conferences if we want to make a visible difference. The research article or technical report should not be the end of the line for any of us. If we think that our discipline is important, then we need to to be confident and decisive in our messages and campaigns, and clever in how we convey them.

It is great to see that our engagement with social media is growing (e.g. LinkedIn, twitter) and that we have had recent public exhibitions. But we need more involvement. We need to be prolific not in how much we write, but in the effectiveness of our communication with decision makers, those affected by our work, and the world at large.

We need to put more effort into the usability of our communication with the world. Think Wordpress, Blogger, twitter, pinterest, Google+, flickr, picaso, Amazon, LinkedIn, Experience Project, e-petitions...as well as letters, magazines, and face-to-face, of course. None of us has 'time' for this, except the time that we prioritise for it. As Jon mentioned, raising awareness isn't just a job for the IEHF. It is for all of us to ensure that our research and practice remains relevant to the world and has broad impact. 

Steve Shorrock

Using the Safety Culture Discussion Cards: Tips from a user

I have received some great practical tips (and considerations for the future) from an ATC Safety colleague who had used the Safety Culture Discussion Cards with several different groups. Thanks to Alfonso Barba Martínez (Head of Regional Safety at AENA, the Spanish Air Navigation Service Provider) for the tips below!

Basically, I find it essential to make an introduction to the aims of the cards and the different ways to use them. What I find could be an improvement is to introduce simple cues for those having to administer them as `facilitators´, because what we have been doing until now is introducing the product to those who can make a better use of it (Instructors, Supervisors, mid-management). This made me think that it might be useful to explain a bit further what the outcome of the cards can result in. So basically I would cue those administering the cards on:

a) Make sure who your audience is, and prepare specific cards for them on each element.
b) The cards are an excellent tool to insert into any meeting as an added practical activity, breaking frequently tedious expositions and offering some `brain refreshment´.
c) Don´t use the cards with more 10-12 people? Otherwise the discussion might drift away very easily.
d) Is anyone supposed to take notes? As facilitator I advised them I would be taking notes, and the different comments and views expressed allowed me to identify weak points in which future safety strategies may be focused on at local level: shifts, fatigue management, airport signaling.
e) Also, I would favour using Option 2 Safety moments at the beginning, as it is a lot easier to engage people in a straight forward discussion on just one item, two maximum.
f) In the `How to use this Cards´ section, Options 1 Comparing views, Option 3 Focus on... and Option 2 SWOT analysis don´t necessarily need to use all cards, but it sounds like you must. It all depends on time available and going through all of them should not be the objective.
g) Be careful with the card selection when mixing groups with different activity areas, i.e. Human Resource, Maintenance, ATS, Financial and again, have relevant questions prepared beforehand.

I hope this can be useful to you. As I say, it is mainly focused on the tasks by those presenting the cards to others who must deal with different audiences, and could perhaps feature as a card of its own at the beginning.

In the near future, I'll be blogging on each of the options for using the cards, and it would be great to hear of any other options for how the cards have been or could be used. In the meantime, you can read more about the cards (and download them all as a PDF via the link at the end of the page) here, and see the individual cards as images here.

Reading on the job: fatigue, boredom and distraction while underloaded

I have received a few queries asking for a view (or "the science") on reading during operational duty, particularly as a possible countermeasure against fatigue during nightshifts. This is in an air traffic control context, meaning reading while a controller on duty at workstation, but it would apply to many other safety-related contexts where people need to monitor.

This is a sticky issue. But it is the kind of issue where a human factors specialist is expected to give advice or at least some succinct guidance or points for consideration. This kind of query is rarely straightforward. The 'science', inasmuch as it is relevant, cannot be simply generalised from laboratories or other contexts, and there is rarely any possibility for controlled experiments to study the problem directly. So, is reading a suitable countermeasure against fatigue? Below are some of my thoughts on this issue.

The task of the air traffic controller, as with many other safety-critical roles, is increasingly visual with high demand for monitoring. Automated tools such as data link and conflict alerts are becoming more common. They usually do not have any associated audible alert or alarm and may not be expected or anticipated. As people naturally become more reliant on such automated aids for normal performance, the need for alertness is even greater.

But there can be fairly long periods of low activity. Reading during operational duty is fairly common during these periods. The motivation to read is one that we can all identify with. We have little tolerance for monitoring when understimulated, and naturally want to fill in the gaps with other activities. Stimulation is often at its lowest during a nightshift (particularly ~0200-0500), when we also experience heightened fatigue. In this scenario, reading may seem like a good idea to maintain alertness. 

But what are we really trying to combat via reading during periods of low activity? Most likely, we use reading to combat boredom rather than fatigue. Boredom is an unpleasant or even distressing state that we naturally wish to avoid. Reading can be stimulating (and effective to counteract boredom) but it can also be visually fatiguing (especially during night shifts). 

Reading, fatigue, boredom and distraction (Photo credit: Ani-Bee http://flic.kr/p/LaxT3 CC BY-NC-ND 2.0)

There is then the issue of distraction. Reading may mean reading from paper (book, newspaper, etc), an e-reader, a smart-phone or a tablet. Reading itself is a primary distraction from operational information, but smartphones and tablets introduce secondary distractions from other applications, such as emails, instant messages, tweets, status updates, notifications, etc - many of which are designed (or can be set) to capture attention (e.g. via pop-up notifications or alerts). 

Reading feels like the most natural way to repel boredom and so maintain stimulation, but at the same time it can increase fatigue and distraction and decrease vigilance and alertness - a situational irony. 

Then there are the legal implications. In the case of a serious incident or accident, reading non-operational material when visual detection may be safety critical could be raised in court, because reading takes attention away from the primary task (e.g. radar monitoring). An argument could be made (ignoring the whole context of the work) that if the person were not reading, then a problem (e.g. conflict) would more likely have been detected. 

It is important to separate the problems of fatigue and boredom because the countermeasures are different. Fatigue requires rest and sleep. Boredom requires stimulation. But often they come as a package, in which case fatigue will usually trump boredom, as fatigue is so much more difficult to counteract without prior planning. A better way to manage fatigue during nights might involve more frequent breaks and scheduled napping, as well as a well-designed shift system (see a EUROCONTROL literature review and Hindsight  Issue 13 on fatigue).

Combating boredom on an operational level also requires frequent breaks - there is positive evidence that light exercise such as a brisk 10 minute walk helps increase arousal. Breaks are only a partial solution, though, because the problem is one of job and task design.While these are not easy to change, while working on position, other tactics might include more active visual scanning (as opposed to reactive scanning), and increasing workload (e.g. combining sectors, offering higher service levels), or rotating activities, tasks or roles (e.g. sectors, positions or non-operational work). A 1996 EUROCONTROL-sponsored research project examined monotony in ATC, and came up with a few recommendations (page 184), but these do not shed much more light on the matter. Note that some other 'common-sense' tactics, such as playing music, can also be counterproductive because of distraction. In the longer term, more strategies are available via personnel selection (some people have a greater tolerance for boredom) and task, job and technology design (for instance, auditory alerts to complement visual information during specified hours), but these options are of little relevance to current operations. 

So reading itself is probably not an effective countermeasure for fatigue while on duty, but we have to recognise that the desire to read indicates another problem that needs to be addressed: boredom. Until we are able to design the boredom out of jobs, it would be useful, then, to share the practical strategies used by different people in different organisations and industries to maintain alertness and vigilance, and fend off boredom, in real world safety-critical environments.

Breaks from operational duty and fatigue management

by Steve Shorrock

(Adapted from Hindsight, Volume 13, Summer 2011, pp. 68-69, EUROCONTROL)

Guidelines from Human Factors research for optimising human performance and reducing fatigue-related risks.

Breaks from operational duty are an important factor in the management of fatigue. But as highly committed and professional operational staff often perform several secondary tasks and activities – inside or outside the ops room – breaks can become a victim. Breaks seem to be used for a wide variety of activities. A controller once said to me that working through breaks was often necessary for him to complete his secondary role…in safety management. This tendency to use breaks to perform tasks that add to fatigue, rather than reduce it, can increase over time without being noticed. In the end it may seem like the normal way of working. It is too easy to forget that the important role of breaks in managing fatigue – visual, mental and physical – and of course in managing stress.

The issue of fatigue is a core issue in human factors. So what activities are acceptable from this point of view? Anyone with some exposure to the field of human factors will not be surprised to hear that there is no black and white answer. But some general guidelines can be offered from the research on fatigue and vigilance.

Let’s start by asking, what is the purpose of a break from operational duty from a human performance viewpoint? Operational tasks are often visually demanding, involving monitoring the outside view, the situation display, or other screen-based work. This work contributes to visual fatigue, even when you are not particularly physically fatigued. Operational work can also involve intense periods of typing, writing or other inputs. You may notice how this brings about feelings of tension in the hands, arms and shoulders. Similarly, the work can often involve static postures, and so more general physical fatigue or discomfort can result. In high workload periods, mental fatigue can build up. But low workload periods can seem even more mentally tiring, and it is during these times when vigilance can really suffer.

So breaks are needed to recover from a variety of demands and the needs from a break will vary. Depending on the operational activity, you may need to rest, or relax, or re-energise during the break. We could put break activities into three categories, and as there are no black and white answers, we’ll call these red, amber and green activities.

 Red activities 

There are some activities that we know (from research and common sense) should be avoided during breaks, or at least minimised. In particular, activities that increase visual or mental fatigue, or increase feelings of stress or pressure, can prevent recovery. Tasks that are likely to be stressful may cause worry or preoccupation and so can even affect your work before the break. In general, such activities should be done outside of break periods where possible. Activities that may fall into this category might include assessments, interviews, important reports, or even difficult conversations. But other types of work which are intense or particularly important are also best done outside of breaks. Managers and staff need to arrange additional time for such activities, where possible.

 Amber activities 

Many activities that are sometimes performed during breaks are often OK in moderation. These really depend on the operational activities before and after the break. In general though, activities should be reduced where they are similar in nature to the operational work, for instance visually demanding, involving a lot of manual inputs, or time-pressured. Let say you have had a very busy period involving high levels of concentration and lots of keyboard or touch screen work. It would make sense, during the break, to reduce or minimise time spent internet surfing or completing detailed computer-based administration. Similarly, if you have had a very busy session on duty, then the break is probably not the time to rush around doing other time-pressed activities. But if you have had a very quiet period, the same activities could help you re-energise. Where it is necessary to do activities that are similar, physically or mentally, to the operational tasks, it is advisable to take a reasonable period of ‘green time’ to recover before going back on duty.

 Green activities 

The best kinds of activities to perform during breaks are usually different to the activities undertaken while on duty. These ‘green activities’ will allow you to rest, relax or re-energise. They might include restful or relaxing personal time, social activities, or light exercise. They may be individual to you, but will certainly depend on the nature of the operational work. A period of ‘green time’ taken immediately before going back on duty, and preferably outside the ops room and away from a computer screen, will help optimise performance.

So when it comes to breaks from operational duty, changes in activity are the key to reducing fatigue-related risks.

Of course, how breaks are spent is only part of the picture. The timing and frequency of breaks are also important and, to a lesser extent, their duration. These broad guidelines would ideally be incorporated in an overall policy or risk management system for fatigue, including provisions for time to perform secondary tasks that may present a fatigue risk. Until then, perhaps managers and operational staff can put the guidelines into practice to help optimise human performance, and maintain safety.

Safety Culture in your Hands

(This article featured in The Controller - Journal of Air Traffic Control, April 2012.)

by Steve Shorrock, Human Factors and Safety Specialist, EUROCONTROL

Are safety issues raised by front-line operational/technical staff given appropriate priority within your organisation? Do you and your team incorporate lessons from incidents into your work? If you saw a colleague doing something you thought was risky or unsafe, what would you do?

These are questions about safety culture. They are just examples of the types of questions that are addressed in the EUROCONTROL safety culture programme for European air traffic management (ATM), which has been underway since 2003.

One thing that is clear from the programme, which involves questionnaires and workshops with operational and non-operational staff, is that controllers have very clear opinions about these sorts of questions. But a survey is fairly pointless without action and continued conversation about the issues that matter to those on the front line of safety.

To help the conversation, the EUROCONTROL Safety Culture Discussion Cards are a practical resource to aid real discussion about safety culture by any person or team within the air navigation service provider (ANSP) organisation, especially operational staff. This article gives you a brief overview of the cards. So what is the point of the cards?

They get people talking. The cards are designed to provoke discussion among both operational and non-operational staff, and have been used successfully with operational staff in safety culture workshops, as well as an aid to team resource management training. They do not give answers, but instead raise questions.

The cards build on what you know already. Operational staff already know about safety culture. They live it and feel it. So the cards build on this understanding. The cards do not use theoretical language. They have been designed and tested with controllers. But still, they are based on the EUROCONTROL approach, which has been used on around 20 ANSP surveys.

The cards help improve safety culture by encouraging discussions on ways to improve safety culture, inspiring action based on the outcome of the discussion.

The physical cards are printed in colour on A6 card (but may also be used digitally, e.g. on smartphones). The first few cards in the pack explain very briefly what safety culture is, show the organisation of the cards, and explain some possibilities for using the cards. Then, the discussion cards are sorted into eight elements:

• Management commitment
• Resourcing
• Just culture, reporting & learning
• Risk awareness and management
• Teamwork
• Communication
• Responsibility
• Involvement

Photo credit: Pamela Ocampo http://flic.kr/p/7xCRdu CC BY-NC-ND 2.0

There are several discussion cards for each element, and each discussion card is designed to catch attention with a photo and headline, raise questions and provoke discussion. There are 83 cards in total – 70 of these are the actual discussion cards, while the rest are explanatory (introduction, photo credits, disclaimer, etc).

So how are the cards used? The cards can be used in any way you can think of, but several ways have been tried. For instance to com-pare views: different individuals or teams sort cards into two piles: ‘What we do well' & ‘What we need to improve', then discuss the piles. Another way is to have 'safety moments: in a small group, take just one card and discuss the card for 10 – 15 minutes. Or have a small group choose a specific element, such as ‘Just culture, reporting and learning', and discuss each card in depth, eg. What and where is our ‘best practice' on this issue? Where do we need to improve? Etc. You could also organise the cards into pat-terns to show how the issues relate to one another in your organisation, unit or team.

You can use any number of cards, from one to the whole set – whatever works for you. Safety culture can seem abstract, fuzzy and hard to break down. The cards provide a way to discuss safety culture in a straightforward and practical way.

The pdf version may be viewed on smart phones and a low resolution version can be found here. For more information and to view the cards as jpeg files, see the SKYbrary article here. High resolution (English and French) print-ready versions are also available: contact steven.shorrock {at} eurocontrol.int.