Monday, November 26, 2012

Using the Safety Culture Discussion Cards to help understand textual data

What we call our data are really our own constructions of other people’s constructions of what they and their compatriots are up to’ (Geertz, 1973)

Probably the most common approach to trying to understand safety culture is via safety climate questionnaires, usually comprising a set of items with a Likert-scale to indicate the level of agreement with each item. Unfortunately, such questionnaires alone do little, if anything, to help understand the meanings that people ascribe to their values, beliefs and behaviour, and so do not explain why we do things, why we do things in the way that we do them, or the conflicts between what we say and what we do. To gain a deeper understanding, a qualitative, interpretive approach is more fruitful, not necessarily to supplant questionnaires, but at least to supplement them. Prior to interactive methods such as focus groups and interviews, one source of data from the questionnaire itself can be a useful starting point to an interpretive approach - the free-text comments written by the respondents.


I recently used the Safety Culture Discussion Cards to help analyse several hundred typed/written unstructured comments from a safety culture questionnaire - a fairly large amount of textual data. Many of the comments were several paragraphs long and referred to a variety of issues, and were mostly very interesting, well thought out and well-written. Making sense of rich textual data is never easy. But a common approach to understanding is via 'content analysis' (Krippendorff, 2004), or textual analysis. This often involves reading the text and applying a set of codes or categories to try to understand the data.

In this case, I decided to try to use the Safety Culture Discussion Cards to help code the data. The aim was to get a detailed understanding of the issues that questionnaire respondents were motivated to comment on - the specific issues, the way the writers related issues to each other, and the number of times that each issue was mentioned. An assumption was that issues mentioned more often by respondents reflect concerns that are important to them.

The cards cover most relevant aspects of safety culture but are (deliberately) not mutually exclusive, so this had to be kept in mind during the analysis. Prior to and during the coding, it was necessary to remove or combine cards as appropriate in order to achieve some satisfactory level of mutual exclusivity.



I started the analysis by reading all of the comments very carefully, and coding pieces of text within each comment using the eight elements of safety culture covered by the cards (Management Commitment; Resourcing; Just Culture, Reporting & Learning; Risk Awareness & Management; Teamwork; Communication; Responsibility; Involvement). Because a person's comment could cover all sorts of issues, it is not possible to apply just one element code to each comment. Even a particular sentence within a comment could cover two or more issues, such as 'Management Commitment' and 'Resourcing'. So at this stage, a sentence or paragraph could be coded using one or more elements.

The next stage was to re-read the comments and now apply more specific codes to the various pieces of text. The specific codes relate to the codes on the safety culture discussion cards, from 1a to 8e, noting also where the text was positive/favourable or negative/unfavourable in nature, or sometimes both. Since some of the cards overlap, where a piece of text could be coded using more than one card (and the cards could not reasonably be mutually exclusive) the codes were combined.

The final stage involved rechecking the use of the codes for each comment to ensure consistency and calculating the usage of each code. (An even more rigorous application of this method would involve having independent coders repeat the exercise with all or some of the text, as I and Amy Chung did when analysing comments relating to HF/Ergonomics practitioners' views on barriers to research application; see Chung and Shorrock, 2010.) This allowed the relative frequency of each issue to be determined, and gave an impression of the perceived pertinence of the various issues.

The frequency of each element as well as the top 20 issues were calculated. The quantitative data, combined with discussion of the actual content of the comments, added substantially to the data received from the Likert-scale standard questionnaire items.

A final interesting output from this exercise is the ability to the the cards to visualise the narratives in the comments by mapping the relationships between issues and the possible meanings emerging. This will be the subject of a different blog entry. The exercise also revealed a few issues that are not covered by the existing cards, as well as the issues that are covered by the cards but that were not mentioned at all by the commenters. This is useful feedback for the further development of the cards.

References

Geertz, C. (1973). The interpretation of cultures: Selected essays. Basic Books.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.
Chung, A.Z.Q. and Shorrock, S.T. (2011). The research-practice relationship in ergonomics and human factors - surveying and bridging the gap. Ergonomics, 54(5), 413-429.

Wednesday, November 7, 2012

Five questions about boredom, fatigue and vigilance

Below are five questions posed by a safety colleague, and the brief responses.

1. How different are boredom and fatigue?
Both affect our ability to pay attention - to notice something that may need attention - but they are different in terms of their causes and can occur completely independently. A person can be bored during a period of low activity, but not fatigued. Prolonged boredom, tends to result in fatigue, but so can high workload, lack of sleep or disruption to sleep patterns, or stress. Other than sleep or rest, there is little that you can do to manage fatigue effectively while on position, while more can be done to tackle boredom and stay in the loop. So preventing and managing fatigue is a key priority to ensure that people remain able to deal with unusual events.

2. Is low workload more dangerous then high workload?
Attention is stretched by both 'overload' and 'underload'. Both require hard work and can be stressful, particularly if there are safety consequences when something is missed. Which is more dangerous will depend on the situation and the person (for instance personality, experience and levels of stress and fatigue), but skilled professionals tend to cope better with higher workload up to the point of overload, when performance degrades more dramatically.

3. How to remain aware and vigilant for unusual situations?
Ask colleagues - people develop different visual and mental strategies that may not be obvious from the outside. But applied research using eye movement tracking gives some tips in terms of scanning. So-called "active scanning" can help to counteract degraded vigilance under low workload situations. With active scanning, people scan displays proactively in sequences or cycles depending on the traffic situation, linking specific information from different information sources. The scanning is more strategic, and helps to anticipate developing situations.

4. When are we most and least vigilant?
In a non-shiftwork environment we could highlight some times of day when we are least alert, especially during the very early morning hours, but shiftwork is a fact of life for many workers working a 24-hour operation. What we can say is that we are most vigilant when well rested, engaged and interested in the activity, not distracted (e.g. TV, radio, visitors) or preoccupied with other thoughts, well hydrated, and well supported by colleagues and supervisors.

5. Is the theoretical human performance knowledge adding value?
Yes, but not nearly as much as it should. So much is known about human performance that it seems that policy and practice are decades behind. But so much that is published is irrelevant to complex systems and activities, does not offer solutions, and technology and practices change fast and do not wait for research to catch up. Much theoretical knowledge in human factors comes from sterile experimental environments, normally focusing on one issue (e.g. vigilance) while 'controlling' (or ignoring) some of the most relevant real-life issues that interact to shape performance in the real world (e.g. motivation, risk, teamwork, supervision, background shift-fatigue). The hard part for practitioners is evaluating what aspects of the research are relevant, piecing them together and drawing out practical implications. With this in mind, the most directly useful human performance knowledge is gained by spending time with end users, listening to and observing them at work, and working with end users and other stakeholders to find solutions to human performance issues.

Wednesday, October 3, 2012

Should the Institute of Ergonomics and Human Factors be more of a campaigning organisation? Yes.

Published in 'The Ergonomist', Newsletter of the Institute of Ergonomics and Human Factors, October 2012, p. 4

In September's The Ergonomist, the President of the Institute of Ergonomics and Human Factors asked whether the IEHF should become more of a campaigning organisation. Assuming that we want to be a relevant organisation, then the answer must be 'yes'. While we have many interesting research findings and effective applications, we rarely seem to communicate our impact in the world.

It is a sad state of affairs that the 'impact' of publicly-funded research is judged primarily by the citation of journal articles by one's fellow researchers and oneself. It is equally sad that we have so few press releases, white papers, blogs or videos of our impactful theories, findings or applications. We seem to put most effort into forms of communication that are least visible to policy makers, decision makers and the public. Perhaps this is why we are still too anonymous to the wider world.

We cannot be content with only writing to each other via Old Media or speaking to each other in closed conferences if we want to make a visible difference. The research article or technical report should not be the end of the line for any of us. If we think that our discipline is important, then we need to to be confident and decisive in our messages and campaigns, and clever in how we convey them.

It is great to see that our engagement with social media is growing (e.g. LinkedIn, twitter) and that we have had recent public exhibitions. But we need more involvement. We need to be prolific not in how much we write, but in the effectiveness of our communication with decision makers, those affected by our work, and the world at large.

We need to put more effort into the usability of our communication with the world. Think Wordpress, Blogger, twitter, pinterest, Google+, flickr, picaso, Amazon, LinkedIn, Experience Project, e-petitions...as well as letters, magazines, and face-to-face, of course. None of us has 'time' for this, except the time that we prioritise for it. As Jon mentioned, raising awareness isn't just a job for the IEHF. It is for all of us to ensure that our research and practice remains relevant to the world and has broad impact. 

Steve Shorrock

Wednesday, July 18, 2012

Using the Safety Culture Discussion Cards: Tips from a user

I have received some great practical tips (and considerations for the future) from an ATC Safety colleague who had used the Safety Culture Discussion Cards with several different groups. Thanks to Alfonso Barba Martínez (Head of Regional Safety at AENA, the Spanish Air Navigation Service Provider) for the tips below!
Basically, I find it essential to make an introduction to the aims of the cards and the different ways to use them. What I find could be an improvement is to introduce simple cues for those having to administer them as `facilitators´, because what we have been doing until now is introducing the product to those who can make a better use of it (Instructors, Supervisors, mid-management). This made me think that it might be useful to explain a bit further what the outcome of the cards can result in. So basically I would cue those administering the cards on:

a) Make sure who your audience is, and prepare specific cards for them on each element.
b) The cards are an excellent tool to insert into any meeting as an added practical activity, breaking frequently tedious expositions and offering some `brain refreshment´.
c) Don´t use the cards with more 10-12 people? Otherwise the discussion might drift away very easily.
d) Is anyone supposed to take notes? As facilitator I advised them I would be taking notes, and the different comments and views expressed allowed me to identify weak points in which future safety strategies may be focused on at local level: shifts, fatigue management, airport signaling.
e) Also, I would favour using Option 2 Safety moments at the beginning, as it is a lot easier to engage people in a straight forward discussion on just one item, two maximum.
f) In the `How to use this Cards´ section, Options 1 Comparing views, Option 3 Focus on... and Option 2 SWOT analysis don´t necessarily need to use all cards, but it sounds like you must. It all depends on time available and going through all of them should not be the objective.
g) Be careful with the card selection when mixing groups with different activity areas, i.e. Human Resource, Maintenance, ATS, Financial and again, have relevant questions prepared beforehand.

I hope this can be useful to you. As I say, it is mainly focused on the tasks by those presenting the cards to others who must deal with different audiences, and could perhaps feature as a card of its own at the beginning.

In the near future, I'll be blogging on each of the options for using the cards, and it would be great to hear of any other options for how the cards have been or could be used. In the meantime, you can read more about the cards (and download them all as a PDF via the link at the end of the page) here, and see the individual cards as images here.

Tuesday, July 17, 2012

Reading on the job: fatigue, boredom and distraction while underloaded

I have received a few queries asking for a view (or "the science") on reading during operational duty, particularly as a possible countermeasure against fatigue during nightshifts. This is in an air traffic control context, meaning reading while a controller on duty at workstation, but it would apply to many other safety-related contexts where people need to monitor.

This is a sticky issue. But it is the kind of issue where a human factors specialist is expected to give advice or at least some succinct guidance or points for consideration. This kind of query is rarely straightforward. The 'science', inasmuch as it is relevant, cannot be simply generalised from laboratories or other contexts, and there is rarely any possibility for controlled experiments to study the problem directly. So, is reading a suitable countermeasure against fatigue? Below are some of my thoughts on this issue.

The task of the air traffic controller, as with many other safety-critical roles, is increasingly visual with high demand for monitoring. Automated tools such as data link and conflict alerts are becoming more common. They usually do not have any associated audible alert or alarm and may not be expected or anticipated. As people naturally become more reliant on such automated aids for normal performance, the need for alertness is even greater.

But there can be fairly long periods of low activity. Reading during operational duty is fairly common during these periods. The motivation to read is one that we can all identify with. We have little tolerance for monitoring when understimulated, and naturally want to fill in the gaps with other activities. Stimulation is often at its lowest during a nightshift (particularly ~0200-0500), when we also experience heightened fatigue. In this scenario, reading may seem like a good idea to maintain alertness. 

But what are we really trying to combat via reading during periods of low activity? Most likely, we use reading to combat boredom rather than fatigue. Boredom is an unpleasant or even distressing state that we naturally wish to avoid. Reading can be stimulating (and effective to counteract boredom) but it can also be visually fatiguing (especially during night shifts). 


Reading, fatigue, boredom and distraction (Photo credit: Ani-Bee http://flic.kr/p/LaxT3 CC BY-NC-ND 2.0)

There is then the issue of distraction. Reading may mean reading from paper (book, newspaper, etc), an e-reader, a smart-phone or a tablet. Reading itself is a primary distraction from operational information, but smartphones and tablets introduce secondary distractions from other applications, such as emails, instant messages, tweets, status updates, notifications, etc - many of which are designed (or can be set) to capture attention (e.g. via pop-up notifications or alerts). 

Reading feels like the most natural way to repel boredom and so maintain stimulation, but at the same time it can increase fatigue and distraction and decrease vigilance and alertness - a situational irony. 

Then there are the legal implications. In the case of a serious incident or accident, reading non-operational material when visual detection may be safety critical could be raised in court, because reading takes attention away from the primary task (e.g. radar monitoring). An argument could be made (ignoring the whole context of the work) that if the person were not reading, then a problem (e.g. conflict) would more likely have been detected. 

It is important to separate the problems of fatigue and boredom because the countermeasures are different. Fatigue requires rest and sleep. Boredom requires stimulation. But often they come as a package, in which case fatigue will usually trump boredom, as fatigue is so much more difficult to counteract without prior planning. A better way to manage fatigue during nights might involve more frequent breaks and scheduled napping, as well as a well-designed shift system (see a EUROCONTROL literature review and Hindsight  Issue 13 on fatigue).

Combating boredom on an operational level also requires frequent breaks - there is positive evidence that light exercise such as a brisk 10 minute walk helps increase arousal. Breaks are only a partial solution, though, because the problem is one of job and task design.While these are not easy to change, while working on position, other tactics might include more active visual scanning (as opposed to reactive scanning), and increasing workload (e.g. combining sectors, offering higher service levels), or rotating activities, tasks or roles (e.g. sectors, positions or non-operational work). A 1996 EUROCONTROL-sponsored research project examined monotony in ATC, and came up with a few recommendations (page 184), but these do not shed much more light on the matter. Note that some other 'common-sense' tactics, such as playing music, can also be counterproductive because of distraction. In the longer term, more strategies are available via personnel selection (some people have a greater tolerance for boredom) and task, job and technology design (for instance, auditory alerts to complement visual information during specified hours), but these options are of little relevance to current operations. 

So reading itself is probably not an effective countermeasure for fatigue while on duty, but we have to recognise that the desire to read indicates another problem that needs to be addressed: boredom. Until we are able to design the boredom out of jobs, it would be useful, then, to share the practical strategies used by different people in different organisations and industries to maintain alertness and vigilance, and fend off boredom, in real world safety-critical environments.

Monday, July 16, 2012

Breaks from operational duty and fatigue management

by Steve Shorrock

(Adapted from HindsightVolume 13, Summer 2011, pp. 68-69, EUROCONTROL)

Guidelines from Human Factors research for optimising human performance and reducing fatigue-related risks.

Breaks from operational duty are an important factor in the management of fatigue. But as highly committed and professional operational staff often perform several secondary tasks and activities – inside or outside the ops room – breaks can become a victim. Breaks seem to be used for a wide variety of activities. A controller once said to me that working through breaks was often necessary for him to complete his secondary role…in safety management. This tendency to use breaks to perform tasks that add to fatigue, rather than reduce it, can increase over time without being noticed. In the end it may seem like the normal way of working. It is too easy to forget that the important role of breaks in managing fatigue – visual, mental and physical – and of course in managing stress.

The issue of fatigue is a core issue in human factors. So what activities are acceptable from this point of view? Anyone with some exposure to the field of human factors will not be surprised to hear that there is no black and white answer. But some general guidelines can be offered from the research on fatigue and vigilance.

Let’s start by asking, what is the purpose of a break from operational duty from a human performance viewpoint? Operational tasks are often visually demanding, involving monitoring the outside view, the situation display, or other screen-based work. This work contributes to visual fatigue, even when you are not particularly physically fatigued. Operational work can also involve intense periods of typing, writing or other inputs. You may notice how this brings about feelings of tension in the hands, arms and shoulders. Similarly, the work can often involve static postures, and so more general physical fatigue or discomfort can result. In high workload periods, mental fatigue can build up. But low workload periods can seem even more mentally tiring, and it is during these times when vigilance can really suffer.

So breaks are needed to recover from a variety of demands and the needs from a break will vary. Depending on the operational activity, you may need to rest, or relax, or re-energise during the break. We could put break activities into three categories, and as there are no black and white answers, we’ll call these red, amber and green activities.

 Red activities 

There are some activities that we know (from research and common sense) should be avoided during breaks, or at least minimised. In particular, activities that increase visual or mental fatigue, or increase feelings of stress or pressure, can prevent recovery. Tasks that are likely to be stressful may cause worry or preoccupation and so can even affect your work before the break. In general, such activities should be done outside of break periods where possible. Activities that may fall into this category might include assessments, interviews, important reports, or even difficult conversations. But other types of work which are intense or particularly important are also best done outside of breaks. Managers and staff need to arrange additional time for such activities, where possible.

 Amber activities 

Many activities that are sometimes performed during breaks are often OK in moderation. These really depend on the operational activities before and after the break. In general though, activities should be reduced where they are similar in nature to the operational work, for instance visually demanding, involving a lot of manual inputs, or time-pressured. Let say you have had a very busy period involving high levels of concentration and lots of keyboard or touch screen work. It would make sense, during the break, to reduce or minimise time spent internet surfing or completing detailed computer-based administration. Similarly, if you have had a very busy session on duty, then the break is probably not the time to rush around doing other time-pressed activities. But if you have had a very quiet period, the same activities could help you re-energise. Where it is necessary to do activities that are similar, physically or mentally, to the operational tasks, it is advisable to take a reasonable period of ‘green time’ to recover before going back on duty.

 Green activities 

The best kinds of activities to perform during breaks are usually different to the activities undertaken while on duty. These ‘green activities’ will allow you to rest, relax or re-energise. They might include restful or relaxing personal time, social activities, or light exercise. They may be individual to you, but will certainly depend on the nature of the operational work. A period of ‘green time’ taken immediately before going back on duty, and preferably outside the ops room and away from a computer screen, will help optimise performance.

So when it comes to breaks from operational duty, changes in activity are the key to reducing fatigue-related risks.

Of course, how breaks are spent is only part of the picture. The timing and frequency of breaks are also important and, to a lesser extent, their duration. These broad guidelines would ideally be incorporated in an overall policy or risk management system for fatigue, including provisions for time to perform secondary tasks that may present a fatigue risk. Until then, perhaps managers and operational staff can put the guidelines into practice to help optimise human performance, and maintain safety.

Wednesday, July 4, 2012

Safety Culture in your Hands

(This article featured in The Controller - Journal of Air Traffic Control, April 2012.)

by Steve Shorrock, Human Factors and Safety Specialist, EUROCONTROL

Are safety issues raised by front-line operational/technical staff given appropriate priority within your organisation? Do you and your team incorporate lessons from incidents into your work? If you saw a colleague doing something you thought was risky or unsafe, what would you do?

These are questions about safety culture. They are just examples of the types of questions that are addressed in the EUROCONTROL safety culture programme for European air traffic management (ATM), which has been underway since 2003.

One thing that is clear from the programme, which involves questionnaires and workshops with operational and non-operational staff, is that controllers have very clear opinions about these sorts of questions. But a survey is fairly pointless without action and continued conversation about the issues that matter to those on the front line of safety.

To help the conversation, the EUROCONTROL Safety Culture Discussion Cards are a practical resource to aid real discussion about safety culture by any person or team within the air navigation service provider (ANSP) organisation, especially operational staff. This article gives you a brief overview of the cards. So what is the point of the cards?

They get people talking. The cards are designed to provoke discussion among both operational and non-operational staff, and have been used successfully with operational staff in safety culture workshops, as well as an aid to team resource management training. They do not give answers, but instead raise questions.

The cards build on what you know already. Operational staff already know about safety culture. They live it and feel it. So the cards build on this understanding. The cards do not use theoretical language. They have been designed and tested with controllers. But still, they are based on the EUROCONTROL approach, which has been used on around 20 ANSP surveys.

The cards help improve safety culture by encouraging discussions on ways to improve safety culture, inspiring action based on the outcome of the discussion.

The physical cards are printed in colour on A6 card (but may also be used digitally, e.g. on smartphones). The first few cards in the pack explain very briefly what safety culture is, show the organisation of the cards, and explain some possibilities for using the cards. Then, the discussion cards are sorted into eight elements:
  • Management commitment
  • Resourcing
  • Just culture, reporting & learning
  • Risk awareness and management
  • Teamwork
  • Communication
  • Responsibility
  • Involvement
Photo credit: Pamela Ocampo http://flic.kr/p/7xCRdu CC BY-NC-ND 2.0

There are several discussion cards for each element, and each discussion card is designed to catch attention with a photo and headline, raise questions and provoke discussion. There are 83 cards in total – 70 of these are the actual discussion cards, while the rest are explanatory (introduction, photo credits, disclaimer, etc).

So how are the cards used? The cards can be used in any way you can think of, but several ways have been tried. For instance to com-pare views: different individuals or teams sort cards into two piles: ‘What we do well' & ‘What we need to improve', then discuss the piles. Another way is to have 'safety moments: in a small group, take just one card and discuss the card for 10 – 15 minutes. Or have a small group choose a specific element, such as ‘Just culture, reporting and learning', and discuss each card in depth, eg. What and where is our ‘best practice' on this issue? Where do we need to improve? Etc. You could also organise the cards into pat-terns to show how the issues relate to one another in your organisation, unit or team.

You can use any number of cards, from one to the whole set – whatever works for you. Safety culture can seem abstract, fuzzy and hard to break down. The cards provide a way to discuss safety culture in a straightforward and practical way.


The pdf version may be viewed on smart phones and a low resolution version can be found here. For more information and to view the cards as jpeg files, see the SKYbrary article here. High resolution (English and French) print-ready versions are also available: contact steven.shorrock {at} eurocontrol.int.

Wednesday, June 27, 2012

Is Ergonomics a Scientific Discipline? No.

This is taken from the article "Head to head: Is Ergonomics a Scientific Discipline?", published in The Ergonomist, Newsletter of the Institute of Ergonomics and Human Factors, July 2012. It is a response to an article by the IEHF President Jon Berman in The Ergonomist, June 2012. The 'Yes' argument was made by Mark Young MIEHF.

by Steve Shorrock MIEHF

Jon Berman’s article in the June edition of The Ergonomist concerned ‘the research base’ and whether it is important for ergonomics to be a scientific discipline. This raises other questions about the nature of science and science communication, the reality of ergonomics practice, and what else characterises our work.

The problems that many practitioners face on a day-to-day basis are usually not problems of science – they are problems of craft and engineering. Many are socio-political in nature. For practitioners working with complex sociotechnical systems, they might include how to gain management commitment to change, how to gain and utilise end-user involvement and input in projects, how to align HF activities with a system engineering or regulatory context, how to balance a change (e.g. a new shift pattern) with its social effects, or how to integrate new interfaces with multiple legacy controls and displays. Some of these issues rest on some scientific knowledge. But for most practitioner needs, the requisite ‘scientific knowledge’ is already there – and has been for years.

The continual enlargement of the scientific research base serves the needs of researchers far more than the needs of practitioners. If I were to take our own journal Ergonomics, then only 10% of articles (maximum) are, or have ever been, relevant to (part of) my practice in complex systems. The same goes for most other journals in our field. For many practitioners the scientific research has clear relevance to initial training, but subsequently diminishing relevance to practice. Even if the topic is relevant, the practitioner needs to access to the research, then consider the representativeness of the tasks, environment and people, think about multifactorial effects, account for publication bias and duplication of reporting, pick out the key implications, and so on. Scientific research in ergonomics, as published, is not ergonomic: it is not designed for those who can best influence the design of real systems – practitioners and policy makers.

Science informs ergonomics, but does not define it – at least not in isolation. Ergonomics is better described as a design discipline that draws on scientific and experiential knowledge. In practice, factors that seem to relate more closely to the effectiveness of ergonomics include empathy, interpersonal and helping skills, design and systems thinking, and understanding of people, technology, organisations and industry. Relevant and useful science clearly has an important role to play in the research base of ergonomics, but craft and engineering play at least equal roles in defining ergonomics in practice.

Saturday, March 10, 2012

Human factors research and practice - Part 2: bridging the gap


This article by Steve Shorrock and Amy Chung was published in The Ergonomist (newsletter of the Institute of Ergonomics and Human Factors), March 2010. It is reproduced here to give more open access and to encourage further conversation on this issue.  Part 1 is here. The full article in the journal Ergonomics is at http://www.tandfonline.com/doi/abs/10.1080/00140139.2011.568636 (ask me for a preprint).

In the last article we outlined some of the quantitative findings from our survey of 587 practitioners regarding the application of ergonomics and human factors (E/HF) research. We concluded that the research practice gaps could be reduced by addressing a number of factors associated with the research, the organisation, and individual practitioners and researchers. In this Part 2, we report on the respondents’ top suggestions for how practitioners and researchers could improve research application. Respondents made 733 suggestions for practitioners and 781 suggestions for researchers. Below, we identify the top 5 suggestions for practitioners and researchers.

How practitioners can improve research application
  1. Increase collaboration and communication between E/HF researchers and/or practitioners and increase contact, conferences, networking and association. Many of the suggestions concerned the relevance of the research focus to the ‘real world’. Typical comments included: “need to communicate real-world problems to researchers so that solutions can be researched”, “communicate the voids in research (as applied to the real world), perhaps in Letters to the Editors”. Other suggestions concerned a more collaborative working relationship, with suggestions such as “practitioners in research laboratories” and “researchers on the project team to provide strategic oversight”. One suggested “links with a research group that share your basic approach and philosophies”, while another suggested that practitioners “consult researchers using helpdesks”. Finally in this category were suggestions for more crossover workshops, dissemination clubs, and forums for discussion of current research.
  2. Read more and be more aware of the research, including more time to read research at work. Many respondents suggested that time to search literature should be built into projects. But several commercial conflicts were noted: “it’s hard to find the time if you work in consultancy; time is money and so if you commit to searching out and then reading papers they had better be good and useful (and a lot are not)!” Some respondents suggested seeking articles for a specific task/ job, but others hinted that this did not meet their needs (e.g. “figure out an easier way to monitor and filter research as it is published instead of searching for it when specifically required”). Some respondents suggested linking reading to personal development planning. 
  3. Promote or obtain better awareness, support, cooperation, permission and authority for research application from decision makers, stakeholders, colleagues, organisation, Societies, other disciplines/ professions and public. One respondent suggested that “practitioners need to educate their customers and ‘translate’ research into everyday English for their customers to aid buy-in of its application”. Another noted that “more industry applications should be based on specific literature; clients should start looking for more than just a guru that says he knows what to do and should insist on evidence”. Other suggestions concerned allied professions (“feed HF into the tools and techniques other technical specialists use and own; translate our discipline into their language!”), and users (“make sure you have a strong link with operational people”). 
  4. Provide more appropriate application of research findings to real problems. Several respondents argued for an evidence based approach (“seek evidence for their interventions – don’t just follow fads”). Another set of suggestions promoted more publishing of application (“provide real world application results in published work”). Other respondents suggested more collaborative application, such as “a mentor coaching relationship with researchers to help find solutions and test ideas”, or “researchers involved in the application environment.” However, several comments indicated that the horse should come before the cart. One respondent warned that practitioners should “address real world problems rather than invented abstractions.” Another stated they should “identify the problem, then identify the solution. Don’t go looking for places to make the research fit.” 
  5. Carry out more research and publication, including replicating or extending research, addressing conflicting results and methodologies and specific themes or topics of research. Several of these emphasised relevance (“write more about ‘real/applied’ problems in practitioner/professional literature, so that researchers can keep up to date with current applied challenges.”) Many of the respondents suggested that practitioners publish case studies showing successful application. However, several highlighted the lack of outlets to do this (“this is a huge problem as practitioners don’t have a venue to do this”).
How researchers can improve research application
  1. Ensure the research focus and methodology is more applied, real-world, relevant, germane, practical and generalisable to organisational environment. Many comments concerned the research focus: “focus research efforts on immediate HF problems relevant to industry”; “understand the needs of practitioners in real industrial environments and aim research at solving real problems”. Respondents frequently mentioned the external validity of methodology, especially the need for more field studies and more representative participants (“more in-situ studies; there is a difference in how people behave at work and in a laboratory”; “stop using simulators as the only means of data gathering”). 
  2. Increase collaboration and communication between E/HF researchers and/or practitioners; increased contact, conferences, networking and association. Many of these had a dual focus on realism (“listen to industry about what research is required instead of deciding what is interesting!”; “by asking practitioners where they think the gaps are in research to back up their practice”). Others focussed on the need to collaborate with regard to outputs (“engage with industry working groups to tailor publications that educate without patronising, i.e. understand that the applied world is not the research world”; “researchers should work together with practitioners on ergonomics projects and stop sitting in their offices publishing loads of theoretical papers”). 
  3. Form more definitive conclusions, and clearer implications, recommendations or applications when reporting. These suggestions were more focussed on the need to provide implications (“describe the outcomes in terms of tangible implications for practitioners”; “provide pragmatic recommendations, guidelines or applications when summarising their research findings”). One focused on the need for “better abstracts” and “clear articulation of ‘so what’ in human language”. Other respondents lamented “silly one-paragraph impact statements” and conclusions indicating that “further research is necessary”. 
  4. Report text and statistical analyses in a different format (e.g. web, magazines, practitioner journals, society publications, summaries, database, email alerts) or in native language. These respondents suggested such alternative non-journal media should be used for “snippets” or “practitioner-friendly versions”. Coupled with the barrier that many practitioners don’t have time to read academic articles, such alterative formats may help uptake of research application if presented “in a form that industry can use”. 
  5. Report text and statistical analyses in a more understandable, clear and readable manner in journal articles. Respondents emphasised the need for writing “in plain English that can be understood outside of their industry” and “dropping the academic jargon”. One responded starkly that academics should “get out of their ivory towers and speak a language that HF people and non-HF people can understand instead of psychobabble”. Another advised researchers to practise what they preach: “Write to your audience, apply the principles of HF to the writing of the article.”

Research and practice are sometimes painted as cosier bedfellows than appears to be the reality in E/HF. This work has highlighted the gaps and bridges between research and practice, but it is up to us to build and cross the bridges. We hope that the data derived from this study may spur motivation for more applicable research and more research application.
This article reported just the top suggestions for improved research application. What more can researchers do to improve application? What can practitioners do to turn theory or research findings into real world application? 

Human factors research and practice - Part 1: surveying the gap

This article by Steve Shorrock and Amy Chung was published in The Ergonomist (newsletter of the Institute of Ergonomics and Human Factors), February 2010. It is reproduced here to give more open access and to encourage further conversation on this issue. The full article in the journal Ergonomics is at http://www.tandfonline.com/doi/abs/10.1080/00140139.2011.568636 (ask me for a preprint).

As a “scientific discipline concerned with the understanding of interactions among humans and other elements of a system”, and a profession that “applies theory, principles, data and methods to design” (according to the International Ergonomics Association), ergonomics and human factors (E/HF) practice should arguably be built upon a solid research base. But do practitioners actually apply research? Do they find it useful and usable? Despite a historical schism between researchers and practitioners, such questions have, perhaps unsurprisingly, been almost completely overlooked in the literature. Yet they are questions that have profound implications for the discipline and profession.

Photo credit: futureshape http://flic.kr/p/S2VLM CC BY 2.0
A widening gap between research and practice has been observed in many disciplines, and especially concerns research published in academic journals. Researchers have found that, while academics place a high value on research, practitioners often do not. Practitioners are often largely unaware of relevant research, or do not implement research findings, or lament that the field is not practical or relevant enough. Understanding whether such a gap exists in E/HF is relevant to all researchers and practitioners. So we conducted a study to examine E/HF practitioners’ application of research findings published in scientific journals, in terms of levels of readership, usefulness, and barriers and facilitators to application.

A questionnaire was developed comprising questions about the practitioner, their access and attitude to published research, and possible barriers and facilitators to using research findings in practice. The questionnaire was targeted at practitioners who apply an understanding of E/HF to the design of interactions in real settings. We wrote to IEA federated E/HF societies to request that they ask for potential respondents. We also advertised the survey on E/HF LinkedIn groups, and emailed LinkedIn members and other professional contacts.

A total of 587 usable questionnaires were submitted from respondents working in 46countries. The top five countries in which respondents worked were US, UK, Australia, Netherlands and Canada, which also approximately reflected national ergonomics society membership. The respondents represented a wide range of organisational types, application areas, qualifications and industries. Respondents’ averaged estimated work time over the most recent 12 months as an active E/HF professional was split fairly evenly between research, application and education.

Respondents read an average of one scientific journal article per month concerning E/HF or related fields over their most recent12 months as an active E/HF professional. The three journals most frequently named as most useful to respondents as practitioners were Applied Ergonomics, Human Factors and Ergonomics. These were also the three most highly rated of the 13 IEA-endorsed journals (see www.tinyurl.com/IEAjournals), and only these journals, plus International Journal of Human-Computer Interaction and International Journal of Industrial Ergonomics were marked as having been read by 50% or more respondents.

Respondents rated E/HF or related scientific journals overall as moderately useful to them as an E/HF practitioner. Statistical analyses showed that reading frequency and overall journal usefulness were significantly higher for:
  • respondents who worked in academic/research institutions, against other organisations; 
  • society members, against non-members; 
  • respondents who spent less than half their time applying E/HF, against those that spent more than half; 
  • respondents who spent more than half their time engaged in research, against those that spent less than half; and 
  • respondents who spent more than half their time engaged in research, against those who spent more than half their time on application. There were no meaningful correlations between years of professional experience in E/HF and reading frequency, or between experience and overall journal usefulness.
The top five barriers to research application for respondents working in academic/research institutions were:
  • “Decision makers won’t allow application of research” 
  • “The research has methodological inadequacies” 
  • “Key stakeholders won’t cooperate with application of research findings” 
  • “I don’t feel I have enough authority to change current organisational processes” 
  • “There’s insufficient time on the job to apply new ideas” 
The top five barriers to research application for respondents working in other organisations were:
  • “The relevant literature isn’t compiled in one place”
  • “Implications for practice aren’t made clear in the article”
  • “I don’t have time to read research”
  • “The research isn’t relevant to my practice”
  • “Journal articles aren’t readily available/accessible”
Four of the five top barriers for those in academic/research organisations were related to characteristics of the organisation, with only one related to the research. However, for non-academics the situation is reversed; four of the top five barriers were related to research, with only one related to the organisation. 

The findings of the survey provide valuable insights into E/HF practitioners’ reported access to, and attitudes toward, research published in journals, and factors that hinder research application. The dominance of barriers related to the research itself for practitioners may parallel the drift from pragmatic to pedantic science in industrial, work and organisational psychology. The secondary organisational barriers may reflect the pressures in organisations to produce rapid results, contributing to a shift from pragmatic to popularist science in practice. 

This research highlighted a number of gaps between research and practice associated with organisational affiliation, research/application involvement, experience, and ergonomics society membership. Those practitioners with the most power to effect change in organisations may, ironically, be benefiting least from E/HF research.

Further research is underway to examine the issues raised in more depth. However, the research-practice gaps could be reduced by addressing a number of factors associated with the research, the organisation, and individual practitioners and researchers. In Part 2, we report on the respondents’ suggestions for ‘bridging the gap’. We hope that this work may encourage a conversation within our discipline about the research-practice relationship.

So, what are the barriers to research application for you? How do you find and access useful and applicable research among the hundreds of HF articles published each month? What value does published human factors/ergonomics research have for humanistic practitioners? What others genres of research might be beneficial to a humanistic approach of human factors?

Wednesday, January 4, 2012

Empathy: A core condition for humanistic design

Many involved in human-centred design (ergonomics, human factors, UX, interaction design, etc) have an internal or external consulting role. Much of the time is spent thinking about task/interaction design, user interfaces, work environment, and many technical and project management issues. In getting caught up with the detail, we can forget something fundamental about consulting. That is, consulting is primarily a relationship business (see Peter Block). If you are a practitioner, client, co-worker or end-user or other project stakeholder, think about the design (or helping) projects that were most successful and enjoyable. Chances are, the quality of the relationships between those involved in the project was a critical factor, and something that you remember fondly. The positive relationships are more often causal to this success than consequential. 

If this is so, what facilitates these productive relationships? Research in the helping professions suggests that a key ingredient in effective relationships is empathy. In the context of therapeutic personality change, Carl Rogers argued that empathy within a relationship one of the ‘core conditions’ which create a climate conducive for growth and development. Daniel Goleman cited empathy is one of the five components of emotional intelligence at work, and empathic listening is one of the ‘seven habits of highly effective people’ according to Stephen Covey

Despite being such a fundamental human capacity, empathy is not often considered in scientific- or engineering-oriented disciplines, but human-centred design consulting involves consulting with people, about people, for people. In effective consulting, perhaps without realising, consultants (and consultees) use empathy to understand the perspectives of others, from their viewpoint. Empathy is a rich and complex concept, it may be viewed as a trait or state of a person, a process and a skill. Empathy is colloquially seen as 'walking in another’s shoes'. In this sense, empathy can be thought of as ‘perspective taking’; the ability to perceive accurately the frame of reference of another person while maintaining a sense of separateness - the 'as if' quality.

In 'humanistic design', we may empathise in several ways, for instance:
  • emotionally, e.g. How might a plant worker be emotionally affected by a change to a new shift pattern? How might automation of the more engaging aspects of flying affect a pilot’s job satisfaction?
  • cognitively, e.g. How might an air traffic controller’s decision making be affected by a new decision support tool? What might be the side effects of a new and more restrictive procedure introduced without worker involvement?
  • physically, e.g. What physical challenges are faced by schoolchildren who must carry very large bags? How are less able travellers ‘disabled’ by the design of buses and trains? 
  • socially, e.g. What might be the effect of family life of a new product for children? How might new working hours or travel requirements affect workers' social and community lives?
  • ethically and morally, e.g. What is the effect of design decisions to improve the user experience on assembly line workers in the manufacturing plant? What ethical or value tradeoffs might be involved in the decision to purchase a product or service that may have contributed to harm to humans, animals, or the environment?
These questions can be addressed formally though scientific inquiry and informally via empathy. There need be no conflict between the two. Insight gained via empathic listening and observation can complement more formally acquired knowledge to better understand the fit between people and activities, artefacts and environments, and build the relationships that are necessary to improve this fit. An empathic approach can also detect the need for more formal scientific inquiry.

While human-centred designers may refine their knowledge about people via formal education, the foundation for this knowledge is our day-to-day experience of the encounters that we have throughout life. With this understanding, and the knowledge developed in formal education, we further our understanding via our professional relationships with the people that we help

Empathy helps to establish rapport, which helps develop an understanding of people’s behaviour as well as their values, attitudes, feelings, knowledge and skills, and the meanings that they ascribe to these. Fulton Suri notes that empathy helps us to learn ‘why’ as well as ‘how and what’ people do, and helps us to take a more holistic approach to the person. In some cases, empathy can help us to understand the person more accurately than the detailed models, and in a manner that is more timely than studies that may take many months or years. So practitioners may use empathy as a qualitative research tool as well as a consulting skill.

The human-centred design disciplines are naturally focused on the 'user' (or rather, the person). But another group is often responsible for making decisions about users: clients (or sponsors, customers, project managers). Empathy with clients is vitally important to achieve change as a result of the intervention – to ensure that a recommendation becomes implemented and delivers the outcome desired. Empathy can help to create a working alliance with clients, and leverage commitment. How often do we consider what it is like to be a client? What are their values? What are their goals and expectations? What pressures and constraints impact their decision making? The same goes for other stakeholders, including those affected by the things - products, processes, procedures, etc -  that we design (beyond the 'user'). Perhaps we can also help decision makers to empathise with those who will be affected by their decisions, thus generating a climate conducive to the acceptance of human-centred advice.

Empathy is a fundamental human capacity, but effective empathy is not as intuitive as we may think. Bohart et al suggests that we can distinguish between three different modes of empathy. Adapted to a consulting perspective, these are:
  • Empathic rapport: the consultant uses empathy to build rapport and support the consultee.
  • Process empathy: a moment-by moment empathy for the consultee’s experience, cognitively, emotionally, and physically.
  • Person empathy: this is known as ‘experiencing near understanding of the consultee’s world’ or ‘background empathy’.
But beware: losing the sense of separateness and impartiality may lead to ‘sympathy’ with the consultee. The consultant may become lost in the consultee’s world, even taking responsibility for the consultee’s decisions. In fact, sympathy can block the capacity for empathy and so can be counterproductive. For instance, while we generally strive to understand ‘end users’, if this leads to strong state of sympathy, then empathy with clients and colleagues (and even the end users themselves) may become very difficult. This will get in the way of acceptance of insights by decision makers and other stakeholders. I have seen this many times (and no doubt fallen for it myself too), not only in design but also in other aspects of human factors practice, such as incident investigation.

So, it can be concluded that empathy is a core condition for humanistic design (necessary but in itself not sufficient). What role does empathy have in training, practice and personal development, and how can empathy be fostered and used effectively? And what are the other core conditions? A little time diverted from our more formal approaches toward this fundamental issue will help us to remain centred on the human, and improve further the effectiveness of design and consulting.

Adapted from Shorrock, S. and Murphy, D. (2008). Empathy: a humanistic take on human factors. The Ergonomist, October 2008, 10-11.

Monday, January 2, 2012

Human Factors & Humanistic Psychology: Distant cousins

Human factors and humanistic psychology are two human disciplines that emerged from adversity during the same era (1940s and 1950s). Both focus on the human, and should have a lot in common. In some fundamental ways they do, but the commonalities are in principle rather than practice. This post begins to explore some possible links. 

Human factors (or ergonomics – the terms are ‘officially’ equivalent) is a discipline concerned with humans as part of sociotechnical systems. According to International Ergonomics Association (to which most other Societies, Associations and Institutes belong), human factors (or ergonomics) “is the scientific discipline concerned with the understanding of the interactions among humans and other elements of a system, and the profession that applies theoretical principles, data and methods to design in order to optimize human well being and overall system performance.” 

The definition does not say anything about the human per se, and seems to formalise how practitioners actually relate to the many and various stakeholders in the process of system (and I mean socio-technical system) design. For me, the definition makes human factors practice out to be rather more Newtonian than is (or should be) the case. One of the essential “methods” that I use is otherwise known as ‘talking to people’, relating to them, listening to them, trying to empathise with them and their unique situation and perspective. This is a very human activity.

This focus on people, and especially the person, can become blurred in an age of advanced technology, large organisations and complex systems and procedures. The focus is not made any clearer by the scientific paradigm in which human factors professionals often operate; it is officially seen as a “scientific discipline” by the IEA but is more properly described as a blend of elements of science (to explain and predict), engineering (to design for improved performance), and craft (to implement and evaluate) (Wilson, 2000).

The craft-side of human factors is often most relevant to how I work as a practitioner, and it is the craft-side of human factors that I perceive to be most instrumental in the outcome of the practice of human factors. It is probably the most familiar role to the very large, perhaps majority, group of ergonomists who provide internal or external consulting services, and yet little is written about it. The science and engineering sides of human factors are essential, but the craft side humanises what we do, and makes the difference between whether decision makers and other stakeholders take notice (in the right way).

Humanistic psychology is a more craft-oriented discipline that could help to rehumanise human factors. While typically associated with counsellors and psychotherapists, humanistic practitioners (or practitioners of anything, working humanistically) may work in fields such as medicine, education, management and social work, and humanistic principles may be applied to organisational functioning. It is harder to pin down in terms of a definition, but the Association for Humanistic Psychology in Britain usefully summarise five basic postulates, which focus on a view of human beings rather than the discipline and profession (very human-centred!). 
  1. Human beings, as human, supersede the sum of their parts. They cannot be reduced to components. 
  2. Human beings have their existence in a uniquely human context, as well as in a cosmic ecology. 
  3. Human beings are aware and aware of being aware – i.e., they are conscious. Human consciousness always includes an awareness of oneself in the context of other people. 
  4. Human beings have some choice and, with that, responsibility. 
  5. Human beings are intentional, aim at goals, are aware that they cause future events, and seek meaning, value and creativity.

What I see from the definition and postulates above is that both human factors and humanistic psychology are concerned with people, relationships and contexts. In principle, both also emphasise wholeness (e.g. the physical, cognitive, emotional, social, and in the case of humanistic psychology, the transpersonal), awareness (e.g. situation, of oneself, of others), choice and responsibility (e.g. safety-related behaviour), intentionality/goal-orientation, meaning (e.g. in interactions and experiences), values (e.g. safety and productivity), and creativity. 

In practice, several issues combine to fragment the human factors approach to the human (and the system), such as: the scientific paradigm of human factors research; the reward structures in academia; the funding mechanisms for research; the industry sector or application domain of practitioners; organisational structures, policies and processes; interdisciplinary divisions; regulatory requirements; and the tendency for specialism among human factors professionals. 

This drift toward fractionalism is manifested in several ways. The person is reduced to a ‘user’ or an ‘operator’. Cognitivism often clouds out other attributes of the person. Human performance variability is transformed to numbers via human reliability assessment. We may ‘analyse tasks’ without understanding meaning and values. Qualitative or action research is harder to publish. Sometimes a reductionist approach may be the only way within the constraints that exist, but often we don’t look for alternatives. Humanistic psychology, which emerged partly in response to the reductionist (behaviourist and psychoanalytic) ‘forces’ in psychology, may be able to reorient us when we become lost in the detail. 

Both human factors and humanistic psychology are about improving the human condition, but they go about this in very different ways. Perhaps we can become more humanistic in our approach to human factors in practice. In future posts, I will look at some aspects of how humanistic practice can be of value to human factors, along with other views on the state of human factors in research and practice.