Performance behaviour in elite sports
Blijlevens, Suzan
DOI:
10.33612/diss.109492160
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2019
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Blijlevens, S. (2019). Performance behaviour in elite sports. Rijksuniversiteit Groningen. https://doi.org/10.33612/diss.109492160
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Chapter 6
The assessment of performance behaviour in
high-potential and elite athletes: instrument
development, content validity and practical
value of a coaching tool
Suzan J.E. Blijlevens, Paul Wylleman, Kayan Bool, Chris
Visscher & Marije T. Elferink-Gemser
Abstract
There is a lack of agreed standards about how psychological competencies can be evaluated within the context of elite sports. The aim of the present study is to develop a practical and valid coaching tool, which enables coaches to assess and develop high-potential and elite athletes’ psychological competencies of
performance behaviour within the context of elite sports. The study is set up in three major parts: (1) the development of the Performance Behaviour Coaching Tool (PBCT), (2) testing the content validity of the tool and (3) establishing the practical value of the tool. The PBCT was developed and tested with the consultation of experts in different fields (e.g. competence-based learning and psychology) and 29 sport coaches with extensive experience at the highest level. The content of the PBCT has proven to be valid and the procedures employed within this study were rated of great practical value by the coaches. The PBCT provides coaches numerous opportunities to get detailed insight in their athletes’ competencies of performance behaviour in a more systematic way, which simultaneously contributes to athletes’ development.
Keywords: coaching tool; performance behaviour; psychological competencies;
Introduction
Coaching generally refers to a leader’s attempts to improve performance by facilitating the acquisition of new competencies and a way of interacting with people (Arthur, Wagstaff & Hardy, 2017). The role of an elite sports coachis to guide high-potential athletes in the acquisition, and elite athletes in the maintenance of excellence and allowing them to repeatedly achieve maximum performance at the highest levels (NOC*NSF, 2019a). Coaching of high potential and elite athletes implies facilitating the acquisition and training of both physical, technical and tactical, as well as psychological competencies (Elferink-Gemser, Te Wierike & Visscher, 2018). Within the Netherlands, the term
performance behaviour describes the psychological competencies that lead to optimal development and maximum performance, enabling high-potential and elite athletes to effectively deal with the challenges of the athletic career
(NOC*NSF, 2019b). Through the duration and intensity of day-to-day interactions with athletes, coaches play a central role in the development athletes’
performance behaviour (Bool, 2018; Wylleman, 2019; Arthur, Callow, Roberts & Glendinning, 2019). Insight in why an athlete does what he does, and how this is reflected in an athlete’s behaviour during daily activities related to their athletic career (both inside and outside the elite sports context) can assist coaches in improving performance behaviour.
Psychological competencies are distinctive elements necessary for expertise that correlate with performance and that can explain individual differences in consistently observable behaviours (APA, 2015; Dohme, Backhouse, Piggott & Morgan, 2017). Common to different definitions is that competencies are (1) based on action in order to respond successfully to a demand or purpose, (2) linked to a specific context, (3) can be learned and evaluated and (4) mobilize and integrate different elements: knowledge, skills and attitudes (Bezanilla et al., 2014; Villa, 2008). A distinction is made between the development of generic competencies, which are common to different sports, and specific competences, belonging specifically to each sport (Bezanilla et al., 2014). A combination of generic and specific competencies allows an athlete to gain relevant knowledge and skills for their athletic career, apply them in complex contexts, and
incorporate them into their own attitudes in their own personal and professional way of acting (Villa, 2008). In the context of elite sports, an athlete needs to be able to apply competencies under high pressure, because several factors are increasing the importance of performing well (Baumeister, 1984; Moore, Vine, Wilson & Freeman, 2015; Otten, 2009). To provide athletes effective guidance and support in the development of the competencies of performance behaviour, coaches would benefit from detailed and qualitative insight in the development and current level of the competencies of performance behaviour.
The assessment of psychological competencies involves establishing an opinion on adequate professional action in a variety of professional contexts (Van Berkel, 2012). Characteristic for the assessment of competencies is the assessment of learning and simultaneously assessment for learning (Stiggins, 2002). Evaluation of competencies should therefore be carefully designed so that it becomes a precise tool that contributes to the attainment of deep learning (Benzanilla et al., 2014). The assessment of psychological competencies is a complex task because it entails the integrated application of knowledge, skills and attitudes in athletes’ behaviour. It should be done through complex and authentic tasks, which are embedded in realistic settings (Ledoux et al., 2013; Villa, 2008). Assessing athletes’ competencies in realistic contexts in which they are being used, will moreover benefit the accuracy of the observations (Ledoux et al., 2013). Serious games, which model real-life situations and are oriented to education, skills training, or the understanding of certain complex processes, are also used to assess competencies. Considering that the elite sports context in itself is complex and provides ample game-like situations (e.g. Bengoechea, Wilson & Dunn, 2017; Fleddermann & Zentgraf, 2018; NOC*NSF, 2019a; McPherson, 1994; Seifert, Araújo, Komar & Davids, 2017), it does provide
opportunities to assess athletes’ psychological competencies through authentic tasks part of their daily practices.
Although psychological competencies can be evaluated against agreed upon standards (Kaslow, 2004), there is a lack of agreed standards about how they can be evaluated within the context of elite sports. In sport, self-reporting
questionnaires are frequently used to get insight in athletes’ behaviours by identifying underlying psychological processes (Meredith, Dicks, Noel & Wagstaff, 2018). In doing so, emphasis is being put on the underlying cognitive processes and to a lesser extent on the athlete’s actual behaviour. They further rely on the athlete’s judgement about his own behaviour, not taking into account the coach’s perspective on the athlete’s behaviour, and therefore may be biased or inaccurate. As an alternative, assessing athlete competencies through observation may be more accurate than self-report methodologies (Holder, Winter & Orr, 2018). Observation can provide objective information with strong internal validity, meaning “what you see is what you get” (McKenzie & Van der Mars, 2015). However, because systematic observation tools make snapshots of a person’s observable behaviour, they often fail to provide information about the underlying cognitive processes and strategies in the establishment of an
athlete’s daily behaviour (e.g. Brewer & Jones, 2002; Cushion, Harvey, Muir & Nelson, 2012; McKenzie, Cohen, Sehgal, Williamson & Golinelli, 2006).
Furthermore, focussing on behaviour as only observable activities, one might reject the private or mental acts as actual cases of behaviour (e.g. trying to solve a problem in his head) (Bergner, 2016). This kind of activities are often left out observation instruments, but actually do contribute to insight in an athlete’s performance behaviour.
Coaches of high-potential and elite athletes work day-in day-out with their athletes and see them in multiple contexts (e.g. training, competition, social life). They therefore have more detailed information about their athletes’ behaviour rather than just what they see in the ‘here-and-now’. A coaching tool for the assessment of athletes’ psychological competencies should therefore allow coaches to integrate observable performance behaviour with their accumulated knowledge about athletes’ mental acts within different situations. From a pragmatic perspective, a coaching tool should ‘work’ in the real world in order to provide solutions to applied research questions (Creswell, 2014). Such a coaching tool should therefore be workable and of practical value to coaches working with high-potential and elite athletes. It moreover needs to be comprehensive (i.e. applicable for the full range of
To support coaches in their daily practice, we aimed to develop a practical and valid coaching tool, which enables coaches to assess and develop high-potential and elite athletes’ psychological competencies of performance behaviour within the context of elite sports. Therefore, the first aim of the present study is to develop the coaching tool for the assessment of athletes’ psychological competencies of performance behaviour. The second aim is to test the content validity and establish the practical value of the tool for coaches of high-potential and elite athletes.
Study set-up
The study is set up in three major parts: (1) the development of the Performance Behaviour Coaching Tool (PBCT), (2) testing the content validity of the tool and (3) establishing the practical value of the tool. The development of the PBCT consisted of several steps. First, the core content (i.e. psychological
competencies) was defined. After the competencies were determined, the tool’s content (i.e. items for each of the competencies) was defined. This led to the initial version of the tool. The initial tool was tested in collaboration with TeamNL coaches. The aim of the initial testing was to test the content validity and practical value of the tool and to define workable procedures in
collaboration with coaches. After testing the initial tool, the final structure of the tool was established. Thereafter, we started the more formal testing of the content validity and the practical value. Because the validation procedures can also provide information about the instrument (Haynes, Richard & Kubany, 1995), the improvement of the tool was an ongoing process. The feedback we received in testing the content validity was used to refine the tool. Last, we formally examined the practical value of the tool.
In total, 29 coaches participated in the study of whom 23 males and 6 females. The coaches all had extensive experience at the highest level. They all joined a three-year development program, called ‘Master coach in sports’, for national coaches working with TeamNL athletes. Coaches need to be nominated and selected to participate in this program (NOC*NSF, 2019a). All coaches worked
with high-potential and/or elite athletes of TeamNL in an individual or team Olympic or Paralympic sports, including male and/or female athletes (see Table 6.1).
Table 6.1. Overview of the coaches (N = 29) and their high-potential or elite athletes. Each coach participated in at least one of the three major parts in this study.
Participant Sport Working with Athletes’ gender IndividualTeam/
1 Gymnastics Elite athletes Female Individual 2 Gymnastics High-potentials Female Individual 3 Field hockey Elite athletes Female Team 4 Water polo Elite athletes Female Team 5 Korfball Elite athletes Both Team 6 Basketball Elite athletes Female Team 7 Checkers High-potentials Male Individual 8 Swimming Elite athletes Female Individual 9 Para athletics High-potentials Female Individual 10 Tennis High-potentials Male Individual 11 Beach volleyball High-potentials Male Team 12 Para basketball Elite athletes Female Team 13 Boccia Elite athletes Both Team 14 Archery Elite athletes Female Individual 15 Basketball Elite athletes Male Team 16 Basketball High-potentials Female Team 17 Para cycling Elite athletes Both Individual 18 Softball Elite athletes Female Team 19 Volleyball High-potentials Female Team
Development of the Performance Behaviour Coaching Tool (PBCT)
Determining the competencies of performance behaviour of the PBCT Based on recent research by Blijlevens and colleagues (e.g. 2018; 2019) among Dutch high-potential and elite gymnasts, beach volleyball and baseball players, TeamNL defined fourteen psychological competencies related to high-potential and elite athletes’ optimal development and maximum performance. Table 6.2 shows an overview of the fourteen competencies of Performance Behaviour. The competencies of Performance Behaviour are generic psychological
competencies, that are required in multiple sports. However, the need and importance of specific psychological competencies might differ per sport and developmental stage of the athlete (Blijlevens et al., 2018).
The fourteen psychological competencies of performance behaviour were leading in this process of determining the core content of the coaching tool. However, instead of developing the tool for all fourteen competencies, the researchers decided to start with developing the tool for a limited number of competencies in first instance. If the tool will proof to be valid and of practical value for coaches, the tool will be extended to the other competencies as well.
20 Judo High-potentials Female Individual 21 Handball High-potentials Female Team 22 BMX High-potentials Male Individual 23 Table tennis High-potentials Male Individual 24 Badminton High-potentials Male Individual 25 Volleyball High-potentials Male Team 26 Gymnastics Elite athletes Female Individual 27 Athletics Elite athletes Both Individual 28 Athletics Elite athletes Both Individual 29 Golf Elite athletes Male Individual
Table 6.2. Overview of the psychological competencies of performance behaviour of TeamNL
Three considerations influenced the choice of competencies. First, coaches gave their feedback about which competencies require most attention in the
development stage of the athletic career (Wylleman & Rosier, 2018). Second, it was discussed which of the fourteen psychological competencies will indirectly be enhanced by working on other competencies. For example, the competency ‘to be able to work process-oriented’ is described as the athlete’s ability to determine how he wants to achieve goals and to work stepwise to achieve it, 1
which results in the athlete having patience and confidence in the process to get there. Enhancing an athlete’s ability to work process-oriented therefore
Psychological competencies of performance behaviour
The ability to focus
The ability to adapt to changing circumstances
The ability to make decisions and oversee its consequences The ability to communicate with your environment
The ability to set and achieve goals The ability to persevere
The ability to set and protect your own boundaries
The ability to create an optimal balance between all activities The ability to make and adhere to your planning
The ability to perform under pressure The ability to identify and solve problems The ability to work process-oriented
The ability to reflect on your development and performance The ability to be self-confident
‘He’ can be replaced by ‘she’ throughout the entire manuscript.
indirectly also strengthens his ability to set and achieve goals. It was therefore decided to give priority to competencies that would also influence the
development of other competencies. Third, it was decided to give priority to competencies which are not developed in other contexts. For example, planning skills can also be developed in education (Meltzer, 2018) and was therefore left out the selection. Fourth, it was discussed whether there is a hierarchy in the development of the competencies. The ability to persevere was for example considered as fundamental for going through the talent pathway and is therefore required at a very young age. It was therefore decided to include the ability to persevere.
Based on these considerations the following six competencies of performance behaviour were chosen: (1) the ability to adapt to changing circumstances, (2) the ability to persevere, (3) the ability to set and protect your own boundaries, (4) the ability to perform under pressure, (5) the ability to work process-oriented, and (6) the ability to reflect on your development and performance.
Determining the items of the competencies of performance behaviour of the PBCT
The second step was to generate items for each of the six competencies. The first author drew from over 5 years of experience as a sport psychologist researcher/ practitioner, as well as the extant literature on sport and exercise psychology, to make a first draft per competency. This process of item generation was informed by a review of several well-established tools on competence-based learning and psychological competencies in elite sports (e.g. Bezanilla et al., 2014; Harwood & Anderson, 2015; Villa, 2008). In line with the tool for competence-based learning in higher education (Villa, 2008), it was decided to describe three levels of mastery. Three levels of mastery give an adequate indication of how the competency’s development takes place, while at the same time making use of the tool not too complex for coaches. The first author first defined the
competencies and described the three levels of mastery in broader lines as shown in Table 6.3.
Table 6.3. Overview of the competencies of performance behaviour and levels of mastery of the PBCT Performance behaviour competencies Level of mastery
Level 1 Level 2 Level 3
The ability to adapt
The athlete shows that he wants to adapt his behaviour when the
circumstances require him to.
The athlete adapts his behaviour when the circumstances require him to.
The athlete purposefully adapts his behaviour in changing circumstances. The ability to persevere The athlete encourages himself to keep on going, also when being confronted with difficulties.
The athlete keeps going when confronted with difficulties.
The athlete shows dedication and perseverance to achieve his goals.
The ability to set and protect your boundaries
The athlete recognizes and can tell other people how he feels right now.
The athlete knows when too much is asked of him (by himself or other people) and communicates this. The athlete proactively protects his boundaries. The ability to perform under pressure The athlete recognizes the effects of stress and is able to stay calm in stressful
situations or under pressure.
The athlete uses techniques to regulate his levels of arousal and stress.
The athlete recognizes the effects of stress on his behaviour and purposefully
continues to perform his task under pressure or stressful situations.
The fist level refers to the knowledge that the athlete needs to develop about (?) the competency in question. Such knowledge has to do with dates, facts, events, features, principles, postulates, theories, etc (Villa, 2008). It also is a first step in self-evaluation, enabling the athlete to know his/her initial level in the
competency in question. The second level describes the way in which athletes’ knowledge, skill or attitude is applied in different situations (analysing, solving, applying, judging, clarifying, etc.). At this level, this is especially reflected in regular situations. The third level indicates the way in which an athlete is able to integrate the ability into his/her life (or some facet thereof: academic,
interpersonal, social etc.) and is able to demonstrate this ability in different circumstances (?). The essential feature of this level is the use that the athlete makes of the competency in question in complex situations (Villa, 2008). As a result, mastery of the competency can also be identified in other situations, such as at school, in contact with family, friends and sponsors, and other social relations.
Thereafter, a number of behavioural indicators were operationalized for each competency and for each level of mastery. The number of indicators per level
The ability to work process-oriented
The athlete explains what he is doing and why he is doing it.
The athlete works patiently and step-by-step on achieving his goals.
The athlete oversees the long-term effects of what he is doing now on his development and undertakes action to improve or speed up the process. The ability to reflect on your development and performance
The athlete knows how to evaluate concrete actions and performances and draws conclusions.
The athlete analyses his development and defines what he needs to improve himself. The athlete continuously and purposefully reflects on his behaviour, performances and development, and acts in order to improve performance
was not set but depended on the need to sufficiently describe athlete’s behaviour at a specific level of mastery.
Testing the initial tool
After a draft instrument was prepared, initial procedures for working with the tool were developed. The draft instrument was first tested with two elite sport coaches working with female gymnasts in collaboration with the first author. As the tool aimed to assess the psychological competencies of both high-potential and elite athletes, the national team coach and a coach working with gymnasts in the junior national team participated (10 to 13 years old). The first author gave them instructions about the aim and the intended use of the tool. Although clear procedures in working with the tool are critical for the quality of the tool (Van Berkel, 2012), it was decided to initially give global instructions to work with the tool. Both coaches had this freedom to work with the tool how they considered useful to do. This provided the researchers also useful information about the coaches’ practices.
Each coach used the tool with two to four gymnasts with whom they were working with more than 21 hours a week. They choose concrete training sessions to focus on observing the gymnasts’ behaviours. After a week, when they had used the coaching tool during 2 to training sessions of 2 hours, the first author met with both coaches face-to-face and discussed their experiences in working with the tool. Both coaches considered the tool as very useful to indicate the current possession of their athletes’ competencies. They further mentioned that it helped them to look at a more structured way to the athlete’s behaviour. As a result, it helped them to define specific points for improvement in terms of behaviour. Both coaches also gave recommendations to rewrite some specific indicators in more everyday language, because there was too much scientific jargon and/or items were too difficult to understand (e.g. “Represents others’ reasoning completely and with organisation well-suited to the type of contents” or “Accurately reconstructs all preconceptions and their origin”). We followed their recommendations and rewrote several indicators. At this point, the content and procedures appeared ready for further testing.
The instrument was then tested by three national team coaches (water polo, field hockey and korfball) in collaboration with the first and second author. The tool was introduced during a face-to-face meeting with all three coaches. The authors explained the aim and the intended use of the instrument. In a follow-up meeting, coaches gave their feedback on the content and the practical value of the tool.
The field testing showed that all three coaches worked with the tool in different ways, while at the same time emphasizing the practical value of working with the tool. The field hockey coach completed the tool for her athletes. She further asked the athletes herself to fill in the tool about their own behaviour.
Afterwards, in a one-on-one meeting, they discussed their findings with each other. The field hockey coach considered these conversations as a very meaningful coaching tool to gain more insight in the athletes’ behaviours and compare this with her own observations. The water polo coach also completed the tool for several of his athletes. He asked his coaching staff to do this as well. They thereafter discussed their findings. He recognized that the scores differed, because some of his staff members had other information about the athletes. He emphasized that the tool gives more insight into athletes’ behaviour. Finally, the korfball coach also completed the tool for his athletes. He asked the players to do this as well. In a team session, they discussed their scores with each other. Within this meeting, he also asked his team which players in the team would score high or low on these competencies. The korfball coach valued his approach, because it provided the players to more insight in each other. For example, it led the team to conclude that each player is different, has his own way of handling situations and has his own points for improvement in terms of behaviour.
The researchers therefore decided to extent the usage in the procedures and manual with different options. The procedures were explained in a five-page manual, including (1) the background and development of the tool, (2) description of the intended use and (3) a user’s guide. The background and development give a short description of TeamNL’s vision on the development of
performance behaviour and the scientific research underpinning the coaching tool. The intended use aims at providing insight in the athlete’s current level of performance behaviour. Last, the user’s guide explains the individual elements of the tool and how the tool can be used. Considering the feedback of coaches during the field testing, suggestions were given how the tool can be used in different situations and different observers.
Final structure of the PBCT
After the initial testing the final PBCT consists of 6 parts labelled as: (1)
performance under pressure, (2) adaptability, (3) process-oriented approach, (4) reflection, (5) set and protect boundaries and (6) perseverance. The six parts can be filled in independently of each other. Each part consists of a definition, examples of situations in which the competency can be observed, and a list of behavioural indicators describing three levels of mastery (indicated by colours). The listed situations are examples, they are not exhaustive. Coaches score whether the behavioural indicators are being observed or noticed. Coaches score the behaviour starting with “the behaviour is observed…” using the following options (1) never/not at all, (2) from time to time, (3) on a regular basis, (4) frequently and (5) almost always. The last column of the tool offers space for additional notes and remarks. The final version of the PBCT is presented in Appendix A . 2
Testing the content validity
The pilot study and field testing supported the content and practical value of the PBCT. At this point, the content of the instrument was finalized, and the
procedures had proven valuable in practice. Thus, the coaching tool was
deemed ready for more formal testing of its content validity, that is the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose (Haynes et al., 1995). The formal assessment of the content validity of the PBCT was done two
This final version of the PBCT includes the changes being made as a result of testing the content
2
ways. First, the PBCT was presented to a panel of experts for review. Thereafter, 20 elite sports coaches systematically evaluated the content validity of the PBCT within their daily practices.
The expert panel included one expert on competence-based learning, one developmental psychologist, one expert in the field of talent development, one sport and performance psychologist working with Olympic athletes and two Dutch National team coaches (i.e. female gymnastics and korfball). This panel represented a range of experience and was considered highly qualified to assess the content validity of the instrument relative to the development and
assessment of psychological competencies and its expression in athletes’ actual behaviour. The first author had face-to-face meetings with all members of the panel over a period of two months. In each meeting, the first author described the purpose and development of the PBCT, provided an overview of the content and described the data collection procedures. Following that introduction, panel members were asked to review the content in detail, ask for any clarification they required, and provide feedback regarding any errors, omissions, or ambiguity. Panel members were also invited to comment on the rigor and feasibility of procedures in using the tool. All panellists provided positive reviews of the content and procedures in these meetings. Only few suggestions were done to reframe or rewrite several behavioural indicators for more clarity or to be more explicit. We followed the experts’ feedback and reworked to the panellists’ satisfaction without altering the original meaning. All panellists were invited to contact the first author if, upon further review, they had additional questions or suggestions; none did.
Subsequently, twenty coaches systematically reviewed the instrument for appropriateness, readability, and comprehension. For each of the six competencies they separately filled in the questions presented in Table 6.4.
Table 6.4. Overview of questions answered by the coaches to establish the content validity of the PBCT (n = 20)
Coaches filled in the questions on paper and gave specific comments/feedback. The authors discussed for each of the coaches’ comments whether the feedback should be considered in reworking the tool. Eventually, after discussing the comments, the first author decided to make the suggested changes. When the authors considered the comments as enhancing the tool’s quality, it was decided to follow the coaches’ suggestions and we rewrote the items. For example, the authors received the feedback that the indicators of process-oriented work can also be observed when imposed by coaches or the staff. In this way, the coach was not able to say whether the athlete shows this behaviour independent and on his own initiative. To improve the tool, we inserted an indicator that describes whether the athlete is able to work in small steps, focussing on the process, on his own initiative (level three of mastery). Further, it was noticed that the comprehensive picture given by all indicators of the ability to perform under pressure showed some overlap with the ability to persevere. The items of both competencies were reformulated to distinguish the two from each other.
When the authors considered that the coaches’ feedback would not increase the quality of the tool, it was decided not to include the suggested changes. For example, one of the coaches had some remarks about the (in)stability and/or
Questions to test the content validity
1. Are there any behavioural indicators missing for this competency?
2. Are there any behavioural indicators included that are not important or do not contribute to the possession of this competency?
3. Do all behavioural indicators together give a comprehensive picture of this competency?
4. Look at the score of your athlete. Do you think the score gives you a correct evaluation of your athlete’s current level of possession of this competency? 5. Remaining feedback
(in)frequency whether specific behaviour is being shown. The authors decided that the coach should consider this point in his score whether and to what extent specific indicators are being noticed. When the athlete shows the behaviour in training and national competition, but not on international level, the behaviour is being observed ‘from time to time’ or ‘on a regular basis’. When the athlete is also able to show the behaviour on international tournaments, the coach should score that the behaviour ‘almost always’ is present. We therefor did not change the items of the tool, but inserted this comment in the description of the tool. Last, one of the coaches remarked that not all indicators were observable. In this case, he needed to search for other ways (e.g. conversations, asking other staff members) to get the required information to fill in the tool. According to the researchers, both examples do not provide information about the content validity of the tool. However, to clarify this for coaches, we described this more explicitly in the manual of the tool.
Establishing the practical value
We aimed to develop a practical and valid coaching tool, which enables coaches to assess high-potential and elite athletes’ psychological competencies of performance behaviour within the context of elite sports. The coaching tool simultaneously allows coaches to integrate both observable behaviours and the accumulated know-how they already have about their athletes’ behaviour within different situations. Other valid tools often lack usability in the context of elite sports because they are too long, unsuitable for a specific sport or athlete, or because they are stripped of context (Brewer & Jones, 2002; Cushion et al., 2011; Horvath & Röthlin, 2017). They are therefore rarely used by coaches. The practical value of this tool is therefore of great importance. Because we collaborated with coaches, the practical value for coaches was continuously monitored during the process of development of the tool and testing the content validity. However, at this point, we wanted to formally get insight in the practical value of the tool.
Six coaches gave their qualitative feedback on working with the tool. The coaches first indicated the current level of mastery of one of their athletes for all
six competencies, on the basis of descriptions in Table 6.3. Subsequently, they defined which behaviour should be developed by their athlete. Afterwards, the authors provided them the PBCT. During the next week, all coaches scored the athlete’s behaviour while working with the PBCT. Then again, they determined which behaviour should be developed within their athletes. The coaches were supported to compare the learning goals deriving from the first step, with the learning goals defined while working with the PBCT. Last, as a measurement of the practical value of the tool, the coaches answered several questions about working with the tool and the challenges they experienced in doing so. Table 6.5 shows an overview of the questions and main answers by the coaches to
establish the practical value of the PBCT.
Table 6.5. Overview of questions and main answers by the coaches (N=6) to establish the practical value of the PBCT.
1. How did you approach the assessment of your athlete’s competencies while
working with the PBCT?
Introduced the tool to the other staff members and asked them to observe, to compare their observations.
Defined key points on which I wanted to focus my observations. Focused on the athletes' typical responses and role related behaviours.
Observed the athlete in his preparation before an important international match. Chose an athlete who I observed in various situations.
2. Which differences did you notice in the first measurement of your athlete’s
competencies in relation to the measurement based on the PBCT?
Both measurements were quite the same, while the measurement with PBCT differentiated the behaviour more explicitly and expressed it more detail. The measurements varied more than I had expected.
The measurement without the PBCT was more negative and unsubtle. The specific way of looking at the athlete's behaviour created a more comprehensive picture of his competencies, in which also the positive elements of his behaviour were highlighted.
I miscalculated the current level of behaviour based in first instance. Through decent observation and focussing on behavioural indicators I was able to do in-depth assessment of the competencies.
Working with the PBCT made me realised that there were more points of improvement that I noticed in first instance.
3. Reviewing both scores, are you able to define what behaviour needs to be
developed?
Yes, for all six coaches. Including specific descriptions what behaviours need to be developed.
4. What challenges did you experience in working with the PBCT?
Finding the time to do the observations during the training or competition. Combining to lead the training, while observing at the same time.
Observing the athlete in a structured manner.
Trying not to influence the athlete's behaviour during observation, while at the same time give the appropriate guidance during the match.
To observe the athletes objectively, while you take part in the situation yourself.
5. What are the advantages of working with the PBCT?
Made the athlete’s behaviour very concrete and measurable
The PBCT forced me to think about and observe the athlete's behaviour Made it easier to discuss observed problem behaviour with my athletes Helps me to look at the athlete's behaviour in a systematic and structured way Provides opportunities to create awareness and speak the same language about performance behaviour with your staff and athletes
Letting the PBCT complete by other staff members helps to increase the nuances in athlete's behaviour
Provides guidance to work systematically on behavioural change A very clear and workable tool
Coaches’ answers shown in Table 6.5 confirmed that the PBCT was of great value to coaches for the assessment of athlete’s competencies and defining points for improvement in terms of performance behaviour of the athletes. Coaches mentioned that the PBCT helped them to look at athletes’ behaviour in a systematic and structured way. Coaches reported that the behavioural indicators of the tool provided them with precise information about what
Sometimes difficult to interpret the different levels of mastery and/or behavioural indicators
Several behavioural indicators were duplicate
You need to read the behavioural indicators carefully before working with the tool during training, because the indicators are sometimes a bit difficult to understand
By looking in this way at athlete's behaviour I had the feeling that I was searching for negative points
It would be helpful when the tool was more sport-specific
I needed more space to write down my additional notes and observations The tool should be made available digitally (it was now on paper)
All behavioural indicators are formulated masculine, which was disturbing when working with female athletes
7. What do you need to develop in your own observation skills?
Create or schedule more time to do the observation decently Learn to look at the athlete's behaviour without presumptions
More often discuss the athlete's experiences of his own competencies in order to be aware of his feelings and thoughts.
To assess the athlete's behaviour more systematically, instead of on the basis of my own feeling, assumptions or interpretation
To formulate more clear learning goals for behavioural change and to work stepwise towards these goals
behaviour they need to look at and which gave them direction in observing the athlete’s behaviour. Furthermore, coaches mentioned that, without the tool, they had a global impression of the athlete’s mastery of a competency. Working with the PBCT provided them more detailed information and a comprehensive picture of the athlete’s performance behaviour. Instead of ‘having an impression’ how the athlete behaves, working with the tool provided them more accurate and objective opinion about the athlete’s performance behaviour. Coaches further mentioned that the tool encouraged and enabled them to communicate with their athletes about their observations. The tool allowed coaches to articulate in a more precise way what they observed and why it need to be developed. As a result, they were able to define more concrete and specific learning goals in terms of performance behaviour with their athletes. Coaches further reported that the tool stimulated them to discuss their observations with their staff members. As other staff members observed athletes in other
situations or contexts, this was of help to get a broader picture of athlete’s behaviour in several situations.
Coaches experienced several challenges in working with the PBCT. One of the challenges was having time to observe athlete’s behaviour and to work with the tool, which was especially reported by team coaches. Coaches emphasized the importance of working with the tool, and they therefore mentioned that they need to create time within their schedule to fill in the tool. Another challenge concerning to observing the athlete in a specific context was that coaches reported that they need to be aware that they are observing the athlete in a specific situation or context, in which coaches also take part in a specific role. Some of the coaches mentioned that it was also challenging to only observe the athlete’s behaviour, without directly intervening the situation in order to change athlete’s behaviour. Also related to changing the athlete’s behaviour, coaches reported that they needed to develop themselves in formulating concrete learning goals in terms of performance behaviour. They mentioned that they need to develop themselves in working step by step on learning to improve athletes’ behaviour, as they are used to with technical or tactical aspects.
why athletes do what they do. During working with the PBCT it became clear for them that they need to discuss these assumptions more often with their
athletes, to check whether they are right or not. These insights were yielded in working with the PBCT, which was appreciated by the coaches. It should be noted that the challenges coaches reported in working with the tool were not related to the age of the athletes. Challenges of the same nature were
mentioned by both coaches of high-potential athletes and of elite athletes. One of the suggestions that was done to improve the tool was the provide the tool online, because then it would be easier to monitor athletes’ development over a longer period. Coaches further suggested to make the tool more sport-specific. They believed that the practical value of the tool would be enhanced by making the behavioural indicators more sport-specific. These recommendations can be used for further improvement of the tool.
Discussion
We aimed to develop a practical and valid coaching tool, which enables coaches to assess high-potential and elite athletes’ psychological competencies of performance behaviour of within the context of elite sports. The PBCT was developed and tested with the consultation of experts in the field of competence-based learning, talent development and psychology (e.g. developmental, sport and exercise psychology) and 29 sport coaches with extensive experience at the highest level. The PBCT provides coaches a systematic and structured way to look at athletes’ performance behaviour, because it gives coaches detailed information about what behaviours they need to focus on while observing athletes’ behaviour. It is the first observation tool developed for coaches to assess psychological competencies within the context of elite sports. Most existing instruments are mainly questionnaires that uses self-reports to indicate athlete’s psychological factors (e.g. Durand-Bush, Salmela & Green-Demers, 2001; Hardy, Roberts, Thomas & Murphy, 2010; Toering, Elferink-Gemser, Jonker, Van Heuvelen & Visscher, 2012). Existing systematic observation instruments are mostly provided for sport psychologist
counselors to code the athlete’s behaviour (e.g. Martin, Toogood & Tkachuk, 1997; Rushall, 1979; Toering et al., 2011). Further, most systematic observation instruments only draw attention to objectively observable behaviour. Vis-à-vis, the PBCT includes items related to private or mental acts (e.g. “can deal with disturbing ideas, thoughts and feelings and overcome mental blockages) as they are also considered as actual cases of behaviour (Bergner, 2016). The PBCT does not only focus on counting the prevalence of the behaviour, but also on the qualitative interpretation of both the coach and athlete. The ‘additional notes’ column stimulate coaches to discuss their observations and interpretations with their athlete, which stimulate the triangulation of the assessment (Holder, Winter & Orr, 2018). Finally, the items of the tool are defined in a way that the tool is applicable in various sports, in contrast to other existing instruments that are only useful in a specific sport (e.g. the Self-Talk and Gesture Rating Calse (STAGRS) by Van Raalte, Brewer, Rivera & Petitpas; 1994).
The PBCT offers coaches insight in athletes’ current mastery of a psychological competency and what behaviour need to be developed in order to enhance athletes’ behaviour. In doing so, the PBCT puts a strong emphasis on the development of psychological competencies. Characteristic for the assessment of competencies is the assessment of and for learning (Benzanilla et al., 2014; Stiggins, 2002). Because the PBCT contains three levels of mastery per competency, it helps coaches to indicate athlete’s current mastery of a competency and to determine the behaviour that needs to be developed. The coaching tool can therefore easily become part of the daily practices of coaches working with high-potential and elite athletes. It can for example be used on a frequent basis (e.g. each three months or half year) to monitor athletes’
development over a longer period. The PBCT can further be used to indicate an athlete’s behaviour in other contexts and situations. For example, it can be used to monitor how athletes behave under high pressure, which is helpful to indicate how athletes’ behaviour differs from how they behave in general situations (such as daily practices without pressure). These detailed insights in athletes’
behaviour can then be used by coaches and athletes to define learning goals for performance behaviour, which need to be embedded in coaches’ daily routines
in working with their athletes. This approach entails with a multidimensional perspective on development of the athletic performance (i.e. athletes should develop themselves on a physical, technical, tactical and psychological level), which is important both in the acquisition of expertise for high-potential athletes and the maintenance of expertise of elite athletes (Blijlevens et al., 2018;
Elferink-Gemser et al., 2018). The strong focus on athlete’s development further align with TeamNL’s mission to achieve optimal development of TeamNL athletes as in function of high performance (NOC*NSF, 2019c).
Although coaches rated the practical value of the tool, they also experienced several challenges in working with the assessment of athletes’ competencies (e.g. limited time, difficulties to observe athletes’ behaviour without
intervening). Considering these challenges, it might be questioned why the responsibility for the assessment and development of an athlete’s performance behaviour is placed on coaches instead of on sport psychologists for example. On the one hand, it sounds reasonable that the responsibility might be put on sport psychologists who are, due to their education and experience (Hutter, Van der Zande, Rosier & Wylleman, 2018), more able to observe an athlete’s
behaviour. On the other hand, coaches are responsible for athletes’ holistic development (Bool, 2018; Gould, Dieffenbach & Moffet, 2002; Wylleman, Rosier, De Brandt, & De Knop, 2016), of which performance behaviour is one
component. They therefore require knowledge of each of the elements of performance (i.e. physical, technical, tactical and psychological) (Elferink-Gemser et al., 2018). Further, the assessment of competencies should be done through realistic and authentic tasks in realistic contexts in which they are being used, because it will benefit the accuracy of the observations (Ledoux et al., 2013). The elite sports context offers a lot of complex and game-like situations, which then can be used on the spot by coaches to assess competencies through realistic and authentic tasks in actual context in which they are being used. The suggested situations within the tool are only a handful among the many opportunities the elite sport context provides. In addition, the development of athletes’ performance behaviour should also take place within the actual context, because the transfer of training of psychological competencies learned
outside the actual context is very low (Burns, Cumming, Cooley, Holland & Beech, 2017). The same situations can also be used to enhance athletes’ performance behaviour.
Considering both the practical value and the challenges coaches experienced in working with the PBCT described above, one of the next steps to take is to further educate and train coaches in observing and enhancing athletes’ performance behaviour. Relevant topics to enhance athletes’ performance behaviour are yet hardly addressed in the education of coaches (NOC*NSF, 2017). Furthermore, research showed that coaches report a lack of confidence and knowledge as barriers in developing psychological competencies (Arthur, Callow, Roberts & Glendinning, 2019; Callow, Roberts, Bringer, & Langan, 2010; Paquette & Sullivan, 2012). It is therefore needed to embed topics of
performance behaviour in the education of coaches working with high-potential and elite athletes at the highest level. This should not only include the transfer of knowledge about the how’s and why’s of specific psychological competencies, but also qualitative insights in how other experienced coaches observe and enhance their athletes’ performance behaviour within their daily practices. Furthermore, another next step to take is to cooperate with coaches to expand the tool for the eight remaining psychological competencies of performance behaviour. To enhance the usability of the tool, the tool needs to be provided digitally, as suggested by several coaches in the present study.
All in all, we consider the PBCT to be designed carefully for the evaluation of performance behaviour, so that it becomes a precise tool which is appropriate for coaches. However, it can also be used by other staff members such as other coaching staff, physiotherapists and sport psychologists to obtain 360 degrees feedback. Broader, as of the competencies are also present in other contexts (e.g. at school, work or home), the tool can also be used by teachers, parents or employers to indicate the someone’s competencies. As participative assessment is characteristic for competence-based learning assessment (Eichbaum, 2017; Jamieson, Jenkins, Beatty & Palermo, 2017), athletes can participate in the assessment themselves, as seen for example seen during the initial testing of the
tool. From a pragmatic perspective, in this way the PBCT provides coaches numerous opportunities to get detailed insight in their athletes’ competencies of performance behaviour in a more systematic way, which simultaneously
contributes to athletes’ development.
Acknowledgement
We would like to thank all the experts and coaches who participated in this study as well as the Dutch Olympic Committee (NOC*NSF) who made this research possible.
References
American Psychological Association. (2015). Competencies for Psychology Practice in
Primary Care. Retrieved from
http://www.apa.org/ed/resources/competencies-practice.pdf.
Arthur, R. A., Callow, N., Roberts, R., & Glendinning, F. (2019). Coaches Coaching Psychological Skills—Why Not? A Framework and Questionnaire Development.
Journal of Sport and Exercise Psychology, 41, 10-23.
Arthur, C. A., Wagstaff, C. R., & Hardy, L. (2017). Leadership in sport organizations. The organizational psychology of sport: Key issues and practical applications. London, United Kingdom: Routledge.
Baumeister, R. F. (1984). Choking under pressure: self-consciousness and paradoxical effects of incentives on skillful performance. Journal of Personality and Social
Psychology, 46(3), 610.
Bengoechea, E. G., Wilson, P. M., & Dunn, S. (2017). From liability to challenge: Complex environments are associated with favorable psychosocial outcomes in adolescent sport participants. Journal of Adolescence, 58, 74-83.
Bergner, R. M. (2016). What is behavior? And why is it not reducible to biological states of affairs? Journal of Theoretical and Philosophical Psychology, 36(1), 41-55.
Bezanilla, M., Arranz, S., Rayn, A., Rubio, I., Menchaca, I., Guenaga, M., & Aguilar, E. (2014). A proposal for generic competence assessment in a serious game. Journal of New
Approaches in Educational Research (NAER Journal), 3(1), 42-51.
Blijlevens, S. J., Elferink-Gemser, M. T., Wylleman, P., Bool, K., & Visscher, C. (2018). Psychological characteristics and skills of top-level Dutch gymnasts in the initiation, development and mastery stages of the athletic career. Psychology of Sport and
Exercise, 38, 202-210.
Blijlevens, S. J., Wylleman, P., Bool, K., Elferink-Gemser, M. T., & Visscher, C. (2019). Acquisition and maintenance of excellence: the challenges faced by Dutch top-level gymnasts throughout different stages of athletic development. Sport in Society, 1-20. Bool, K. (2018). De weg naar het podium. Talentherkenning en -ontwikkeling in de
Nederlandse topsport. Nieuwegein: TeamNL/Arko Sports Media.
Brewer, C. J., & Jones, R. L. (2002). A five-stage process for establishing contextually valid systematic observation instruments: The case of rugby union. The Sport
Burns, V., Cumming, J., Cooley, S., Holland, M., & Beech, N. (2017). ‘Skills don’t transfer themselves’: Translating training courses into lasting behaviour
change. Education, 3(1).
Callow, N., Roberts, R., Bringer, J.D., & Langan, E. (2010). Coach education related to the delivery of imagery: Two interventions. The Sport Psychologist, 24, 277–299.
Creswell, J. W. (2014). Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches. Thousand Oakes, CA: Sage.
Cushion, C., Harvey, S., Muir, B., & Nelson, L. (2012). Developing the Coach Analysis and Intervention System (CAIS): Establishing validity and reliability of a computerised systematic observation instrument. Journal of Sports Sciences, 30(2), 201-216. Dohme, L. C., Backhouse, S., Piggott, D., & Morgan, G. (2017). Categorising and defining
popular psychological terms used within the youth athlete talent development literature: a systematic review. International Review of Sport and Exercise
Psychology, 10(1), 134-163.
Durand-Bush, N., Salmela, J.H. and Green-Demers, I. (2001) The Ottawa Mental Skills Assessment Tool (OMSAT-3). The Sport Psychologist, 15, 1-19.
Eichbaum, Q. (2017). Acquired and participatory competencies in health professions education: Definition and assessment in global health. Academic Medicine, 92(4), 468-474.
Elferink-Gemser, M., Te Wierike, S., & Visscher, C. (2018). Multidisciplinary Longitudinal Studies: A Perspective from the Field of Sports. In K. Ericsson, R. Hoffman, A. Kozbelt, & A. Williams (Eds.), The Cambridge Handbook of Expertise and Expert
Performance (pp. 271-290). Cambridge: Cambridge University Press.
Fleddermann, M. T., & Zentgraf, K. (2018). Tapping the full potential? Jumping
performance of volleyball athletes in game-like situations. Frontiers in Psychology, 9, 1375.
Gould, D., Diffenbach, K., & Moffett, A. (2002). Psychological characteristics and their development in Olympic champions. Journal of Applied Sport Psychology, 14, 172– 204.
Hardy, L., Roberts, R., Thomas, P.R., & Murphy, S.M. (2010). Test of Performance Strategies (TOPS): Instrument refinement using confirmatory factor analysis. Psychology of Sport and Exercise, 11, 27-35.
Harwood, C., & Anderson, R. (2015). Coaching psychological skills in youth football: Developing the 5Cs. Bennion Kearny.
Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological
assessment, 7(3), 238.
Holder, T., Winter, S., & Orr., B. (2018) Seeing Athletes on the Field. In J. Taylor (Ed.),
Assesment in Applied Sport Psychology (pp. 115-126). Champaign, United States:
Human Kinetics.
Hutter, R. V., van der Zande, J. J., Rosier, N., & Wylleman, P. (2018). Education and training in the field of applied sport psychology in Europe. International Journal of Sport and
Exercise Psychology, 16(2), 133-149.
Jamieson, J., Jenkins, G., Beatty, S., & Palermo, C. (2017). Designing programmes of assessment: A participatory approach. Medical Teacher, 39(11), 1182-1188. Kaslow, N. J. (2004). Competencies in professional psychology. American
Psychologist, 59(8), 774.
Ledoux, G., Meijer, J., Van der Veen, I., Breetvelt, I., Ten Dam, G., & Volman, M. (2013). Meetinstrumenten voor sociale competenties, metacognitie en advanced skills. Een Inventarisatie. Amsterdam, Nederland: Kohnstamm Instituut.
Martin, G.L., Toogood, A., & Tkachuk, G.A. (1997). Behavioural assessment forms for sport psychology consulting. Winnipeg, Canada: Sport Science Press.
McKenzie, T. L., Cohen, D. A., Sehgal, A., Williamson, S., & Golinelli, D. (2006). System for Observing Play and Recreation in Communities (SOPARC): reliability and feasibility measures. Journal of Physical Activity and Health, 3(s1), S208-S222.
McKenzie, T.L., & van der Mars, H. (2015). Top 10 research questions related to assessing physical activity and its contexts using systematic observation. Research Quarterly for
Exercise and Sport, 86, 13-29.
McPherson, S. L. (1994). The development of sport expertise: Mapping the tactical domain. Quest, 46(2), 223-240.
Meltzer, L. (2018). Executive function in education: From theory to practice. New York: Guilford Publications.
Meredith, S. J., Dicks, M., Noel, B., & Wagstaff, C. R. (2018). A review of behavioural measures and research methodology in sport and exercise psychology. International
Review of Sport and Exercise Psychology, 11(1), 25-46.
Moore, L. J., Vine, S. J., Wilson, M. R., & Freeman, P. (2015). Reappraising threat: How to optimize performance under pressure. Journal of Sport and Exercise
NOC*NSF (2017). Kwalificatiestructuur Sport 2017. Retrieved April 24, 2019 from https:// www.academievoorsportkader.nl/stream/kp-bundeling.pdf.
NOC*NSF (2019a). Master Coach in Sports, Programma 2013-2016. Retrieved April 24, 2019 from https://www.nocnsf.nl/mastercoach.
NOC*NSF (2019b). TeamNL performance behaviour. Retrieved April 24, 2019 from
https://www.nocnsf.nl/teamnl-experts/performance-behaviour.
NOC*NSF (2019c). TeamNL High performance team. Retrieved April 24, 2019 from
https://www.nocnsf.nl/highperformanceteam.
Otten, M. (2009). Choking vs. clutch performance: A study of sport performance under pressure. Journal of Sport and Exercise Psychology, 31(5), 583-601.
Paquette, K.J., & Sullivan, P. (2012). Canadian curling coaches’ use of psychological skills training. The Sport Psychologist, 26, 29–42.
Rushall, B. (1979). Psyching in sport: The psychological preparation for serious competition in sport. London: Pelham Books.
Seifert, L., Araújo, D., Komar, J., & Davids, K. (2017). Understanding constraints on sport performance from the complexity sciences paradigm: An ecological dynamics framework. Human movement science, 56, 178-180.
Stiggins, R. J. (2002). Assessment crisis: The absence of assessment for learning. Phi
Delta Kappan, 83(10), 758-765.
Toering, T., Elferink-Gemser, M.T., Jonker, L., van Heuvelen, M.J.G., & Visscher, C. (2012). Measuring self-regulation in a learning context: Reliability and validity of the Self-Regulation of Learning Self-Report Scale (SRL-SRS). International Journal of Sport and
Exercise Psychology, 10(1), 24-38.
Toering, T.T., Elferink-Gemser, M.T., Jordet, G., Pepping, G.J., Jorna, C., Visscher, C. (2011). Self-Regulation of Practice Behavior Among Elite Youth Soccer Players: An
Exploratory Study. Journal of Applied Sport Psychology, 23(1), 110-128.
Van Berkel, A. (2012). Kritische reflectie op competentietoetsen in het hbo. Tijdschrift
Onderwijsinnovatie, 2, 17-26.
Van Raalte, J.L., Brewer, B.W., Rivera, P.M., & Petitpas, A.J., (1994). The relationship between observable self-talk and competitive junior tennis players’ match performances. Journal of Sport and Exercise Psychology, 16, 400-415.
Villa, A. (2008). Competence Based Learning. A Proposal for the assessment of generic
Wylleman, P. (2019). An organizational perspective on applied sport psychology in elite sport. Psychology of Sport and Exercise, 42, 89-99.
Wylleman, P., & Rosier, N. (2016). Holistic perspective on the development of elite athletes. In M. Raab, P. Wylleman, R. Seiler, A.-M. Elbe, & A. Hatzigeorgiadis (Eds.),
Sport and Exercise Psychology Research: From Theory to Practice (pp. 270–282).
London: Elsevier.
Wylleman, P., Rosier, N., De Brandt, K., & De Knop, P. (2016). Coaching athletes through career transitions. In R. Thelwell, C. Harwood, & I. Greenlees (Eds.), The Psychology of
