Master of Business Administration in Healthcare Management
Thesis
Making science more efficient as a measure to increase the competitive advantage of the Netherlands.
by
Jort Kropff student number: 5667933
March 5th, 2017
University supervisor, Professor dr. R.J.M.M. Does: R.J.M.M.Does@uva.nl Company supervisor, Dr. M.E.A. Stouthard: m.e.stouthard@amc.uva.nl
2
Executive Summary
Modern societies, and the organizations that build these societies, depend more and more on knowledge generation for maintaining their competitive advantage. Although the Dutch economy is highly competitive, innovation, a requisite for future competitive advantage is lagging behind. Internationally science reform projects have focused on PhD-programs as a measure to increase the efficiency of science as a whole.
This case-study at the University of Amsterdam Academic Medical Centre, investigated whether use of the Lean Six Sigma methodology could improve the efficiency of the PhD graduation process of full-time PhD-programs. The project aimed to reduce the time between enrolment and graduation of the PhD-candidate. A project team was formed including the dean of the faculty of medicine (Champion), principal investigators and a black belt. The project team aimed to reduce the graduation duration from the current median 4.4 years to a median of 4 years, increasing the efficiency of science.
A qualitative analysis of the literature, exit-survey data and panel discussions with PhD-candidates and principal investigators provided the basis for a survey created to assess potential process influence factors. A quantitative association analysis of the survey data of recently graduated PhDs indicated a significant reduction of graduation duration with: 1) candidates with a higher level of project management skills, 2) a graduating fast culture at the local department, 3) older age of the candidate at the start of the programme, 4) PhD-programmes that were finalized within employment, 5) an intellectually stimulating climate and larger research group size. The results of this project and resulting improvement proposals provide opportunities to reduce the current PhD-program duration significantly.
This case-study is an example of an improvement project that supports a culture of continuous improvement and learning. Responsibility for continuous improvement of the research process should be shared between the end-user and the central organization. The central organization should define process goals, assure transparent process performance data and prioritize improvement projects according to organizational goals. End-users should be enabled to signal inefficiencies and share learning experiences within the organization. Then the full-potential of a learning organizations can be unleashed.
This project indicates that focus on continuous improvement can increase the efficiency of science and eventually might contribute to the innovative strength of the Dutch economy as a whole.
3
Contents
Executive Summary ... 2 I. Introduction ... 4 II. Framing ... 8 A. Setting ... 8 B. Methods ... 8C. Empowerment as a measure to maintain control ... 10
III. Case description ... 12
A. Define ... 14 B. Measure ... 15 IV. Results ... 18 A. Analyse ... 18 B. Improve ... 20 C. Control ... 22
V. Summary, discussion, conclusions and recommendations ... 24
A. Summary ... 24 B. Discussion ... 25 C. Conclusions ... 28 D. Recommendations: ... 29 References ... 30 VI. Appendix: ... 34
A. SIPOC Process description ... 34
B. Graduate school Exit survey (PREQ) themes ... 35
4
I.
Introduction
Science aims to build knowledge in the form of testable falsifiable explanations and predictions about the world around us. Modern societies, and the organizations that build these societies, are more and more dependent on knowledge generation for maintaining their competitive advantage (Porter (1985)). This concept was popularized by Peter Drucker as the knowledge economy in 1969 (Drucker (1969)) and entails a transition from agricultural- or labour-intensive economies to knowledge driven economies which are propelled by innovation (Dutta and Soumitra (2012)).
The World Economic Forum and society of Dutch collaborating universities (VNSU) warned recently about the ability of the Dutch economy to maintain its´ competitive advantage (Porter (1985)) based on results presented in the global competitiveness report (Schwab (2012)). This report indicates that although the Dutch economy is highly competitive with a top 5 highest ranking economies of the world, innovation, a requisite for future competitive advantage is lagging behind (8th position) (VSNU (2016)). In 2015 in the Netherlands over twelve billion euro was invested in Research and Development (R&D)(CBS statline (2016). Nonetheless R&D spending in the Netherlands was lower than expected based on the overall ranking and has its impact on the countries’ innovation potential (Rathenau Institute (2016)). Furthermore R&D spending has been 0.5% below the 2.5% GDP goal of the Dutch government in the last years.
If research is performed efficiently, a relative underspending does not have to result in less scientific output and innovation. Consequently in a sector with such high relevance to the country’s economy, the efficiency at which research is performed is of utmost importance. Considering their eighty percent market share in research output, universities would be the primary organizations to target for efficiency improvements in science and consequent innovation. The VSNU supports this notion and stresses the importance of efficient science and knowledge valorisation (VSNU (2017)).
Internationally government-lead programs have been developed aiming to increase the efficiency of science (Alberts et al. (2014), Bergin et al. (2016)). In the United States of America (USA) the National Institutes of Health (NIH) roadmap for medical research aims to create maximal value from biomedical research investments. While the definition of clinical and translational research is still being debated, there is broad consensus that formal and sustained processes are needed. The topic of science reform and productivity was also vocalized recently in the proceeding of the USA national academy of sciences by highly respected
5
scientists (Alberts et al. (2014)). Fundamental flaws in the research process were noted, including the position and (in)effectiveness of PhD-candidates and postdocs.
At universities, research is conducted in a learning environment, meaning that the majority of research is performed by doctor of philosophy (PhD) candidates with help of their supervisors and promotor, a professor in the applicable field (VSNU, Wopi (2017), Alberts et al. (2014)). The degree of doctor which is generally considered the highest academic degree is granted after the PhD candidate has proven his competence in science by completion and defence of a thesis or dissertation. Due to the practical nature of the PhD-candidates training PhD-candidates effectively run the scientific experiments. As such the success of the universities at knowledge generation and the efficiency of the PhD-candidate are interrelated. Increased efficiency of the PhD-candidates daily work might improve the research output and reduce research cost. As such it is not surprising that the PhD-programme and hind lying processes have received attention in the USA research reform programs (Council of Graduate Schools (2016)). In the Netherlands, a contrary development can be distinguished with an increase in the number of PhD-candidates without an equal growth in the number of supervisors. Potentially reducing the time available for mentorship, furthermore supervisors are often dependent on external funding for their employment, increasing the risk of a switch of supervisors during a PhDs program (VSNU (2016-2), Berger, de Jonge (2005)). Both developments further indicate the need for increased efficiency in science to maintain scientific output.
In industry, especially in the 20th century much time was invested in improving the companies most important processes, i.e. the processes responsible for generating competitive advantage over other businesses (Porter (1985)). Methods such as Lean manufacturing, developed after the second world war in Japan and Six Sigma developed in the 1980s in the USA have helped organizations to continuously improve their processes enabling them to outpace their competitors (De Mast et al. (2012)). In the last decade a synthesis of these two approaches (Snee and Hoerl (2004), Hoerl (2004)) known as Lean Six Sigma (LSS) was created. The LSS methodology can be seen as the accumulation of principles and tools concerning efficiency and quality improvement, combining the principles and tools of lean with the scientific and managerial methods of six sigma (De Koning et al. (2006)). In line with Stewart’s focus on process quality control (Shewart and Deming (1939)), LSS enables an organization to improve their processes as a basis for competitive advantage. Specifically by unravelling the processes problems’ root causes instead of responding to incidents (firefighting)
6
and performing frequent inspections as measures of quality control (De Mast et al. (2012)). Furthermore LSS helps to focus on customer oriented outcomes enhancing the quality of the research, and facilitating the successful adoption of biomedical research findings into practice (Schweikhart and Allard (2009)).
Although LSS has proven its effectiveness in industry (Snee (2010)), little evidence is available for success of improvement of research processes using LSS. In industry processes are generally well defined and standardized and can be more easily improved. As such it can be debated whether LSS improvement methods could also be used for improvement of non-industry processes, such as healthcare and research (De Koning (2006), Schweikhart and Allard (2009)). Nonetheless at a meta-level science can be described as process with selected observations, hypothesis forming, development of scientific experiments to test these hypothesis and generation of new theories or rejection of theories (Fig 1). Also at a lower level, the work of PhD-candidates for a large part consists of repetitive tasks that could possibly be optimised. Focus on process improvement using
industrial improvement methods such as LSS might therefore pose a viable opportunity for science reform. The industrialization of science can be viewed as a conversion of artisan methods to more efficient, cost-effective, streamlined systems for the delivery of products or services (Heskett, Sasser and Schlesinger (1997), Levitt (1976), Schweikhart and Allard (2009)). The effectiveness of these methods in healthcare has recently been proven, results in science are more preliminary (De Koning (2006), Schweikhart and Allard (2009)).
The question remains whether methods that focus on process improvement and reduction of waste that
have been effective in industry can be applied to a research environment. The mere complexity of the research process itself indicates that opportunities for efficiency improvement must exit. Consequently in this thesis, the objective was to assess the efficiency of the PhD-graduation process defined as the time until PhD-graduation and to assess which factors impact this process outcome. A process-improvement project was undertaken at the Academic Medical Centre at
Figure 1. Abstract depiction of the scientific method
7
the University of Amsterdam in collaboration with the local graduate school using the LSS methodology.
8
II.
Framing
This chapter describes the concepts and frameworks that were used to study the case used in this thesis. The choice of the setting for the case is described in the first paragraph, the methods for continuous improvement and case analyses are discussed in the second paragraph. Lastly governance models with different levels of control and hierarchy are described and compared.
A. Setting
The Academic Medical Centre (AMC) is the second largest university medical centre in the Netherlands with 438 scientific staff members and yearly about 220 PhD graduations spread over 10 divisions and many more departments. Eighty percent of research in the Netherlands is performed at universities, the AMC has a 19.8% share of university medical centre’s scientific publications included in the Centre for Science and Technology Studies analysis (AMC (2015)). All types of scientific research including basic science, translational science, clinical science and social sciences are performed at the AMC. The complexity and size of the organization is representative of academic science in the Netherlands providing opportunities for externalization of the projects results.
B. Methods
B.1 Continuous improvement
Organizations aim to continuously improve their key processes to maintain their competitive advantage over other companies, the same goes for science where innovation can be accelerated if the supporting processes themselves are improved (Schweikhart and Allard (2009), Kaplan and Norton (2004), Dekker (2014)).
The case was studied using the LSS improvement method consisting of the Lean manufacturing (short: Lean) tools and Six Sigma methodology. Lean focuses mainly on the identification of ways to streamline processes and reduce waste, by elimination of non-value adding activities, enabling customer centric production of any good. Six Sigma predominantly aims to make processes less variable and more precise through the application of statistical methods. The approach used in Six Sigma resembles that of medical practice where relevant information is gathered to come to a diagnosis. After diagnosis, treatment is proposed and implemented, eventually the patients’ health is monitored and new measures are taken if needed.
9
A rigorous step-wise approach is part of the methodology creating familiar but also strict guidelines for project management. LSS is mostly used in industry but more recently also in healthcare and science (De Mast (2012), De Koning (2006), Schweikhart and Allard (2009)).
The problem solving strategy of LSS distinguishes five phases, define, measure, analyse, improve and control (Bisgaard (2009), De Mast et al. (2012)). The strict order of steps guides the black belt through the process, increasing the chance of success. In the define phase, a charter is created that includes a cost-benefit analyses. Part of the define phase is the formation of a project team, including a champion who bares end responsibility for the project and the affected process(es), a (master)black belt or yellow belt, executes the project and orange and yellow belts support the project if needed. The black belt (BB) reports to the champion, to keep track of progress and to establish whether other projects are affected by the current project. In the measure phase, baseline data are gathered and preliminary analyses are performed. The problem is defined in quantifiable terms using critical to quality (CTQ) characteristics (Furterer (2004)). Then in the analyse phase the process characteristics are specified further, baseline CTQ data are measured and possible relationships between the process variables and the predefined outcomes are described. If the diagnosis is completed, based on the results of the analyse phase improvements are proposed and implemented. A control plan is written to monitor the process outcome and to support continues improvement (De Koning (2006), De Mast et al. (2012)).
B.2 Analysing methods
Several techniques are used for the numerical and graphical description of the current process performance, the required process performance and factors that impact the process performance. A process that has limited variability performs reliably, limited variability and noise increases the chance of discerning patterns that can be linked to assignable causes providing opportunities for improvement (De Mast et al. (2012).
The control chart provides a graphical overview of the CTQs variability and average performance and allows for discerning patterns that guide further analysis of assignable causes that impact the process performance. Current process performance is quantified by the lower and upper control limit (LCL/ UCL). Requirements for the process performance are based on the requirements of the customer and the organization and are objectified by the lower and upper specification limits (LSL/ USL) or Service Level Agreement (SLA). Comparing the
10
current performance (LCL/ UCL) against the LSL/ USL provides insight in the eventual need and opportunities for process improvement.
In addition to GEMBA-walks qualitative data from literature, interviews and panel-discussions and complaint/ support data, can provide information on causes for process performance variability. The collected data can be used to create a survey that acts as an indirect measure to assess the impact of influence factors on the CTQ by use of correlation analyses between the CTQ and included survey items (factors). Comparison of performance of the CTQ in groups of respondents that strongly disagree with those that strongly agree with a survey item (factor) provides insight in the potential impact of the factor on the CTQ. An Out of Control Action Plan (OCAP) can be formulated to provide information on how to act if a process is out of control.
C. Empowerment as a measure to maintain control
To assure continuity of the business, organizations need to continuously attain and exploit new resources and adapt to changing circumstances (The resource based firm) (Fligstein, N. (1990), Pfeffer, J. Salancik, G.R. (2003), Strikwerda (2014)). Organizations that innovate their processes, improve their value proposition and satisfy their customers more easily (Kaplan and Norton (2004)).
Science has long been typified by high individual liberty and goal oriented professionals. In response to external regulations and societal pressure organizations have increased the number of internal regulations (tight control) as a way to remain in-control creating mechanistic organizations. Mechanistic organization are characterized by highly centralized authority, formalized procedures and practices, and specialized functions. Although often more easy to manage, mechanistic organisation have difficulties coping with change (Prahalad & Bettis (1986), Christensen (1997), Burns and Stalker (1961)). It can be argued that mechanistic organizations use of tight control actually leads to reduced control since it reduces the ability of employees to innovate and play into the customer’s needs reducing the company’s ability to adapt to change (Strikwerda (2014)).
Organic forms of organization as compared to mechanistic organization have substantial benefits in terms of flexibility of the organization and personnel satisfaction. In organic organisations professionals are given space to make their own decisions within the limits and goals of the organization (Strikwerda (2014)). These organizations empower their personnel
11
and can be more successful since they make effective use of the local knowledge of the front-line worker and increase alignment with the customer need. Organic structures have greater sensing capability and process and distribute information faster, resulting in an increased ability to adapt to changes in the environment.
The need for the line-workers discretional freedom (empowerment) is recognized by the Netherlands scientific council for government policy as a measure to increase the problem solving capacity of the government (Meurs 2014). Personnel can be considered empowered if the internal organisation provides rights to the individual to initiate projects and make decisions based on the expected contributions to the organizations goals (Strikwerda (2014), Prahalad and Bettis (1986)). An organization that is capable of creating empowerment successfully is in-control due to its ability to sustain itself by adapting to changes in its environment (Strikwerda (2014)).
The following prerequisites can be defined for effective empowerment in an organic organization ((Prahalad & Bettis (1986), Christensen (1997), Burns and Stalker (1961), Strikwerda, (2014)):
- The goals and mission of the organization are clearly defined at the organizational and local level. A value hierarchy is defined guiding the individuals decision making process. - Information is shared openly. Information is available for every employee to make an
informed decision and to learn from the positive or negative outcome of this decision. - Performance of the individual employee is measured and its contribution to the
12
III.
Case description
The AMC is the University hospital at the University of Amsterdam. Next to its hospital function, educational programs in life sciences are provided and academic research is performed. PhD-candidates and academic staff report that they are often confronted with the complexity of the organization and the bureaucracy that is involved in research. Employees describe the AMC culture as rigid systems and processes, lack of long-term planning, ‘kingdoms’ (AMC Strategie 2011-2015).
The AMC has a division structure in which departments are set in divisions. Supporting functions are organized in services, directorates and support groups. The organogram provides further information on the organization structure.
13
Scientific research in the AMC is concentrated on fundamental translational research and clinical research. Research is organized in the AMC Research institute which is governed by the Research Council which advices the board of directors of the AMC. Furthermore the graduate school and the board of graduate schools are involved in teaching activities, research facilities, evaluations and generation of strategy in collaboration with the dean of the faculty of medicine. Research is organized around research themes that form the main strengths and topics for further development for the organization. But a high degree of freedom for principal investigators to organize their research and choose their research topic remains especially if the investigator has been granted funding by external organizations. The principal investigators primarily answer to their respective supervisors organized in the department-division structure which does not directly follow the research themes structure. Recently the following network institutes were formed: Amsterdam Neuroscience, Amsterdam Public Health, Amsterdam Cardiovascular Research, Amsterdam Reproduction and Development, Amsterdam Cancer Research, Amsterdam Movement, Amsterdam Infection and Amsterdam Immunity and Metabolic and Amsterdam Gastro-intestinal diseases.
The AMC has 438 full-time equivalent scientific staff members and about 1500 PhD-candidates that continuously work at the AMC. Yearly about 220 PhD-PhD-candidates graduate. The PhD-program starts officially when registered at the AMC graduate school and is finalized when a thesis is successfully defended against a committee of professors and doctorates. Several phases and activities can be distinguished as part of the PhD-programme including learning activities, formulating hypotheses, designing studies, running clinical/laboratory experiments, data-cleaning and analysis, reading and writing scientific papers, presenting at (international) meetings, organizing meetings, clinical work etc. PhD-candidates often research groups of variable size and are provided supervision by up to two promotors, a professor in the field, and up to two (daily) supervisors.
The goal of the project was to improve the efficiency of the PhD-graduation process defined as the time between employment and graduation. For this project a requirement of €100,000 yearly benefit was predefined by the business school. The project was officially started November 2016 and was finalized 31st of January 2017. The case methodology and the results of the project are described based on the LSS DMAIC phases.
External supervision for use of the LSS methodology in this project was provided by prof. dr. R.J.M.M. Does, MBB since LSS is not used as a general method for improvement projects at the AMC.
14
A. Define
In the define phase the project is identified and described, and the scope of the project and anticipated benefits are stated. The measurable outcomes are objectified using critical to quality descriptions (CTQ). Finally the process involved is described and current state evaluated. Also the organization of the project is objectified.
The to be improved process are all activities between time of enrolment as a PhD-candidate and successful thesis defence. A set minimum of three peer reviewed first author publications are required to qualify for thesis defence. The focusses on full-time PhD-candidates since this group represents the majority of PhDs and the causes for graduation delay are expected to be less heterogeneous in this group than with part-time PhD-candidates.
The CTQ to be improved is the time between enrolment at the graduate school and graduation. The efficiency of the candidates graduation is secondary to quality of the PhD-candidates scientific output. As such project improvements that can be expected to negatively impact the PhD-candidates learning experience or quality of scientific output will not be implemented. Control variables could be the number peer-reviewed publications per graduation, the combined citation impact of these publications and the quoted learning experience of the PhD-candidate. The process to be improved is given in a suppliers, inputs, process, outputs, and customers (SIPOC) chart. The projects SIPOC can be found in the manuscripts appendix. The relation between the strategic focal point, the project objective and the CTQ is depicted in the CTQ Flow Down.
Figure 3. CTQ (critical to quality) flow down chart
Strategic Focal Point: Research output
Project objective: Increased revenue Faculty of Medicine
15
Both financial, scientific and human benefits can be expected as a result of the process improvements. The faculty of medicine is awarded a fee per graduation. A reduction of time until graduation increases the associated cash-flow of the faculty. Potentially sustained increases in the revenue can be expect if efficiency improvements can be reached resulting in more publications, funding and/ or graduations per year. A reduction in PhD graduation would be in line with the request of over half (52%) of PhD-candidates which would like to continue their career in medicine faster. Elimination of waste and rework in the graduation process might increase the PhDs work pleasure. Finally society could benefit if more PhD-candidates graduate in a shorter period of time and consequently less money is spent or productivity increases.
The project deliverables were discussed with the main stakeholders including the dean of the faculty of medicine (Champion), the director of the AMC Graduate School for Medical Sciences and the former chair of the board of graduate studies. The BB collected the data, performed the analysis and wrote the report. The BB reported 3 times to the head of the graduate school, and 4 times to the MBB and preliminary results of the project were discussed with the champion. The project set-up was extensively discussed with the project members.
Figure 4. Project organization and communication structure
B. Measure
In the second phase of DMAIC, measure we measure the process performance in its current state in order to understand the problem. Furthermore in this phase it is checked whether
16
sufficient data can be collected. Resulting in a focussed problem statement (De Koning and De Mast (2007)).
Graduation of full-time PhD-candidates at the faculty of medicine takes a median of 4.4 years which is significantly longer than the advocated upper specification limit of four years and varies considerably between graduate candidates, departments and universities indicating opportunities for process improvement. A reduction of 0.4 years PhD-graduation towards a median of 4.0 years would results in a 50% of candidates graduating in 4 years, i.e. within USL/SLA. This would result in a single increase in cash flow of 5.63 million euro taking into account that the faculty of medicine receives a fee of €93000,- per graduate and currently about 220 candidates graduate each year. Substantial additional benefits could be valorised if increased efficiency results in higher number of graduates per year and increased scientific output (formula 1).
𝐹𝑜𝑟𝑚𝑢𝑙𝑎 1, 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑏𝑒𝑛𝑒𝑓𝑖𝑡 𝑟𝑒𝑎𝑙𝑖𝑠𝑎𝑡𝑖𝑜𝑛: (220 ∗ €93000 ∗ 4.4
4)/ 4 years = €5.626.500,-.
PhD-candidates are required to enrol before the start of their PhD, the graduation date is registered after receipt of a formal confirmation of the graduation date by the University bureau of the beadle. This formal data are required for reporting at the VNSU and assessment of the faculties scientific performance. A data sample of 20 students was compared against self-reported PhD-duration, this indicated high validity of the data. GEMBA walks did indicate that a small minority of candidates start their PhD some time before registering at the graduate school. As such the current data might underrepresent the duration of graduation. This can be expected to result in shorter PhD duration and consequently benefit the faculty financially.
The following forms of data-collection were performed to gather potential influence factors. 1) literature review: from literature potential data on potential influence factors were gathered. Using these preliminary data themes were defined that point towards influence factors. 2) data available from the graduate schools exit survey were analysed to assess the association between potential influence factors and the CTQ. Factors that were associated with variability in the CTQ with a significance of p<0.2 were selected. 3) subsequently process users and experts, including PhD-candidates, scientists and managers were interviewed or invited for panel discussions. The discussions were semi-structured, open discussion was stimulated. The BB asked for clarification if needed, but remained neutral and did not participate in the discussions.
17
All conversations were recorded, the frequency of influence factors being mentioned were recorded by the BB. Factors that were mentioned at least two times in separate interviews were added to the list of potential influence factors. 4) Anonymised PhD Candidate Advisor individual support meetings data were analysed. All collected factors were ordered according to the predefined themes from the literature, new themes were added if needed. Factors that did not seem influencable in any way were excluded from the list. Table II. Provides an overview of the data collection plan.
Table I. Data collection plan
P OT EN TI AL IN F LU EN C E F ACTORS
Source Data selection
1. Literature PICO/ Relevance
2. Exit- survey Association between factor and CTQ with p<0.2 3. Interviews-panel
discussions
Factors that were mentioned at least two times in separate interviews
4. PhD Candidate Advisor support meetings report
Factors that were mentioned at least two times in separate support meetings
A survey was created using aforementioned data sources and data selection methods. The survey was checked by a native speakers and was finally sent out to of PhD-candidates that had finalized their PhD in the past year or had a PhD-graduation data recorded (< 6 months before graduation) using a digital form. All factors were assessed using a 5 point likert scale (from strong disagree to strongly agree + does not apply).
18
IV.
Results
In the next three phases analyse improve control of the DMAIC the results of the process analysis, influence factors and improvement measures are presented. Finally a control plan is discussed to maintain (potential) improvements. Aiming to reduce waste and eventually limiting time between enrolment and graduation of the candidate.
A. Analyse
In this phase, the collected data is analysed using statistical analyse tools to identify root causes, the diagnosis is made. Table II and Figure 5 provide an overview of the current process performance.
Table II. Current process performance with compliance with SLA Overall n=282 Part-time n=117 Full-time n=165 PhD-duration* 4.5 5.8 4.4
% Graduation within four years 27.7 29.7 24.8
Source AMC Graduate School 2014-2016. n= number. * Median time in years between registration as a PhD-candidate and graduation.
19
The data collection phase including literature search, interviews and panel discussions and survey analysis resulted in a survey containing the following themes. The full list of survey items are given in the manuscripts appendix.
General, e.g. the age of the PhD-candidate
Questions on your PhD research work, e.g. the type of research Induction program, e.g. the introduction at the department
Supervision, e.g. the frequency and the quality of the supervision Skills development, e.g. the project management skills
Intellectual climate, e.g. the number of PhD-candidates at the research group Research Infrastructure, e.g. the availability of biostatistical expertise
Goals and Expectations, e.g. the number of publications that are required
The Survey was sent out to 209 PhDs that graduated in 2016 and 80 with a registered graduation date within 6 months. Responses were received from 112 individuals of which 96 full-time and 12 part-time candidates with a resulting response rate of 39%. Data from full-time candidates were analysed. The median duration of graduation of the full-time participants of this survey was 4.7 years which is in line with the results of the graduate school. Fifty-two percent of PhD-candidates would have wanted to graduate faster. Those that wanted to graduate faster differed significantly in graduation duration (p=0.001, median 5.2 years versus 4.4 years, more for those that did want to graduate faster). Graduation duration differed significantly between departments.
20
Survey data of 96 full-time PhD-candidates was analysed using an association analyses between variation in the factor and variation in the CTQ. Factors with a p-value of 0.05 or less were reported as influence factors. To get an indication of the possible impact of the influence factor on the CTQ, results from participants that reported to totally disagree were compared to those that totally agreed with the survey item. Table IV. Provides an overview of the influence factors that significantly affected the CTQ. It should be noted that it is unlikely that these factors have cumulative effects.
Table III. Factors with significant impact on PhD duration
Influence factor Significance
(p-value)
Potential impact on CTQ*
1. My project management skills were sufficient to perform my tasks Ω
0.031 7.9 – 4.6 = 3.3 years
2. Employment at division B (versus the mean of other divisions) (fig 5)
0.008 4.7 – 2.7 = 2.0 years
3. Attaining a PhD within 4 years is considered the standard at my departmentΩ
0.021 4.7 – 3.4 = 1.3 years
4. Age at start PhD-program (22 vs 30 years) <0.001 5.64 – 4.36 = 1.28 years
5. Percentage of your thesis that was accepted or submitted before the end employment? (0 vs 100 %)
<0.001 5.35 – 4.55 = 0.8 years
6. What is the number of PhD-candidates in the research group that you consider yourself part of? (0 vs 25 candidates)
0.014 4.95 – 4.45 = 0.5 years
7. My supervisor/s helped to structure and plan my activities to meet longer term goalsΩ
0.024 4.4 – 4.3 = 0.1 years
Ω
Strongly agree versus strongly disagree. * Median difference.
B. Improve
In the improve phase, the influencable factors with the most impact are selected. Their effects are modelled on the CTQ. Improvement is identified and prioritized.
21
The designed improvement initiatives were reviewed and the following topics selected.
1) Project management skills
The work of PhD-candidates is complex and requires a vast amount of skills. During panel discussions and interviews, project management skills were recognized as an important success factor for PhD-candidates. It was mentioned frequently that, although a project management course exists at the graduate school, the frequency of the course and the number of available places is limited. Furthermore it was mentioned that PhD-candidates realize the relevance of the course to late.
Advised course of action: increase the availability of project management course. Evaluate the quality of the course. Consider including the course in the PhD-candidates induction program.
o Responsible party: graduate school
2) A graduating fast culture
Cultures between departments vary significantly in terms of accepted PhD-duration and measure taken to reduce PhD-duration. The department of surgical specialties discusses the graduation duration and date frequently. Graduating fast is the standard here since it is considered important to start specialty training before the age of 30.
Advised course of action: include the graduation date and thesis planning at a meta-level in formal year discussions.
o Responsible party: dean via heads of departments
Make graduation in four years the standard and communicate this in the organization. o Responsible party: research council
3) Older age of candidates, recruitment
Higher age is associated with faster graduation in full-time graduates. Currently no supportive recruitment procedures are in place for selection of PhD-candidates. Except for age other human factors can also be expected to influence the PhD-projects duration.
Advised course of action: discuss the opportunities for recruitment support with department of human resources AMC. Inform principal investigators of relevance of
22
effective recruitment procedures and support recruitment and selection of PhD-candidates.
o Responsible party: HR AMC
4) Finalizing your PhD during employment
The minority of PhD-candidates finalized their manuscripts within employment while writing the last manuscript outside of the faculty without employment and/ or next to another job significantly delays the graduation. Currently the local PI does not have a financial incentive to help faster graduation.
Advised course of action: if the faculty aims to increase the speed of graduation it must put in place (financial) incentives for PIs to counterbalance the potential extra salary spent if PhDs graduate within employment time.
o Responsible party: dean of faculty of medicine.
5) The research group climate and size
PhDs that graduate in a large research group graduate faster. Multiple factors are at play, interviews and panel discussion indicated the importance of a good induction program including the availability of a predecessor that the candidates works with on its first research project(s).
Advised course of action: consider stimulating the formation of larger research groups. The research schools might play a vital role in the exchange of (practical) knowledge between PhD candidates.
o Responsible party: research council.
C. Control
In the final control phase a plan is presented to improve process control and the project is closed. The actions that lead to the process improvement need to be implemented in the process so that the improvements are maintained and measures are taken if the process is registered to be out of control. In the end the financial controller verifies whether the financial goals of the project are met. The BB is then discharged of the DMAIC improvement plan.
The responsibly for the process monitoring and improvement of the process will have to be delegated to one of the partners and formally recorded. It would be advised to delegate this role to the research council. The graduate school and medical library could aid in the collection of
23
the required data. The CTQ (duration of graduation) should be continued to monitored and checked against the norm (graduation < 4 years). Furthermore the number of publications published at pubmed should be recorded per PhD-candidate over time and compared to a publication timeline. Available data could be used to build a prediction model that assesses the chance of PhD-graduation within four years based on the number of publications over time. Based on pre-defined risk cut-offs the PhD-candidate and PhD-supervisors receives an automated notification if graduation within 4 years is at risk. At high risk this could be escalated to the department head. PhD-candidates at risk of delay need to be provided support by the graduate school early in the process to increase the likelihood of PhD-success.
An Out-of-Control Action (OCAP) needs to be formulated which can be viewed as the companion to the control chart. The OCAP guides the actors reaction´s to the out-of-control situation. The OCAP would preferably be refined by the research council that would be advised to analyse the data and propose improvement projects.
Table IV. Data monitoring and preliminary out of control action plan.
PRO
CE
SS
C
O
N
T
RO
L
Party responsible for data collectionAction
1. Graduation duration
Graduate School RC, propose measures to reduce graduation duration 2. No of 1st-author publications per candidate over time Graduate school in collaboration with the medical library
GS, contact candidate and supervisors if projected graduation duration is above SLA.
GS, provide support to increase PhD-programs support.
Although the improvement results cannot be measured yet due to the long time between improvement and result, based on the improvement actions it can be safely said that it should be possible to achieve a reduction of 0.4 years (9.1%) graduation duration. Resulting in the calculated €5.626.500,- increase in cash flow which by itself represents over €200.000,- at an annual percentage rate (APR) of 4%. Substantial additional benefits could be valorised if increased PhD-work efficiency results in higher number of graduates per year and increased scientific output.
24
V.
Summary, discussion, conclusions and recommendations
A. Summary
This case-study at the University of Amsterdam Academic Medical Centre, investigated whether use of the Lean Six Sigma methodology could improve the efficiency of the PhD graduation process of full-time PhD-programs. The project aimed to reduce the time between enrolment and graduation of the PhD-candidate. A project team was formed including the dean of the faculty of medicine (Champion), principal investigators and a black belt. The project team aimed to reduce the graduation duration from the current median 4.4 years to a median of 4 years, increasing the efficiency of science.
A reduction of 0.4 years PhD-graduation towards a median of 4.0 years would results in a 50% of candidates graduating in 4 years, i.e. within USL/SLA. This would result in a single increase in cash flow of 5.63 million euro taking into account that the faculty of medicine receives a fee of €93000,- per graduate and currently about 220 candidates graduate each year. Substantial additional benefits could be valorised if increased efficiency results in higher number of graduates per year and increased scientific output.
A qualitative analysis of the literature, exit-survey data and panel discussions with PhD-candidates and principal investigators provided the basis for a survey created to assess potential process influence factors. A quantitative association analysis of the survey data of recently graduated PhDs indicated a significant reduction of graduation duration with: 1) candidates with a higher level of project management skills, 2) a graduating fast culture at the local department, 3) older age of the candidate at the start of the programme, 4) PhD-programmes that were finalized within employment, 5) an intellectually stimulating climate and larger research group size. The results of this project and resulting improvement proposals provide opportunities to reduce the current PhD-program duration significantly.
An effective process performance monitoring and out of control action plan was formulated including clear role definitions and process end-responsibility. Measure include monitoring of PhD-gradation duration over time and implementation of an early warning system that informs the organization, the PhD-candidate and supervisors in case of expected graduation duration longer than four years. This allows for early support and intervention if needed increasing the likelihood of PhD-success.
The case study results confirm the effectivity of LSS improvement methodology as a measure to improve efficiency of science.
25
B. Discussion
Improvements in the efficiency of science could increase the competitive advance of the Netherlands over other competing knowledge economies ((Michael Porter) 1969 Drucker (1988)). Universities play a vital role in knowledge generation and innovation in the Netherlands. In line with the aim of the VSNU to improve the efficiency of science and knowledge valorisation (VSNU, “Onderzoek En Valorisatie”) measures to improve the efficiency of science at universities could benefit the Netherlands as a whole. PhD-candidates run the majority of scientific experiments. This case study aimed to improve the PhD-graduation process resulting in process improvement actions that could result in significant financial and soft benefits.
Science exists due to the generation of knowledge, innovations and translation to other fields of expertise. It seems straightforward that an organization that fosters these skills in his personnel not only generates knowledge about the world around us, but also about the work process itself and how it can be perfected. The work environment should enable the professional to achieve their goals and continuously improve its skill and knowledge. Organizations that do continuously improve their processes and that enable their personnel to experiment and innovate are more successful (Kaplan and Norton (2004), Prahalad and Bettis (1986), Christensen (1997), Burns and Stalker (1961)). As such we need to evaluate practices in continuous learning and organizational design and empowerment with the learnings of the case-study in mind to pursue learnings for more efficient knowledge generation in science.
B.1 Continuous improvement
Vast amounts of tax payers money is spent on science leaving the academic community with the responsibility to spent this money wisely and use it efficiently. Employees experience waste and inefficiencies in their daily work, thus opportunities for improvement exist around us every day but only with the right culture and personnel with the right tools these opportunities can be turned into activities that lead to streamlined processes. Process improvement can increase the customer value proposition (Kaplan and Norton (2004), Kaplan and Porter (2011), Slater (1997)).
This case-study is an example of an improvement project and use of an improvement methodology that can support a culture of continuous improvement and learning. The report indicates that the LSS improvement methods can be used for improvement in a
non-business-26
setting, in-line with previous reports (De Koning (2006), Heskett, Sasser and Schlesinger (1997), Levitt (1976), Schweikhart and Allard (2009)).
To allow for continuous improvement a culture needs to exist in which mistakes are taken as opportunities for learning and improvement instead of blaming and shaming. Such a culture provides opportunities for efficiency improvement and growth (Dekker 2006). In this case Lean Six Sigma was used as a method to analyse current process performance, find influence factors and improve the process (De Koning (2006), De Mast (2012)). Learning from mistakes in science is more difficult than in manufacturing since the activities are knowledge intensive, highly heterogeneous and the time between production and quality control (publication) is long. To improve in such an environment high knowledge of process thinking including the concept of influence factors, variability and specification/ control limits is required. It’s the processes that link the goals of the organization and the outcome of its’ activities. Improving these processes through plan do check act (PDCA) cycles and including these improvements in the standard operating procedures lays the foundation for future improvements allowing for continuous improvement. As such process deserve our fullest attention (Ghoshal and Bartlett (1999), Kaplan and Norton (2008)). But improvement without clear vision and goals lead to nothing as indicated by W.E. Deming ‘it is not enough to do your best; you must know what to do and then do your best’.
Clear goals and mission of the organization are a prerequisite for any organization. This should include a value hierarchy since it guides the employees in the decision making process (Collins and Porras (1996)). Goals should be defined and communicated at an overarching business level but also at an departmental and individual level (Strikwerda (2014)). The AMC formulates in its’ 2011-2015 strategy that it aims to excel at translational research and aims to focus its activities to create mass and synergies. It mentions in the strategy that it will focus more on awarding (fees) for finalizing graduates thesis. The interviews and panel discussions provided no evidence for any incentive other than the composite measure of scientific performance mainly consisting of the Hirsch factor. PI’s performance is ranked, but no performance goals are communicated. This lack of clearly defined goals at meta level and lower levels hampers the effectiveness of the business policy. This is in line with the perception of AMC employees describing that they perceive a lack of clear long term goals (AMC Strategy 2011-2015). Even if goals are set and improvement measures are taken to achieve these goals, without measurement of the outcomes an organization cannot continue to improve.
27
To allow for organizational learning performance measurement needs to be executed and has to be included in the business control cycle (Slack et al. (2015)). Then key processes can be monitored and if out of control can be steered using PDCA cycles and OCAP. If needed processes can be improved using Kaizen improvement actions and DMAIC projects (De Koning (2006), Strikwerda (2014)). Using performance measurement, individual and department performance can then be linked to the performance of the business and can be appraised if appropriate.
`Collective learning across the corporation´, a core competence of successful organizations can only be effective if information flows freely in the organization (Prahalad and Hamel (1990)). PhD-candidates perceive a lack of information about practical skills and procedures. No platform exists where PhD candidates can share practical information, protocols and standard operating procedures are hard to find, let alone improved resulting in PhDs finding out the same wheel over and over again. Principal investigators report that ‘how to manage a research group efficiently’ or how to ´supervise a PhD-candidate´ is not discussed with colleagues or between departments. (sub)Departments are ´reported be ran like small kingdoms’ (strategie 2015). If information is not shared, the organization cannot learn from its’ mistakes. (Drucker (1988)).
The aforementioned topics reduce the ability of the organization to learn and improve continuously. The LSS methodology could support the AMC in its intention for continuous improvement (AMC (2015)). The organization currently has no experience with the LSS projects increasing the difficulty of the project. Especially due to the complexity of Six Sigma methods. The experience with Lean could be used to further build the employees knowledge on improvement methods. It would be advised to use Kaizen improvement projects, as part of the Lean methodology for simpler improvement projects and retain LSS for complex overarching projects (De Mast et al. (2012)). Personnel needs to be trained to allow for continuous improvement in the organization.
B.2 Science in an organic empowering organization
In line with the creative and individualistic nature of the origin of science, science has been loosely coupled, i.e. coincided with large individual responsibility and limited regulations. In the past decade regulations, internally and externally enforced, have increased as a measure to reduce risk of failure and to remain in control. It can be debated though whether an increase in regulations leads to better business control (Strikwerda (2014)). A better way to remain in-control as a business is to make sure that the customer is served what it needs, over and over
28
again. The business should support the professional to add value to the processes to create the ideal customer proposition (Kaplan and Norton (2004), Kaplan and Porter (2011), Slater (1997)).
Organically developing organisations that stimulate empowerment can help reduce regulatory pressure without reducing or even increasing business control ((Prahalad & Bettis, (1986), Christensen (1997), Burns and Stalker (1961), Strikwerda (2014)). If the organization aims to reduce the graduation duration this should be transferred in an measurable organizational goal and data should be collected at an organizational and departmental level to assess the process performance. This could include registration of individuals contributions and (financial) appraisal if appropriate.
Overarching goals can only be set if the process (graduation) is recognized as a goal and responsibilities are clearly defined (Strikwerda (2014)). Currently the responsibility for parts of the process is organized by separate organizations. Information is not openly shared by these organizations and goals are not aligned. The responsibility for the process is laid at the local level (principal investigator) without providing the PI the information about the performance of the process as a whole. The organization does not have an overview of the activities that contribute to the process and thus is incapable of improving the process. At the same time the research process is perceived by the organisation as bureaucratic and slow (interviews this thesis, AMC (2011-2015)).
Decentralized organisation of science does not by itself produce the fruit that organic organized organisations can produce. As discussed, overarching goals should be clearly defined, processes defined and responsibility of the process as a whole taken so that the professional can be empowered to reduce waste and share learning experiences with the rest of the organization.
C. Conclusions
This case study shows that an structured analysis of an complex and vitally important process can lead to significant improvement activities that can be expected to improve the graduation process as a whole. If measures are successfully implemented single cash flow benefits of up to €5.626.500,- and reduced financing cost of up to €200.000,- annually can be expected. Graduation of PhD-candidates and the activities that lead to graduation are vitally important to the faculty of medicine. Graduation fees provide an annual income of over €19 million euro from the university and additional efficiency improvements in the research process could raise the number of publications potentially at lower cost.
29
Looking at graduation and the underlying activities as a process instead of through the lens of organizational structures like departments and research lines raises awareness of the need for process goals, performance measurement and improvement activities. Responsibilities need to be clearly defined and performance appraised to achieve optimal process outcomes. To allow for continuous improvement learning information such as failures and best practices needs to be shared within the organization. Then professionals can be provided with the needed freedom to perform their daily activities, signal improvement opportunities and design improvement activities linked to the organizational goals in an environment that supports them to do so.
D. Recommendations:
Faced with an immense debt and destroyed industry after the second world war Japan searched for opportunities to revitalize its industry just as the US searched for an increase in efficiency after the great depression. Both found tools that were combined in the Lean Sig Sigma methodology that helped organizations to recognize and continuously improve key processes eventually resulting in better value for the customer. In the past decade other industries such healthcare have started to adopt these improvement methods. Time has come to review opportunities for process improvement in science.
Provided that process goals are clearly defined at the organizational, departmental and individual level and clearly linked to the organizations mission and vision, professionals can use their expertise to continuously improve processes. Schooling personnel in process analysis and optimization is vital for the success of the process especially since process improvement in science can be extra complex since tasks are variable and knowledge intensive. Performance should be measured and results should be available throughout the organization. The professional or any other line worker should be stimulated to undertake improvement projects and should be helped to disseminate learning experiences through the organization. Information should be accessible to anyone irrespective of function and should be organized over the boundaries of structures such as divisions, departments or research lines. Employees that are tought to analyse and improve, that have insight in de goals of the organization and are stimulated to experiment, are empowered to increase the value and efficiency of science. The LSS methodology can aid in achieving this goal.
30
References
Alberts, Bruce et al. (2014), “Rescuing US Biomedical Research from Its Systemic Flaws.” Proceedings of the National Academy of Sciences of the United States of America 111.16 (2014): 5773–5777.
AMC (2015), “Maatschappelijk Jaarverslag AMC”, Web accessed 12-12-2016 https://www.amc.nl/web/Het-AMC/Organisatie/Kerngegevens/Jaarverslagen.htm.
AMC (2011-2015), Strategie, AMC.
Berger, de Jonge (2005), “Rendement verkend. Succes- en faalfactoren van promotietrajecten aan Nederlandse universiteiten” Zoetemeer, Onderzoek voor Bedrijf & Beleid.
Bergin, Shane et al. (2016), Bratislava Declaration of Young Researchers, website accessed 03-01-2017, http://www.eu2016.sk/data/documents/bratislava-declaration-of-young-researchers-final.pdf.
Bisgaard, Søren (2009), “Solutions to the healthcare quality crisis; cases and examples of lean six sigma in healthcare”, Millwaukee: ASQ Quality Press.
Burns, T. & Stalker, G.M. (1961), The Management of Innovation, Tavistock, London.
CBS statline (2016), “Research en Development; Financiering Uitgaven per Sector van Uitvoering.” CBS statline.
Christensen, C.M. (1997), “The innovators dilemma: when new technologies cause great firms to fail. Boston, Mass.: Harvard business school press.
Collins J., Porras J. I., (1996), “Building Your Company’s Vision” Harvard business review sept-oct.
Council of Graduate Schools (2016), “Ph.D. Completion and Attrition: Findings from Exit Surveys of Ph.D. Completers” Web accessed 12-12-2016 http://www.phdcompletion.org/. De Koning, Henk et al. (2006), “Lean Six Sigma in Healthcare.” Journal for healthcare quality : official publication of the National Association for Healthcare Quality 28.2 (2006): 4–11.
De Koning, H. de Mast, J. (2007), “The CTQ flowdown as a conceptual model of project objectives. Quality Management Journal”, 14(2), 19-28.
De Mast, Jeroen, Ronald J.M.M. Does, Henk de Koning and Joran Lokkerbol (2012), Lean Six Sigma for Service and Healthcare, Alphen aan den Rijn, Beaumont Quality Publications.
31
Dekker, Sidney (2014), The field guide to understanding ‘human error’, Surrey: Ashgate Publishing Limited.
Drucker, Peter F. (1969), The Age of Discontinuity. Transaction Publishers.
Drucker P.F. (1988). “The coming of the new organization.” Harvard Business Review (Jan-Feb).
Dutta, Soumitra, (2012), "The Global Innovation Index 2012: Stronger Innovation Linkages for Global Growth, World Intellectual Property Organization.
Fligstein, N. (1990), “The transformation of coporate control”. Cambridge, Mass: Harvard University Press.
Furterer S (2004), “Implementation of TQM and lean Six Sigma tools in local government: a framework and a case study”, Total Quality Management & Business Excellence, Vol. 16-10. Ghoshal, S. , Bartlett C.A. (1999), “The individualized corporation; great companies are defined by purpose, process, and people” New York: HarperBusiness.
Heskett, J. L., Sasser, W. E., Jr., & Schlesinger, L. A. (1997), The service profit chain. New York: Free Press.
Hoerl, R.W. (2004), ‘One perspective on the future of Six Sigma’, Int. J. Six Sigma Competitive Advantage, Vol. 1, No. 1, pp.112–119.
Kaplan, R. S. and David P. Norton (2004), “The strategy map: guide to aligning intangible assets” Strategy & Leadership, 32 (5), 10-17.
Kaplan, R. S. and David P. Norton (2008), “Mastering the management system,” Harvard Business Review, 86 (1), 62-77.
Kaplan, R. S. and M. E. Porter (2011), “How to solve the cost crisis in health care” Harvard Business Review, 89 (9), 46-64.
Levitt, T. (1976). The industrialization of service. HarvardBusiness Review, September– October, 54, 63–74.
Meurs P.L. (2014), “Van regeldruk naar passende regels. Vertrouwen, veerkrachtverantwoordelijkheid en vrijheid.” Web accessed 06-01-2017,
32
Pfeffer, J. Salancik, G.R. (2003), “The external control of organizations: a resource dependence perspective. Stanford, Calif.: Stanford Business Books.
Porter F. (1985), Competitive Advantage, Creating and Sustaining Superior Performance. The Free Press.
Prahalad, C.K., Bettis, R.A. (1986), “The dominant logic: a new linkage between diversity and performance. Stretagic Management Journal, 7(6), 485-501.
Prahalad, C. K., Hamel G., (1990), “The Core Competence of the Corporation,” Harvard Business Review, 68 (3), 79-91.
Rathenau Institute (2016), "Totale Investeringen in Wetenschap En Innovatie 2014-2020".
Schwab K. (2012), The Global Competitiveness Report. World Economic Forum
Schweikhart SA, Allard ED (2009), “The Applicability of Lean and Six Sigma Techniques to Clinical and Translational Research” J Investig Med. 2009 October ; 57(7): 748–755.
Shewart, W.A., and W.E. Deming (1939), Statistical Method from the Viewpoint of Quality Control. Dover publications Inc. New York.
Slater, S. F. (1997), "Developing a customer value-based theory of the firm". Journal of the Academy of Marketing Science, 162-167.
Slack, Nigel, Alistair Brandon-Jones, Robert Johnston and Alan Betts (2015), Operations and process management; principles and practice for strategic impact. Harlow: Pearson Education Limited.
Snee, R.D. and Hoerl, R.W. (2004), Six Sigma Beyond the Factory Floor, Upper Saddle River, NJ: Pearson Education.
Snee, R.D. (2010), “Lean Six Sigma – Getting Better All the Time.” International Journal of Lean Six Sigma 1.1: 9–29.
Strikwerda, H. (2014), Bespiegelingen over governance, bestuur, management en organisatie in de 21e eeuw, Assen: Koninklijke Van Gorcum.
VSNU (2016), “Factsheet Top 5 Kenniseconomieën. ”
VSNU (2016-2), “Verhouding Vast En Tijdelijk Personeel.” Web accessed 22-12-2016 http://www.vsnu.nl/nl_NL/f_c_verhouding_vast_tijdelijk.html
33
VSNU, Wopi (2017), “Onderzoek En Valorisatie.” Web accessed 10-01-2017 http://www.vsnu.nl/valorisatie.
VI.
Appendix:
A. SIPOC Process description
SIPOC
Suppliers Inputs Process Outputs Customers
Departments PhD-students From student to doctorate Doctor PhD's AMC medical research Principal investigators Learning experience Scientific output Dean
Principal investigators Department
1: Thesis definition 2: Thesis planning 4: Collecting Data 6: Writing & Publishing 5: Processing & Analysis 9: Committee & Assessment 10: Dissertation defence Iterative steps 0: PhD-student registration as employee & other (administrational requirements before work can be started) 3: Study preparation 8: Thesis conclusion 7: Archiving
& Open Data
Appendix Figure 1: A SIPOC chart provides an overview of the process to be improved including its suppliers, inputs, process, outputs, and customers.
B. Graduate school Exit survey (PREQ) themes
Supervision
- 1. Supervision was available when I needed it
- 7. My supervisor/s made a real effort to understand difficulties I faced - 13. My supervisor/s provided additional information relevant to my topic - 17. I was given good guidance in topic selection and refinement
- 21. My supervisor/s provided helpful feedback on my progress - 24. I received good guidance in my literature search
Skills development
- 6. My research further developed my problem-solving skills
- 10. I learned to develop my ideas and present them in my written work - 14. My research sharpened my analytic skills
- 20. Doing my research helped me to develop my ability to plan my own work - 26. As a result of my research, I feel confident about tackling unfamiliar problems
Intellectual climate
- 5. The department provided opportunities for social contact with other postgraduate students
- 9. I was integrated into the department’s community
- 16. The department provided opportunities for me to become involved in the broader research culture
- 22. A good seminar programme for postgraduate students was provided - 23. The research ambience in the department or faculty stimulated my work
Infrastructure
- I had access to a suitable working space
- 8. I had good access to the technical support I needed
- 12. I was able to organise good access to necessary equipment - 18. I had good access to computing facilities and services
- 27. There was appropriate financial support for research activities
Thesis examination process
- 2. The thesis examination process was fair
- 15. I was satisfied with the thesis examination process
- 25. The examination of my thesis was completed in a reasonable time
Clarity of goals and expectations
- I developed an understanding of the standard of work expected - 11. I understood the required standard for the thesis
36
C. Collected influence factors and Developed Survey
General Type Association
G1 Year of birth: Numeric NA
G2 If employed by AMC or AMR, in which AMC division do/did you work?
Cat Significant
G3 When did you start your PhD? DATE NA
G4 When is/was your PhD graduation date (date) DATE NA
Questions on your PhD research work Type Association P1 Did you do your PhD research: full-time/
part-time
NA
P2 Would you describe your PhD research as mainly: Fundamental/ Translational/ Clinical/ Epidemiological
Cat NS
P3 My project was part of a larger research line that has ran for several years
Likert NS
P4 My PhD consisted of mainly high risk projects Likert NS P5 At the start of my PhD, my project goals and
timeline were clearly defined?
Likert NS
P6 My scientific projects were carefully thought through before study execution
Likert NS
P8 How many hours per week do you spent working as a PhD candidate?
Numeric NS
P9 What percentage of your average work week was spent on activities that did not directly contribute to your PhD thesis (e.g. teaching, running errands, clinic work )
Numeric% NS
