If the SCBA-plus approach is to be established as the standard way in which ESA evaluates the economic benefits of public investments in space, this approach and the activities it constitutes become part of the normal operations executed by ESA. Although at first the application of
38 Financial Analysis does not need data on investments related to the space programme and economic statistics for the space sector and for all sectors.
39 If indirect effects are estimated by applying Computable General Equilibrium analysis, Input/Output tables and economic statistics at a relatively detailed level are required as well.
40 See Appendix F for a description of how ESA establishes an investment programme.
SCBA-plus will likely be organised in an ad-hoc fashion to evaluate a specific (set of) programme(s), in the long run, the steps as presented in section 5.2 will need to be integrated in the day-to-day activities undertaken by ESA, and in the way ESA communicates with its stakeholders and other external entities. This way a body of knowledge can be built up, which enables better analysis over time.
The first three steps of the SCBA-plus approach are (see subsection 5.2.1):
Step 1. Definition of aim and scope of evaluation
Step 2. Identification and characterisation of investments
Step 3. Identify assessment criteria: costs, possible effects, other criteria, and actors
Although in this report we focus on the ex post analysis of investments in space, a first iteration of the above three steps could be executed around the time a new investment programme is established. At that moment the basic input to execute these steps is already available. Such attention for economic benefits early-on in the establishment of an investment programme also helps to flag the start of data collection over the lifetime of the respective investment programme.
Here, a distinction between the application of the SCBA-plus method to ESA’s own investment programmes and the application to other public investments in space is to be made, since it is likely that ESA will have a much more detailed insight into its own programmes than into third party programmes. One can assume that the amount of control that ESA will have over an overall assessment process like the SCBA-plus approach, is much lower (if at all present) for public space investment programmes that are executed by other entities than for those programmes executed under the ESA umbrella. It is therefore advised that ESA focuses its efforts in this area on its own investment programmes and communicates to external entities both on the steps taken and on the results found. In other words “to lead by example”.
When looking at the steps of the SCBA-plus approach in more detail, it can be observed that the first stage in establishing an investment programme focuses on defining the aim and scope of this programme. Step 1 of the SCBA-plus approach (“definition of aim and scope of evaluation”) provides an opportunity to also consider at an early stage the options to evaluate to what level the original aim and scope (the goals) are achieved over the course of an investment programme. The SCBA-plus approach will in itself provide a generic set of assessment criteria which may even help in identifying new goals for the investment programme under consideration. It also provides the opportunity to include in the investment programme itself agreements and instructions on for instance data collection efforts or feedback on project results.
By integrating the SCBA-plus approach at an early stage in the specification of an investment programme, the identification and characterisation of the investments – step 2 of the SCBA-plus approach – is made more easy. Stakeholders in the investment programme (for ESA in particular the Member State delegates) are closely involved in the definition of the programme and can bring their inputs more readily to the table then at a later stage. Also, the identification of assessment criteria such as costs, possible effects, other criteria, and actors – step 3 – is best closely linked to the definition of programme details. Again, the SCBA-plus needs and
PROPOSED METHODOLOGY: SCBA-PLUS 79
requirements in themselves may help in establishing the investment programme, for instance by bringing forward effects or actors that may otherwise have been overlooked.
Considering the continuous nature of most space programmes (for instance the ESA General Science and Technology Programme which has been running for many years and is updated regularly) it is advisable to schedule individual assessments at regular intervals (for instance bi-annual) rather than at the end of a specific programme. After each interval, the results of the steps 1, 2 and 3 can be revisited and then the remaining steps of the SCBA-plus approach are to be undertaken:
Step 4. Quantifying effects (SCBA/MCA)
Step 5. Valuation of effects (SCBA/MCA)
Step 6. Calculating outcomes (SCBA/MCA)
Step 7. Sensitivity analysis (SCBA/MCA)
Step 8. SCBA-plus: presenting the combined results, including non-monetized effects
Step 9. Evaluation
It is likely that at least some of the specific assessments included in the overall SCBA-plus approach yield results that are more widely applicable than for a single investment programme alone. Therefore by scheduling an SCBA-plus assessment at regular intervals, it can cover more than just a single programme in an efficient way. Furthermore, this allows for a comprehensive reporting on public space investments whereby both the individual programmes and the public space investment at ESA or even European level can be addressed and compared. On a basic level, regularity in this type of assessments provides a clear long term perspective and supports the buiding up knowledge and – not in the least – the results in the overall management processes of ESA.
When scheduling the execution of the SCBA-plus assessments at regular intervals, the data collection efforts can be aligned to these intervals and become part of ESA’s business routine.
This build-up of a body of knowledge is of great importance to ensure continuity over time of both the collected data and the assessments themselves. Much of the data that is currently already available within ESA is collected on an annual or multi-annual basis. For instance the EMITS Entity Questionnaire is to be updated annually by all entities doing business with ESA, providing input on e.g. employment, turnover etcetera. Also programme status updates are provided to the ESA Member State delegates at regular intervals. If other entities (such as the OECD, European Union, national space agencies or Eurospace) decide to adopt a similar approach to evaluate the space investment programmes under their control, regular reporting intervals facilitate the exchange of economic data with these entities.
PUBLIC INVESTMENTS IN SPACE 81
6 Implementation
If you cry "Forward,"
you must make plain in what direction to go.
Anton Chekov In the short term, a simplified version of the ideal methodology presented in chapter 5 can be applied. Another step towards better evaluation is to collect better data, in particular on the composition, economic relations and impacts of the space sector, and on complementary investments made by other actors. Also, ESA could improve its own data and use these data not only for decision making and administrative purposes, but also for ex post evaluation.
For the first follow-on activity it is advisable to apply the SCBA-plus method to two of the current ESA programmes.
This chapter starts off from the current situation of less-than-complete data availability. This deviates from the ‘ideal’ situation that has been assumed in chapter 5. The SCBA-plus methodology can still be applied, be it in modified form because of missing data. A data collection effort is needed in order to get to (or at least in the direction of) the ideal situation that has been assumed in chapter 5.
Section 6.1 describes what the SCBA-plus methodology looks like in the current case of less-than-complete data. Section 6.2 defines the data collection effort for the long term, and sketches possibilities for the medium term. Section 6.3 turns to improvements in ESA’s own data. Section 6.4 concludes with the first steps that can be taken.