• No results found

The Usage of e-Government Services in the Baltic States: How the Usability of Government Websites Contributes

N/A
N/A
Protected

Academic year: 2021

Share "The Usage of e-Government Services in the Baltic States: How the Usability of Government Websites Contributes"

Copied!
106
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

The Usage of e-Government Services in the Baltic States: How the Usability

of Government Websites Contributes

Claudia Scandol s1341707 Leiden University

MSc Public Administration: International and European Governance Master’s Thesis

Supervisor: Dr. Sarah Giest First Reader: Dovilė Rimkutė Second Reader: Nadine Raaphorst

(2)

The Usage of e-Government Services in the Baltic States 2

Abstract

The internet has changed the way people interact with each other. With the increase in communications between people and their governments, it has become increasingly important that online government services are provided in an easy to use way to different cross-sections of the population. In order for governments to be able to provide their services in the most effective way, it is vital to understand the factors that contribute to usable websites. This thesis looks at the bottom-up and top-down factors that contribute to different rates of usage of online government services among citizens. This leads to a more comprehensive understanding of the requirements to implement e-government on a wider scale. The research looks into the relationship between the usability of government websites and their usage rates. By using statistics from the European Union, we were able to paint a picture of the penetration rates of e-government in the case studies of Estonia, Latvia and Lithuania. Usability is conceptualised through the functionality, accessibility and provision of services, which are theories based on the extensive literature on this topic. These were then observed (or not observed) on the Ministry of Health, Finance and Education websites of the three case studies. In order to show the usability of the websites, we sought answers to FAQs on the websites to see how accessible information was. By using a comparative case study of the three Baltic states, we were able to look into the differences between the levels of usage by citizens of the three countries, as well as the different levels of usability seen on three government websites. The study ascertained that there was a relationship between usability and usage in the case study of Estonia, whereas in the cases of Latvia and Lithuania, it was not as evident. Latvia had higher rates of usage; however, Lithuanian websites performed better in the observations during the study. This leads to the conclusion that there are further factors at play than just usability in these cases. It provides an interesting premise for further study as the experiment can be easily replicated for other cases and analyses the most important factors of usability of government websites within the frameworks of institutional capacity, data readiness and literature on accessibility and the digital divide.

(3)

The Usage of e-Government Services in the Baltic States 3

Contents

Contents 3 1. Introduction 6 2. Literature Review 8 2.1 E-government 9

2.1.1 Definition and Context 9

2.1.2 Citizens and Open Communication 10

2.1.3 Government Provision of Information and Services 12

2.2 Institutional Capacity 15

2.2.1 Government Infrastructure 16

2.2.2 Governmental Capabilities and Capacity 17

2.3 Data Readiness 20

2.4 Access to Public Services and the Digital Divide among Citizens 24

2.4.1 Digital Divide 25

2.5 Usage of Online Services by Citizens 27

2.6 Place of the Research in Wider Literature 30

3. Research Design and Case Discussion 32

3.1 Research Design 32

3.1.1 Research Goals 32

3.1.2 Theory and Causal Mechanisms 33

3.1.3 Research Approach and Design 33

3.1.4 Unit of Analysis, Relevant Populations and Case Selection 34 3.1.5 Key concepts and their operationalisation into variables 35

3.1.6 Common Threats to Inference 37

3.2 Case Discussion and Results 38

3.2.1 Estonia 39

(4)

The Usage of e-Government Services in the Baltic States 4 3.2.3 Lithuania 49 3.2.4 Graph 55 4. Analysis 56 4.1 Accessibility 58 4.2 Functionality 60 4.3 Provision of Services 61 4.4 Answering FAQs 63 4.5 Final Thoughts 65 5. Conclusion 67 6. References 71 7. Appendices 79

7.1. Estonian Government Websites 79

7.1.1 Ministry of Social Affairs (Healthcare) 79

7.1.2 Obtaining information about the flu 80

7.1.3 Ministry of Finance 81

7.1.4. Obtaining information about the budget 82

7.1.5 Ministry of Education and Research 83

7.1.6 Obtaining information about scholarships 84

7.2. Latvian Government Websites 86

7.2.1 Ministry of Health 86

7.2.2 Obtaining information about the flu 87

7.2.3 Ministry of Finance 90

7.2.4 Obtaining information about the budget 91

7.2.5 Ministry of Education 92

7.2.6 Obtaining information about scholarships 93

7.3. Lithuanian Government Websites 95

(5)

The Usage of e-Government Services in the Baltic States 5

7.3.2 Obtaining information about the flu 96

7.3.3 Ministry of Finance 101

7.3.4 Obtaining information about the budget 102

7.3.5 Ministry of Education and Science 104

(6)

The Usage of e-Government Services in the Baltic States 6

1. Introduction

E-government is one of the buzzwords of 21st century public administration and addresses the increasing usage of the internet in people’s everyday lives. With the changing ways that citizens interact, it is vital that governments are up to date with the newest technologies in order to provide their services more effectively. However, this approach must be multifaceted - governments must be providing services that are accessible, functional and easy to use in order to engage citizens. Open communication between governments and citizens is a vital aspect of e-government (Björklund, 2016) and underpins the most functional workings of online public services. From the other angle, citizens themselves make up an important part of a culture that is open to welcoming to new technologies. In order to harmonise the organisational capacity of these two actors, it is vital that the platforms on which they interact are accessible, open and simple to use. Institutions such as the European Union recognise the importance of the impending digitalisation of governments which is analysed within the framework of the Digital Single Market - one of the key policy foci of the European Union, and one of the central themes of Estonia’s Presidency of the Council of the European Union in 2017 (Tangi et al., 2017; “Priorities of the Estonian Presidency”, 2017). The focus on digitalisation of public services manifests itself in many ways in modern society such as transparency and functionality as well as closing of the digital divide between the user and non-users and the gap between well performing countries and others. Currently the average penetration rate of online government services around Europe is 52%, however this differs greatly from country to country (Tangi et al., 2017) and the average usage rate is 49% (Eurostat, 2017). The use of online government services reflects an increasingly digitalised society and is therefore it is important to discuss how governments are implementing their digital policy.

This research looks into the factors that make a website usable and places it in the wider literature of online government services. It aims to answer the research question: “How does the usability of government websites contribute to their usage rates by citizens?” By studying the canon and the most important contributions to the literature on usability, the research will then apply this to the case studies of Estonia, Latvia and Lithuania in order to see whether there is a relationship between the number of users interacting with the government online and the assessed usability of these websites. The research involves observing the presence (or lack) of certain factors dictated to influence the usability of government websites as theorised by Bertot et al. (2014), Nam (2014), and Bekkers and Homburg (2007) in order to assess the quality of

(7)

The Usage of e-Government Services in the Baltic States 7

the website and the rate of usability, as well as tracing answers to Frequently Asked Questions (FAQ’s) on the websites to ascertain to ease of information attainment. This data is then analysed within the theoretical frameworks of institutional capacity, data readiness and the digital divide in order to gain a more in depth understanding of the reasons behind the actions and capabilities of different government departments. The research hypothesises that the more accessibility, functionality and provisions of services on a government website, the more usable it is, thus leading to an increase in usage rates.

This research is of societal and theoretical relevance to the overarching discussions on e-government, the online provision of government services and the increasing digitalisation of our society as it places the usability and citizen experience at the forefront of our understanding. With the overall increase in usage of European citizens to access government services online (Eurostat, 2017), it is vital to comprehend the factors that make the aforementioned government services accessible, functional and viable. Current research has looked into different aspects of online public services; however, this study aims to bring together existing approaches to measuring usability in the specific cases of the Baltic states to assess the relationship between the usability of government websites and their overall rates of usage. This is significant for future policymaking and can be utilised by different actors interested in how to use the internet effectively in the provision of government services. By analysing the way in which the usability of websites contributes (or does not contribute) to usage levels, we can understand how to implement best practice on government websites. This is socially relevant for a society that is ever increasing the extent to which they rely on the internet for day-to-day tasks. It is important that governments are not left behind in this discussion as communication with the public is of utmost importance to effective governance.

1.1 Thesis Roadmap

This thesis will be laid out as follows: the research begins by giving an overview of the literature pertinent to the discussion in Chapter 2. Starting with the literature on e-government (2.1), its history, current challenges and the human aspects of the digitalisation of online services, it will provide an overview to the topics of institutional capacity (2.2), data readiness (2.3) and the literature on accessibility in public services and the digital divide (2.4). The reader will fully be able to understand the context in which governments are implementing digital changes to their provision of online services to citizens. It will also give an overview into the

(8)

The Usage of e-Government Services in the Baltic States 8

literature on usage of online public services until now (2.5), as well as its place in the wider canon (2.6). In Chapter Three the reader will be shown through the thorough research design (3.1) dictating exactly how and why the research was carried out, the processes used and the limitations of the research. 3.2 will introduce the case studies of the countries of Estonia, Latvia and Lithuania and the results from the empirical research along with their context. The results and consequences will be thoroughly analysed in Chapter 4. Chapter Four analyses the case studies in a regional context before the individual hypotheses (4.1, 4.2 and 4.3) and the results of the information locating exercise (4.4) before final thoughts on the analysis and the comparative case studies (4.5) and the conclusions in Chapter Five.

2. Literature Review

The literature on e-government is extensive, even when taking the relatively modernity of the concept. In order to fully understand how e-government technologies manifest themselves in modern society, it is vital to understand the theoretical underpinnings of its implementation. This literature review will give an overview of the core concepts that the thesis will deal with such as institutional capacity, data readiness and the digital divide, and how they are closely related to the variables and factors that will be studied in depth later on. The research question will be understood and analysed through hypotheses related to how the usability of government websites affects their overall levels of usage. Key theoretical frameworks that underpin the study of e-government are data readiness, institutional capacity and research on accessibility and usage. These outline the factors that are reflective of effective utilisation of e-government in the public sector and are closely related to the governmental support the provision of more efficient public services. In addition to providing a substantial theoretical framework to the topic at hand, this review will also provide an overview of the literature on e-government and its usage in the public sector, the digital divide and the ways in which it is used. The literature is broadly split into the two overarching themes of top down government initiatives and bottom up citizen usage, which is reflected in the discussion. It will place this research inside the established framework, identify how the current literature does not adequately cover this question and assess how it is relevant to potential further works. By combining approaches from different authors, this research allows for a more nuanced discussion on how different factors come together to represent usability of online government services. A variety of literature from primary and secondary sources was consulted in the process of this review,

(9)

The Usage of e-Government Services in the Baltic States 9

sourced from renowned, peer reviewed journals or official government publications. This is important to ensure the quality of the research and accuracy of the information.

2.1 E-government

2.1.1 Definition and Context

As the central theme of this thesis, it is important to have a comprehensive understanding of how e-government functions in the modern world. E-government has multiple definitions in the literature and this thesis will conceptualise the term in accordance with Björklund’s (2016) definition, whereby e-government refers to “the use of information and communication technology within the public sector [and] entails digitalization of governmental records and national data as well communication between the government, public administration, and citizens” (p. 914; also outlined in Barbosa, Pozzebon and Diniz, 2013). This concept was chosen as it makes specific reference to the digitalisation of public information and the importance of the communicative relationship between government and its citizens. The open communication and the relationship between governments and their citizens is important to the provision of online public services as it is the basis for their usage and engagement, making this definition a strong reference point for this thesis and our contextual understanding of the concept. E-government has also been theorised as “a vehicle for fostering customer-orientation in public agencies” (Bekkers and Homburg, 2007, p.374); the use of any IT in government (Bannister and Connolly, 2015; Carrizales, Melitski, Manoharan and Holzer, 2011) and a way of transforming traditional government services making it accessible, transparent and effective (Haque and Pathrannarakul, 2013). It can be therefore seen that the literature on the delivery of e-government services is extensive and has developed in different ways over the last two decades.

In order to understand the development of e-government, it is important to consider the literature on New Public Management. New Public Management supplied the basic current thoughts on e-government as it set a discussion on how governments could treat the provision of services to the public in a similar manner to business models. Foci in New Public Management are related to result-oriented management, effectiveness in market practices and integration of organisational aspects (Barbosa, Pozzebon and Diniz, 2013). Another important factor of New Public Management is the identified “trinity of macro-themes” of disaggregation, competition and incentivisation (Margetts and Dunleavy 2013, p.2) which is indicative of

(10)

The Usage of e-Government Services in the Baltic States 10

business functionality being applied to the public service. E-government is considered to be a shift from traditional New Public Management as it is based on the same underlying concepts but has developed in a different direction since then due to the fundamentally distinct factor of computer and ICT capabilities being central to policymaking (Margetts and Dunleavy 2013). The paradigm shift is also discussed by Haque and Pathrannarakul (2013), where they outline the differences between the previous bureaucratic paradigms to the newer e-government paradigms such as the change between hierarchical to multidirectional organisational structures, from centralised to decentralised decision making and the move between formal and informal communication channels (p.28). These integral organisational changes are vital to understand how e-government operates in modern democracies. With this context of development in mind, it can clearly be seen that the addition of ICT to New Public Management has reshaped and restructured the way in which we see the government as a service provider. Having understood the paradigm shifts of the 21st century, this research will now go on to discuss e-government from a human perspective in regards to citizens’ values and the provision of information and services. By analysing the literature from a bottom up and top down approach, we will be able to clearly assess the different ways e-government manifests itself in modern society.

2.1.2 Citizens and Open Communication

One of the key factors behind the implementation of e-government is the values behind transparency and open information for citizens of a country. Open information and communication is an important discussion point when it comes to e-government as it underpins many of the services governments can provide. Bertot et al. (2014) outline the biggest concerns for the public in regards to open data sources as being privacy, data re-use, data accuracy, preservation and archiving, data curation and support of libraries (pp. 13), whereas Mergel (2015) focuses on the changing nature of how citizens are accessing information. With the increase in the use of social media in addition to the internet by citizens, the demand for open and free data and information from the government is becoming an ever more important requirement. This is emphasised by Tolbert and Mossberger (2006) who assert the importance of open government as being inextricably linked with citizen confidence in their actions. They identify the promise of e-government as reflective of open and accessible data has on disseminating information to citizens1. Citizen values of openness and transparency are key to

(11)

The Usage of e-Government Services in the Baltic States 11

understanding how and why governments engage in certain policies as it dictates the level of engagement and services desired. Cullier and Piotrowski (2009) identified the four main types of public access to government information as 1) proactive dissemination by the government (dossiers, publications etc.); 2) information per request of citizens; 3) leaks from whistlblowers; and 4) open public meetings (p. 442). These become especially relevant within e-government due to its intrinsic ability to collect and organise information in a way that is easily accessible to curious citizens. The increasing openness of information is a central aspect to the functioning of e-government in modern societies and strongly reflects the human values that underpin them.

Central to the values that governments are placing into e-government systems is the idea that the online provision of services is associated with modernity and progress (Bekkers and Homburg, 2007) as well as effective service delivery for citizens (Åström, Hinsberg, Jonsson, and Karlsson, 2013), which is vital to its overall success. The characterisation of modern e-government technologies is being linked to the “move to customer-centric models of service delivery for the public sector organization” (Haque and Pathrannarakul, 2013, p. 33) and is closely related to how the consumers of e-government perceive the possibilities for use that are connected to other aspects of modern life. The connection to modernity is important to take into consideration as it represents the intrinsic appeal of e-government systems. This is an important factor in regards to the discussion on usage as the reasons behind uptake of online services can help understand the most effective provision of them. E-government is considered to be transparent, accessible and efficient, client oriented, cohesive, democratic and responsive (Bekkers and Homburg, 2007). By associating the processes as new and modern, e-government enjoys the perception of being progressive, even when actual implementation is slower and less effective than desired (Bekkers and Homburg, 2007). The acquisition of usable e-government systems is closely linked with the ideas of modernity and progress and belief in these concepts has helped a more comprehensive roll out in many places.

The value of e-government can be understood from both a top down and a bottom up approach as governments and citizens realise the benefits of online services for their everyday life. Beyond this, communication between the two parties can be improved as e-government increases citizen trust in government (Tolbert and Mossberger, 2006), transparency and accountability (Giest, 2017; Björklund, 2016). These are central concepts to the understanding of how e-government goes beyond just providing services to citizens and can actually impact

(12)

The Usage of e-Government Services in the Baltic States 12

their daily life and dealings with government on a local, national or even regional level. Bekkers and Homberg (2007) identified the three factors that contribute to e-government’s better and increased communication with citizens as being “more responsive to the needs of citizens and enterprises [...] more democratic and [...] more efficient (p. 373; also discussed in Tolbert and Mossberger, 2006). This is closely linked to how Cullier and Piotrowski (2009) identified the idea of open government as being key for reducing corruption and increasing trust in government. These concepts are inextricably linked as they reflect the idea of how citizens are intrinsically more trusting of governments they can see, understand and feel are acutely representing their interests as well as being responsive to their requests. When different departments of government are given the possibility to work together, it increases this productivity and allows for better services as intergovernmental cooperation is at the centre of efficient systems that provide more streamlined assistance (Bekkers and Homburg, 2007). Beyond improving the quality and services and money saved, e-government is seen as being responsible for more effective policy and increased participation of citizens (Helbig, Ramón Gil-García and Ferro, 2009, p. 89), contributing to a better provision of services and wider citizen trust. From the perspective of citizens, the fundamental ideas behind e-government such as open information are key drivers behind usage and contribute to a more positive impression of the processes.

2.1.3 Government Provision of Information and Services

In order to understand how e-government permeates modern society, the discussion on e-government technologies as the provision of information and services must be assessed. Tolbert and Mossinger (2006) identified the premise upon which e-government aims to present itself in modern societies as both the deliverer of online transactions as well as the dissemination of information about the government and its activities (p. 354). By using these technologies, governments are giving themselves a platform to communicate directly with citizens, disseminating their policies, information and other priorities. This has led to an increase in interest in the studies of how governments engage with their citizens (Carrizales, Melitski, Manoharan and Holzer, 2011; Giest, 2017; Johnston, 2015). This is closely related to the provision of services and how governments put their policies into practice in an online setting. This is central to this research, as it represents the extent to which e-government has penetrated society. Bekkers and Homberg (2007) identify four types of information services that cover the way in which governments provide: information services, contact services, participation services and data transfer services (p.374). These terms closely cover the range

(13)

The Usage of e-Government Services in the Baltic States 13

of services that governments can offer their citizens, although the extent to which they do tends to vary. Transitioning to comprehensive e-government is high priority for a number of countries but is not as easy to implement on a wider scale as hoped (Baldersheim and Oegard, 2008, p. 125) - this will be discussed further in the sections on institutional capacity and data readiness. E-services in the public sector have become one of the most discussed aspects of e-government (Barbosa, Pozzebon and Diniz, 2013) as it is the factor that has the most impact on citizens’ day-to-day life and their perception of their government. One of the key characteristics of e-government according to Haque and Pathrannarakul (2013) is its customer-centric models in its service delivery (p. 33), something that strongly affects how people perceive the levels of organisation within government.

Central to the usage of e-government services is the human factor. Without citizen interest and government willingness, e-government has very little purpose in society. Vital to this is the idea that digitisation of public services is framed in a discourse of progress and advantage (Björklund, 2016). With the ubiquitous use of internet in everyday society, it leads to the assumption that people will continue to actively seek out e-services where they can when it comes to government interactions as well. The increase in online interactions also manifests itself in modern society with the decreasing costs of communication technologies and the further dissemination of these tools on the internet (Johnson, 2015). This is relevant to the discussion on online government services as it represents changing attitudes to how the internet is used in our everyday lives and how the government views its role in this position. This discussion is continued by Mergel (2015) who recognises the rapidly changing attitudes that people have towards information and their online selves. She discusses how governments recognise that information is not only accessed by their online official sources, but also by methods such as social media (p. 322). This represents a challenge in the way in which governments must grow and adapt in their provision of information and services in order to remain relevant and in order to reach the maximum amount of people. The ability of citizens to simply seek out relevant information to their lives is central to the changing paradigms of public service delivery and how people perceive access to their information (Cullier and Piotrowski, 2009; Mergel, 2015). This is closely related again to Björklund’s (2016) idea of the discourse of progress - as also discussed by Bekkers and Homburg (2007) - there is this strong belief and trust in ICT’s and the potentials of progression as brought along by internet and the information society (p. 378). Human interest is a major part in e-government usage as individuals are fundamentally the ones who have to use, engage with and make the active

(14)

The Usage of e-Government Services in the Baltic States 14

decision to learn about how these technologies work in order to be able to get the most out of them (Bekkers and Homburg, 2007, p.379). This applies not just to citizens who wish to engage with their governments online, but also to governments who look to provide these services and implement these changes.

An overview of the literature surrounding implementation and effectiveness is important in the understanding of how government institutions are actively implementing changes, and some of the challenges they face. Service delivery by governments for authors is the central aspect government digitalisation (Bekkers and Homburg, 2007; Baldersheim and Oegard, 2008) as it enables governments to communicate directly with their people. This is mainly implemented through front office technologies, which enable easy communication. Bekkers and Homburg (2007) discussed the case of Canada as a case of a country that is trying to implement effective e-government that identified their main priorities as being “aligning various ICT structures, developing a world-class ICT workforce within government, the improvement of the management and success rate of ICT investments, and the minimizing of risks of ICT Projects” (p. 378). These backend tasks are central to effective implementation and are the backbone on the discussion of implementation. This thesis will not be focussing on the practical/technical components of the implementation of e-government via usable and accessible websites for citizen services, however it is important to have a theoretical understanding of government priorities when implementing new policies in regards to technology, as it is one of the underlying factors leading to wider usage and adoption by the public. Behind the implementation factors is also the paradigm shift evident in public sector service delivery as discussed by Tapscott and Caston (1993) in Haque and Pathrannarakul (2013) which outlines the fundamental changes behind e-government paradigm in comparison to bureaucratic paradigm. These shifts, such as: in ‘orientation principle’ from production cost efficiency to user satisfaction and control; in the ‘management principle’ from management by rule and mandate to flexible management and interdepartmental teamwork; and in the ‘communication principle’ from centralised and formal to fast feedback and multiple channels (p. 28) represent the internalised changes within governments that will be discussed in further detail in the sections on data readiness and institutional capacity. However, it is also important to recognise the literature on government implementation and attitude is central to the e-government canon as it provides a functional basis for work surrounding the topic.

(15)

The Usage of e-Government Services in the Baltic States 15

As can be seen in the previous paragraphs, literature on e-government is extensive, covering many bases, many contexts and provides an interesting framework through which to analyse the modern electronic provision of government services. Concepts such as open communication, transparency, access to information, provision of services, the idea of modernity and progress and the challenges of implementation come together to give a robust understanding of the literature canon surrounding the topic. However, as Björklund (2016) noted, many studies on the effects of e-government have been case based (p. 915), which has impacted wider theoretical understanding of the topic as it is intrinsically contextualises related issues. This thesis aims to provide a concrete understanding of how e-government services are provided to the citizens and the factors that make lead to higher levels of uptake and usage while commenting on features that make government websites more functional and accessible in the provision of online services. This fits into the wider literature on e-government and online service provision as it seeks to look into the factors that contribute to how and why people use e-government websites.

2.2 Institutional Capacity

Institutional capacity is at the centre of discussions on how to implement e-government as it represents the fundamental reasoning and decision-making that underpins how governments tackle challenges. The term in this case refers specifically to the capacity of governments to implement e-government policies as the human capacity for citizens and usage will be addressed in detail during the sections on digital divide and accessibility. Government institutional capacity is important to understand, as it is the backbone of the effective provision of online services as it dictates the extent to which e-government policy can be implemented. Capabilities of governments to implement successful e-government tools on a wide-reaching scale are often restricted by institutional factors that can have a negative or positive effect on the overall effectiveness of these systems. The literature on this topic is wide ranging and focuses on how external and internal factors come together to contribute to overall decision-making, policy and implementation. This part of the review will cover the extensive literature and research on the factors that contribute differing levels of institutional capacity between different groups. The themes central across include infrastructure, expertise, governmental and policy capacity and the different types of capabilities necessary to produce the most effective policy (Haque and Pathrannarakul, 2013; Giest, 2017; Bekkers and Thaens 2002; Struijs, Braaksma and Daas, 2014; Howlett, 2015; Layne and Lee, 2001; Christensen and Lægreid, 2010).

(16)

The Usage of e-Government Services in the Baltic States 16

2.2.1 Government Infrastructure

Infrastructure is the backbone of institutional capacity in governments as it underpins the capabilities of governments within their specific contextual frameworks. Within the context of e-government, the level and quality of the infrastructure can make or break how people use its services. In many countries in the world, development is limited by a lack of infrastructure and human capital capabilities that represent a major hurdle for governments to make the most out of the benefit of modern technology (Haque and Pathrannarakul, 2013). With the increase in data, technology and information, governments must be prepared to be able process the large amount of data in an acceptable time frame and using the best quality infrastructure (Daas, Puts, Beulens and van den Hurk, 2015), therefore governments’ IT capabilities are vital to their ability to be able to process them. A topic of much discussion is stovepipe systems - a system whereby information between different government departments is not shared. Originally created to avoid complexity in government processes, with the increase in communication between departments and the ever-present presence of technology this has created confusion, slow process and obstacles (Giest, 2017; Kim, Trimi and Chung 2014). This is one of the factors that is most preventative to a concise e-government system that works across departments and functions. When presenting e-government platforms, Bekkers and Thaens (2002) mention the example of mijnoverheid.nl, the Dutch online platform for government services, which is indicative of the technological infrastructure required to have a comprehensive e-government platform. The discussion on how different government departments can be more integrated is important to the discussion on infrastructure as it represents the underlying government values and capabilities. The challenge of processing, storing and transferring data and information between government departments is technically difficult, expensive and requires a large commitment from governments (Struijs, Braaksma and Daas, 2014), which is highly relevant to the discussion on institutional capacity. By looking at the role of infrastructure in institutional capacity in e-government literature from a theoretical perspective, we can see the integral role it plays in the capabilities of governments when it comes to implementing e-government solutions.

At the centre of institutional capacity in regards to e-government is the technical expertise required on a personnel level. In order to have a high-functioning e-government system in place, there needs to be advisors, policymakers and IT professionals who understand best practice, the technical requirements for complex systems and how to make systems usable

(17)

The Usage of e-Government Services in the Baltic States 17

and accessible (Daas, Puts, Beulens and van den Hurk, 2015). There is a high level of technical expertise necessary when dealing with all the different types of data and information that governments currently have access to. When attempting to do something meaningful with the large amounts of data, knowledge of high level computing systems and engineering is necessary to be able to process the information in a timely and useful matter (Daas, Puts, Beulens and van den Hurk, 2015). Literature on this topic is mainly focussed on the requirements for processing big data in government institutions but the concepts are closely related to those in e-government implementation as the technical and IT skills are similar, as well as the progressive mind-set required in order to embrace these technologies (Bekkers and Homburg, 2007). Part of the discussion is how these services are introduced - how do governments access the required amount of IT professionals in a sector that is not traditionally technical? In many cases around the world, IT is being outsourced to private companies and delivered by private stakeholders - this has a number of impacts, such as the increased role of businesses and industry and performance factors (Dunleavy et al., 2006 as quoted in Giest, 2017, p. 372). This changes the fundamental shape and workings of governmental institutions as the increased reliance and cooperation with external parties can bring in different knowledge, opinions and way of workings. Another factor that is discussed in the conversation on expertise’s’ influence on institutional capacity is the divide between developing and developed countries as capacity can be severely limited by a need for excessive technical expertise (Davis, 2005). In this discussion, it is important to factor in how differing levels of expertise can have an impact on the infrastructure surrounding e-government. Quality of infrastructure can affect the extent of a government’s institutional capacity as it makes it far-reaching policy reforms more difficult to attain. In the context of e-government services, this can be make-or-break if the infrastructure does not support the desired changes.

2.2.2 Governmental Capabilities and Capacity

The intrinsic capabilities of governments in regards to institutional capacity is an important concept to factor into this discussion as it is an underlying factor that influences how governments make decisions and come to policy decisions. Howlett (2015) identifies the three types of capabilities as being individual, organisational and system, and the main competences as analytical, managerial and political (p.175). These competencies and capabilities intersect with each other to create a number of skills and resources dimensions that can be beneficial for institutional capacity as they represent the different ways in which governments, civil servants and society interact. E-government has increased the complexity of public institutions in a time

(18)

The Usage of e-Government Services in the Baltic States 18

when they were already becoming more complex - dealing with “conflicting ideas,

considerations, demands, structures and cultural elements at the same time” (Christensen and Lægreid, 2010, p.407). This increasing complexity can have an impact on government capabilities, however also represents an opportunity to grow and develop new types of governance in response to our increasingly diverse societies. Christensen and Lægreid’s (2010)

discussion on cultural complexity that influences government institutions provides an interesting standpoint as to how informal, cultural norms and values influence how governments function. They to discuss how this can vary between countries, with path dependency explaining the unique features of national reform. With different countries and institutions having different socio/historical/cultural traditions and norms (Christensen and Lægreid, 2010, p.11), the direction that countries can take differs vastly. This idea has an important place in the discussion of e-government implementation and usage as different countries have different levels of capacity to integrate their systems and create usable programmes. When considering this angle, it is important to compare and contrast how different approaches between different countries manifest themselves in modern governance values.

Governmental policy capacity can be heavily influenced by the structure and openness of said government and is an important aspect of government capabilities. Policy capacity, which refers to the “competencies and skills of policymakers and the capabilities or resources required to exercise them” (Howlett, 2015, p. 173), is central to the ability of governments to implement effective and easy to use e-government systems. The concept goes beyond just policymaking and implementation in order to fully assess capabilities on a day-to-day basis - how competent are their systems? How easy are new ideas to implement? Are they effective and legitimate? (Howlett, 2015). Policy-making capabilities are important to the discussion of how e-government can be as effective as possible as it represents the extent to which the government as an institution is prepared to implement (in some cases radical) new ideas into their everyday usage. Howlett (2015) theorises that organisations that have a strong policy analytical capability are therefore more likely to be successful when it comes to implementing outcomes than those with a lower level (p. 174). This may appear relatively straight forward, but it manifests itself in a number of different ways - in many cases, this has required structural transformations, whereby internet-based government models have tried to combine and work with traditional public administration systems (Layne and Lee, 2001, p. 123). This argument by Layne and Lee (2001) about structural transformation is important in the canon of

(19)

The Usage of e-Government Services in the Baltic States 19

technological process and how governments are facing struggles vis-a-vis traditional methods of public administration in comparison with newer, electronic based methods. The level of policymaking capacity is a factor that perpetuates itself in the overall institutional capacity of government organisations and is a vital concept to the discussion on how governments are restricted in their structural transformations.

Government capacity is vital to achieving complex outcomes in public administrations. In regards to e-government, these outcomes can be particularly difficult due to their intrinsically technical nature. Institutions need to be able to “process information and realise desired outcomes by employing staff, creating agencies and building up standard operating procedures” (Dunleavy et al., 2006, p.21 as quoted in Giest, 2017, p. 369). This is vital to understanding how governments of different levels of capacity can achieve outcomes that are in line with their desires. Of course, there are multiple institutional barriers to change and progress that have been widely discussed in the literature due to its importance to the wider canon. Barriers to change include factors such as risk-averse cultures (accountability is central, there is extensive red tape and weak links) these factors contribute to more recalcitrance to innovation. In addition, other identified barriers such as task complexity, use and participation, legislation, information quality and technical hurdles (Janssen, Charalabidis and Zuiderwijk, 2001, p. 261) can contribute to a lack of progress in the field of e-government. The extent of barriers is important to take into account as they dictate the degree to which governments are capable of implementing diverse policies. Gil-García and Pardo (2005) discuss the idea that institutions are not only the laws and regulations that underpin governments, but also the “norms, actions and behaviours that people accept as good or take for granted” (p.193), meaning the way that institutions interact with the public can influence and shape the level of capacity they enjoy. Barriers to governmental capacity reflect negatively on e-government implementation and usage as it can result in a number of negative consequences. If a government is committed to digitalising its services, it must bear in mind the full extent of its needs. Privacy and security related issues (Gil-García and Pardo, 2005) are issues that are at the forefront of users’ experiences and concerns when engaging with digitalised data and present a challenge that implementers face when dealing with a number of new threats. Other values that are well looked upon in government include openness, accountability, transparency, participation and collaboration (McDermott, 2010), while the fundamental implementation of these in an open e-government presents challenges that need solid institutional capacity on the governmental level to ensure the highest quality. This discussion is particularly important as it

(20)

The Usage of e-Government Services in the Baltic States 20

gives the opportunity to analyse the way in which governmental capabilities affect e-government application and usage.

By discussing and understanding the literature canon on institutional capacity, it can be seen that the structural factors that contribute to e-government implementation and effective usage are multi-faceted, contextual and are related to how different levels of expertise and infrastructure can influence the abilities and capabilities of governments. When taking government and policymaking capabilities into account, we can see how barriers to change, path dependency and a lack of resources can influence and affect the abilities of governments to implement newer and progressive technologies on a wider scale. This discussion is vital to understanding how governments can be restricted in their desired changes and implementations due to structural factors that can slow down or prevent aspired adoption and usage.

2.3 Data Readiness

In order to understand how e-government is effectively implemented, a contextual analysis of how governments are prepared to bring technologies into their everyday actions is required. By understanding how data readiness manifests itself in government arrangements, it is vital to look into how governments are dealing with the increase in data and information that they have access to. Key elements of discussion include the processing of data, government policy on information and data, the human aspect and data culture and the concept of big data. Factors that contribute to and restrict data readiness can have a profound effect on how e-government is implemented and has an important role in the literature canon, as it dictates the way in which governments use the large amounts of new information.

The data readiness concept assesses organisations capabilities in relation to big data - more specifically the idea of organisational alignment, capabilities and maturity (Giest, 2017, p. 369). These concepts refer to the ability of organisations to collect, manage and utilise their data in a certain way, in line with their principles. More specifically, alignment refers to whether big data is a good fit with the organisation’s structure and main activities; capabilities refer to the organisation’s use of big data lined to IT, data science expertise, legal expertise and data governance and; organisational maturity is the maturity of e-government initiatives within the organisations (Giest, 2017, p. 369; Klievink and Janssen, 2009). This represents an integral part of how governments interact with their different departments when attempting to implement e-government policy. The development of the proper data structures in public

(21)

The Usage of e-Government Services in the Baltic States 21

organisations is vital to successful IT initiatives, especially when it comes to cross-departmental initiatives and platforms (Gil-García and Pardo, 2005, p.193). Cooperation between departments can be a great opportunity, however enabling it to happen between multiple organisations can lead to difficulties (Gil-García and Pardo, 2005). For many governments this is worth it, as a free and open public service culture is well perceived, overall positive and considered a project in democracy and accountability in the long term (Hardy and Maurushat, 2017, p. 36) as well as representing a participatory and collaborative conversation for citizens and policymakers (Evans and Campos, 2012, p.173). For data on a wider scale to be effective for the community however, it needs to be accessible to the public, by avoiding technical terms, acronyms and overly legalese texts (Evans and Campos, 2012). Understanding how governments use the data for the public on an internal and external level is an important aspect of their data readiness as it shows how mature their organisations and initiatives are on a longer-term scale.

Government policy on information is an important part of data readiness as it dictates the way in which governments actively pursue to use data and how they seek to utilise it on an interdepartmental level. The trend of open data is growing exponentially as it is seen to be accessible, low cost, innovative, accessible and re-usable for a multitude of purposes (Hardy and Maurushat, 2017, p. 30). With its ever-increasing popularity, it is important to consider the added benefits and drawbacks of open data and this is central to how governments are addressing the data readiness. Hardy and Maurushat (2017) identify three major benefits to open data as improving the effectiveness and efficiency of government services; gains in transparency and accountability; and improving democratic participation (p. 32-33). However, implementation in practice can be a more problematic issue due to differing levels of data readiness. The United States for example has a complex and constantly changing set of policies that cover the process from creation through to archiving; however, this system often does not keep up with the technological advancements and is problematic for its overall functioning (Bertot et al., 2014, p.9). This is however not a US only issue - many governments’ adoption of technology is lagging behind the use of technology by private citizens, often in a sort of “catch-up mode” (Margetts and Dunleavy, 2013, p.13). This is problematic in many situations, - beyond playing catch up to citizens’ everyday technological use, there are also legal ramifications (Struijs, Braaksma and Daas, 2014, p.3), which come from the government’s inability to predict upcoming changes and the factors that will become the most important and integral to future systems. Government lags behind the private technological sector have also

(22)

The Usage of e-Government Services in the Baltic States 22

been identified as leading to low levels of technological and computer literacy (Giest, 2017, p.369). Government policy towards data readiness is an extremely important discussion in this literature as it details the level to which governments are committed to their data usage and implementation.

Data culture refers to the “importance of civil servants and policymakers understanding how to find, analyse and utilize big data and the institutional structure to support this through, for example, training or sharing of data among government department” (Giest, 2017, p.372). This concept is central to the way governments are able to fulfil their goals as the way in which staff are trained, their backgrounds and expertise can make or break policy changes within government. With a lack of data culture, organisations and governments can find themselves unable to deal with the sometimes-convoluted systems that complex data requires. In the discussion on government provisions of online services, it is important to have an understanding of how data culture manifests itself in the relevant policy changes required to implement e-services as it affects every level of government.

Another angle that has been extensively discussed in the literature is the human aspect of data readiness, whereby the public’s view and understanding of new technologies is central to their perception and usage of online government services. Digitalisation of services has had a profound effect on the public and how they interact with their governments and other authorities. With the new skills required in order to be active and informed with government services, the human aspect of data readiness cannot be underestimated (Haque and Pathrannarakul, 2013 as quoted in Björklund, 2016). Open data policies have been considered to increase interaction, self-empowerment, social inclusion and participation of citizens and providers among other benefits (Zuiderwijk and Janssen, 2014, p.17). This aspect of human empowerment creates an interesting debate on how these services are created in order to be usable by the wider public. Closely linked to this concept is the idea of cultural resistance when manifests itself in a number of ways such as generational preferences, concerns about the quality and accuracy of information, fears, limited understanding of the benefits, lack of leadership in driving open government (Hardy and Maurushat, 2017, p.35) and citizen trust (Malatras et al. 2017). The cultures surrounding data readiness are an important aspect to take into account when assessing the discussion on open data in the public sector as the users stand central to it overall usage.

(23)

The Usage of e-Government Services in the Baltic States 23

When considering data readiness in the modern public sector it is impossible to avoid the topic of big data. Big data refers to the large amounts of data which is being processed from multiple ranges of sources at high speeds at real time - often defined with the words ‘volume’, ‘variety’ and ‘velocity’ (Hardy and Maurushat, 2017, p.31; Giest, 2017, p.367-8). This has become a staple of modern day life due to the way in which technology permeates our everyday actions. Relevant factors related to dealing with big data were identified as “IT governance, IT resources, internal attitude, external attitude, legal compliance, data governance and data science expertise as relevant factors for dealing with big data” (Klievink et al. 2016 as quoted in Giest, 2017, p.372). When accounting for these factors in the discussion on big data, it is important to assess how these manifest themselves in the institutional context of the governments dealing with big data on a daily basis. Literature indicates that many governments have a limited capacity for dealing with this issue as well as a lack of tools and frameworks (Struijs, Braaksma and Daas, 2014). The perception of governments to be able to deal with the organisational challenges of big data management is important, as trust is central to the government having access to sensitive information. This relates closely to the usage of this data for evidence-based policymaking, which relies on levels of societal trust to be able to be fully utilised (Struijs, Braaksma and Daas, 2014). Therefore, central to the discussion on big data is the way in which government utilises this information for policy reasons and how this is contextualised in institutional, organisational and social systems (Dawes and Helbig, 2015, p. 31). The literature on big data extends far beyond the data readiness framework however; this is beyond the scope on this thesis. Addressing big data in the framework of data readiness shows the way that governments are dealing with and managing the big data they have access to due to the provision of online public services.

Data readiness is one of the key concepts that provides an understanding of the contextual reasons that contribute to acquisition and functional usage of e-government in many countries. By analysing the literature existent on this matter, it is possible to see how the existing issues facing other cases studies and theories can contribute to an increased understanding of the challenges and opportunities presented by increased use of data in the public sector. Important factors to take into consideration include the way in which governments process the data they obtain, government policy towards data and information, the human aspect behind open data and the concept of big data. This presents an interesting theoretical view into how governments enact change related to the increase in open and big data in modern societies.

(24)

The Usage of e-Government Services in the Baltic States 24

2.4 Access to Public Services and the Digital Divide among Citizens

Accessibility is a vital aspect of the provision of e-government services as it is an indication of the extent to which citizens are able to and interested in, engaging with their governments online. By discussing this topic within the context of the extensive literature on the matter, it can be see how important access is to the general population as they attempt to navigate new systems of technological process. The main topics featured by authors in this field are related to accessibility issues, the idea of have and have-nots in society and the digital divide. The number of barriers that exist to access provide an interesting backdrop to a discussion on how governments can implement measures that are more effective in order to reach bigger audience.

Bertot, Gorham, Jaeger, Sarin and Choi (2014) identify one of the central factors to the utilisation of e-government on a wider scale as accessibility. In order for online governmental services to be appropriate for wider use, they must also be accessible for people with disabilities, people whose mother tongue is not that of the titular land and those without high levels of IT skills. Bekkers and Homberg (2007) identify the need for e-government to enhance the access, quality, efficiency and transparency of public administration. Factors that come into play when assessing the operability of e-government in the public sector are closely related to education and age, whereby it is recognised to what extent these two factors impact on accessibility. Younger and middle-aged people with a higher level of education are more likely to recognise the importance of e-government and the need for governmental transparency via open portals accessible to all (Cullier and Piotrowski, 2009). It is also closely related to trust as citizens perceive well-functioning e-government to effective, efficient and participatory (Tolbert and Mossberger, 2006) making the importance of its accessibility vital to its success. Another factor that is taken into account by Haque and Pathrannarakul (2013) is the accessibility for people in developing countries; how governments can implement far-reaching policies that are feasible for places with less internet access. The importance of accessibility is key to the penetration of e-government on a wider scale as it allows for government tools and services to engage with a more extensive range of citizens.

Accessibility issues refer to the way in which citizens have the ability to use and engage with online services. This is a multifaceted concept that can refer to both whether people have access to the services and how they go about using them. The diversity of users means that governments cannot make a one size fits all approach to citizens, especially when there are

(25)

The Usage of e-Government Services in the Baltic States 25

multiple organisations and departments involved (Gil-García and Pardo, 2005). Nam (2014) identifies four different types of access: motivational, physical, skills and usage (p.212). These represent the way in which people may perceive access to public services as potentially limited due to their personal circumstances. In comparison, Helbig, Ramón Gil-García and Ferro (2009) identify the dimensions of digital inequality as autonomy of use, skill, social and purpose technology as being important factor that contribute to lowered barriers (p.92). When combined with the discussion on structural factors such as political and economic disenfranchisement, limited educational opportunities and social mobility and how they actively encourage and discourage interaction with internet technologies in the public sector (Helbig, Ramón Gil-García and Ferro, 2009, p.92), it is possible to see how barriers to access manifest themselves across different social groups. When barriers to access become too problematic and infiltrate large sectors of society, it can create issues for a government that is attempting to instigate wide reaching technological reforms, as IT initiatives are essentially useless if there is a lack of ability to use it on a wider social scale (Helbig, Ramón Gil-García and Ferro, 2009, p.89). Barriers to access such as a lack of computers, internet access and basic electrical and telecommunications infrastructure (Jaeger and Bertot, 2010, p.373) have a large impact on how some of the aforementioned social and economic structural factors (Helbig, Ramón Gil-García and Ferro, 2009, p.92) can manifest themselves in government aims when it comes to e-government initiatives for public services.

2.4.1 Digital Divide

Closely related to barriers is the discussed concept of the “haves and have nots” and the digital divide in society (Helbig, Ramón Gil-García and Ferro, 2009; Nam, 2014; Epstein, Newhart, and Vernon, 2014) which refers to the gaps and differences between different groups in society. This is not a simple phenomenon and manifests itself in across multiple sectors of the population, sometimes with consequences for governments that try to engage across groups with the same tactics. It is important to remember that users are not always a homogenous group (Helbig, Ramón Gil-García and Ferro, 2009, p.93) which means that they have to be addressed in different ways according to their needs and experiences. E-government is intrinsically reliant on citizen willingness to adopt and use these technologies (Nam, 2014, p.217), which means that there has to be attempts from the government side to approach users that may sit outside their realm of interaction. Otherwise, governments run the risk of only engaging and mobilising those with advanced or at least average technical skills, while disenfranchising those with little technical experience (Nam, 2014). Epstein, Newhart and

(26)

The Usage of e-Government Services in the Baltic States 26

Vernon (2014) expand on this notion with a discussion on how those traditionally marginalised in other areas of life are more likely to be marginalised when society moves online as well. This presents a dilemma for governments as they attempt to make wide reaching policies that do not seek to create a further divide within society. This factor of barriers to access and inequality within society are important to the further reaching discussion on e-government implementation and usage.

The digital divide is one of the key concepts of accessibility as it represents the fundamental ability of users to engage with online content. The concept goes beyond the idea of “haves” and “have nots” in modern literature as it goes into further detail about disparities between and within countries and regions (Epstein, Newhart, and Vernon, 2014, p.338). The literature has taken a number of different approaches to the way in which this concept manifests itself in wider society. For Helbig, Ramón Gil-García and Ferro (2009), there are three main approaches: an access digital divide (those who have and those who do not have); the multidimensional approach (between economic opportunities, develop(ed/ing) nations and technical skills) and; the multi perspective approach (types of government, values and beliefs, social structure and institutions). These present different approaches for analysis, focusing on the different levels at which these disparities may occur. Norris (2001) on the other hand, addresses the concept from three different approaches - the global approach (between developed and developing countries); a social divide (the information rich and information poor) and; the democratic divide (between those that engage the internet in public life and those that don’t). A third interpretation, from Epstein, Newhart, and Vernon (2014) separates the concept in three other approaches and identify the following: the motivational divide (the interest of users in engaging with and learning about new technologies); the access divide (operational, information and strategic skills) and; the democratic divide as ascribed by Norris (2001). These approaches show how multifaceted the digital divide has become in recent years with the proliferation of technologies on a wide scale in society. With accessibility being an intrinsic part of electronic services, it is important that it enables even the most disenfranchised users (Jaeger and Bertot, 2010) in order to ensure widespread receptiveness.

The discussion on accessibility of online public services and e-government is vital to this literature review as it highlights the challenges that implementers of technology-based policymaking face when dealing with vastly heterogeneous groups of citizens. By approaching this concept from a number of angles, it can be seen to what extent accessibility challenges

(27)

The Usage of e-Government Services in the Baltic States 27

manifest themselves in e-government implementation and usage from a citizen perspective. Barriers to accessibility such as language skills and disability, the concept of “haves” and “have nots” and the digital divide all provide an interesting framework to discuss the challenges and benefits of technologically provided government services.

2.5 Usage of Online Services by Citizens

Usage of internet-based public services has been discussed extensively in the wider literature on e-government and represents how effective methods from the government level are applied and utilised on the citizen level. Theory on this topic has mainly focused on the ease of use, participation by population group and the reasons for usage. By taking these factors into account, we can collate a thorough understanding of the ways in which people are utilising e-government measures in their everyday lives, in order to later study which external and organisational components influence people’s usage.

Ease of use is one of the factors that heavily influences the extent to which the public is involved with e-government technologies on a daily basis. Carrizales, Melitski, Manoharan and Holzer (2011) define usability as “the degree of comfort of citizens using the website” (p.937), which is the fundamental reasoning behind the design of government websites. Factors such as the number of clicks required to get information (Mangule, 2016) contribute to the overall ease of use, particularly for those lacking high levels of technical expertise. Ease of use differs from accessibility, as it also includes users already engaged in the online world to those just entering. It is important as it represents the way in which everyday citizens can interact with the services they desire on a more basic, more functional level. Carrizales, Melitski, Manoharan and Holzer (2011) discussed factors that make for an easy to use website, such as screen length, availability of PDF files, customised links, branding, structure and hard- and software requirements, showing that the intrinsic design of the website is vital to success. This is backed up by Baldersheim and Oegard (2008), who postulate that a well-designed website that functions well is “an indication of capability of deep-doing organisational change” (p.126). The underlying factors behind usable websites can closely be linked to institutional capacity and data readiness, as it shows the interest in governments to make easy to use, accessible services for their citizens. Ease of use in some countries also has a linguistic notion - whether public services are also available in languages other than the titular (Brubaker, 2011; Bertot et al., 2014). This is important to take into account in countries with large minority populations,

Referenties

GERELATEERDE DOCUMENTEN

For example, Dechow and Sloan (1991) find that CEOs approaching retirement cut R&D spending but equity incentives help to reduce this career horizon problem..

Bij personen die onschuldig zijn en dus valse bekentenissen afleggen wordt een lagere reactietijd op de relevante items verwacht dan bij de ware bekentenissen, omdat de informatie

Purpose: This interview’s feedback will serve as a part of the verification process for the developed framework for decision-making purpose in selecting a proper projects to fill

To ensure a good combination, the three Ts need to be considered, namely: temperature - high enough for ignition purposes; turbulence - to ensure proper mixing of O 2 and fuel;

There are good evidence-based guidelines for platelet transfusion thresholds in different situations, but the limitations must be taken into account, namely that data are

A question therefore arose: Why are nurses not using the Mindset Health e-Learning system effectively for their professional development in a public hospital

Bij afwijkende bevindingen in staande houding of bij specifieke klachten wordt het onderzoek uitgebreid met onderzoek van de abductie en kniehoogte in liggende houding. Bij een

By means of the user tests insight is provided into the kind of problems users of similar websites encountered while looking for volunteer opportunities.. Other volunteer centers