Hacking collaborative innovation : a multiple case study on the aims, process, characteristics and outcomes of a Hackathon

(1)

Hacking collaborative innovation;

A multiple case study on the aims, process,

characteristics and outcomes of a Hackathon

By: Maarten van Bemmel – 11163216 Final Version

Expected date of submission 24/06/2016

MSc Business Administration: Entrepreneurship & Innovation University of Amsterdam

First supervisor: dr. Ileana Maris de Bresser Second reader: Prof. Peter van Baalen

(2)

(3)

Statement of originality 

This document is written by Maarten van Bemmel who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

Abstract

In this thesis, a qualitative study is conducted in order to find out what hackathons are, why and how are they organised in businesses, and with what effects. Build on literature in the field of collaborative innovation and co-design, a multiple case study of four hackathons was conducted, consisting of interviews with the organisers of these hackathons, observations during the hackathons, and a document study. In addition, three experts on this topic were interviewed. The findings of this study tell us that the process and characteristics of a hackathon can influence the functioning of it, and that therefore the organiser should carefully consider how to organise a hackathon. Moreover, it is found that the main reason for organizing a hackathon is to bring separated parties together, and that organisers indicated that this was accomplished by organizing a hackathon. In addition, this study found three novel characteristics of a hackathon – structure, experts & workshops, and facilities – and examined their influence on the functioning of the hackathon.

(4)

Acknowledgements

This thesis would not have been possible without the help of some people, and therefore I would like to thanks them is this way. Firstly, I would like to thank my supervisor dr. Illeana Maris de Bresser for supporting me in this process with her very helpful feedback an asking the questions

that helped me to write what I wanted to write. Moreover, I would like to thank Britta

Schaffmeister, Chris Koehler and Mark Lengton for their support and helpful feedback. But most of all, I would like to thank the respondents of this study, who allowed me to explore the world of hackathons. They have been very open and helpful, and I’m grateful for the fact that I could be present during these four amazing events. Without them, this thesis would not have been possible.

(5)

Introduction

A hackathon is an event of any duration - mostly between 24 hours and a week - where a diverse set of people from various backgrounds come together to solve a certain challenge or problem. The concept originates from the developer’s world, where “hackers” came together to generate and co-create new (software) applications. So not only are new ideas generated, like in more “traditional” brainstorms, but also are the ideas prototyped into a working first version. A famous example is Facebook’s “Like-button”, which is invented and build during an internal hackathon. Nowadays, hackathons are getting more and more popular, not only in IT-related departments or companies. Examples range from the city of Amsterdam who organised a hackathon to find out how Amsterdam could become a smart city, to British Airways who organised a hackathon on a transatlantic flight from San Francisco to London (Constine, 2013), to Rabobank who organises hackathons to accelerate its digital transformation (Keuning, 2014).

Today, being able to innovate quickly and cheaply, test digital products and services in the market, refine them, and release them on a regular basis has become a competitive advantage (McKinsey, 2015). For a long time, centralized R&D, product development, and process engineering groups within firms were the most economical way to design mass-produced products and related production processes (Chandler, 1977 ; Hounshell 1985). However, organizations and companies are facing rapidly and radically changing conditions in which they operate nowadays (Baldwin & von Hippel, 2011), which forces them to move away from this centralized innovation approach. This has caused more and more parties in the private (Baldwin & von Hippel, 2011), and public (Hartley et al., 2013) sector to move from producer innovation to collaborative innovation, which opens the innovation process to a diversity of actors across hierarchies and organisations (Nambisan, 2008). This collaborative innovation thus includes multiple parties to collaborate on a particular topic, where the selection, prototyping, and testing of promising ideas is strengthened by the diverse set of actors who help asses gains and risks (Bommert, 2010).

(7)

Since hackathons connect actors from multiple parties and functional backgrounds, and they entail a process of accelerated ideation and prototyping, it can serve as an excellent collaborative innovation tool in this rapidly changing environment in which organizations have to operate today. This study aims to build on the current literature in the field of collaborative innovation by studying this ‘newcomer’ to this field; the hackathon. The articles and reports that have been published tend to only describe – in very little detail – the process and characteristics of a hackathon (e.g.,Briscoe & Mulligan, 2014; Komssi et al, 2015). Moreover, these articles do not discuss how the process and characteristics actually influence the functioning of a hackathon. In addition, the rising number of hackathons being organised today (O’Connell, 2015) causes to wonder why these “hack-marathons” are organised. Is it because of the output that they produce, is it because of the cross-functional collaboration that they often promote, or are there any other reasons? Moreover, what is the difference between the expected outcome of the hackathon by the organiser, and the actual outcome of a hackathon? So once the hackathon is over, what is the result? Until today, no single academic article has attempted to find out what the reason is for organising a hackathon, and whether it can deliver the desired outcome.

Considering this gap in literature related to the aims, processes and outcomes of a hackathon, this research has the following research objects:

• To find out what type of problem leads to organising a hackathon. • To find out what the reasons are for organising a hackathon. • To find out what the process of a hackathon is.

• To find out what the characteristics of a hackathon are, and how these influence the functioning of the hackathon.

• To find out what the outcome can be of a hackathon.

• To find out whether there is a difference between the expected outcome of an organiser and the actual outcome of a hackathon.

(8)

Please note that the objectives are grouped in three subsets. Each subset of objectives represents a different phase. The first subset consists of research objectives that aim to answer questions relevant before the hackathon. The second subset aims to do this for objectives related to the process and

characteristics during the hackathon, and the third is related to what occurs after the hackathon. Both the literature review and findings section will be structured in this way. A more detailed description of the structure of this thesis can be found and the end of the introduction.

To meet the research objectives, four case studies of hackathons are examined. Theoretical insights from the field of collaborative innovation, (co-) design and hackathons are used to offer an explanation of the aims, processes and outcomes of a hackathon. The Research Question that guides this study is:

• What are hackathons, why and how are they organised in businesses and with what effects?

Finding an answer to this question is important both in terms of theory and practice. The theoretical relevance of this thesis is that it builds on the current knowledge of hackathons and collaborative innovation that is available today. It is a topic that is not widely discussed in academic literature yet, while the hackathon is widely used by business and organizations. This is why there is a high theoretical relevance to get a better understanding of this new phenomenon, so that literature on the topic of collaborative innovation stays up to date.

There are several reasons why this thesis is of practical relevance. Firstly, the vast number of hackathons being organised by business could cause practitioners to wonder whether they should also organize a hackathon or not. Therefore, it is crucial to understand for what kind of challenges hackathons are organised nowadays, and what can be achieved by organising a hackathon. If known, managers can decide themselves whether a hackathon could add value to their organization or not. If decided that a hackathon should be organised, it is necessary to have an understanding of what a hackathon looks like in practice, and what its elements and characteristics are. Thus, this thesis will

(9)

help practitioners to better understand when and how to organize a hackathon and is therefore of high practical relevance.

The thesis starts with a review of the academic literature in the field of collaborative innovation and hackathons. Based on the literature review, a conceptual model will be presented which will serve as the guideline for the rest of the thesis. In the methodology section, a short overview of the studied cases will be given together with a detailed description of the methodology used for this thesis. The collected data will be presented in de findings section, followed by the discussion in which the research question to be answered. The thesis ends with a conclusion, and references and appendices can be found in the back of the thesis.

(10)

Literature Review

The literature starts with an overview of the main themes studied in the field of collaborative innovation. Then, a description and a definition of the hackathon will be given, followed by a review of literature about the type of problems that can lead to organising a hackathon. Then, the process and characteristics of the hackathon are described based on current literature on hackathons. Lastly, a description of the possible outcomes/effects of a hackathon will be given. In each section it will be stated how the concepts presented during the literature review will be used in the study that is being done to answer the research question.

Collaborative Innovation

Theories in the field of collaborative innovation have so far mainly focused on co-creation (Prahalad and Ramaswamy 2004), and open innovation (Chesbrough 2006). As Prahalad and Ramaswamy (2004) state, co-creation is about joint creation of value by the company and the customer. It is not the firm trying to please the customer, but it allows the customer to co-construct the service experience to suit his or her context. This co-construction can also be found in open innovation, although open innovation is centred around the exchange of information between two producing parties rather then a producing and a consuming party (Chesbrough, 2006). In his widely cited article, Chesbrough (2006) divides open innovation into inbound and outbound open innovation. He states that a competitive advantage often comes from inbound open innovation, which is the practice of leveraging the discoveries of others. Thus, so he argues, companies should not rely exclusively on their own R&D. In addition, outbound open innovation suggests that rather than relying entirely on internal strengths, companies can look for external organizations with business models that are better suited to commercialize a given technology (Chesbrough, 2006).

(11)

Although there are differences to be found in these different innovation approaches, they both share the underlying assumption that tapping into the vast innovation assets across organizational boundaries will increase the quantity and quality of innovations. (Bommert, 2010). The generation of new and creative solutions is enhanced when different ideas are developed, combined, challenged, and built on (Hartley et al., 2013). The principal idea of collaborative innovation is to open the innovation process to a large group of actors, to internalize external ideas but also to leverage internal knowledge externally (Hartley et al., 2013).

In his his review on the benefits of collaborative innovation, Bommert (2010) argues that collaborative innovation can be beneficial in various ways. Firstly, he argues that collaborative innovation strengthens the ideas generated, due to the variety of knowledge and expertise involved. When selecting ideas, collaborative innovation can prevent “groupthink” due to the fact that multiple parties are involved in the decision, so it prevents people to easily agree with each other. Lastly, it is stated in this article that collaborative innovation benefits the implementation and diffusion of the selected idea, since is actors who have participated in the idea generation and/ or selection process are more likely to accept and promote innovations, because of having ownership and responsibility.

Knowing the benefits of collaborative innovation, we must now understand in what ways collaborative innovation can be facilitated. In other words, in what forms is this collaborative innovation taking place? In their article, Antikainen et al. (2010) propose several types of online toolkits based on a case-study amongst open innovation communities, but in order to gain a better understanding of this offline collaborative process we must understand the work that has been done in the field of co-design.

In their article, Cross & Cross (1995) were pioneers in researching co-designing as a social process. Where literature mainly focused on individual design before, they observed teams of designers in order to gain a better understanding of this collaborative design process. They observed 6 aspects in their study; (1) roles and relationships, (2) planning and acting, (3) information gathering and sharing, (4) problem analysing and understanding, (5) concept generating and

(12)

adopting, (6) conflict avoiding and resolving. They found that informal roles are established in the beginning of the design process, and that teams have different approaches in terms of how they plan their time. Since design behaviour is characterised by ‘opportunistic’ behaviour – meaning that designers tend to deviate from planned activities in order to pursue ideas as they occur –, the authors assume that this can create difficulties in teamwork. However, there study finds that although teams have different approaches to their planning, there is still room for unplanned activities in order to pursue a particular idea. Moreover, they find that the designers are willing to trust each other on personal knowledge and information that has been gathered. In terms of problem understanding, there tends to be a gap between the understanding of the given problem statement, and the internalized conceptualizing or grasping of the problem so that the whole group agrees on the ‘real’ problem. The latter was found to be lacking when designing in a group. In terms of concept generating, it was found that the groups tended to be perform rather well on this, being able to build upon the ideas of others. Finally, their study found that disagreements occur, but they were resolved or avoided in a socially skilful way.

In order to compare the hackathon with other collaborative innovation techniques, this study will compare the three benefits of collaborative innovation (Bommert, 2010) and the six aspects of co-design (Cross & Cross, 1995) with the findings of this study on hackathons.

The “Hackathon” phenomenon

It is hard to tell when the first hackathon was organised, but it was in 1960 already when students of computing classes at MIT worked in “marathon bursts” in order to complete a line of coding (Levy, 2010).The term hackathon was firstly introduced on the 4th of June 1999, during a gathering where developers of both Sun Microsystems (nowadays part of Oracle) and OpenBSD came together collectively to write a computer program in the Java programming language for the new Palm V handled computer (Briscoe & Mulligan, 2014). Soon others followed in organising short hack-marathons to write new code in a short amount of time, and the hackathon was born. From the 2000s

(13)

on hackathons became more wide spread, and began to be increasingly viewed by companies and venture capitalists as a tool to quickly develop new software technologies, and to locate new areas for innovation and funding (Briscoe & Mulligan 2014).

The hackathon has ever developed from then on, and has already spread beyond the conventional tech world (Leckart, 2012) resulting in a wide variety of hackathons that are being organised today; hackathons to improve education (Hunckler, 2015), to help veterans (Mochari, 2016), to promote clean energy (Zapico et al., 2013), to fight autism (Tam, 2013) and women-only hackathons (Kaul, 2016). These are just a few examples of the wide variety of hackathons being organised today. Fortune 500 companies have also started to pick up on the hype. Firms like Microsoft, Nokia and even big food and goods retailers like Unilever have organised hackathons recently (Leckart, 2012).

Definition

There is no generally accepted definition – yet – of a hackathon in today’s academic literature. Although most definitions are fairly similar, small but not unimportant differences can be seen between these definitions. Rossell et al. (2014) define a hackathon as an event where people come together to collaboratively build and launch a new application or finished good, aimed at solving a particular problem on top of new or existing technology enabler. Mohajer Soltani et. al. (2014) define people more specific; they define a hackathon as events in which programmers, developers and

sometimes individuals from other disciplines collaborate on a software project in a friendly environment by generating a solution to a beforehand specified problem. Chowdhurry (2012) adds that a hackathon is focused on rapidly iterative (software) development trough which groups design, code and build testable prototypes. An overview of all academic definitions of a hackathon can be found in table 1.

(14)

Type of design problems

In order to find out what kind of challenges will cause people to organize a hackathon as a tool for a possible solution, we must first gain a better understanding of what type of design problems there are. In other words, what kind of challenge can possibly lead the organiser to organise a hackathon? In the field of design theory, these problems are called design problems, and several types of design problems can be identified; simple, complex and wicked. In her article, Roberts (2000) describes these three design problems in more detail. According to her, simple problems have consensus on a problem definition and solution. Problem solving in this case is straightforward and little conflict will take place among those involved. Within a short period of time, problem solvers recognize what the problem is and activate established routines and standard procedures to deal with it. An example of this is an electrician who has to fix a broken light. He/she identifies that the light is broken, and replaces it with a new one. One can argue that in this case the problems are determinate (Buchanan, 1992), because they are bound by definite conditions.

Article Definition of a hackathon

Chowdhurry (2012)

A small event where, over the course of day or week, programmers and developers collaborate intensely to build prototypes, focused on rapidly iterative software development through which groups design, code, and build testable prototypes.

Mohajer Soltani et al. (2014)

An event in which programmers, developers and sometimes individuals from other disciplines collaborate on a software project in a friendly environment by generating a solution to a beforehand specified problem.

Rossel et al. (2014)

An event where people come together to collaboratively build and launch a new application or finished good aimed at solving a particular problem on top of new or existing technology enabler.

Lodato & DiSalvo (2016)

A rapid design and development events at which volunteer participants come together to conceptualize, prototype, and make (mostly digital) products and services.

(15)

When facing a complex problem, problem solvers agree on what the problem is, but there is no consensus on how to solve it. Therefore, conflict might arise amongst problem solvers when discussing the solution. These complex problems are therefore undetermined (Buchanan, 1992), requiring further investigation by the problem solvers to make them determinate. An example of a complex problem is when the public roads are unsafe, but problem sovlers do not agree on how the roads can become safer. Is it by limiting speed, safer cars, more space etc. Thus, multiple reasons can be thought of to explain unsafety of the road. Complex problems are thus similar to simple problems in the sense that the problem is agreed upon, but they differ from simple problems due to the fact that problem solvers do not agree on the solution.

Lastly, she defines a wicked problem as: there is no agreement on the problem, nor on the solution. The introduction of wicked problems into modern literature dates back to the late sixties, when Churman (1967) first introduced the topic in his article in Management Science, and Rittel and Webber (1973) later defined ten characteristics of wicked problems, followed by Concklin

(2005) who was able to define a wicked problem by only six characteristics (see Table 1). In his article, he describes the process of solving a wicked design problem in detail. He mentions a study that was conducted among designers at the Microelectronics and Computer Technology Corporation (MCC) in 1980, where it was found that designers worked simultaneously on understanding the wicked problem and formulating a solution. In addition to this finding, Dorst & Cross (2001) found in their study that [creative] design seems more to be a matter of developing and refining together both the formulation of a problem and ideas for a solution rather than these two to be separated. Since it is expected that the problems will be formulated by the organisers of the hackathon prior to the event, it will be interesting to explore how this is dealt with by the participants of the hackathon.

1. The problem is not understood until after the formulation of a solution. 2. Wicked problems have no stopping

rule

3. Solutions to wicked problems are not right or wrong

4. Every wicked problem is essentially novel and unique

5. Every solution to a wicked problem is a ‘one shot operation.’

6. Wicked problems have no given alternative solutions

Figure 1: The 6 characteristics of a Wicked problem (Concklin, 2005)

(16)

Buchanan (1992) states that design problems are by definition wicked problems, characterised by the indeterminate nature of problems, meaning that there are no definite conditions or limits to design problems.

This thesis builds upon the current knowledge of design problems by investigating what kind of design problems leads to organising a hackathon; a simple, complex, or wicked problem, and whether the problem and solution definition are defined simultaneously or separately.

Process of a Hackathon

According to Komssi et al. (2015), a hackathon typically begins with the process of team forming and ideation. These ideas can be generated prior or during the hackathon, depending on the preference of the host. After the idea selection phase, the prototyping phase starts where the teams aim to build a prototype which they will then present during the last pitching phase of the hackathon. Below a short overview of literature on each of these phases is given, except for the pitching phase which is not included in the study. This is done because because the study of pitching is a topic too distant from the main objective of this study.

Team forming

In the field of management science and occupational psychology, the influences of team diversity, and the novelty of group members on team performance have been widely discussed. In their article on the influence of team diversity on team performance, Mannix & Neale (2005), argue that there is a difference between diversity on the surface level, and underlying differences. They state that differences on the surface level – differences in race, gender and age – negatively influence team performance, commitment and satisfaction of them team. In contrast, underlying differences like functional background, education and personal differences have a positive influence on creativity and problem solving. This is important to know, since especially creativity and problem solving are needed during a hackathon. Therefore, it is expected that teams during a hackathon are diverse in

(17)

Choi & Thomson (2005) have examined a relationship between the novelty of team members and and the creativity of a team in their study. The positive effects according to them are that new team members bring new perspectives into group, and that it diversifies the knowledge and skills of a group. Moreover, it stimulates social processes since group members are new to the group, which in turn might improve performance of the group. However, the negative effects might be that the newness of the group causes conflicts to arise, and that is is harder to build up working styles and routines in the group. Therefore, it will be interesting to look at which groups during the hackathons perform better; groups that are formed prior to the hackathon, or groups that are formed during the hackathon.

Ideation

The ideation - or idea generation - phase is the phase where the group members will come up with ideas, often referred to as brainstorms. It was advertising agent Osborn (1957), who firstly introduced the term brainstorming to the wider public in his book Applied Imagination. He argued that working in teams leads to multiple creative stimuli and to interaction among participants, resulting in a highly effective process. According to Osborn (1957) people have a natural tendency to be selective in the expression of their ideas. Thus, he argued, people always generate fewer ideas than they could have generated. Collective – in a group – brainstorming is designed to take away this selective behaviour and to maximise one’s creative productivity. As the number of generated ideas or solutions increases, it becomes more likely that some of the best possible solutions are among those ideas. Note that the emphasis here is on the quantity of ideas rather than quality of ideas.

Since then, many scholars - mainly in the social psychology and design literature - have discussed the concept brainstorming, and later other types of idea generation techniques. Diehl & Strober (1987) argued that that the concept of brainstorming is ineffective, and that an individual method should be used instead. They found that individuals generate more ideas than groups because

(18)

like the idea), and production blocking (only one person at a time can speak, causing the others to wait, limiting the number of ideas that can be spoken of) that all occur in a group setting. Combining both an individual and group approach, Stroebe & Diehl (1994) came up with a hybrid form, where a combined approach results in better results according to their study. Girotra et. al, (2010) have shown in their empirical study on the quality of the best ideas within both team and hybrid settings, that the quality of the best ideas is highest in a hybrid setting. Moreover, they found that building upon others’ ideas is counterproductive. Teams using this technique neither create more ideas, nor are the ideas that build on previous ideas better.

This study aims to find out what ideation techniques are used during a hackathon; collective, individual or hybrid brainstorming.

Idea selection

While generating ideas is of obvious importance for a hackathon, selecting the the best idea to continue with is just as crucial. In general, organizations strive to select the best ideas and eliminate the worst. Therefore, the procedure used to select ideas to be implemented is of concern for both practitioners and theorist. Surprisingly though, academic literature has been mainly focused on the topic of idea generation. It was only recently that this issue has been addressed by several scholars. These recent studies have have provided better insight into how individuals (Rietzschel et al., 2010) and groups (Faure, 2004), select ideas. In their extensive empirical study, Rietzchel et al. (2010) found that generation of creative ideas does not automatically lead to the selection of creative ideas and that creative idea selection benefits most from specific selection criteria. In addition, people appear to have a strong preference for ideas they believe can and should be adopted, and seem to believe that this is incompatible with the selection of original ideas. Their study found that an idea that is not very original, but very familiar, may be judged more favorably than an original, but unfamiliar idea. Moreover, the results showed that people tend to select ideas they believe in, rather than creative ideas.

(19)

Yet in organizations, a consensus must often be reached about the idea(s) selected and thus a group selection of ideas might be more appropriate than an individual selection (Faure, 2004). In her

study, Faure (2004) investigated the effects of two factors – generating ideas interactively or not and judging one’s own ideas or ideas from another group – on a number of outcomes, including performance at the idea generation stage, quality of the idea selection process, and satisfaction with both the ideas selected and with the group. During the selection process, the common belief is that the best practice for commitment purposes is to let the groups evaluate the ideas that they themselves have generated. But the results in this study show a more complex picture. On one hand, while the originality and practicality of the ideas selected does not differ for groups selecting from their own or from another group’s set of ideas, groups selecting from their own sets were found to select more effective ideas than groups selecting from a set of ideas generated by another group (Faure, 2004). This indicates that a group should therefore aim to select their own ideas.

Thus, based on literature it can be stated that it is best for groups to let groups select the ideas that they generated themselves (Faure, 2004), without the use of any specific selection criteria, since this hinders the selection of the best idea (Riezchel et al., 2010). In this study these two factors – individual or group selection & own or other’s ideas – will be used to find out how ideas are selected during a hackathon.

Prototyping

Once the idea that the team wants to continue working with is selected, participants start working on a prototype. Although many scholars do explicitly not state one particular definition of a prototype due to its dynamic nature (Lim et al., 2008), Beaudouin-Lafon & Mackay (2003) have a provided a definition that provides good guidance. They define a prototype as a concrete representation of part or all of an interactive system. It is a tangible artifact, not an abstract description that requires interpretation. Designers, as well as managers, developers, customers and end- users, can use these artifacts to envision and reflect upon the final system.

(20)

Beaudouin-Lafon & Mackay (2003) have defined three characteristics of a successful prototype, which helps us to understand how a prototype can help participants in a hackathon. Firstly, a prototype supports creativity, helping the participant to capture and generate ideas, facilitate the exploration of a design space and uncover relevant information about users and their work practices. A prototype also encourages communication, helping designers, engineers, managers, software developers, customers and users to discuss options and interact with each other. Lastly, it permits early evaluation since a prototype can be tested in various ways, including traditional usability studies and informal user feedback, throughout the design process – or in this case the hackathon.

Knowing that developing a prototype can help participants to capture on ideas, communicate them and evaluate them in an early stage, it is now important to understand the distinction in the fidelity of a prototype. The concept of fidelity indicates the similarity of a prototype and the final product. Prototypes are usually classified as low-fidelity or high-fidelity (Rudd et al., 1996). Low-fidelity prototypes showcase design alternatives, concepts and screen layouts rather than describing a

systems functionality in detail. It is often rapid and it occurs early in the design process. Low-fidelity prototyping methods are popular for validating designs and uncovering requirements with very low cost early in the design process (McCurdy et al., 2006). High-fidelity prototypes already include the complete functionality and interactive features of a service or a product, and it is therefore hard to tell the difference between a fidelity prototype and a finished product. In addition, producing a high-fidelity prototype is significantly more time consuming an expensive compared to low-high-fidelity prototypes. Interesting to note is that according to Sauer et al. (2010), users are actually able to imagine how the final product would appear based on the lower fidelity prototypes as well. An overview of the advantages and disadvantages of low- and high-fidelity prototypes can be found in Table 2 below.

(21)

Table 2: Advantages and Disadvantages of Low- and High-Fidelity protoypes (Rudd et al., 1996)

The tools used to create these low- or high fidelity prototypes are categorized depending on their fidelity. Carter & Hundhausen (2010) defined six classes for different types of prototyping tools based on an empirical study amongst UX designers: art supplies, graphics editing software, presentation software, HTML software, programming language and prototyping software. Their study also found that art supplies are by far the most widely used prototyping medium, this is most likely due to the fact that art supplies are widely available, quick, and easy to use.

Low-fidelity tools are quick and easy to use for creating early designs and promoting collaborative sketching. They are also better than high-fidelity tools at generating design feedback, because users often mistake high-fidelity prototypes for the final product and are therefore not in the best possible mindset to provide feedback. In turn, low-fidelity tools can not model complex interactions. (Carter & Hundhausen, 2010). High-fidelity tools are good for creating rich user experiences by accurately modeling the final product. Also a higher level of reuse is possible with the high-fidelity tools

Advantages Disadvantages

Low-Fidelity Prototypes

• Lower development cost. • Evaluate multiple design

concepts.

• Useful communication device. • Address screen layout issues. • Useful for identifying market

requirements.  • Proof-of-concept.

• Limited error checking. • Poor detailed specification

to code to. • Facilitator-driven. • Limited utility after

requirements established. • Limited usefulness for

usability tests.

• Navigational and flow limitations. High-Fidelity Prototypes • Complete functionality. • Fully interactive. • User-driven. 

• Clearly defines navigational scheme.

• Use for exploration and test • Look and feel of final product. • Serves as a living

specification.

• Marketing and sales tool.

• More expensive to develop • Time-consuming to create.

Inefficient for proof-of-concept designs. • Not effective for

(22)

compared to low-fidelity tools. On the other hand, many high fidelity tools are time consuming and difficult to learn.

In this study it will be investigated whether low-fidelity or high-fidelity prototypes are being created during a hackathon.

Characteristics of a hackathon

Hackathons can be an internal or external event (Komssi et al, 2015). During an internal hackathon, employees of an organisation aim to “hack” the challenges of the company they work for, without any presence of outsiders. An external hackathon on the other hand involves people from all different kind of companies, and participation is not restricted to employees only.

For large organisations in particular, hackathons can serve to accelerate the process of digital transformation. They are less about developing new products, but more about “hacking away” old processes and ways of working. According to a McKinsey (2015) report, giving management and employees the ability to actively work together during a 24-hour (or more) hackathon will help big organisations delivering breakthrough innovation at start up speed. This same report states several characteristics of a hackathon. These characteristics will be used during this study for two reasons. Firstly, these characteristics serve as a guideline in this fairly new topic, and are being used to explore whether these characteristics are also to be found in this multiple case study, and whether new characteristics are to be added to the current list. Secondly, this study builds upon this McKinsey (2015) report, by investigating how these characteristics influence the way the hackathon functions. The first characteristic of a hackathon is that it is focused purposive on a single challenge and it supports a clear business target - e.g. speed, revenue growth, or customer experience. In this study, it will be looked at to what extent the challenges of the hackathons are focused on a single problem, or on multiple problems, and whether it supports a clear (business) target. Secondly it is deeply cross-functional, bringing together people from across different sectors to force different ways of working

(23)

influences the way the hackathon functions. Thirdly, its stimulated to build ideas from scratch and to reimagine an idealised method for a given customer need. In this study it will be examined how participants are stimulated by the organisation of the hackathon to toss aside traditional notions of how things are done, and to work on truly new concepts. Another characteristic of a hackathon is that the output includes a clear development path, stating all the stops that should be taken after the hackathon in order to embed the prototype in the organisation. In this study, it investigated whether the included hackathons actually have a clear development path. Lastly, - as mentioned earlier - hackathons should be concrete and focused on output. They start with ideas, but end with a working prototype. Since this overlaps with the process of the hackathon that is already described, this characteristic will not be considered in this study.

Possible outcomes of a Hackathon

Now another question arises: Why are hackathons actually being organised, and what do the participants and organisers of a hackathon get out of it? We do know a little about the reasons for attendance of participants. According to Briscoe & Mulligan (2014), the main reasons for attending

a hackathon are learning and networking. The first can be explained given that the nature of software development has a strong element of life-long learning, caused by the continuing emergence of new technologies. Other reasons for attending are ‘changing the world’ and winning prices.

Yet, it is still unknown what outcome the organiser aims to achieve by organising a hackathon, and what happens with the output created during a hackathon once finished. Based on a review of literature of both hackathons and collaborative innovation, the following could be the reasons to organise a hackathon.

New products/solutions

As mentioned before, the main goal of a hackathon is to generate ideas, develop them into a prototype, and being able to present a working version at the end of the hackathon. This indicates that the main reason for organising a hackathon is generating new prototypes. In the healthcare industry for

(24)

example, hackathons have proven to be very successful so far (Chowdhury, 2012). Both heath professionals and technology experts are trained in their own domains, each with their own jargon and ingrained methods and perspectives (Chowdhurdy, 2012). Bringing these two groups together can lead to totally new and unexpected solutions. Like in many industries, those who have the frontline experience (Medical experts) are normally separated from those with the technical skills to build solutions (Coders, Developers, Programmers). During Hacking Health hackathons these two parties are brought together, taking away the physical barrier to collaborate and draw upon each other’s skills and knowledge . But it is hard to draw any conclusions on what happens after the hackathon. Are some of these new prototypes actually implemented? Chowdhury (2012) clearly states that the lack of empirical data on the long-term impact of hackathons in generating lasting entrepreneurial activity is an important area for future research.

Fostering cross-functional collaboration

During a case study, Raatikanen et al (2013)found out that one of the major benefits, as perceived by the participants of an internal hackathon at a security software firm, was the fact that they met colleagues from other departments. After the hackathon, they had a better idea of that kind of people they were and what kind of work they were doing, providing seeds for more efficient collaboration in future projects. Rosell et al. (2014) mention that organising a hackathon can bring together employees from various organizations in a large corporation to dedicate energy to a creative endeavour outside their normal work responsibilities, and that it resulted in a wide range of valuable ideas and innovations that would otherwise not have come to light.

Recruiting talent

Many firms, of which Google and Facebook are two of them, organise college hackathons as a recruitment tool. The students will evidently have a chance to work on an interesting challenge, but for the hosting party the underlying reason for organising a hackathon can be to recruit students or participants who they see as a potential employee for their own enterprise. Rather than asking for

(25)

examples of the past to test a candidate’s capabilities, recruiters can actually observe real skills and behaviour during a hackathon in real live (Purohit, 2014). This might be the reason that the hackathon as a recruitment tool is rising in popularity.

Creating awareness

Another reason for organising a hackathon might simply be the awareness that it creates around the hosting company or organization. Companies like British Airways, Microsoft, Nokia, and many more have organised external hackathons - of which some on very exclusive locations like a Boeing 747 - and they are certainly not reluctant when it comes down to sharing this with the wider public. Therefore, one might argue that the positive image that organising a hackathon creates might be of more importance to these organising parties then the actual outcome of the hackathon itself.

These are possible reasons for organising a hackathon based on the literature review, and hence there might be other reasons for organising a hackathon than the ones stated above. This is what this thesis aims to find out. In addition, it will be examined whether there is a gap between the expected outcome of an organiser and the actual perceived outcome of the hackathon. In order words, does the hackathon deliver what the organising party aims for?

(26)

Conceptual Model

Like the research objectives, the literature review and the findings, the conceptual framework is divided into three phases: before, during and after the hackathon.

In each of the phases it is shown what this study aims to find out. Firstly, this thesis aims to find out what type of challenges are aimed to be solved by organising a hackathon. These can either be simple, complex or wicked. Then, both the process and the characteristics will be described based on collected data. This will also allow us to compare this process and these characteristics as described by the available literature with the findings of the study, and provides room for possible additions or adjustments to what we currently know. In addition, the influence of the characteristics on the functioning of the hackathon will be studied. Lastly, the possible outcomes that are based on the literature study will be compared with the actual outcomes that will be given in the findings, and the expected outcome of the organiser of a hackathon will be compared with the actual outcome. By doing this, we will be able to answer whether there is a difference between the expected outcome and the actual outcome of a hackathon.

(27)

Methodology

Research type and design

Since this study aims to gain a better understanding of this new phenomenon, an interpretive qualitative study will be conducted, consisting of multiple cases – hackathons –. An interpretive study will help to discover and understand the phenomenon, process and perspectives of people involved in the topic (Merriam, 2002). Moreover, case studies provide an opportunity to gain a deep holistic view of the research problem, and may facilitate describing, understanding and explaining a research problem or situation (Tellis, 1997; Baxter & Jack, 2008). This suits the novelty of the subject, and the interpretive nature of the study. Descriptive case studies try to completely describe different characteristics of a phenomenon in its context and so they are also mainly used for theory building (Yin, 2009). In addition, case studies are appropriate for answering how and why questions. The specific questions can be identified and developed by closely examining any previous studies and identifying any suggestions/opportunities for future research (Tellis, 1997).

Preliminary Study

As a preliminary research, I attended a hackathon organised by the University of Amsterdam, The Amsterdam University of Applied Science, The Amsterdam University of the Arts and the Gerrit Rietveld Acadamy. During this 3-day long Grit Project, students from these different universities and different backgrounds gathered to come up with innovative solutions for clients like energy provider Nuon and theatre De Melkweg. At the end of this event, teams had to pitch their solution to a panel

of professionals, and a winner was chosen. This preliminary study allowed me to gain some first insights on what a hackathon looks like, and what an interesting research questions would be. Description of Cases

Four hackathons were chosen for this multiple case study. Each of these hackathons was organised around a particular issue or topic. The hackathons that will be included in my study are the Climathon,

(28)

a Smart Road Hackathon, the EduChallenge and a Hackathon organised around the disease Multiple Scleroses; The MS Hackathon. In table 3, an overview of the selected hackathons is given.

The hackathons were selected based on the the fact that they are all hackathons, and all organised around a particular topic or theme. Moreover, they are related to four different themes (Climate change, Mobility, Health, Education), which will give me a wider coverage than simply focusing on one type of issue-oriented hackathon. I gained access trough the first three hackathons by directly contacting the organisers of the hackathons, who were all very willing to cooperate on this new topic. Moreover, offering to provide feedback on their hackathon once the thesis is finished made it even easier to gain access (Walsham, 2006). The last case was even easier to be involved in, since it was facilitated by PwC.

The Climathon was a hackathon that was organised by the municipality of Vejle City (Denmark) because the city has to cope with flooding due to increasing cloud bursts, rising sea level and an increased volume of river water. The Climathon was organised in order to find ways to cope with this pressing situation.

The Smart Road Hackathon was organised during the Dutch Innovation Expo, which was held in Amsterdam to celebrate the Dutch presidency of the European Union. It was organised with the aim start thinking about how the Dutch Ministry of Infrastructure should prepare the Dutch roads for self-driving cars. Specialists from different industries – finance, energy, mobility, construction,

Theme Duration Teams

Team-formation in advance? Ideation in advance? Number of sub-challenges

Climathon Climate _Change 24 hours (non-_stop) 6 No No 3

Smart Road

Hackathon Mobility Smart 8 hours 9 No No 6

EduChallenge Education 24 hours (3x 8 _hours) 7 Only 1 team No 3

MS Hackathon Healthcare 36 hours

(non-stop) 21 Yes Yes 1

(29)

government – gathered to collectively come up with innovative ideas to be used by the Dutch government.

The EduChallenge was organised by the Utrecht Medical Centre and the University of Utrecht surrounding the topic of innovation of education in the Netherlands. All students studying in The Netherlands were invited to come to the EduChallenge to hack into the current educational system.

The challenge was to build an education technology concept and prototype on the basis of own experiences, ideas, and problems

The MS Hackathon, facilitated by PwC, was organised in order to find innovative ways for people with the Multiple Sclerosis disease and medical experts to cope with this disease.

Data collection methods

Yin (2005) argues that any case study findings are likely to be more convincing and accurate if theyare based on several different sources of information. Therefore, the study consists of both semi-structured interviews, direct observations and a document study. While interviewing is often an efficient and valid way of understanding someone’s perspective, observation can enable you to draw inferences about this perspective that you couldn’t obtain by relying exclusively on interview data (Maxwell, 2012). A document study is done because the use of additional data of the websites of the hackathons and the guidebooks for participants gave interesting insights.

Interviews

Semi-structured interviews based on the interview techniques as proposed by Leech (2002) were conducted with the organisers of the four hackathons that are included in this study, and with three experts in the field of hackathons.

The interviews that were conducted with the organisers of the four included hackathons were conducted before and approximately a month after the hackathon (8 interviews in total). This approach was chosen to be able to compare the expected outcome of each individual organiser had prior to the event with the actual perceived outcome of the organiser a month after the event. The

(30)

interviews were conducted according to the General Interview Guide Approach, which is more structured than an informal conversational interview, but it still provides quite a bit of flexibility in its composition (Turner, 2010). This approach was chosen due to the newness of the topic, and the interprative nature of this study. This approach helped to ensure that the same general areas of information was collected from every single interviewee, but it still allowed a degree of freedom and adaptability during the interview. An overview of the interview objectives and examples of questions can be found in the table below.

Table 4: Objectives and Example questions of interviews with organisers of the 4 included hackathons

In addition, I conducted interviews with three experts on this topic. ‘Expert 1’ was a Director from PwC in the US who had attended and organised multiple hackathons. The second expert (Expert 2) I interviewed was the Marketing Director from BeMyApp, a fast-growing French start-up that organizes hackathons for multinational companies. The last expert (Expert 3) was from a multinational Dutch-based bank, who have organised multiple hackathons as well. These experts were not related to one of the four hackathons who were part of the case study. By interviewing these

Interview before the

hackathon Interview after the hackathon

Objectives To find out:

• What the challenge is that lead the organiser to host a hackathon • Why the organiser chose for a

hackathon as tool

• What the organiser expects from the hackathon itself

(Process/Characteristics)

• What the organiser wants to get out of the hackathon (Outcome)

To find out:

• How the organiser experienced the hackathon • What happened with the ideas/prototypes

generated during the hackathon • To which extend the hackathon met the

expectations that the organiser had before the hackathon

• How the process and characteristics influenced the function of the hackathon • What is achieved with the hackathon

Example questions

1. What challenge or problem have led you to organize a Hackathon? Can you explain the challenge?

2. What do you expect the upcoming hackathon to deliver?

3. What does the hackathon needs to deliver in order for you to be satisfied with the result?

1. Did the hackathon meet your expectations? 2. What happened with the ideas/prototypes

generated during the hackathon thus far? 3. How did the process and characteristics

influence the hackathon?

4. What did you (or your organisation achieve by organising this hackathon?

(31)

experts, I aimed to get a better understanding of the concept of hackathons by including more data sources than just the four cases. They were selected based on their vast experience with hackathons and I gained access to these experts by contacting them directly.

The interviews all took places as expected on beforehand, with the note that some interviewees were more elaborate in their wording than others, which required the use of additional questions were needed. Since semi-structured interviews were conducted, there was room for these additional questions.

Eight out of eleven interviews were done via Skype, and recorded via recording software. The others were conducted face-to-face. On of these face-to-face meetings was with the expert of the big bank, and this interview took place at his office. The other two were with the organiser of the Smart Road Hackathon, of which one took place in a coffee bar, and the other in a co-working space. These face-to-face interviews were recorded via an audio recorder. The audio files – of both the Skype interviews and the face-to-face interviews – were transcribed, which allowed me to code the data in specific coding software. A further explanation on the coding of the data is given in the data analysis section. The interview guides can be found in the appendix.

Observations

In general observations are used as a research method in two distinct ways – structured and unstructured. Where a structured observations guide the observant by the means of an observation scheme, this is not the case with unstructured observations. These unstructured observations are commonly used to understand and interpret a particular – and new - phenomena (Mulhall, 2003). Therefore, unstructured observations were conducted for this study.

During all four hackathons, I personally observed directly to find out what the phases and characteristics of a hackathons were, and how these characteristics influenced the functioning of a hackathon. Everything that was observed was recorded verbally on a tablet, and these field notes were later processed on a computer. Added to these field notes were memos with ideas and

(32)

preliminary conclusions that occurred during the observations. Doing direct observations allowed me to fully experience the hackathons, and to observe the phases and characteristics of a hackathon that would not have been able to notice when only conducting interviews.

Document study

Besides the collected data trough interviews and observations, this study also included the analyses of documents. These documents were gathered during the study, and were related to the four cases used in the thesis. These included: websites of the hackathons, project plans, agendas, slides that were used during the hackathons, and evaluation forms filled in by participants. Due to privacy reasons these documents are not included in the appendix, but codes derived from the documents can be found in the codebook.

Data analysis techniques

The data obtained by transcribed interviews, observations and a document study (including web pages) was analysed according to a coding process which includes open, axial and selective coding (Strauss & Corbin, 1990; Mortelmans, 2009). During this process the data is broken down in many small ‘pieces’, after which it is rebuild in a structured manner. This was done by defining ‘codes’ which are assigned to the data in order to assign it to a particular definition (deductive coding) or found phenomena (inductive coding). The coding was divided into three phases; open, axial and selective coding. According to Mortelmans (2009), open coding is the process of breaking the ‘raw’ data into chunks which are then coded with a particular theme or topic. These chunks are then bound tougher during the axial coding process, and concepts are defined on the basis of the open codes. In the last step the concepts are bound together, forming the basis to draw conclusions about the data. The codebook can be found in the appendix.

In order to draw conclusions from multiple cases in this study, a within-case analysis is done for every single case, followed by a cross-case analysis that in order to find overarching patterns. As Eisenhardt (1989) describes, the purpose behind the within case analysis is to gain familiarity with

(33)

data and preliminary theory generation, while the cross-case analysis forces the investigator to look beyond initial impressions and see evidence thru multiple lenses.

Construct, Internal and External Validity & Reliability

In their widely cited article, Gibbert, M., & Ruigrok (2010) provide two strategies for case study researchers to ensure construct validity, which have been used in this study as well. Firstly, an the triangulation of different sources of data is used in this study, by including interview data, direct observations and a document study. This triangulation of data contributes to good construct validity. It is important to note that the the interviews and direct observations were all personally conducted, and not by others. This also positively contributes to construct validity (Gibbert & Ruigrok, 2010). The second strategy, as suggested by the authors, was to establish a clear chain of evidence to allow the reader to reconstruct how the researcher went from the initial research questions to final conclusions. This study has aimed to achieve this by not only providing a clear description of the findings in their context, but also by including detailed descriptions of the four hackathons and transcriptions of all the conducted interviews in the appendices.

In order to establish internal validity this study compared empirically observed patterns during the hackathons with predicted ones and previous studies. This comparison of the between the data of this study and of previous research will increase internal validity (Eisenhardt, 1989). Moreover, like for construct validity, triangulation of data sources enables a researcher to verify findings by adopting multiple perspectives (Yin, 2009).

In terms of external validity – or generalizability – Eisenhardt (1989) argued that case studies can be a starting point for theory development and suggests a cross-case analysis involving 4 to 10 case studies that may provide a proper basis for analytical generalization. Please note the difference between statistical generalization (form observation to population) that can be applied to quantitative studies, and analytical generalization (from observation to theory) which is applicable to qualitative (case) studies and thus also to this study. Since four hackathons were included in this multiple case

(34)

study, one might argue that the conclusions can be generalized into a theory. However, the wide variety of hackathons being organised must be kept in mind, and therefore this study might not be applicable to every single hackathon. This is despite the fact that four very diverse hackathons were included in this study.

Silverman (2000) suggests that reliability can be ensured by a principle he calls ‘‘low-inference descriptors. In his article he argues that no piece of research can be free from the underlying assumptions that guide it, and that detailed data presentations that make minimal inferences are always preferable to researchers’ presentations of their own (high-inference) summaries of their data. In terms of interview data in particular, Silverman (2000) suggests a number of means to increase reliability which have been adopted in this study. These included tape recording of all face-to-face interviews and Skype interviews, transcribing these, and adding these in the appendices of the thesis. By these means this study aimed to be reliable to its best possibilities.

(35)

Findings

The findings will be presented according to the same structure as the literature, although the characteristics of a hackathon are combined with the type of problems and phases of a hackathon. This, because the data showed that particular characteristics were to be found in particular phases, and therefore they are presented simultaneously. The types of problems that have led the organisers to organise a hackathon will be presented in combination with findings on the characteristic focused purposive on a single challenge. This is then followed by a detailed description of the process of the

hackathon with its corresponding characteristics. Besides challenging or confirming the process and characteristics as defined by the literature based on the collected data, we build upon previous literature by discussing their influence on the functioning of the hackathon. In addition, three novel characteristics that were found in this study are presented. Lastly, the expected outcomes of the organiser are compared with the actual outcome of each hackathon, to gain a better understanding whether the hackathon actually delivers what the organising party aims for. Findings from all four cases will be presented in each section, followed by a cross-case analysis of all four cases. A more detailed description of every individual hackathon can be found in de appendix.

Type of design problem & Focused purposive on a single challenge

The Climathon was organised from 17-18 March 2016 in by the municipality of Vejle in Denmark - in collaboration with Climate KIC –to find ways to cope with severe flooding that this city has to cope with due to accelerated climate change. As the organiser stated it: “Vejle is placed in a Fjord close to the sea, and it has been flooded several times… Now we want to find another way to find

solutions for this issue” Moreover, more and heavier cloudbursts are impacting the city, and due to

its location between to steep slopes, water drowning into the city. These three factors – rising sea-level, cloudbursts and the location of the city – are thus known to cause the problem, and the solution can be to just build dykes around the city to protect it. However, the municipality of Vejle clearly indicated that this was not an option, since it would significantly damage the liveability of the city.

(36)

Therefore, other solutions for this problem have to be thought of, and this was the reason to organize the hackathon. Since the problem is known, and possible solutions as well, it could be argued that it is a simple problem, and definitely not a wicked problem. However, since the it was not an option to build dykes, the solution was undetermined, and therefore it the problem can be seen as complex. Three sub-challenges related to the floods were given to the participants prior to the hackathon. The first challenge was to find new ways to manage the water – divert it from the city centre, delay it upstream or store it for a time – and to effectively communicate to citizens when the water is entering the city. The second concerned the urban transition the city had to make. Here the challenge was to find out how the city can cope with the flooding risk and rising water levels, while utilise already existing green and recreation spaces. The third challenge was about liveability and citizen engagement. Participants were challenged to think about how citizens can take responsibility for the city during both floods and between floods, and act to help themselves and their neighbours in times of crisis. The participants could choose one challenge they wanted to work on at the beginning of the hackathon. The chosen theme served as a guidance throughout the hackathon. It could be observed that the concepts that the groups came up with during the hackathon were not necessarily related to the chosen theme. Some of the concepts were, some were not.

The Smart Road Hackathon was organised on the 14th

of April 2016 by Holland ConTech for the Dutch Ministry of Infrastructure with the aim to start thinking about how the Ministry of Infrastructure should prepare the Dutch roads for self-driving cars. Specialists from different industries – finance, energy, mobility, construction, public sector – gathered to collectively come up with innovative ideas to be used by the Dutch government. As the organiser of this hackathon stated: “The hackathon is about smart roads, and how we can influence smart roads when we take in mind

the rise of the self-driving car”. Since the event itself was organised to find out how this rise of the

smart-car will influence the way Dutch roads needs to be build in the future, it can be stated that the problem, as well as the solution were not yet clear, and the nature of the problem was indeterminate. Therefore, it can be stated that the type of problem in this case is a wicked problem. During the Smart

Hacking collaborative innovation : a multiple case study on the aims, process, characteristics and outcomes of a Hackathon