iPad apps:
The teachers of the 21
st
century
Author: Lucie Vránová (6191614) Supervisor: dhr. prof. dr. J.W.J. Hans Beentjes Study: Communication Science, Youth and Media
Institution: University of Amsterdam Place: Amsterdam
Abstract
This explorative study looks into educational applications for toddlers in order to investigate if they contain features that support learning of 21st century skills. The study is important and relevant for a number of reasons. First, previous research (Duncan et al., 2007) has confirmed that learning in such an early age has got impact on school readiness and later education as well. Secondly, 21st century skills are vital for everyone who wants to be successful in work and life in the modern world. Thirdly, the iPad is a perfect device for toddlers to play around with thanks to its touch-screen format which allows intuitive operating. This research is founded on the Framework of 21st Century Learning, cognitive developmental theories and
“Kijkwijzer” (the Dutch rating system for audiovisual productions). In this research a codebook was assembled. This codebook served for a content analysis of 52
educational applications targeted on toddlers. The results confirmed presence of features enhancing 21st century skills in the educational apps. However, not all of the skills were supported equally, some of them were not supported at all. The most frequently supported skill was “communicate clearly”, belonging in the most frequently supported group of “Learning and innovation skills”. The results and the implications derived from them can serve parents as a guideline while choosing an application for their child, and to the creators of educative applications as a wake up call to focus more on the development of 21st century skills while creating an educative content.
First of all I would like to thank my supervisor, Prof. Hans Beentjes, for his inspirational discussions, valuable time and feedback, simply an exceptional guidance
throughout the whole research.
My thanks also belong to my spouse, Edwin van Manen, who has been incredibly supportive, tolerant and loving.
Introduction
The discussion about whether toddlers should be allowed to use digital applications has shifted into one about what kind of content is suitable and beneficial for such young children (Lieberman, Bates & So, 2009; O' Hare, 2014; Pijpers & Van Den Bosch, 2014; Vincent, 2015). Research about the influence of media content on toddlers has been focused mainly on television (Valkenburg and Cantor 2001; Valkenburg & Vroone, 2004), a few studies looked into the use of digital media by toddlers, but with no specific focus on analyzing the content (Rideout, 2014), and a number of studies looked into what sort of digital content might be beneficial for such young children (Hirsh-Pasek et al., 2015; O' Hare, 2014). However, there are no studies investigating the content of digital apps as they are today.
The aim of this study is to look into the content of digital apps and investigate whether or not they have any educational potential. The results can contribute to the scientific field about young children and digital media, where so little is known
specifically about the impact of iPads and their apps on young children. Moreover, the results can act as a foundation for further research pertaining to the impact of the educational content on young children.
There are more than 80,000 apps on the Apple App store targeted at children (Apple, 2015). Some of those apps claim educational value so that parents might choose to download this particular app over all of the others available, leaving them dependent on the subjective description of the app by its makers. However, there is no research confirming that those “educational” applications have any educational value and/or can have the potential to educate toddlers at all. This explorative research tries to
investigate if educational apps do indeed have valuable educational potential, specifically as it pertains to the needs of 21st century economy and society. iPad applications have been chosen to be the subject of this investigation for a number of reasons.. First, the touch screen enables intuitive interactions with this media. This format is easy to use even for children of a very young age. Second, children learn the most effectively when they are cognitively and emotionally involved (Valkenburg, 2014), which could be the case when an application is well designed with proper care and expertise. Finally, the increasing rate in in which such devices – most notably the iPad – are used in households are used in households results in an increasing opportunity of out-of-school learning through these digital educational apps (Hirsh-Pasek et al., 2015).
Results from longitudinal research confirmed (Duncan et al., 2007), that school readiness is predictive of later achievement. If digital educational apps can be used to improve young children’s skills and school readiness, it might have long-term impact on the later achievement of those children (Goldin et al., 2014). This could imply that these digital educational applications can have an impact on the rest of these
children’s lives, which makes it important to investigate their content and discover if they encourage learning or improve the skills that are most relevant in these days. Considering the amount of the applications for children on the Apple’s App Store, there is a plenty of content into which to look. Scientific research cannot keep up with the speed with which technology changes and the delivery of new apps on the
marketplace in these days (O' Hare, 2014). The result is that most of the content is both unregulated and untested (Hirsh-Pasek et al., 2015). It is socially desirable to help the parents and their kids with choosing good quality content delivered in an age
appropriate form. For the creators of the apps, investigating the content can bring insight into ways in which the educational potential can be improved, in turn enhancing the overall quality and perceived value of the app.
In order to provide advice for all involved parties, the following research question has been formulated:
RQ:What are the features of good educational iPad applications for toddlers?
A good educational app might encourage toddlers to enhance their 21st century skills and that could make them better prepared for their life and the opportunities of the 21st century. These skills could be learnt or improved by certain features, which may be found in educational apps. Said features could be grouped according to a skill or the whole category of skills that they support. Educational apps can enhance a varying degree of 21st century skills. For example, some applications might focus more on communication skills and less on creativity, and vice versa.
Most parents are lost in an abundance of apps without having clear guidelines or objective metrics on what provides good educational value for their child. This makes it hard to know what to expect from an educational app. The parents could consider, for example, looking at the price or the topic, or they may revert to review websites that might be subjective. The decision is therefore made for them in a way, instead of them determining for themselves what might be best for their child. The parents might believe that the price could be associated with the quality or that some topics might be more educational than others. Review websites are a common source of information for parents selecting an app. This research takes into account a popular review website “Common Sense Media”, which is created by the parents themselves, specialists in child education, specialists in media and other associated matters.
Comparing the educational potential assigned by Common Sense Media and features associated with 21st century skills could validate on what basis the educational reviews are made.
To answer the research question, content analysis was chosen as the most suitable method. This method gives direct insight into the content features of the applications, where the emphasis of this research also lies.
Theory
What could be considered “good” and what is beneficial for children
This section describes a normative theory that outlines of what a good educational application for toddlers should consist. As part of this, the Framework of 21st Century Learning was chosen. This framework was created in order to help to reform
education, where students of the 21st century no longer need to learn what students in the 20th century did. It describes the most relevant and useful skills that universities and modern working opportunities demand. The focus of the Framework of 21st
Century Learning lies in 21st century skills, which are cross-disciplinary skills and can be implemented in all educational, career and civic settings in all different disciplines (Greenhill, 2010). These skills are therefore considered vital for everyone hoping to succeed in the 21st century (Trilling & Fadel, 2009).
Furthermore, this section will describe of what a good educational app must not consist, namely what might be perceived as harmful for toddlers to encounter. For this reason, a publication about Kijkwijzer from authors Valkenburg, Beentjes, Nikken and Tan (2002) was selected as it provides information about possibly harmful effects of audio-visual productions on young people.
Additionally this section will clarify why it is important for toddlers to come in contact with such applications; how such applications might benefit children’s cognition and its development. Finally, the cognitive development of children
between two and three years old will therefore be outlined based on theories described by experts in this field.
Before the RQ could be answered, it must first be understood what exactly is and is not considered “good” in terms of applications for toddlers. Moreover, it needs to be understood what the children need to know in these days, in order to know what can improve their cognitive skills and prepare them for school and future education. This can be summarized in the following two sub-research questions that will help to outline the main terms and concepts:
SRQ1: What is considered to be “good” in regards to applications for toddlers? SRQ2: What is beneficial for toddlers?
Both of these sub questions could be judged based on the three following distinct aspects of the apps:
- Content - Form - Not harmful
All of these three aspects will be explained, one after another.
In addition to the research question and its sub research questions, this research attempts to address all of the discussed topics related to the parents’ search for good educational applications. Providing the answers on the following sub research questions can establish clear guidelines, which can assist the parents in choosing
educational apps based on features that they consider to be the most important for their child. The additional sub research questions are as follows:
SRQ3: What categories of 21st century skills will be enhanced the most in the apps for toddlers?
SRQ4: Which applications have the potential to enhance the most 21st century skills?
SRQ5: What is the relation between the potential to enhance 21st century skills and learning potential indicated by Common Sense Media website?
SRQ6: What is the relation between the potential to enhance 21st century skills and the price of the app?
SRQ7: What is the relation between the subject matter of an application and a 21st century skills category?
Content
Content refers to what exactly should be taught to the toddlers. A normative theory that explains what is important to know in the 21st century in order to be successful in work and life in general was found. This theory is referred to as the Framework of 21st Century Learning (Greenhill, 2010). This framework was created to answer what the purpose of the public education should be, to reflect the specific demands of the 21st century’s highly competitive, complex, knowledge-, information- and technology driven society and economy. The purpose of the public education lies within preparing the students for higher education and modern career and not just in obtaining a school diploma (Partnership For 21st Century Learning, 2009). This framework consists of core subjects and 21st century themes, and skills referred to as 21st century skills. Core subjects and 21st century themes form one group and 21st
century skills are divided into three groups: 1. Core subjects and 21st century themes
a. The core subjects include: world languages, reading or language arts, arts, mathematics, economics, science, geography, history, government and civics. b. 21st century themes are categorized into: global awareness, financial, economic, business and entrepreneurial literacy, civic literacy, health literacy and environmental literacy. These should be naturally woven into the core subjects.
2. 21st century skills
a. Learning and innovation skills consist of creativity and innovation, critical thinking and problem solving, communication and collaboration. According to the Partnership For 21st Century Skills, these are essential skills that separate students into a group that is prepared for life and work in the 21st century and a group that is not. b. Information, media and technology skills include information literacy, media literacy and ICT (Information, Communications and Technology) literacy. Adapting these skills is necessary as life in 21st century is filled with media and technology. This also results in the possibility of obtaining a huge amount of information, possibility to collaborate on an unprecedented scale and fast changing technology (Partnership For 21st Century Learning, 2009).
c. Life and career skills cover flexibility and adaptability, initiative and self-direction, social and cross-cultural skills, productivity and accountability, and, last but not least, leadership and responsibility. Understanding how to manoeuvre within the complex life and work environments is required in these days (Partnership for 21st Century Learning, 2009).
Integrating these skills into apps for toddlers might be beneficial for such young kids. A number of studies confirmed that teaching young children with well-designed digital media can be effective in enhancing communication skills, such as language acquisition or reading (Fish, Akerman & Levin, 2002; Segers & Verhoeven, 2002; Shute & Miksad, 1997). A study from Subrahmanyam, Greenfield, Kraut and Gross (2001) confirmed that digital devices can help enhance critical thinking, analyzing and evaluation in regards to information problem solving by children from the age of two through to eighteen. According to Hong and Trepanier-Street (2004) well-chosen educative technology tools can enhance, among others, planning. A number of studies found that collaboration and social interaction could also be improved by
well-designed digital media (Bers, New & Boudreau, 2004; Shahrimin & Butterworth, 2001). Finally, a child can improve their ability to be a self-directed learner with proper educational software (Lieberman & Linn, 1991).
Yet there is no empirical study that investigates direct influence of educational apps on toddlers. The empirical purpose of this study, as explained above, is investigating if the apps labelled as “educational” enhance the 21st
century skills, thus validating or disproving the claims of the creators of the apps. If the educational value is validated, future research can build on this foundation and look at the impact on the children, eventually investigating how much such a young child can learn about 21st century skills. The importance of learning at this young age is therefore established in the impact it can have on the rest of their life (Duncan et al., 2007).
The focus of this explorative research lies on 21st century skills; Learning and
innovation skills, ICT skills and Life and career skills. This choice is made in order to keep the scope of this research feasible. Investigating the whole Framework of 21st Century Learning would not be possible within the given time.
It was chosen specifically for 21st century skills because they are considered as the most relevant, useful and university-applicable and they should be prioritized in today’s education. As discussed before, 21st
century skills are cross-disciplinary skills, containing knowledge, skills, working manners and personality traits. 21st century skills could be implemented in all educational, career and civic settings. Moreover, failing to acquire these skills can have an impact not only on an individual but also on the economy, society and democracy (Partnership for 21st Century Learning, 2009).
As 21st century skills may be taught either at school or in non-school settings, perhaps digital applications could be a suitable medium to bring these skills to toddlers and prepare them for the 21st century. This research looks at the degree in which those skills could be enhanced through educational apps for toddlers using tablet devices such as the Apple iPad.
Form
Form indicates how the content should be taught: what it should look like in order to be understood by toddlers and to be appealing for them. According to Vossen, Piotrowski and Valkenburg (2014), the developmental capabilities of children predefine their media use and their media preferences. Therefore, the developmental stage of the toddlers should be explored. A moderate discrepancy hypothesis
explained and approved by several authors (Mccall, Kennedy & Applebaum, 1977; Valkenburg & Cantor, 2000) will then help to explain what kind of features make the apps engaging for toddlers. The information about developmental capabilities, media preferences and features that are engaging for this age group, can give insight into what a suitable form of the applications for toddlers might be.
Despite the extensive amount of studies and theories dealing with childhood cognitive development, there is not a consensus of how to label children in between the age of two and three. Bukatko (2007) classifies children of this age in the early childhood phase. Valkenburg and Vroone (2004); Vossen et al. (2014) claim that two-year old children are still in the toddlerhood phase. Other authors, such as Copple, and Bredekamp (2009) define children from the age of one till three as toddlers. For the purpose of this research, children between the ages of two and three will be referred to as toddlers.
Abilities and disabilities of toddlers are described by a number of developmental psychologists. However, the foundational ideas were elaborated by Jean Piaget (Bukatko, 2007). Moreover, the Piagetian developmental approach is recommended and used by several experts as a good guide for judging and developing children’s media (O' Hare, 2014). This study is going to look into the Piaget’s theory of cognitive development on the elements that are relevant for media use by toddlers, specifically focusing on digital media.
According to Piaget, toddlers belong to the so-called pre-operational stage (Bukatko, 2007). This stage is characterised by several important milestones in children’s development: attention, knowledge and language acquisition increase considerably (Bukatko, 2007). Children start to use symbols, which can be seen while they play and pretend that one object represents another one, for example that a pillow is a dog with which they can cuddle. This might be implemented in an app environment as a creative task like assembling a creature from pictures of daily objects, such as a picture of a light bulb that would represent a head and a milk carton that would represent a torso etc.
Another important aspect of this stage is role-playing, while children pretend to be someone else from their familiar environment (such as their mother or a teacher). Role-playing can be easily realized in an app. Toddlers can pretend to be just about any character from the app that their imagination allows. Moreover, with characters from the app they can do things that would not be possible in real life, nor with most of the other media, such as television or a book.
Despite the possibilities for role-playing, toddlers are not able to see the perspective of other people yet. Piaget calls this “egocentrism.” In the context of playing with applications this means that a child can only understand the point of view of their character in the app and would not understand that using other characters in the same game might bring different perspectives.
Another important cognitive feature is focusing on an individual or a prominent characteristic of an object, defined by Piaget as centration (Bukatko, 2007;
Valkenburg & Cantor, 2001). Centration would lead children to focus on only one aspect of a problem. Therefore an app for toddlers must not ask to solve a complex problem, because toddlers can only focus on one, striking feature/issue at the time. Furthermore, toddlers have difficulty distinguishing between fantasy and reality (Bukatko, 2007; Valkenburg & Cantor, 2001). They believe that the characters from the media and their adventures are real, which makes the fantasy characters just as attractive and engaging as real-life characters (Vossen et al. 2014).
It is apparent Piaget has brought several important concepts and terms into depicting children’s development that are broadly used and respected. However, he is also criticised for his explanation of some of the underlying mechanisms (Bukatko, 2007). In short, it is his “domain general” approach, where he generalizes that all domains
are dependent on the developmental stage. In the opposition stand authors such as Hirschfeld and Geldman (as cited in Bukatko, 2007, p. 280) that advocate the
“domain specific” approach. This means that thinking in some domains evolves faster than in others. For example children can learn quickly and already at a very young age about different kinds of animals or plants. These topics could then be used in the applications targeted towards very young children.
While discussing milestones of development, it is important not to forget individual differences. Those can pertain to different origins (e.g. cultural, socioeconomic etc.) and tmay have a crucial impact on reaching these milestones (Bukatko, 2007). Keeping this in mind, it can be assumed that some toddlers would already be able to understand apps aimed at older children, while some might experience difficulties while interacting with apps designed for their own age.
Presumably, applications that contain features supporting 21st century skills in an understandable form might have an educational potential. But the apps also need to be engaging in order to be successful in educating young children (Hirsh-Pasek et al., 2015). What is specifically engaging for toddlers can be explained based on the moderate discrepancy hypothesis (Mccall et al., 1977). This hypothesis was formulated in order to explain why a child is interested in a stimuli that is just moderately different from what it already knows, which can be at least partially incorporated within the existing conceptual framework (Valkenburg & Cantor, 2000). For example if a toddler is familiar with a cat, it will be more interested in animals more similar to cats (such as rabbits etc.) than in an exotic reptile. This might suggest that the creators of the apps should be careful with using less common objects in their apps and rather use ordinary objects of daily use in different kinds of representations. The moderate-discrepancy hypothesis can also explain the shift in interest from
salient to non-salient features. Toddlers’ attention can be caught by salient features, however, further engagement is ensured by non-salient features, such as a storyline (Valkenburg and Vroone, 2004). A simple storyline can be already followed due to the toddlers’ cognition.
The elements of the toddlers cognitive development described in this section provided implications about what a form of the apps for toddlers should look like. Additionally, practical examples were included as well. These characteristics combined outline what is considered to be a good form for applications for toddlers.
2.1.3. Not Harmful
Besides suitable content and form, good applications for toddlers must not be considered harmful. According to the Dutch rating system “Kijkwijzer”, further explained by Valkenburg, Beentjes, Nikken and Tan (2002), there are six main categories of content considered to be harmful: violence, frightening content, sexual content, discrimination, coarse language, use of hard drugs and excessive use of soft drugs and alcohol. This kind of content has, according to many authors (Gosselt, Jong & Van Hoof, 2012; Sternheimer, 2003; Strassburger et al., 2013; Valkenburg, et al., 2002), a negative influence on its audience.
There are several theories that explain how content can influence an audience. Perhaps the most broadly used theory, describing the mechanisms regarding the influence of media on children, is social learning theory invented and broadly tested by Bandura (1977). In his theory, Bandura explains how a simple observation can be powerful and how learning through the observation could be reinforced by reward or reduced by punishment. According to this theory, a child can become violent by
simply seeing violent content in an application. Receiving a reward within the app, such as applause would intensify the learning process.
Additionally Bandura’s theory was incorporated into the “Differential Susceptibility to Media Effects Model” (further only DSMM model) by Valkenburg and Peter (2013). This model incorporates a number of other theories as well and basically describes important moderators and mediators (e.g. dispositional, developmental and social susceptibility) between audience, media use and its effects. The effect of learning by pure observation proclaimed by Bandura is hereby weakened and the emphasis is on individual differences associated with the above indicated moderators and mediators. By applying violent content on this model, a child would not learn violence as straightforward as Bandura describes in his theory. Many factors would have an impact on this learning process, such as personality, emotional state or attention to the content (Valkenburg & Peter, 2013).
Method
Sample
The sample in this study is assembled of 52 iPad apps aimed at toddlers. The website www.commonsensemedia.org was used as a guideline for selecting the best-rated apps for toddlers. Common Sense Media is an organisation where experts in education, health, media, psychology etc. work. The applications themselves are reviewed and rated by researchers (among others university professors) in the media and technology branch. This website provides a list of the best-rated applications for different platforms and targeted on audiences with different ages. This website was chosen rather than a website that indicates the most downloaded (a.k.a. popular) iPad
applications because there is a bigger chance to get applications with some educational value.
The sample was extracted on 20th April 2015 in two steps. First, the data was filtered for the best rated apps for children of the ages of two and three. Furthermore, the data was filtered by specifying the filter criteria that the apps would need to have an educational value. It was then checked if the app was available to download from Apple’s App Store, from which a list of applications was created that will be investigated with content analysis.
First of all, a pilot test was executed in order to test the clarity of the codebook. This pilot test was executed by the author of this study and a second trained coder (a software and mobile application developer). Ten applications were randomly chosen from the list and coded by both coders. This pilot test helped to discover some
uncertainties and weak points of the codebook and so the codebook could be adjusted (see the appendix C: Codebook).
Design
This content analysis uses a primarily deductive approach; it uses the Framework of 21st Century Skills. However, as there was no existing resource available in the literature to perform a content analysis on educational apps, this research also has an explorative character. The research proposes its own codebook (see the appendix C: Codebook), which can be used in future research of educational apps, regardless of the age of the target audience.
In the coding process, a total of 52 apps were coded. After coding this amount it became apparent that the coding process was saturated. While the author of this study
the pilot test again with the adjusted codebook plus five extra applications so that the reliability of the coding could be checked.
Agreement between the coders was checked with the Krippendorff’s alpha (also referred to as Kalpha). In some cases Kalpha returned under acceptable reliability score and in some cases the Kalpha could not be calculated at all. Both of these outcomes could be explained by the sensitivity of the Kalpha to the appearance of very rare values, especially with binary variables, which is the case in this study. When some of the values appear very rarely, Kalpha returns a low score even when the coders disagree only on a few cases (Hayes & Krippendorff, 2007). When one of the binary values does not occur at all, SPSS is not able to calculate Kalpha.
Therefore, the inter-coder reliability was also calculated in a conservative way with absolute values in percentages of the agreement. The range of the percentages is 80% – 100%%, which is an acceptable level of agreement (Lombard, Snyder-Duch & Bracken, 2002). The overview of all inter-coder reliability scores could be found in the Appendix A: Inter-coder reliability.
Operationalization
This section describes how the individual concepts are measured as variables. The first part discusses the research units and concepts applied from the Common Sense Media website and Apple’s App Store. The second part explains how the 21st
century skills are measured.
iPad Applications for toddlers (N = 52) are in this study the research units, categorical variables.
Learning points (M = 2.62, SD = 1.33) is an ordinal variable that signifies how many points an app scores on learning through the Common Sense Media website’s ratings.
With these learning points the website indicates the learning potential of the title. The variable is expressed through a value from one to five. Five indicates maximum learning value, whereas one was assigned to all apps where the learning value was not indicated with learning points but merely in its verbal description. The reason why a “one” was assigned instead of a “zero” is, because a “zero” would be associated with no learning value at all. Yet the description on the Common Sense Media website is not always consistent and even when an application is described verbally as having a learning value, it does not always reflect the description in their “learning points”. Price (M = 2.78, SD = 1.36) is amount of money (American Dollars) paid for purchasing an application in the Apple App store. The scale of this variable is ratio. Topic Common Sense Media is a categorical variable, inspired by the topics that Common Sense Media assigned to the application. Some of the categories were merged together for practical reasons, such as not having many categories containing only a few apps: in the most of the cases calculations cannot be performed if the category/categories contain low numbers. Only the categories with related topics were merged together: “Horses and farm animals” and “Wild animals” fell together in the category “Animals”. “Music and sing-along” and “Arts and dance” together created the category “Music & Arts”, “Friendship” and “Adventures” were merged together, plus a topic “Socializing” was added in order to create “Friendship, socializing & adventure”. New topic “Shapes” was added in order to cover topics of all investigated applications. In summary this variable contains 6 categories (Mo = 1).1 = “ Book characters”, 2 = “Numbers & letters”, 3 = “Music & Arts”, 4 = “Animals”, 5 = “Shapes”, 6 = “Friendship, socializing & adventure”.
All variables in this section are taken over from the 21st century skills framework definition and therefore their measurements are based on the definitions embedded by Partnership for 21st Century Skills (2009). Most of the features are presented by one variable and others by two variables to improve the clarity of the codebook.
LEARNING AND INNOVATION SKILLS are divided into three main groups:
Creativity and innovation, Critical thinking and problem solving and Communication and collaboration. Each of these three groups is characterized by a number of
features. Each feature is represented by a variable. All of these variables have binary distribution, pointing out whether that certain feature is absent (0) or present (1). 1. Creativity and innovation contains three subgroups; depicted as three variables:
a. Think creatively is defined as using idea creation techniques such as brainstorming, creating new valuable ideas and elaborating and evaluating own ideas to enhance the creative efforts.
b. Work creatively with others means developing, carrying out and communicating new ideas to others, being open and responsive to the variety of novel perspectives and incorporating those as well as group input into the work. Additionally, displaying innovation in work and understanding the limits while adopting new ideas is also concerned. Last but not least, it is important to view failure as an opportunity to learn. c. Implement innovations is seen as putting the creative ideas into practice for useful means.
2. Critical thinking and problem solving is divided into four subgroups. Each single group represents a separate variable.
a. Reason effectively is defined as using a variety of reasoning (inductive, deductive etc.) as appropriate to the situation.
b. Use systems thinking means understanding how different parts work together as a whole.
c. Make judgments and decisions is defined as analyzing and evaluating evidence, arguments, claims and beliefs, making connections between them, interpreting information and drawing conclusions based on the analysis. Moreover, it needs to be reflected on learning experiences and processes.
d. Solve problems is about figuring out an array of non-familiar problems in both conventional and innovative ways, as well as the ability to explain different points of view in order to be able to choose a better solution.
3. Communication and collaboration consists of two subgroups; two variables.
a. Communicate clearly is considered as effectively expressing ideas and thoughts, using verbal and non-verbal (e.g. facial expression) communication skills of diverse forms and contexts. Using communication for many purposes, such as asking, entertaining, persuading. Understanding and making use of various media and technologies, knowing how to communicate in diverse environments. Additionally, effective listening is important in order to decode the meaning of the information. b. Collaborate with others involves being able to work with diverse teams, being flexible and make compromises in order to reach a mutual goal. Realize shared responsibility and appreciate contributions made by the team members.
All these nine variables are computed into another dependent variable representing Learning and innovation skills. This variable (M = 2.17, SD = 1.12) has got an
interval scale, indicating how many features from Learning and innovation skills category were present in an app.
INFORMATION, MEDIA AND TECHNOLOGY SKILLS consist of three main groups: Information literacy, Media literacy and ICT (Information, Communications and Technology) literacy. Just like the Learning and innovation skills, certain features are assigned to each of these three groups and each feature represents a binary variable indicating the absence (0) or presence (1) of this feature.
1. Information literacy
a. Access information is about accessing information in a time efficient and source effective manner.
b. Evaluate information needs to take place critically and competently.
c. Use information is defined as a proper and creative use of information for the issue at hand.
d. Manage information involves handling information from an array of sources and applying ethical and/or legal issues regarding the access and use of the information while doing so.
2. Media literacy
a. Analyze media is to understand how and why media messages are created, taking into account that individuals can decipher messages in different ways and that media can influence beliefs and behaviors, and understanding the ethical/legal issues while approaching and utilizing media.
b. Create media products stands for grasping the most suitable media tools, as well as understanding and using the most suitable expressions and interpretation in multi-cultural environments.
3. ICT (Information, Communications and Technology) literacy
a. Apply technology effectively is understood as using technology as a means for studying, systematizing, evaluating and communicating information. Using digital technologies (e.g. mobile devices) and social networks in order to reach out for information, process it and create new information, keeping in mind ethical/legal issues at the same time.
In order to create a dependent variable that would embrace all features belonging to INFORMATION, MEDIA AND TECHNOLOGY SKILLS, variables representing these features were computed together. The target variable is called Information, media and technology skills (M = .69, SD = .94) and its scale is interval. The value of this
variable indicates how many features related to Information, media and technology category are present in an app.
LIFE AND CAREER SKILLS are split into five main groups: Flexibility and adaptability, Initiative and self-direction, Social and cross-cultural skills,
Productivity and accountability and Leadership and responsibility. Certain features characterize each group and these features are represented by variables with a binary distribution, marking the absence (0) or presence (1) of this feature.
1. Flexibility and adaptability
a. Adapt to change is interpreted as accommodating to diverse roles, job
b. Be flexible is described as knowing how to make the best out of given feedback, both positive and negative. How to understand, evaluate and negotiate different views in order to come to viable results.
2. Initiative and self-direction
a. Manage goals is explained as setting goals with the potential for success, balancing between tactical and strategic goals.
b. Manage time is associated with using time efficiently in order to manage goals within the required timeframe.
c. Work independently is seen as searching for, defining and accomplishing tasks on their own.
d. Be self-directed learners involves deepening one’s skills in order to enhance learning and/or expertise, reflecting on gained experiences to improve the learning process and seeing learning in general as a lifelong journey.
3. Social and cross-cultural skills
a. Interact effectively with others covers knowing when to speak and when to listen, presenting oneself as a professional.
b. Work effectively in diverse teams is defined as being able to work productively with people from different backgrounds, respecting their culture, being open to different values and ideas and integrating them in order to work most efficiently.
a. Manage projects is explained as establishing and carrying out the goals despite of difficulties on the way, setting the priorities, planning and managing work to
accomplish the results.
b. Produce results means displaying extra qualities that promote producing high quality products; effective time management, collaboration with(in) diverse teams, multitasking, working ethically and positively, active participation, punctuality, reliability, professionalism and responsibility for the results.
5. Leadership and responsibility
a. Guide and lead others is interpreted as guiding others toward a goal using interpersonal and problem-solving skills. The leader needs to inspire, influence and use the strengths of the team members while he or she remains ethical and selfless. b. Be responsible to others says that one has to behave responsibly and be concerned with the interests of the whole community.
Just like with the two previous groups of skills, the dependent variable with interval scale was computed by summing up all binary variables. This variable called life and career skills (M = .52, SD = .67) indicates how many features related to the life and career skills category can be found in an app.
21st century skills (M = 3.38, SD = 1.93) is a variable computed by summing up the variables learning and innovation skills, information, media and technology skills and life and career skills. This variable has an interval scale and it indicates the absolute amount of all features supporting 21st century skills. In this way it indicates the potential to enhance 21st century skills through an app.
Analyse and Results
In order to answer the research question, all data from the codings were set into the statistical programme SPSS. First, all features were ordered by the sum of
occurrences from high to low. This provided an overview of which skills are enhanced the most and which the least in the applications for toddlers, of the total amount, n = 28 skills. The most represented feature was the one enhancing
“communicating clearly”, the second was “thinking creatively” and the third
“problem solving”. For the complete overview of all the features of good educational applications for toddlers see the Table B1 in the Appendix B.
In order to answer the SRQ3: “What kind of 21st century skills categories will be enhanced the most in the apps for toddlers?”, the values of the variables, Learning and innovation skills, Information, media and technology skills and Life and career skills were compared together in the Table 1. To make the picture complete, 21st century skills variable was added as well.
Table 1
21st century skills enhanced by the apps
Skill category M SD Min Max
Learning and innovation skills 2.17 1.12 1 5
Information, media and technology skills
.69 .94 0 3
Life and career skills .52 .67 0 1
21st century skills 3.38 1.93 1 7
From Table 1 it is evident that the most enhanced skills in the apps for toddlers were the learning and innovation skills.
In order to answer SRQ4: ”Which applications have the potential to enhance the most of 21st century skills? The apps were compared based on their scores on the 21st
century skills variable. The highest score was a total number of 7 features found in the following applications: ABC Food, ABC Go; ABC Music; ABC Play; ABC Wildlife; Daniel Tiger's Grr-ific Feelings. No application scored 0 on the features enhancing 21st century skills. For the full overview see the appendix B, Table B3.
To answer SRQ5: “What is the relation between the potential to enhance 21st century skills and learning potential indicated by Common Sense Media website?”,
correlation between the 21st century skills and learning point variables was
calculated. As the first variable had a ratio scale and the latter an ordinal five-point scale, the Pearson’s correlation coefficient could be calculated. A significant moderate positive correlation was found, r = .35, p = .011. The more 21st century skills an app contained, the higher it also scored on learning points assigned by Common Sense Media website.
To answer SRQ6: “What is the relation between the potential to enhance the 21st century skills and a price of the app?”, correlation between variables 21st
century skills and price was calculated. The first variable has got interval scale, the second one ratio and therefore the Pearson’s correlation coefficient could be calculated. There was no significant association found, r = .22, p = .121. A more expensive application does not necessarily have a higher potential to enhance 21st century skills. To answer SRQ7: “What is the relation between the topic of an application and a 21st century skills category?”, three cross-tabulations were calculated: the first between learning and innovation skills and topic Common Sense Media (Table 2), the second between information, media and technology skills and topic Common Sense Media
(Table 4) and the third one between life and career skills and topic Common Sense Media (Table 6). Additionally, the total amounts of features supporting learning and innovation skills in the particular topics is depicted in Table 3, Table 5 and Table 7. In order to make a clear overview of the distribution of the Common Sense Media topics categories in the sample of this study, Figure 1 was created.
Figure 1
For a clear overview of the values of the skills categories in the sample of this study, see the Table 1 above.
16 13 8 6 5 4
Frequencies of the applications' topics
Book characters
Numbers & Letters
Friendship, Socializing & Adventure
Shapes
Animals
Table 2
Learning and innovation skills by topic Common Sense Media
Topic Common Sense Media
Total Book characters Numbers & Letters Music &
Arts Animals Shapes
Friendship, Socializing & Adventure Learning and innovation skills 1.00 n 7 1 1 2 3 4 18 % 43.8% 7.7% 25.0% 40.0% 50.0% 50.0% 34.6% 2.00 n 6 2 3 3 1 0 15 % 37.5% 15.4% 75.0% 60.0% 16.7% 0.0% 28.8% 3.00 n 1 9 0 0 1 2 13 % 6.3% 69.2% 0.0% 0.0% 16.7% 25.0% 25.0% 4.00 n 0 1 0 0 1 2 4 % 0.0% 7.7% 0.0% 0.0% 16.7% 25.0% 7.7% 5.00 n 2 0 0 0 0 0 2 % 12.5% 0.0% 0.0% 0.0% 0.0% 0.0% 3.8% Total n 16 13 4 5 6 8 52 % 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Table 3
Total amount of features supporting learning and innovation skills in the topics Book
characters
Numbers & Letters
Music &
Arts Animals Shapes
Friendship, Socializing & Adventure
Learning and innovation skills in total
From Table 2 is evident that learning and innovation skills were supported by all the applications topics, yet in varying degree. Most of the applications scored one on the features that enhances learning and innovation skills (18 out of total 52 applications). Table 3 indicates that the topic most associated with learning and innovation skills are “Numbers & Letters”, with a total number of 36 features in 13 applications and topic ”Friendship, Socializing & Adventure with a total of 18 features in 8 applications The lowest scored topic was “Animals” with 8 features in 5 applications.
Table 4
Information, media and technology skills by topic Common Sense Media
Topic Common Sense Media
Total Book characters Numbers & Letters Music &
Arts Animals Shapes
Friendship, Socializing & Adventure Information, media and technology skills .00 n 11 5 1 4 5 2 28 % 68.8% 38.5% 25.0% 80.0% 83.3% 25.0% 53.8% 1.0 0 n 5 2 3 1 1 5 17 % 31.3% 15.4% 75.0% 20.0% 16.7% 62.5% 32.7% 2.0 0 n 0 1 0 0 0 1 2 % 0.0% 7.7% 0.0% 0.0% 0.0% 12.5% 3.8% 3.0 0 n 0 5 0 0 0 0 5 % 0.0% 38.5% 0.0% 0.0% 0.0% 0.0% 9.6% Total n 16 13 4 5 6 8 52 % 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Table 5
Total amount of features supporting information, media and technology skills in the topics Book
characters
Numbers & Letters
Music &
Arts Animals Shapes
Friendship, Sociali-zing & Adventure
Information, media and technology skills in total
From Table 4 is evident that 28 applications out of the total scored 0 on information, media and technology skills. They do not support these skills at all. Moreover, features are not spread equally across the topics. Table 5 displays that the topic most associated with information, media and technology skills is “Numbers & Letters”, with total of 19 skills found in 13 applications. The least supportive to the
information, media & technology skills are topics “Animals” and “Shapes”, they both have 1 application associated with 1 feature supporting information, media and technology skills.
Table 6
Life and career skills by topic Common Sense Media
Topic Common Sense Media
Total Book characters Numbers & Letters Music &
Arts Animals Shapes
Friendship, Socializing & Adventure Life and career skills .00 n 8 5 3 4 4 5 29 % 50.0% 38.5% 75.0% 80.0% 66.7% 62.5% 55.8% 1.00 n 6 8 1 1 2 2 20 % 37.5% 61.5% 25.0% 20.0% 33.3% 25.0% 38.5% 2.00 n 2 0 0 0 0 0 2 % 12.5% 0.0% 0.0% 0.0% 0.0% 0.0% 3.8% 3.00 n 0 0 0 0 0 1 1 % 0.0% 0.0% 0.0% 0.0% 0.0% 12.5% 1.9% Total n 16 13 4 5 6 8 52 % 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Table 7
Total amount of features supporting life and career skills in the topics Book
characters
Numbers & Letters
Music &
Arts Animals Shapes
Friendship, Socializing & Adventure
Life and career skills in total
Table 6 displays that most of the applications (29) scored 0 on life and career skills. They do not support these skills at all. 20 applications contain 1 feature supporting life and career skills. These occurrences are equally distributed across all the topics. From Table 7 is evident that two of the topics support life and career skills modestly more than the rest: “Book
characters” and “Numbers & Letters”, scoring 10 features found in 16 applications, and 8 features in 13 applications respectively. The lowest scores “Animals” with 1 feature in 1 app out of 5 and “Music & Arts” with 1 feature in 1 app out of 4.
Conclusion and Discussion
The answer to the research question
What are the features of good educational iPad applications for toddlers?
is answered based on analyzing content, form and whether or not the application may be considered “harmful.” Additionally, topics that might be associated with parents’ search for a good and the most suitable educational application were also addressed.
As explained in the theoretical section, 21st century skills can form a good base for the creation of educational content (Greenhill, 2010). The content analysis in this study indicated that indeed, features supporting the learning of 21st century skills are present in educational applications for toddlers, some of them more frequent than others.
“Learning and innovation skills” were discovered to be the most enhanced in the apps for toddlers, as pertaining to SRQ3:” What categories of 21st century skills will be enhanced the most in the apps for toddlers?” The first one among the features were those supporting “communicate clearly”. These features are suitable for toddlers’ apps because of the fast language development typicalof this age (Bukatko, 2007). Features supporting “Thinking creatively” and “Problem solving” scored highly as well.
Many of the 21st century features were not found in the apps at all. There could be a number of reasons why. Firstly, the creators might be aware that some skills, such as time
young children due to their limited cognition (Bukatko, 2007; Valkenburg & Cantor, 2001; Vossen et al. 2014). Secondly, the creators did not integrate some of the 21st century skills into the applications perhaps because these skills could be pose too many difficulties due to the format and technological constraints. Thirdly, the creators did not focus on 21st century skills at all while creating the apps. Finally, the creators lacked the expertise to create educational applications.
The consistent lack of some of the 21st century skills discovered through this study across all the investigated apps should serve as a wake up call to the creators of the apps for toddlers. In order to meet the demands of relevant education in these days, app creators and educators should explore whether it is possible to address a greater variety of 21st century skills. Until they do so, parents need to look elsewhere to educate their children with all of the age-appropriate 21st century skills.
The study also looked into educational value of individual applications; SRQ4: “Which applications have the potential to enhance the most 21st century skills?” Firstly, all of the investigated applications contained at least one feature that supports 21st century skills. Secondly, the highest scored apps were ABC Food; ABC Go; ABC Music; ABC Play and ABC Wildlife; which are applications created by Peapod Labs LLC. The amount of
applications with high scores on enhancing 21st century skills indicates that this creator is an up-to-date expert in creating educational applications.
Furthermore, SRQ5: “What is the relation between the potential to enhance 21st century skills and learning potential indicated by Common Sense Media website?” was investigated. The correlation found implies that Common Sense Media website takes into account 21st century skills while evaluating applications. When the parents need to turn to a review website in order to find an application containing 21st century skills, they can safely turn to the Common Sense Media.
This study looked at the price/quality ratio as well; SRQ6: “What is the relation between the potential to enhance 21st century skills and the price of the app?” From the results it is evident that price is not a good indicator of the educational potential of the app. Parents should not assume that an expensive app will help their children enhance their 21st century skills more than a cheap one.
differences found across the topics. The most striking was the association of topic “Numbers & Letters” and “Information, media and technology skills”. If the parents want their child to stimulate these skills, they should, unsurprisingly, purchase an app from the “Numbers & Letters” topic.
What a surprising result was that only one app from the “Shapes” topic supported
“Information, media and technology skills”. One would expect a positive relation between these two concepts. A possible explanation is that a combination of ”Shapes” and
“Information, media and technology skills” is too complex to make and for toddlers to understand.
“Numbers & Letters” were scoring relatively well also on the life and career skills, but the topic “Book characters” scored even better. Perhaps the best way to learn about life and career is from classic stories transformed into apps. As learning and innovation skills were supported by all apps in the research sample, parents can pick any topic their children like if they want to support these skills in particular.
The form most suitable for toddlers was not an object of the empirical investigation in this study. However the literature review in the theoretical section has brought to light some implications. Due to evolving yet limited language and cognitive abilities (Bukatko, 2007) it is advisable to stimulate learning new words with, for example verbal labeling, using slow pace and repetition. Egocentrism disables the children from seeing the world from a different perspective than their own. This implies that the environment of the application must be done in a simple way. Using, for example, a 3D environment would be confusing. Centration, then, makes children focus on a striking feature of an app. They will enjoy characters with big eyes, striking colours or other such feautres. Moreover they will be only able to focus on the most salient feature of a complex problem, which leads to an inability to solve such
problems. The tasks for the toddlers need to be simple; eventually a more complex problem needs to be divided into simple steps for them. Toddlers’ difficulty in distinguishing between fantasy and reality (Bukatko, 2007) gives the creators of the app an opportunity to choose between real world characters or made up ones from the fantasy world. The child will enjoy both of them.
Hirschfeld and Geldman’s domain-specific approach indicates that applications using domains such as animals or flowers will be successful amongst toddlers.
An engaging form needs to meet the criteria of being neither too simple nor too complex (Valkenburg & Cantor, 2000). A familiar environment within the app is optimal, such as simple house or a garden, with a number of unknown features that might be incorporated into the child’s world. As the toddler starts to be interested in non-salient features, simple narrative is a way to keep the attention.
The attribute of not being harmful was also not empirically investigated in the scope of this study. To answer what is considered as harmful as well as why it could be harmful was found in the literature. Things such as violence, frightening content, sexual content, discrimination, coarse language, hard drugs use and excessive use of soft drugs and alcohol could cause harm (Valkenburg et al., 2012). The ways in which media content can influence the viewer was explained based on Bandura’s social learning theory and DSMM model invented by Valkenburg and Peter (2013).
A number of limitations of this study lie within the method that was used. An insufficient number of coded applications resulted in low numbers in the cells of the cross tabulations. No standard association coefficient was therefore calculated. The result would not have any statistical power. The cross tabs would be left only to interpretation.
Another limitation associated with the method is that the applications were played and investigated by adults. Adults cannot see the world through the eyes of two or three year olds and therefore cannot experience and feel what the young ones do. An adult can miss things in an app that a toddler would see and vice versa. Besides that, the adult cannot experience how engaging the application is unless the toddler plays it and shows interest in it, or not. Previous research claimed that apps need to be engaging in order to be successful, but this research did not take this variable into account.
Future research could perhaps investigate the interactions of toddlers with educational apps where the features supporting 21st century skills were found. It could investigate which of these features toddlers can understand and learn as well as how engaging the different groups of 21st century skills are for toddlers. A longitudinal study could then investigate what impact apps supporting 21st century skills have on school readiness and later education.
Another limitation of this study is its focus only on the 21st century skills. These were chosen as they are considered to be the most important skills of these days, but that does not mean that other skills and knowledge are not important. Therefore, the lack of 21st century skills in
an app does not mean that the app has no educational value at all. There could be other subjects that these applications enhance, such as geography or mathematics.
The importance of this research lies within its explorative character on a field so important as education for toddlers is as of yet still relatively unexplored regarding new technology, which is used by young children. The results from this research can help to build a foundation for future research and together create more and better learning opportunities for these little ones. The world offers tremendous numbers of opportunities – let’s allow the children to learn about that.
Reference list
Apple (2015). iPad in education. Retrieved from: http://www.apple.com/education/ipad/apps-books-and- more/
Bandura, A. (1977). Social learning theory. Retrieved from:
http://www.esludwig.com/uploads/2/6/1/0/26105457/bandura_sociallearningtheory.pd f Bergin, D. A., Ford, M. E., & Hess, R. D. (1993). Patterns of motivation and social behavior associated with microcomputer use of young children. Journal of Educational
Psychology, 85(3), 437. DOI: 10.1037/0022-0663.85.3.437
Bers, M., New, R., & Boudreau, L. (2004). Teaching and learning when no one is expert: Children and parents explore technology. Early Childhood Research and Practice, 6(2). Retrieved from: http://ecrp.uiuc.edu/v6n2/bers.html
Bukatko, D. (2007). Child and adolescent development: A chronological approach. Florence, KY: Cengage Learning
Copple, C., & Bredekamp, S. (2009). Developmentally appropriate practice in early
childhood programs serving children from birth through age 8. National Association for the Education of Young Children. 1313 L Street NW Suite 500, Washington, DC 22205-4101 Côté, S. M., Mongeau, C., Japel, C., Xu, Q., Séguin, J. R., & Tremblay, R. E. (2013). Child care quality and cognitive development: Trajectories leading to better preacademic
skills. Child development, 84(2), 752-766. DOI: 10.1111/cdev.12007
Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., ... & Japel, C. (2007). School readiness and later achievement. Developmental
psychology, 43(6), 1428. DOI: 10.1037/0012-1649.43.6.1428
Fisch, S. M., Shulman, J. S., Akerman, A., & Levin, G. A. (2002). Reading between the pixels: Parent-child interaction while reading online storybooks. Early Education and Development, 13(4), 435-451. DOI: 10.1207/s15566935eed1304_7
Goldin, A. P., Hermida, M. J., Shalom, D. E., Costa, M. E., Lopez-Rosenfeld, M., Segretin, M. S., ... & Sigman, M. (2014). Far transfer to language and math of a short software-based gaming intervention. Proceedings of the National Academy of Sciences, 111(17), 6443-6448. DOI: 10.1073/ pnas.1320217111
Gosselt, J., Van Hoof, J., & De Jong, M. (2012). Media rating systems: Do they work? Shop floor compliance with age restrictions in the Netherlands. Mass communication and
society, 15(3), 335-359. DOI: 10.1111/j.1460-2466.2011.01597.x
Greenhill, V. (2010). 21st Century Knowledge and Skills in Educator Preparation. Partnership for 21st Century Skills. Retrieved from: http://files.eric.ed.gov/fulltext/ED519336.pdf
Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication methods and measures,1(1), 77-89.
DOI:10.1080/19312450709336664
Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., Robb, M. B., & Kaufman, J. (2015). Putting Education in “Educational” Apps Lessons From the Science of
Learning. Psychological Science in the Public Interest, 16(1), 3-34.
Hong, S. B., & Trepanier-Street, M. (2004). Technology: A tool for knowledge construction in a Reggio Emilia inspired teacher education program. Early Childhood Education
Journal, 32(2), 87-94. DOI: 10.1007/s10643-004-7971-z
Lieberman, D. A., Bates, C. H., & So, J. (2009). Young children's learning with digital media. Computers in the Schools, 26(4), 271-283. DOI: 10.1080/07380560903360194 Lieberman, D. A., & Linn, M. C. (1991). Learning to learn revisited: Computers and the development of self-directed learning skills. Journal of research on computing in education, 23(3), 373-395. DOI: 10.1080/08886504.1991.10781968
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication. Human communication research, 28(4), 587-604. Retrieved from:
http://www.researchgate.net/profile/Cheryl_Bracken/publication/227605219_Content_Analys is_in_Mass_Communication_Assessment_and_Reporting_of_Intercoder_Reliability/links/0d eec52e147934dd04000000.pdf
Mccall, R. B., Kennedy, C. B., & Applebaum, M. I. (1977). Magnitude of dis- crepancy and the distribution of attention in infants. Child Development, 48, 772-786. DOI:
O' Hare, E., & by Cinekid, C. (2014). Mobile Apps for Children. Criteria and Categorization, Cinekid. Retrieved from:
http://www.cinekid.nl/sites/cinekid.nl/files/bijlage/Research%20Report%20-%20Mobile%20Apps%20for%20Children.pdf
Partnership for 21st century learning. (2009). P21 Framework Definitions. Retrieved from: http://www.p21.org/storage/documents/docs/P21_Framework_Definitions_New_Logo_2015. pdf
Pijpers, R. & van den Bosch, N. (Eds.). (2015). Positive digital content for kids. Amsterdam: POSCON, Mijn Kind Online.
Rideout, V. (2014). Learning at home: Families’ educational media use in America. New York: Joan Ganz Cooney Center, January. Retrieved from:
http://www.joanganzcooneycenter.org/wp-content/uploads/2014/01/jgcc_learningathome.pdf Shahrimin, M. I., & Butterworth, D. M. (2001). Young children's collaborative interactions in a multimedia computer environment. The Internet and higher education, 4(3), 203-215. DOI:10.1016/S1096-7516(01)00068-9
Shute, R., & Miksad, J. (1997). Computer Assisted Instruction and Cognitive Development in Preschoolers. Child Study Journal, 27(3), 237-53. Retrieved from:
http://search.ebscohost.com.proxy.uba.uva.nl:2048/login.aspx?direct=true&db=aph&AN=97 11241882&site=ehost-live
Segers, E., & Verhoeven, L. (2002). Multimedia support of early literacy
learning. Computers & Education, 39(3), 207-221. DOI:10.1016/S0360-1315(02)00034-9 Sternheimer, K. (2003). It's not the media. The truth about pop culture's influence on children. Boulder, CO: Westview Press.
Strasburger, V. C., Wilson, B. J., & Jordan, A. B. (2013). Children, adolescents, and the media (3rd ed.). Thousand Oaks, CA: Sage.
Subrahmanyam, K., Greenfield, P., Kraut, R., & Gross, E. (2001). The impact of computer use on children's and adolescents' development. Journal of Applied Developmental
Psychology, 22(1), 7-30. DOI:10.1016/S0193-3973(00)00063-0
Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. John Wiley & Sons.
Valkenburg, P. (2014). The Teletubies controversy & the value of the entertainment. In R. Pijpers & N. van den Bosch (Eds.), Positive digital content for kids (pp. 15-23). Amsterdam: POSCON, Mijn Kind Online.
Valkenburg, P., Beentjes, H., Nikken, P., & Tan, E. (2002). Kijkwijzer: The Dutch rating system for audiovisual productions. Communications, 27(1), 79-102. DOI:
10.1515/comm.27.1.79
Valkenburg, P. M., & Cantor, J. (2000). Children’s likes and dislikes of entertainment programs. Media entertainment: The psychology of its appeal, 11, 135-152. Retrieved from: http://xa.yimg.com/kq/groups/23323224/1850488634/name/Media+Entertainment.pdf
Valkenburg, P. M., & Cantor, J. (2001). The development of a child into a consumer. Journal of Applied Developmental Psychology, 22(1), 61-72. DOI:10.1016/S0193-3973(00)00066-6
Valkenburg, P. M., & Peter, J. (2013). The differential susceptibility to media effects model. Journal of Communication, 63(2), 221-243. DOI: 10.1111/jcom.12024
Valkenburg, P. M., & Vroone, M. (2004). Developmental changes in infants' and toddlers' attention to television entertainment. Communication Research, 31, 288-311. DOI:
10.1177/0093650204263435
Vincent, J. (2015). Mobile opportunities: exploring positive mobile opportunities for European children. Retrieved from:
http://eprints.lse.ac.uk/61015/1/__lse.ac.uk_storage_LIBRARY_Secondary_libfile_shared_re pository_Content_POLIS_Vincent_Mobile-Opportunities_2015.pdf
Vossen, H.G.M., Piotrowski, J.T., Valkenburg, P.M. (2014) Media use and effects in childhood. In J.F. Nussbaum (Ed.), The handbook of lifespan communication (pp. 93- 112). New York: Peter Lang Publishing. Retrieved from:
http://www.researchgate.net/profile/Helen_Vossen2/publication/268681548_Media_Use_and _Effects_in_Childhood/links/547308b60cf216f8cfae9677.pdf
Appendix A: Inter-coder reliability
Table B1
Inter-coder reliability
Variable Frequency Liberal:
Krippendorff’s alpha Conservative: Absolute Value thinking.creatively 35 .82 93,33% work.creatively.with.others 0 n.c.* 100% implementing.innovations 0 n.c.* 100% reasoning.effectively 4 .00 93.33% use.of.system.thinking 4 -.07 80% making.judgements.and.decisions 4 1 100%
problems.solving 29 .30 80% communicate.clearly 37 .86 93.33% collaborate.with.others 0 n.c.* 100% access.information 7 .71 86.67% evaluate.information 0 n.c.* 100% use.information 0 n.c.* 100% manage.information 5 1 100% media.analyzing 8 .85 93.33% creating.media.products 7 .00 93.33% apply.technology.effectively 9 .82 93.33% adapting.to.change 3 1 100% flexibility 0 n.c.* 100% manage.goals 1 n.c.* 100% manage.time 0 n.c.* 100% working.independently 0 .00 93.33% self.directed.learner 10 .86 93.33% interacting.effectively.with.others 11 -.04 86.67% working.effectively.in.diverse.teams 0 n.c.* 100% manage.projects 0 n.c.* 100% produce.results 0 n.c.* 100% guide.and.lead.others 1 n.c.* 100% responsible.to.others 1 n.c.* 100%
* Could not be calculated in SPSS
Appendix B: Results
Table B1Ranking of the skills
Rank Feature representing skills Sum
1. Communicate clearly 37
2. Thinking creatively 35
