Koli Calling
International Conference on Computing Education
Proceedings
20th Koli Calling Conference on Computing Education Research Koli Calling 2020
November 19-22, 2020 Virtually hosted from Koli, Finland
Conference chairs
Nick Falkner, University of Adelaide, Australia Otto Seppala, Aalto University, Finland
Organised by
University of Adelaide, Australia Aalto University, Finland University of Eastern Finland, Finland
In cooperation with ACM/SIGCSE
The Association for Computing Machinery 2 Penn Plaza, Suite 701
New York New York 10121-0701
ACM COPYRIGHT NOTICE. Copyright © 2020 by the Association for Computing Machinery, Inc.
Permission to make digital or hard copies of part or all of this work for
personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request
permissions from Publications Dept., ACM, Inc., fax +1 (212) 869-0481, or permissions@acm.org.
For other copying of articles that carry a code at the bottom of the first or last page, copying is permitted provided that the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923,
+1-978-750-8400, +1-978-750-4470 (fax).
ISBN 978-1-4503-8921-1 Koli Calling 2020
Koli Calling 2020 – Foreword
Welcome to Koli Calling 2020, the 20th Koli Calling International Conference on Computing Education Research. While Koli is traditionally held in the Koli National Park in Finland, this year it was held on-line due to the COVID-19 Pandemic. Koli Calling is a single-track conference which focuses on education research in the computing disciplines and aims to enhance our understanding of the learning and teaching issues affecting computing. The single-track was maintained despite the remote delivery model used, which led to some ingenious duplication of sessions across timezones to ensure this.
This conference attracted topics on a variety of themes, including: computing in schools; motivation and course design; supporting program and programming comprehension (with foci on both teaching and the tools employed); K-12 and higher education, conceptions and misconceptions in computing;
and exercises and feedback.
We were fortunate to have Professor Shuchi Grover from Stanford University as our Keynote Speaker, delivering an address entitled “Teaching frontiers of CS in secondary school: gender equity, data science, transdisciplinary learning and teacher preparation”.
We had 92 paper submissions, of which 20 were accepted as full papers and 16 as short papers, with a further 7 as posters and demonstrations (included in these proceedings as two-page abstracts), an overall acceptance rate of 39% for papers. Each paper was double-blind reviewed by at least three program committee members.
The conference would not have been possible without the hard work of the organising committee, to whom we extend our appreciation and thanks. The Senior PC members provided guidance and oversight, in addition to their specific roles as meta-reviewers. Many reviewers joined us for the first time, given the larger than usual number of submissions and we thank all of them for their professionalism and adaptability.
The Koli Calling 2020 Chairs would like to thank the team of volunteers who have helped to make the conference a success.
Nick Falkner and Otto Seppälä
Koli Calling 2020 – Committees Program Chairs
Nick Falkner University of Adelaide, Australia
Otto Seppälä Aalto University, Finland
Local Chair
Ilkka Jormanainen University of Eastern Finland
Senior PC
Mark Guzdial University of Michigan Petri Ihantola University of Helsinki
Lauri Malmi Aalto University
Robert McCartney Department of Computer Science and Engineering, University of Connecticut Andrew Petersen University of Toronto
Program Committee
Thushari Atapattu The University of Adelaide
Erik Barendsen Radboud University & Open University Mike Barkmin University of Duisburg-Essen
Roberto A. Bittencourt UEFS - State University of Feira de Santana Francisco Enrique Vicente Castro Worcester Polytechnic Institute
Miguel Ángel Conde University of León
Cornelia Connolly National University of Ireland Galway David Cuartielles Malmö University
Rodrigo Duran IFMS Instituto Federal de Mato Grosso do Sul Stephen Edwards Virginia Tech
Katrina Falkner The University of Adelaide Francisco José García Peñalvo University of Salamanca Lassi Haaranen Aalto University Sally Hamouda Rhode Island College Brian Harrington University of Toronto Matthias Hauswirth University of Lugano Arto Hellas Aalto University
Timo Hynninen South-Eastern Finland University of Applied Sciences (XAMK) Hannu-Matti Järvinen Tampere University
Johan Jeuring Utrecht University and Open University NL Mike Joy The University of Warwick
Antti Knutas Lappeenranta University of Technology Emmanuel Awuni Kolog University of Ghana Business School
Juho Leinonen University of Helsinki Andreas Muehling Kiel University Solomon Sunday Oyelere University of Eastern Finland Anne Peters Uppsala University
Marian Petre The Open University Keith Quille TU Dublin
Guido Rößling TU Darmstadt
Kate Sanders Rhode Island College
André Santos ISCTE - Instituto Universitário de Lisboa Mohammed Saqr University of Eastern Finland
Carsten Schulte University of Paderborn Jane Sinclair The University of Warwick Juha Sorva Aalto University
Calkin Suero Montero University of Eastern Finland Jarkko Suhonen University of Eastern Finland Erkki Sutinen University of Turku, Finland Matti Tedre University of Eastern Finland Neena Thota University of Massachusetts Markku Tukiainen University of Eastern Finland Rebecca Vivian The University of Adelaide Carol Zander University of Washington
Table of Contents
“Teaching frontiers of CS in secondary school: gender equity, data science, transdisciplinary learning
and teacher preparation.” ...viii Shuchi Grover, PhD, Stanford University and Looking Glass Ventures
Full and short papers (in the order they were organised for presentation as, due to time zones, order may have been different for some attendees)
Teaching Programming
On Programming Competence and its Classification ... Article 1 Natalie Kiesler
Exploring the Bug Investigation Techniques of Intermediate Student Programmers... Article 2 Rifat Sabbir Mansur, Ayaan M. Kazerouni, Stephen H. Edwards and Clifford A. Shaffer
Preprocessing for Source Code Similarity Detection in Introductory Programming ... Article 3 Oscar Karnalim, Simon and William Chivers
Student Refactoring Behaviour in a Programming Tutor ... Article 4 Hieke Keuning, Bastiaan Heeren and Johan Jeuring
Should Explanations of Program Code Use Audio, Text, or Both? A Replication Study ... Article 5 Albina Zavgorodniaia, Arto Hellas, Otto Seppälä and Juha Sorva
Learning to Program Hands-on: A Controlled Study ... Article 6 Kristina von Hausswolff, Anna Eckerdal and Michael Thuné
K-12
Semi-Automatically Mining Students’ Common Scratch Programming Behaviors ... Article 7 Minji Kong and Lori Pollock
Student’s Rating of Contexts for Teaching Data Literacy at School Regarding the Context Characteristics Relation
to Everyday Life and Uniqueness ... Article 8 Stephan Napierala and Torsten Brinda
Escape Room Game for CT Learning Activities in the Primary School ... Article 9 Juho Kahila, Tuomo Parkki, Anssi Gröhn, Atte Karvinen, Elmeri Telimaa, Pekka Riikonen, Riku Tiitta,
Pasi Haantio, Anssi Keinänen, Tero Kerkkänen, Ilkka Jormanainen, Silja Penttinen and Matti Tedre
Fostering Knowledge of Computer Viruses Among Children: The Effects of a Lesson with a Cartoon Series ... Article 10 Katerina Tsarava, Manuel Ninaus, Tereza Hannemann, Kristina Volná, Korbinian Moeller and Cyril Brom
To Be or Not to Be a Teacher? Exploring CS Students’ Perceptions of a Teaching Career ... Article 11 Sabiha Yeni, Efthimia Aivaloglou and Felienne Hermans
High School Teachers’ Understanding of Code Style ... Article 12 Diana Kirk, Tyne Crow, Andrew Luxton-Reilly and Ewan Tempero
Concepts and Cognition
Mindset and Study Performance: New Scales and Research Directions ... Article 14 Mikko-Ville Apiola and Erkki Sutinen
Investigating Students’ Preexisting Debugging Traits: A Real World Escape Room Study ... Article 15 Tilman Michaeli and Ralf Romeike
Digging into Computer Science Students’ Learning Journals ... Article 16 Paramvir Singh, Ewan Tempero, Andrew Luxton-Reilly and Shuxiang Zhang
Curriculum
Mathematics, Computer Science and Career Inclinations — A Multi-Institutional Exploration ... Article 17 Jacqueline Whalley, Andrew Petersen and Paul Denny
The Value of Aligning Your Course for Curricular Improvement ... Article 18 Noha Elsherbiny and Stephen Edwards
From the Mathematical Impossibility Results of the High School Curriculum to Theoretical Computer Science ... Article 19 Rafael del Vado Vírseda
Equalizing Data Science Curriculum for Computer Science Pupils ... Article 20 Koby Mike, Tamir Hazan and Orit Hazzan
FIRST Principles to Design for Online, Synchronous High School CS Teacher Training and
Curriculum Co-Design... Article 21 Shuchi Grover, Veronica Cateté, Tiffany Barnes, Marnie Hill, Akos Ledeczi and Brian Broll
Access and Skill Development
Aligning Theory and Practice in Teacher Professional Development for Computer Science... Article 22 Veronica Catete, Lauren Alvarez, Amy Isvik, Alexandra Milliken, Marnie Hill and Tiffany Barnes
Adapting Student IDEs for Blind Programmers ... Article 23 Emmanuel Schanzer, Sina Bahram and Shriram Krishnamurthi
Lessons from Teaching HCI for a Diverse Student Population ... Article 24 Bowen Hui
Course Design
Combining Ideas and Artifacts: An Interaction-Focused View on Computing Education Using a
Cybersecurity Example ... Article 25 Lutz Terfloth, Lea Budde and Carsten Schulte
Complex Online Material Development in CS Courses ... Article 26 Lassi Haaranen, Giacomo Mariani, Peter Sormunen and Teemu Lehtinen
Will Students Write Tests Early Without Coercion? ... Article 27 John Wrenn and Shriram Krishnamurthi
A Proposal to Use Gamification Systematically to Nudge Students Toward Productive Behaviors ... Article 28 Stephen H. Edwards and Zhiyi Li
Students’ Preferences Between Traditional and Video Lectures: Profiles and Study Success ... Article 29 Petri Ihantola, Juho Leinonen and Matti Rintala
Feedback and Assessment
Towards an Assessment Rubric for EiPE Tasks in Secondary Education: Identifying Quality Indicators
and Descriptors ... Article 30 Renske Weeda, Cruz Izu, Maria Kallia and Erik Barendsen
Analysis of Programming Assessments—Building an Open Repository for Measuring Competencies ... Article 31 Mike Barkmin and Torsten Brinda
Crowdsourcing in Computing Education Research: Case Amazon MTurk ... Article 32 Arto Hellas, Albina Zavgorodniaia and Juha Sorva
Motivation and Computational Thinking
Weekly Open-Ended Exercises and Student Motivation in CS1... Article 33 Sadia Sharmin, Daniel Zingaro and Clare Brett
Employability Through Imagination, Alignment, and Engagement—Students’ Prospects and Change During
Their First Year in Computing Education ... Article 34 Gunhild M. Lundberg and Birgit R. Krogstie
Computational Thinking Interventions in Higher Education: A Scoping Literature Review of Interventions
Used to Teach Computational Thinking ... Article 35 Imke de Jong and Johan Jeuring
Infusing Computing: A Scaffolding and Teacher Accessibility Analysis of Computing Lessons Designed
by Novices... Article 36 Veronica Cateté, Amy Isvik and Tiffany Barnes
Abstracts of Posters
Towards Understanding the HCI Education Landscape ... Article 37 Keyvan Khademi and Bowen Hui
GitCanary: A Tool for Analyzing Student Contributions in Group Programming Assignments ... Article 38 Jan Jaap Sandee and Efthimia Aivaloglou
Inferring Students’ Tracing Behaviors from Interaction Logs of a Learning Environment for Software
Design Comprehension ... Article 39 Prajish Prasad and Sridhar Iyer
An Online Tool for Analyzing Written Student Feedback ... Article 40 Niku Grönberg, Antti Knutas, Timo Hynninen and Maija Hujala
What Competencies Do Student Teachers Have? — Developing a Test Instrument to Assess
Algorithm-Related Competencies ... Article 41 Benjamin Voorgang and Torsten Brinda
Presenting Basic CS Concepts: A Content Analysis of AP CSA Textbooks ... Article 42 Julie M. Smith
Detecting Students’ Affective States in Industry-focused Projects ... Article 43 Veronica Liesaputra and Claudia Ott