1
A study into how use of a webcam and direct feedback can reduce the amount of performance mistakes
SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER OF SCIENCE
Bonita van Veldhoven
11272619M
ASTERI
NFORMATIONS
TUDIESH
UMAN-C
ENTEREDM
ULTIMEDIAF
ACULTY OFS
CIENCEU
NIVERSITY OFA
MSTERDAMDate of defence: 26-07-2018
1st Supervisor 2nd Supervisor
dr. F.M. (Frank) Nack prof. dr. T.M. (Tom) van Engers Faculty of Science, Faculty of Law
1
Program design for practicing signs used in Dutch Sign Language
A study into how use of a webcam and direct feedback can reduce the amount of performance mistakes
Bonita van Veldhoven
University of Amsterdam The Netherlands
bonita.vanveldhoven@gmail.com
ABSTRACT
Dutch sign language students do not have always someone to practice with besides their lessons and therefore they cannot always receive feedback. This can be demotivating or result in them performing the signs incorrectly. A possible solution to this problem would be that students can practice signing in front of a webcam, while they can see themselves and get direct feedback on their signs. By doing field research, this research explores if there is need for a system that will give the students direct feedback while they are practicing signs at home, in front of a webcam. In addition it will be investigated what this program needs to offer and how it should work and a prototype was developed. This research showed that a combination of textual and visual feedback is needed to meet the user needs and to reduce the amount of performance mistakes. The corresponding experiment indicated that students indeed benefit from the designed program that offers this feedback.
KEYWORDS
Dutch Sign Language, Dutch with Signs, Sign Language learning environment, Language practice program, automatic feedback, Sign language course
1. INTRODUCTION
In the Netherlands there are several providers of sign language courses. In most of these courses, students attend classes given by a teacher. Between the lessons the students need to do homework, practice the signs (lexicon) at home and try to use Sign Language in the daily life to make as much progress as possible. However, students do not always have someone to practice with and cannot always receive feedback. This can be demotivating or result in them performing the signs incorrectly.
A possible solution to this problem would be that students can practice signing in front of a webcam, while they can see themselves and get direct feedback on their signs. This research explores if there is need for a system that will give the students direct feedback while they are practicing signs at home, in front of a webcam. In addition it will be investigated what this program needs to offer and how it should work. An experiment will be carried out to check whether students indeed benefit from the designed program and whether it helps them to reduce the amount of performance mistakes.
The paper starts with describing the context and stating the problem statement in chapter 2. The related work is discussed in Chapter 3. This includes background information about Dutch Sign Language, the role of visual feedback in learning Sign Language and technical background information that is important for the development of an automatic feedback program to practice signs. Chapter 4 explains the theory this research is built on. In chapter 5, the process and the results of the research are explained. The research is divided in three phases. Phase I is a field study, Phase II consists of the prototype design and development and Phase III is the final experiment. Subsequently, the conclusion of this research can be found in chapter 6. Hereafter, in chapter 7, the discussion, the results will be interpreted and the limitations of the research will be discussed. The thesis closes with chapter 8, future work. 2. CONTEXT AND PROBLEM STATEMENT The Dutch Sign Centre is the national independent expertise centre in the field of Dutch Sign Language (Nederlandse gebaren taal, NGT) and Dutch with signs (Nederlands met gebaren, NmG) and has been recognized by the government since 2004 as the Lexicographic Institute [1]. They collect signs from users of the Dutch Sign Language and make the data accessible through, for example, an online sign language dictionary. In addition, they develop courses and (digital) educational materials [1].
The courses of the Dutch Sign Centre are offered by GGMD (Geestelijke Gezondheidszorg en Maatschappelijke Dienstverlening), a foundation and healthcare provider for anyone with a hearing, speech or language limitation [2]. They support people – who are deaf or have a hearing problem – and their families and friends by offering Sign Language courses given by a qualified teacher. These courses are reimbursed by the health insurance of the client when he/she has a hearing loss at both ears exceeding 35 dB [2]
Courses consist of weekly two hour lessons given by a teacher. In these lessons the teacher shows and explains the lexicon (a list with signs in a certain theme) for that week. Various exercises are being performed to practice with all signs. Furthermore, grammatical elements and signs in sentence context are practiced. Between the lessons, the students need to do homework from a workbook and practice the signs (lexicon) at home.
To practice and test signs at home, the Dutch Sign Centre offers an online platform1 on their website2 with three forms of quizzes. These are multiple choice questions, open questions and a test yourself quiz. For example in the multiple choice quiz, a movie of a sign is showed with four possible answers. The student has to choose the right one (fig. 1).
Figure 1. Screenshot of a wrong answered multiple choice question
In the test yourself quiz, the student sees a written word and has to reproduce the corresponding sign (figure 2). Subsequently he or she can watch the movie of the sign. The action is not observed by anyone, the student can only look at the video and check with themselves if he or she thinks they performed it right or wrong.
Figure 2. Screenshot of the test yourself quiz
During the classes, a teacher gives feedback on the performance of the signs. However, when students are practicing at home, they cannot always receive feedback. Students do not always notice their own performance mistakes while practicing signs, sometimes not even if they watch videos of the correct performance of the
1 http://leeromgeving.gebarencursus.nl/ 2 https://www.gebarencentrum.nl/
signs in the online environment by doing the quizzes. As a consequence they continue to learn and perform signs incorrectly.
A possible solution to this problem would be that students can practice signing in front of a webcam, while they can see themselves and are getting direct feedback on their signs. This as an addition to a Dutch Sign Language (NGT) or Dutch with Signs (NmG) course.
3. RELATED WORK
3.1. Background on Dutch Sign Language
There are two different forms of Dutch Sign Language. Sign Language of the Netherlands (NgT) is a complete language that also includes its own grammar. Besides this there is Signed Dutch, which is Dutch supported Sign Language (NmG) [3]. In this form of Dutch Sign Language, the speaker articulates spoken Dutch, but supports his or her sentences with gestures. NmG includes little special grammar [3]. This form is easier to learn for people that have been born hearing and became deaf or hard of hearing later in their life. Also, for hearing people this form is easier to learn. The gestures that are standardized for both forms of Dutch Sign Language are the same.
It is important to know that right-handers and left-handers always signing with their best hand [3]. Unless of course, the sign has to be made with both hands.
A sign exists of five different components, the basic elements [4]. To execute a sign properly, all the basic elements must be correct. The elements are:
Handshape - there are many standardized hand shapes, that are used in sign language
Place - e.g. next to the ear, before the mouth or in front of the chest.
Palm and finger orientation - this indicates in which direction the palm and the fingers point, for example to the left or to the floor.
Movement - in almost every sign, a special movement is made. For example, a displacement of the hand or a movement with the fingers.
Non-manual part - simultaneity is very important in Sign Language. Simultaneity means that information can be transmitted at the same time through the hands and through other parts of the body. This can be by mimicry, body posture and head movement, an oral component or a spoken component. This is called the Non-manual part of a sign [4].
3.2. Feedback on Sign Language
Emmorey et al. studied the role of visual feedback in imitation learning of American Sign Language (ASL) and how signers monitor their language output, so that errors are detected by the comprehension system before they are actually articulated [5]. With the combined results of two experiments they concluded that a lack of visual feedback is not detrimental. Novice Sign Language learners may even improve more without visual
July 2018 Bonita van Veldhoven
3 feedback because they will not be distracted by a mismatch
between the visual perception of the sign model and the visual perception of their own signing [5].
Huenerfauth et al. [6] evaluated, in two phases, three different methods of feedback that can be given on a recording of an American Sign Language student.
Phase 1:
VIDEO: Just replay the recorded video to watch. NOTES: A written message which indicates errors or
correctly performed linguistic elements.
POPUP: A replay of the video with time synchronized, written popups that indicates errors or correctly performed linguistic elements.
Phase 2:
POPUP: as described at phase 1.
SIMPLIFIED POPUP: the written POPUP was textual simplified.
SIMPLIFIED POPUP + PHOTO: a replay of the video with time synchronized, simple written popups and photos that illustrate specific aspects of ASL grammar.
They concluded that American Sign Language students “prefer time-synchronized feedback on videos of their signing, and they also benefit from visual illustration of how to perform aspects of ASL correctly when they correct these errors” [6].
3.2. Related technical research
In 2008, Kelly, McDonald and Markham [7] presented a computer vision based system that could give real time feedback on gestures used in sign language. This system analysed the position, shape and movement of the hand. The system worked on every standard Microsoft Windows based computer, with a standard webcam. It was build using C++ and made use of OpenCV functions. The accuracy was tested on 9 signs from the Irish Sign Language alphabet. The system classified 93,8% of correct signs as correct and just 9,1% of incorrect signs as correct [7].
Since 2008, there has been much more research regarding gesture recognition and hand tracking, for example the research by Mazumdar, Talukdar, & Sarma [8]. They compare free and gloved hand tracking. This research shows that for a reliable feedback system on signs, wearing a glove seems necessary. Otherwise it is difficult to separate the hand from the face when a sign is made in front of it. In comparison with free hand tracking, gloved hand tracking is not dependent on the background, works better under different lighting conditions and is not dependent on skin color [8].
Yeo, Lee and Lim [9] present a design and method “of a robust marker-less hand/finger tracking and gesture recognition system using low-cost hardware” [9, p. 2687]. This method allows fast and robust hand tracking, even with a complex background and motion blur, which is important when students are practicing signs at home.
4. THEORY & HYPOTHESIS
The literature study did not give a clear indication of the influence of visual feedback in Sign Language learning. Emmorey et al. [5] concluded that a lack of visual feedback is not detrimental and can even distract students by a mismatch between the visual perception of the sign model and the visual perception of their own signing [5]. While Huenerfauth et al [6] concluded that students benefit from visual illustration of how to perform aspects of ASL correctly when they correcting errors [6].
Because these different studies contradict each other and because there are no comparable studies on this subject about Dutch Sign Language, this research will address the following research question:
What kind of direct feedback do Dutch Sign Language students need on their signs while they practice in an online environment in front of their webcam, to reduce the amount of performance mistakes?
For being able to answer the research question a set of sub-questions will be investigated:
What do sign language teachers and students think about a system that can provide immediate feedback on signs made before a webcam?
What are common mistakes made by Sign Language students? / What kind of feedback is necessary to reduce the amount of performance mistakes?
How should feedback, on signs, be given in an online environment according to Dutch sign language teachers and students?
To what extent does the program, that is designed based on the needs of the Dutch Sign Language students, help in reducing performance mistakes?
The goal is to design a program that will reduce the amount of mistakes while performing signs.
5 RESEARCH
This research has been conducted by performing the following procedure, divided in three phases.
5.1. Phase I: Field Research
As the literature is indecisive about the value of self-observation of sign performance via a webcam environment, the first step in this research was to create an overview of the different ways to practice gestures at home. What kind of online practicing possibilities are already used for Dutch Sign Language? For establishing a broad view on this issue we interviewed sign language teachers and performed an online survey among sign language students.
The target group
Unfortunately no statistics about the target group can be found through desk research. However, a direct request for statistics of the GGMD provided the following information about statistics of Sign Language students.
The clients of the GGMD can request a course if they themselves have a hearing impairment. The courses are given in the home situation together with the client's relatives. In the first quarter of 2018, 43 new courses started. Group sizes vary from 1 to a maximum of 8. The GGMD only has the data of the client, not from the other participants. In the courses that they give the majority of the group can hear. Often, only the client is hearing impaired / deaf. The age varies enormously. Most participants are between 20 and 60 years of age.
This information still does not give an indication of the total target group (everyone that does follow Dutch Sign Language Lessons or Dutch with Signs lessons). Therefore 9 teachers were asked how many students they have in their groups, on average. Based on those numbers it is estimated that there are on average 5 students per group.
Teacher interviews
Interviews were held with Sign Language teachers. The goal of these interviews was to get insight in:
How a teacher gives feedback to the students now (as inspiration for a feedback design)
What kind of mistakes are often made by sign language students (to find out what kind of feedback the system must provide)
How they feel about a direct-feedback system The ideas teachers have for a direct- feedback system The seven teachers that participated in this research were sampled by a convenience sampling. Their e-mail addresses, through which they could be approached, were found online (via LinkedIn
3
and Google search4). Six out of the seven teachers are not hearing impaired of deaf and teach both Dutch Sign Language and Dutch with Signs. One teacher, who is deaf, only teaches Dutch Sign language. The teachers have on average of 6,7 years of teaching experience and all teach according to the method of the Dutch Sign Centre. In addition, 3 teachers also use other methods or write methods themselves.
The interviews were semi-structured with pre-formulated, open questions which can be found in table 1. This made sure that all the desired information was obtained, but left space for supplementary questions and for the teachers to come up with additional information.
3 https://www.linkedin.com 4 https://www.google.com/
Nr. Question Reason of the question 1 Which form of sign language do you
teach? NmG or NGT?
To break the ice and get an overview of the teacher
2 For how long are you teaching sign language lessons?
To break the ice and get an overview of the teacher
3 Do you teach according to a certain teaching method?
To get an overview of the teacher and to see if/how practicing in front of a webcam can fit in the teaching method. 4 How do you advise your students to
practice at home?
To find out how students are practicing at home now and what the teacher find good methods of home practicing 5 What difficulties do students face
while practicing at home, in your opinion?
To get an insight in the needs sign students have, when they practice at home
6 Do you still see many mistakes / points of improvement in the performance of the signs when students have practiced at home?
To find out where the system should give feedback on
7 What kind of performance mistakes do students make often?
To find out where the system should give feedback on (maybe they think about more things than in the previous question. Or otherwise for triangulation) 8 Do you ever give feedback on
students online or via a digital road? If so, how?
To find out how teachers give feedback in digital ways
9 To what extent do you think that extra feedback on the sign performance during practice at home, can help students?
To get a sense whether or not they find this kind of practicing at home useful.
10 The goal of this research is that sign language students can practice signs before the webcam, while receiving direct feedback. What kind of possibilities do you see in such a type of system?
A direct question to find out what teachers expect of a real-time feedback system and if the teachers have an ideas for it
11 What do you think is a good way of giving feedback in this setting? How do you imagine a program like that?
To find out if the teachers have an idea for a real-time feedback system
Table 1: pre-formulated interview questions
The interviews were transcribed using the transcription software AmberScript5 (a example can be found in APPENDIX III). Subsequently the transcribed interviews were coded using Atlas TI and an Excel Quotation Report was exported from this program. This report can be found in APPENDIX IV. Here follows a description of the most important findings:
According to the interviews, all teachers think that the program should give feedback on the basic elements (which are explained in chapter 3.1). Ideas about how feedback should be given differ, some teachers say textual, some say in Sign Language (movie) or a combination of those two. One says by an virtual teacher (avatar). Most teachers think that a movie of the correct performance of the sign needs to be shown, but that only this movie is not enough.
In addition, the mistakes of the performance need to be pointed out. Some teachers imagine a sort of list that indicates which elements of a sign are right and which are wrong. Two teachers think that it would be good to show the student the movie
July 2018 Bonita van Veldhoven
5 of their own performance in which with the help of a circle can be
indicated where the student made a mistake. Another teacher thinks that an image can be used to show what kind of mistake is made. However, according to the teachers, feedback should be given in a positive or constructive way at all times. This is mentioned by all teachers because they are afraid that the program can work demotivating if only negative feedback is given.
At the end of the interview, all teachers note that they are positive about a feedback system. Nevertheless, some teachers are afraid that people will think that they can learn sign language just with the program and do not need real classes. They say that feedback from a teacher is always necessary but the program can complement a course. Finally, the teachers think that the program should be only used to give feedback on separate signs and not on sentence level, they are afraid that this will be too difficult for a computer to understand and give proper feedback on.
Online survey
An online survey was held among Sign Language students, the target group for the outcome of our research project. The goal of this survey was to find out how students like to practice at home, what they have difficulties with and of course, if there is need for a program that gives them direct feedback. For the latter we were interested in what their vision is towards a feedback program. The survey has been distributed in three ways:
It was sent to students of the teachers that were interviewed On Facebook in the following groups:
Doven en Slechthorenden [NL/BE]6 Stichting Plotsdoven7
At the University of Applied Sciences Utrecht, among the students of the minor Taal en Cultuur Nederlandse Gebarentaal (minor language and Culture Dutch Sign Language)
The survey questions, structure, results and analysis can be found in APPENDIX II, the Design Process (page 7 – 18). Most questions were open or checkbox questions because this allowed the respondents to explain how they practice now, what they need and what they think about a feedback program without while being influenced as little as possible. An example of one of the questions is: What extra help, feedback or tips would you like to have/had while you are/were practicing at home?
The survey is completed by 47 respondents from which 35 (68%) are currently following sign language lessons. 12 respondents (26%) have followed lessons in the last 3 years and 3 respondents (6%) try to teach themselves. The respondents are between the age of 20 and 70 but there is a bias towards students between 20 and 30. Most of the respondents are hearing. This corresponds with the statement of the GGMD, which also indicated that most sign language students are hearing. Unfortunately, it is not possible to indicate to what extent the total amount and distribution of the respondents represent the target group, as there are no statistics.
6 https://www.facebook.com/groups/dovenensh/ 7 https://www.facebook.com/groups/stichtingplotsdoven/
The most important conclusion that can be drawn is that there is need for and enthusiasm towards a feedback program. Students face various difficulties while practicing. They are not always certain if they are performing a sign right and even when students think they are doing it right, sometimes in class it appears that they did not. Explanations for this are that details of signs can be difficult to see in movies (which is the most popular way of practicing) and also that they miss getting feedback or being corrected.
The majority of the respondents think that a feedback program will probably help to:
- Reduce the possibility to learn signs in the wrong way; - Help to focus on the details of the signs and perform
them cleaner; - Reduce insecurity.
The respondents want the program to give an indication if they perform a sign right or wrong. However, only an indication is not enough, they want an explanation of what they did wrong, for example by getting feedback on the basic elements of a sign. Some also mentioned that they want an explanation of the sign itself.
Based on the survey analysis, the following user needs were / requirement list was established:
- The students want to see movies of the correct performance of the signs.
- They want to see the details of a sign pointed out in addition to the movies.
- Users want an indication of what they do right and wrong.
- They want to get feedback in the form of movies and textual.
5.2. Phase II: Prototype design & development
Based on the results of Phase I, the design for the support tool has been developed. The design is made by following the “ Five Planes” model of Jesse James Garret [10] as a process guideline. This model was chosen because it is user-centred and its planes have helped to not overlook important steps and considerations.
Figure 3 and 4 show the wireframes of the interface design. Within the description of the different planes we outline the main design considerations. The complete description of the design process can be found in APPENDIX II.
Strategy plane
In the strategy plane the product objectives and the user needs need to be established. The product objectives describe what the “owners of the final product” want to get out of it. The relevant needs are those established in the requirement list of Phase I. In addition, we aimed at providing an interface that sign language teachers and course developers can have confidence in. The most important product objective to achieve this is to provide feedback on the basic elements in a constructive, positive way.
Figure 3. Wireframe: record a sign
Figure 4. Wireframe feedback on a sign Scope plane
The scope plane is meant to define the functional specifications and the content requirements of the system, based on the user needs [10]. Table 2 gives an overview of the functions that the system will provide and which content is necessary.
Structure plane
In the structure plane it is devised how the functionalities fit together. For this system a sequential structure is used (figure 7). This means that the users of the program will practice all the signs of a certain set in a sequential (random) order. First they practice with sign one, for example the sign for the word “lesson”. The user performs that sign while recording him- or herself. Subsequently the sign will be analyzed by the computer and feedback will be shown. The user can take a look at the feedback and additional information regarding the sign and decide if he or she wants to try and perform the sign again.
Functional specifications: Content requirements:
A function that allows the user to record a movie while he/she is performing a sign
Movies of the correct performances of the signs
A database which contains the information about the signs (see content requirements)
Images of the handshapes that are used within the signs
A function that helps the user to compare the movie of the sign he/she made, with a movie of the correct performance of the sign
Textual explanation or/and image that explains the movement made in the sign A function that makes it possible for the
user to access additional information about the sign performance
Textual explanation or/and image that explains the place where the sign has to be made
A function that gives the researcher the possibility to give textual feedback on the sign performance
Textual explanation or/and of the movements that are made in the sign A function that gives the researcher the
possibility to indicate which basic elements were performed right and/or wrong by the user
Explanations of the non-verbal elements that are included in the sign
Table 2: functional specifications & content requirements of the feedback system
Skeleton plane
In the skeleton plane is defined how the system will present the information and how the user can navigate through the system [10]. In this plane, wireframes have been made to give a schematic overview of the system layout (see Figure 3 and 4). Surface plane
In the surface plane the design is finished. The so called sensory design [10] has been conceived. An example of an important decision in this plane was the colour of the basic element buttons on the feedback screen. Despite the fact that the color red is associated with faults or errors, the button of a basic element turns orange if an element is performed wrong by the user. The color orange will be less negative to the students than red, but still indicates that the element needs improvement.
During this plane also the different feedback sentences are defined. It has been tried to make the sentences as positive as possible. The sentences only indicate which basic elements are performed wrong and right. The user needs to take a look at the movies and the additional information to find out what he or she did wrong in the particular elements. For example if the user has more than one element correct the following sentence (translated to Dutch) will be used: Well done: 3 of the elements are correct.
Do you pay attention to <basic element> and <basic element>?
July 2018 Bonita van Veldhoven
7 Figure 5. Sign-feedback prototype: record a sign
Figure 6. Sign-feedback prototype: feedback on a sign
After the design has been made, the prototype is programmed with HTML, CSS, JQUERY and JavaScript. The prototype works offline on a 15 inch ACER laptop with a built-in webcam. A visualisation of the prototype is presented in figure 5 and 6.
By linking a second screen, keyboard and mouse to the laptop, automatic feedback could be simulated. In another room, the researcher could watch the test person practice on the second screen, which duplicated the main laptop screen. The researcher was able to give the feedback trough the control panel (with the pink background) at the right side of the interface. This control panel was not visible to the test person itself but hidden under a black piece of paper.
5.3. Phase III: Experiment
The feedback program is designed to help a student practice signs and reduce the amount of performance mistakes. To test if the program works, a controlled experiment was set up.
Experiment set up
For a controlled experiment at least two groups are required [11]. In this experiment there is a feedback group and a non-feedback group. The feedback group would practice a set of 15 signs with the program designed by the researcher. The non-feedback group would practice on a recreated prototype, of the “sign-self quiz” as provided by the Dutch Sign Centre. So both groups have practiced the same set of signs. The program they use for practicing is therefore the independent variable in this experiment [11].
The groups were composed of test persons, sampled by virtue of a maximum variety sample [12, p. 419] to be able to represent the variety of students of the GGMD, who are using the learning methods of the Dutch Sign Centre.
The GGMD indicated that they started 43 courses in the first quarter of 2018. A course can be requested by a client if he/she themselves have a hearing impairment. The course is given in the home situation together with the client's relatives. The group size varies from 1 student to a maximum of 8. In the courses they give, often, only the client is hearing impaired or deaf, the other students of the group are not hearing impaired. The age of the participants varies enormously but most participants are aged between 20 and 60 years old.
The two groups were approximately equally divided in terms of age and level of education. The study level was rated as follow:
finished high school finished MBO finished HBO
finished University bachelor/master
Table 3: demographics Table 4: demographics feedback group non-feedback group
Experiment sessions
The experiment was carried out in 6 sessions with 4 test persons per session. Two belonged to the feedback group and two to the non-feedback group.
Sign language lessons provided by the GGMD are given at the client’s home. In addition students practice at home between the lessons. Therefore test session took place at the researcher’s home, where an informal setting was created.
During the session the test persons needed to learn and practice a set of 15 signs. 11 of these signs existed of four basic elements, namely handshape, place, movement and orientation. Four signs also included a specific facial expression (mimicry) that needed to be performed.
Feedback group Non-Feedback group
Testperson Age Studylevel Testperson Age Studylevel
1 26 2 2 27 3 3 26 4 4 24 3 5 56 2 7 21 3 6 54 2 8 23 2 9 54 2 10 53 2 12 59 1 11 63 4 15 36 3 13 43 2 16 21 2 14 23 2 18 62 2 17 55 1 20 35 2 19 37 2 23 25 3 21 58 3 24 23 1 22 60 2 Avarage 39,75 2,166667 Avarage 40,58333 2,416667 StdDev: 15,27593 0,799305 StdDev: 15,87691 0,759203
Each session was structured as follows.
- First the test persons were given a sign-lesson. In this lesson the basic elements of a sign were explained and the 15 gestures they had to practice were presented by the researcher, who took the task as teacher upon herself.
- The participants were split up. All test persons had to practice with the signs twice, on an Acer laptop. During the sessions, the screen recordings were made with the consent of the participants.
- Between practicing they had a break, during which they were not allowed to practice. To distract them, they were required to read magazines.
- After two times practicing and another break, they made a test on the laptop and recorded movies of themselves performing the 15 signs.
- After making the test they could go home.
The lesson took place in the living room. Upstairs in two different rooms the laptops were placed on a desk with a chair. The second screen, with which the researcher could give feedback, was placed in a third room on the same floor, out of sight of the test persons. Figure 8 shows a systematic overview of the structure of a session.
Figure 8: test session structure Experiment results
To identify possible usability problems of the program, a questionnaire regarding the usability was filled in by the feedback group. This questionnaire consisted of 10 questions (see table 5). The answers can be found in APPENDIX VII.
No usability issues were pointed by the test persons in the questionnaire. However they seemed not to have noticed that the dotted line, which formed a silhouette on the recording frame, was meant to indicate that they needed to sit at a certain distance from the webcam, in order to film their complete upper body. They only saw the silhouette as an indication to sit in the middle of the frame. For this test this was no problem since the researcher had no difficulty with giving feedback on the signs. However, when the movies are analyzed by the computer itself this will probably cause problems because sometimes signs fell outside the frame.
Nr. Question
1 Below you see a screenshot of the practice program. Can you explain what was indicated with the dotted line?
2 Was it immediately clear how you could record a movie of yourself?
3 Are there elements that were not clear to you or difficult on this page? If so, can you describe them?
4 Can you explain the goal of this page?
5 Can you explain the meaning of the different of the buttons? Green & orange
6 Did you understand that you could click on the buttons for more information about the sign?
7 Did you appreciate that you could see movies of the correct performance of the sign?
8 Did you appreciate that you could see yourself back on the movie
9 Have you compared your own movie with the movie of the correct execution of the sign? Why did or didn't you do this?
10 Q10: Are there elements that you did not find clear or difficult on this page? If so, can you describe it?
Table 5: questions of the usability questionnaire
In order to determine the results of the experiment, the researcher watched at all screen recordings of the practice sessions of the feedback group and the test of both groups. The researcher assessed the sign performance by grading all the basic elements as correct or incorrect. Grading the signs immediately gave a first insight into the effect of the feedback program. The researcher noticed that the feedback group performed the signs more clear than the non-feedback group.
Subsequently the results were analyzed. First only the results of the feedback group will be discussed. Table 6 and 7 show the results of the two practicing sessions and the final test result. The variable ‘total elements correct’ shows how many elements (out of 64) the test person performed correctly. If all the elements of a sign were correct then the sign itself counted as correct.
Table 6: Feedback-group Table 7: Feedback-group elements correct complete signs correct
Feedback Group: Total elements correct Feedback Group: Signs correct
Testperson Practicing session 1 Practicing session 2 Test Testperson Practicing session 1 Practicing session 2 Test 1 51 53 57 1 6 8 10 3 38 61 62 3 7 14 13 5 38 42 51 5 1 3 9 6 47 51 62 6 5 7 13 9 33 34 42 9 1 2 9 12 48 55 56 12 7 7 10 15 59 60 63 15 11 12 14 16 60 61 63 16 11 12 14 13 38 54 58 13 0 7 9 20 49 58 62 20 6 11 13 23 54 58 63 23 9 10 14 24 35 39 45 24 1 4 6 Avarage 45,8333333 52,16666667 57 Avarage 5,41666667 8,08333333 11,1666667 Feedback Group: Total elements correct Feedback Group: Signs correct
Testperson Practicing session 1 Practicing session 2 Test Testperson Practicing session 1 Practicing session 2 Test 1 51 53 57 1 6 8 10 3 38 61 62 3 7 14 13 5 38 42 51 5 1 3 9 6 47 51 62 6 5 7 13 9 33 34 42 9 1 2 9 12 48 55 56 12 7 7 10 15 59 60 63 15 11 12 14 16 60 61 63 16 11 12 14 13 38 54 58 13 0 7 9 20 49 58 62 20 6 11 13 23 54 58 63 23 9 10 14 24 35 39 45 24 1 4 6 Avarage 45,8333333 52,16666667 57 Avarage 5,41666667 8,08333333 11,1666667
July 2018 Bonita van Veldhoven
9 Figure 8 and 9 display the results per practicing session of the
feedback group and shows therefore their progression.
Figure 8: Progression of the feedback-group (elements correct)
Figure 9: Progression of the feedback-group (complete signs correct)
By looking at the progression (figure 8 and 9) it can be concluded that practicing with the feedback program has a positive influence on the number of correctly performed signs and the amount of correctly performed elements. On average, the test persons increased the amount of correct elements by 5.583 per practicing session. This resulted in the fact that they had on average 2.875 more signs completely correct per practicing session.
The final test results of the feedback group are compared with the test results of the non-feedback group (see table 8). This has been done carrying out statistical calculations using IBM SPSS Statistics 25. The sample size of this experiment was too small for the results to be translated to the entire population. However the results can be seen as the beginning for a larger study and give a first indication of the effects of the program. The results can be used to decide if it is useful to continue research to this program.
The following tests were executed:
- To compare the results of the two groups and test the hypothesis, a t-test for independent means is carried out (for signs correct and elements correct)
- A linear regression analysis is carried out to measure the influence of feedback on signs correct
- A multiple regression analysis is then carried out to show to what extent this influence holds up when age and study level is also taken into consideration. For these measurements the significance level of 0.05 will be used, to lower the risk of a type 2 error [12, p. 349] while maintaining a high level of accuracy.
The variable Practice Program was recoded into 1 and 2. Non-feedback was coded as 1, Non-feedback was coded as 2, at nominal value. This was needed because SPSS could not otherwise analyze this variable. It is labeled PraProRec in the calculations (see APPENDIX I).
Table 8: Test results non-feedback group
t-test for independent means
The null hypothesis is that there is no difference between getting feedback during the practice sessions and not getting feedback. The alternative hypothesis is that there is a positive correlation between getting feedback and having more signs correct. The table Group Statistics (APPENDIX I) shows the sample parameters for non-feedback (PraProRec 1.00) and feedback (PraProRec 2.00) and how many signs were correct during the final test.
The ‘equal variances assumed’ is significant on 0,666 which is higher than 0,05 if we use a 95% confidence level. This shows that we must assume that the population is equal, and we should look at the top half of the table. The ‘Sig. (2-tailed)’ shows a significance level of 0,051 for a two-tailed test. Because the hypothesis is that the group that used the practice program with feedback did better at the test and had more signs correct a one-tailed test is needed. To do this 0,051 is divided in half to 0,026. This is lower than 0,05 for a 95% confidence level. This shows a statistical significant correlation between getting feedback while practicing signs and scoring better at a test for signs afterwards, the null hypothesis is rejected. This only shows the fact that there is correlation present. The next step is checking how large the effect is. For this, both a single regression model and a multiple regression model are carried out.
Non Feedback group : test results
Testperson Elements correct Total signs correct 2 63 14 4 57 11 7 58 10 8 55 10 10 53 8 11 50 8 13 58 9 14 59 10 17 39 5 19 56 11 21 40 5 22 47 6 Average 52,9166667 8,916666667
Single regression analysis
The adjusted R Square (which can be found in APPENDIX I) has a value of 0,124. Because the sample is rather small and the adjusted R Square tries to account for this when translating to a population, adjusted R Square is used in favour of the regular R Square. It means that 12,4% of the signs correct is predicted by whether or not a person gets feedback during practice.
The ANOVA table (see APPENDIX I) shows that the regression model significantly predicts the dependent variable. The significance is 0,051, which is just above 0,05 which is required for 95% confidence. However, since there is a directional hypothesis (that getting feedback increases test results) the significance is divided by two into 0,0255, which is below the required 0,05.
Based on the Coefficients table the following equation is concluded:
Signs correct = 6,667 + 2,250 * Practice Program.
This equation is based on the regression model and tries to predict how many signs a person has correct based on the values and the knowledge of whether or not someone received feedback during practice. With a constant of 6,667 signs correct, getting feedback while practicing provides for 2,250 more signs correct during a test session, and therefore 2,250 more signs learned within a set of 15 signs. In other words, within the set of 15 signs that the groups had to perform during the test, they both had a base amount of sign correct of 6,667 (this is a fictional number). To fill in the equation for a person who received feedback (coded as 2,00) during practice:
Signs correct = 6,667 + 2,250 * 2.00 = 11,167.
Following this equation it can be predicted that a person who receives feedback during practice will get, on average, 11,167 signs correct.
Multiple regression analysis
With the multiple regression, the model for predicting how many signs correct is improved by adding Age and Study Level. This increases the predictive power of the model and also takes into account these two factors in showing how great the effect of the feedback program is. In other words, by taking into account Age Study level, we get a better measure of the influence of the feedback program. In comparison with the single regression model, the factor is increased from 2,250 to 2,492.
The regression model summary (see APPENDIX I) shows the adjusted R Square for explanatory power. Again the adjusted R Square is used, this time because there is more than one independent variable. The adjusted R Square is 0,467, meaning that the independent variables predict for 46,7% how many signs a person has correct during the test session.
The ANOVA table (see APPENDIX I) shows the significance of the prediction of the regression model. At F(3,20) = 7,730,
p < 0,001, it is lower than the required F = 3,10, p < 0,05 for 95% confidence. The independent variables significantly predict the dependent variable; the regression model is a good fit for the data.
Based on the Coefficients table (see APPENDIX I) the following equation is formed:
Signs correct = 6,702 + 2.492 * Practice Program Recoded – 0,080 * Age + 1,236 * Study Level.
All the independent variables as well as the Constant have a significance level below 0,05, indicating there is a statistical significant influence between them and the dependent variable. The unstandardized coefficients in the Coefficients table (APPENDIX I) show how much the dependent variable changes for one independent variable with the other independent variables being held constant. In other words, everything else constant, getting feedback increases signs correct during the test by 2,492. An increase in study level, on its own, increases signs correct by 1,236. Because there are three possible increases in study level (1 to 4) and no one had study level 0, the maximum positive influence going up in study level a person can have is 3,708 (1,236*3). A person with the highest study level has, on average, 3,708 more signs correct than a person with the lowest study level. The age variable is negative, indicating that for each year older, a person will get 0,080 less signs correct. For example, on average, a person of 60 will get 3,20 less signs correct compared to a person of 20 (0,080*40=3,20).
For example, there is a person of 45, with study level 3, who received feedback during the practice session (2.00). If this is filled into equation we get:
Signs correct = 6,702 + 2.492 * 2.00 – 0,080 * 45 + 1,236 * 3 = 11,794.
Using the multiple regression model (see APPENDIX I) we can predict that this person will have on average 11,794 signs correct. 6. CONCLUSION
This research was conducted to investigate whether a program that can provide immediate feedback on signs used in sign language, can help to reduce the amount of sign performance mistakes.
The first sub question of this research was: what do sign language teachers and students think about a system that can provide immediate feedback on signs made before a webcam? The answer was positive. Both Dutch sign language teachers and students have a positive attitude towards a program that gives direct feedback on signs and see advantages. Teachers say that the more feedback students receive the better as long as the feedback is reliable. Students think that a feedback system can help them to reduce the possibility to learn signs in the wrong way, to focus on the details of the signs and to reduce their insecurity.
The second sub question was meant to find out what are common mistakes are made by Sign Language students. The answer on this question gave an indication what kind of feedback
July 2018 Bonita van Veldhoven
11 is necessary to reduce the amount of performance mistakes. It was
found that most mistakes are made regarding the basic elements. Often students perform some but not all elements right. However, a sign is only correct if all elements are performed correct.
The third sub question was: how should feedback on signs be given in an online environment according to Dutch sign language teachers and students? Opinions differ on this, but a number of ways have often been mentioned. The students want to get feedback in the form of movies and textual, they also want an indication of what they do right and wrong. In addition they want to see movies of the correct performance of the signs and the details of a sign pointed out in addition to the movies. Teachers find it important that students receive feedback in a positive way on the basic elements of the signs.
Based on the insights of the teachers and the needs of the students a design and prototype has been made for such a program. Therefore the last sub question of this research was: to what extent does the program help in reducing performance mistakes?
The experiment results show that the program had a positive influence on the sign performance of the 12 test persons who practiced with this program. On average their performance was better, in comparison with the sign performance of an approximately equivalent group of persons that practiced with an existing non-feedback program.
Together the sub questions answered the following research question: What kind of direct feedback do Dutch Sign
Language students need on their signs, while they practice in on an online environment in front of their webcam, to reduce the amount of performance mistakes?
This research and corresponding experiment showed that a combination of textual and visual feedback is needed to meet the user needs and to reduce the amount of performance mistakes. Textually the program needs to indicate which basic elements were performed right or wrong. In addition the students need to have the possibility to compare their performance with a movie of the correct performance of the sign. Furthermore textual descriptions of the sign, per basic element are desired to explain the details of the signs.
In other words a combination of different kinds of feedback is necessary to reduce their amount of performance mistakes:
Textual and trough color changing buttons this function draws the students attention to the wrong performed basic element(s)
Possibility to compare a correct movie with their own recorded movie
With this function the students can check what they did wrong and how they can do it better
Textual descriptions of the sign per basic element (including images of handshapes and mimicry) To explain the details of the signs, which are sometimes not clear to see in a movie
7. DISCUSSION
This research program is designed for practicing separate signs that are used in Dutch Sign Language and Dutch with Signs. It is however not a program to learn this languages. There must be no confusion about the fact that this is a sign-practice program and not a Sign Language learning program.
The results of this experiment give the indication that a feedback program has a positive effect on the sign performance while learning and practicing them. It lowered the amount of performance mistakes by the test persons and seemed to make them sign clearer than users that practiced with the existing program in which they only can watch movies of the correct performance to control themselves.
However the sample size is too small to make predictions regarding the effect that this program can have on the sign performance of NGT and NmG students in general. In other words, the program seems to increase the sign performance of the students and reduce the amount of performance mistakes. Nonetheless, more research needs to be done to determine whether or not this holds true on the entire population. However this research was a first step in that direction and the results showed that this program does have potency.
8. FUTURE WORK
The feedback program needs to be tested on a larger sample or tested throughout a whole NGT or NmG course. For example two groups that following the same course can be compared if one group uses the existing learning environment of the Dutch Sign Centre for practicing and the other group uses the same environment supplemented with the feedback program.
In addition, the potential role that this program can play in addition to a course can be investigated. A possibility is for example, that the program can remember the signs that the student finds hard and performed wrong (perhaps multiple times). Then these particular signs can be practiced again. Also, it would be good that, when the program is used in addition to a course, the teacher can see their progress and also has the possibility to give additional feedback on the movies that are recorded by the students.
Finally the technical aspects behind the program need to be examined. How exactly will the program analyze the movies recorded by the users and provide them with the correct feedback? Previous studies show that a program like this is technically possible (see related work). However, there needs to be established how this particular program, with all designed functions can be developed and added to an online environment.
REFERENCES
[1] Nederlands Gebarencentrum, "Organisatie," [Online]. Available: https://www.gebarencentrum.nl/organisatie/organisatie/. [Accessed 11 June 2018].
[2] GGMD, "Communiceren met Gebaren; verschillende trainingen in gebaren," 01 2018. [Online]. Available:
https://www.ggmd.nl/wordpress/wp- content/uploads/2018/01/180125-flyer-communiceren-met-gebaren.pdf. [Accessed 25 05 2018].
[3] Stichting Nederlands Gebarencentrum, "Werkboek voor cursisten; NmG-AB1 voor algemeen belangstellenden," Stichting Nederlands Gebarencentrum, Amersfoort, 2009.
[4] T. Schermer and C. Koolhof, "Enkele basiskenmerken van de Grammatica van de Nederlandse Gebarentaal (ngt)," in Van Dale
Basiswoordenboek Nederlandse Gebarentaal, Utrecht, Van
Dale, 2009.
[5] K. Emmorey, R. Bosworth and T. Kraljic, "Visual feedback and self-monitoring of sign language," Journal of Memory and
Language, vol. 61, no. 3, pp. 398-411, 2009.
[6] M. Huenerfauth, E. Gale, B. Penly, S. Pillutla, M. Willard and D. Hariharan, "Evaluation of Language Feedback Methods for Student Videos of American Sign Language," Transactions on
Accessible Computing, vol. 10, no. 1, pp. 1-30, 2017.
[7] D. Kelly, J. McDonald and M. Markham, "A System for Teaching Sign Language using Live Gesture Feedback," in
Automatic Face & Gesture Recognition, Amsterdam, 2008.
[8] D. Mazumdar, A. K. Talukdar and K. K. Sarma, "Gloved and free hand tracking based hand gesture recognition," Emerging
Trends and Applications in Computer Science (ICETACS), pp.
197-202, 2013.
[9] H.-S. Yeo, B.-G. Lee and H. Lim, "Hand tracking and gesture recognition system for human-computer interaction using low-cost hardware," Multimedia Tools and Applications, pp. 2687-2715, 2015.
[10] J. J. Garret, The Elements of User Experience; User-Centered Design for the Web and Beyond, Berkeley: New Riders Publishing, 2011.
[11] A. Grubišić, S. Stankov, M. Rosić and B. Žitko, "Controlled experiment replication in evaluation of e-learning system's educational influence," Computers and Education, vol. 53, no. 3, pp. 591-602, 2009.
[12] A. Bryman, Social Research Methods, New York: Oxfort University Press Inc., 2012.
[13] R. M. Gurav and P. K. Kadbe, "Real time Finger Tracking and Contour Detection for Gesture Recognition using OpenCV,"
Industrial Instrumentation and Control (ICIC), pp. 974-977,
2015.
[14] A. Haria, A. Subramanian, N. Asakkumar, S. Poddar and J. S. Nayak, "Real time Hand Gesture Recognition using different algorithms based on American Sign Language," Imaging, Vision
& Pattern Recognition (icIVPR), pp. 367-374, 2017.
[15] OpenCV team, "About OpenCV," 2018. [Online]. Available: https://opencv.org/. [Accessed 16 01 2018].
[16] C. John, "SignAssess – Online Sign Language Training Assignments via the Browser, Desktop and Mobile," Lecture
Notes in Computer Science, vol. 7383, no. Part 2, pp. 253-260,
2012.
APPENDIX I : STATISTICAL TABLES P. 13
APPENDIX II : DESIGN PROCESS P. 15
APPENDIX III : A EXAMPLE OF A TRANSCRIBED INTERVIEW P. 39 APPENDIX IV : QUOTATION REPORT TEACHER INTERVIEWS External file
This is an Excel file and therefore an external appendix
APPENDIX V : STUDENT SURVEY RESULTS External file
This is an Excel file and therefore an external appendix
APPENDIX VI : EXPERIMENT PRACTICING & TEST RESULTS External file
This is an Excel file and therefore an external appendix
APPENDIX VII : EXPERIMENT RESULTS USABILITY QUESTIONAIRE External file
This is an Excel file and therefore an external appendix
APPENDIX VIII: FEEDBACK PROGRAM PROTOTYPE External file
This prototype exists of a set of html, css and javascript files
All external files can be found in the attached folder or requested per email. Bonita.vanveldhoven@gmail.com
July 2018 Bonita van Veldhoven
13 APPENDIX I : STATISTICAL TABLES
t-test for independent means
Group Statistics
PraProRec N Mean Std. Deviation Std. Error Mean
Signs correct
1,00 12 8,92 2,678 ,773
2,00 12 11,17 2,657 ,767
Independent Samples Test
Levene's Test for Equality of Variances
t-test for Equality of Means
F Sig. t df Sig. (2-tailed)
Mean Difference Std. Error Difference 95% Confidence Interval of the Difference Lower Upper Signs correct Equal variances assumed ,191 ,666 -2,066 22 ,051 -2,250 1,089 -4,509 ,009
Equal variances not assumed
-2,066 21,999 ,051 -2,250 1,089 -4,509 ,009
Single regression analysis
Regression model summary
Model R R Square Adjusted R Square
Std. Error of the Estimate
1 ,403a ,162 ,124 2,668
a. Predictors: (Constant), PraProRec
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 30,375 1 30,375 4,268 ,051b
Residual 156,583 22 7,117
Total 186,958 23
a. Dependent Variable: Signs correct b. Predictors: (Constant), PraProRec
Coefficientsa Model Unstandardized Coefficients Standardized Coefficients t Sig.
95,0% Confidence Interval for B
B Std. Error Beta Lower Bound Upper Bound
1 (Constant) 6,667 1,722 3,871 ,001 3,095 10,238
PraProRec 2,250 1,089 ,403 2,066 ,051 -,009 4,509
Multiple regression analysis
Regression model Summary
Model R R Square Adjusted R Square
Std. Error of the Estimate
1 ,733a
,537 ,467 2,081
a. Predictors: (Constant), Study level, PraProRec, Age
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 100,382 3 33,461 7,730 ,001b
Residual 86,576 20 4,329
Total 186,958 23
a. Dependent Variable: Signs correct b. Predictors: (Constant), Study level, PraProRec, Age
Coefficientsa Model Unstandardized Coefficients Standardized Coefficients t Sig.
95,0% Confidence Interval for B
B Std. Error Beta Lower Bound Upper Bound
1 (Constant) 6,702 2,425 2,764 ,012 1,644 11,759
PraProRec 2,492 ,862 ,446 2,893 ,009 ,695 4,289
Age -,080 ,028 -,449 -2,906 ,009 -,138 -,023
Study level 1,236 ,553 ,350 2,235 ,037 ,082 2,390
15
APPENDIX II : DESIGN PROCESS
The idea for this project is to design a program, in which Sign Language students can practice signs (separate from each other) while they receive automatic feedback from the computer. During this design process the five planes of Jesse James Garret were followed.
Table of contents Strategy plane ... 16 Teacher interviews ... 17 Pre-formulated questions ... 17 Interviews results ... 18 Analysis ... 19 Conclusion ... 20 Student survey ... 21
Questions and structure ... 21
Results ... 25
Survey result summary/analysis ... 31
Conclusion ... 32
User needs & product objectives ... 32
Scope plane ... 33 Functional specifications ... 33 Content requirements ... 33 Structure plane ... 34 Skeleton plane ... 35 Surface plane ... 37
In the strategy plane the product objectives and the user needs, need to be established. The product objectives describe what the “owners of the final product” want to get out of it [1]. As this is an individual thesis project, there is no real owner that wants something to get out of the project. However, to make a program like this succeed, sign language teachers and course developers need to have confidence in the program. They must stand behind the program in order to use it in their courses and recommend it to their students to practice with. Therefore, Sign Language teachers were interviewed to establish the product objectives. The interviews were semi-structured with pre-formulated questions. This made sure that all the desired information was obtained, but left space for supplementary questions and for the teachers to come up with information I did not think about yet.
The users of the desired system are Dutch Sign Language or Dutch with Signs students. Therefore, a survey was held among them. The goal of this survey was to find out how students like to practice at home, where they have difficulties with and of course, if there is need for a program that gives them direct feedback and what their vision is towards a program like that. There has also been tried to retrieve statistics about these users. Unfortunately no statistics about the target group were found through desk research.
The GGMD (Geestelijke Gezondheidszorg en Maatschappelijke Dienstverlening) is a foundation and healthcare provider for anyone with a hearing, speech or language limitation [2]. They support people who are deaf or have a hearing problem and their loved ones by offering Sign Language courses to their clients. These courses are reimbursed by the health insurance of the client when he/she has a hearing loss at both ears for more than 35 dB [2].
At GGMD, qualified teachers work sign language. They work with teaching materials from the Dutch Sign Center. A direct request for statistics to the GGMD provided the following information about statistics of Sign Language students.
The clients of the GGMD can request a course if they themselves have a hearing impairment. The courses are given in the home situation together with the client's relatives. In the first quarter of 2018, 43 new courses started. Group sizes vary from 1 to a maximum of 8. The GGMD only haves the data of the client, not from the other participants. In the courses that they give, the majority of the group is hearing. Often, only client is hearing impaired / deaf. The age varies enormously. Most participants are between 20 and 60 years of age. This information does still not give an indication of the total target group (everyone that does follow Dutch Sign Language Lessons or Dutch with Signs lessons). Therefore 9 teachers were asked how many students they have in their groups, on average. Based on those numbers it is estimated that there are on average 5 students per group.
17
Teacher interviews
Seven teachers have been interviewed. Two of them were interviewed at the same time. Because one of them could only speak Sign Language, the other teacher was able to translate her answers for the researcher, as her Sign Language level is not high enough to understand everything.
Pre-formulated questions
The goal of these interviews is to get insight in:
How a teacher gives feedback to the students now (as inspiration for a feedback design)
What kind of mistakes are often made by sign language students (to find out what type of feedback
the system must provide)
The ideas teachers have for an real-time feedback system
The information provided by the interviews will be used as inspiration for the for the feedback design to be developed.
Each interview consisted of the same questions (see table 1). But also supplementary questions were asked if this was needed to clarify something of when something interesting came up.
Nr. Question Reason of the question
1 Which form of sign language do you teach?
NmG or NGT?
To break the ice and get an overview of the teacher
2 For how long are you teaching sign language
lessons?
To break the ice and get an overview of the teacher
3 Do you teach according to a certain teaching
method?
To get an overview of the teacher and to see if/how practicing in front of a webcam can fit in the teaching method.
4 How do you advise your students to practice
at home?
To find out how students are practicing at home now and what the teacher find good methods of home practicing
5 What difficulties do students face while
practicing at home, in your opinion?
To get an insight in the needs sign students have, when they practice at home
6 Do you still see many mistakes / points of
improvement in the performance of the signs when students have practiced at home?
To find out where the system should give feedback on
7 What kind of performance mistakes do
students make often?
To find out where the system should give feedback on (maybe they think about more things than in the previous question. Or otherwise for triangulation)
8 Do you ever give feedback on students online
or via a digital road? If so, how?
To find out how teachers give feedback in digital ways
9 To what extent do you think that extra
feedback on the sign performance during practice at home, can help students?
To get a sense whether or not they find this kind of practicing at home useful.
10 The goal of this research is that sign language students can practice signs before the webcam, while receiving direct feedback. What kind of possibilities do you see in such a type of system?
A direct question to find out what teachers expect of a real-time feedback system and if the teachers have an ideas for it
11 What do you think is a good way of giving
feedback in this setting? How do you imagine a program like that?
To find out if the teachers have an idea for a real-time feedback system
The interviews were transcribed using the transcription software AmberScript. Two examples can be found in APPENDIX III. Subsequently the transcribed interviews were coded using Atlas TI and an Excel Quotation Report was exported, which can be found in external APPENDIX IV.
On the first tab in the Excel Quotation Report, the CodeTable can be found (see table 2). This table shows which codes were used and how often and where they occur in the different interviews. The other tabs in the Excel file are named after the different code numbers. Each tab gives an overview of all the content that was coded by a particular code. Figure 1 shows an example of the content of tab Q1.
Table 2: CodeTable Interviews
Figure 1: Tab Q1 of the Excel Quotation Report
Code Codename Marlies van der Plas Lorre Willems Mariëlle Stoop Marijke Scheffener & Ulrika Klomp Petra Bouterse Soraya luijks Totals
Q1 Which form of sign language do you teach? NmG or NGT? 1 1 1 0 1 1 5
Q2 For how long are you teaching sign language lessons? 2 1 1 0 1 1 6
Q3 Do you teach according to a certain teaching method? 1 2 1 1 1 3 8
Q4 How do you advise your students to practice at home? 5 3 3 5 3 4 23
Q5 What difficulties do students face while practicing at home, in your opinion? 10 1 1 2 5 1 20
Q6 / Q7
Do you still see many mistakes / points of improvement in the performance of the signs when students have practiced at home? / What kind of performance
mistakes do students make often? 5 4 8 1 3 4 25
Q8 Do you ever give feedback on students online or via a digital road? If so, how? 4 2 1 4 4 4 19
Q9
To what extent do you think that extra feedback on the sign performance during
practice at home, can help students? 1 6 2 1 3 1 14
Q10
The goal of this research is that sign language students can practice signs before the webcam, while receiving direct feedback. What kind of possibilities do you
see in such a type of system? 0 2 2 1 4 2 11
Q11
What do you think is a good way of giving feedback in this setting? How do you
imagine a program like that? 5 6 5 8 3 2 29
AC1 sees a potential advantage for a feedback system 1 1 2 9 0 0 13
AC2 What students find hard in the lessons 0 1 0 0 1 0 2
AC3 explanation about other learningmethod than from the Dutch Sign Centre 2 2 0 0 1 3 8
AC4 expresses fear of a digital automatic feedback system 0 1 3 2 3 2 11
AC5 How the teacher gives feedback during the lessons 2 3 0 0 0 0 5
AC6 additional suggestions 1 0 0 2 0 3 6
