An Investigation into Statistical Language Learning Using Click Detection

(1)

MA Linguistics

Language & Society

An Investigation into Statistical Language Learning Using Click Detection

Lia Morrissey

11759771 January 17th, 2019 18 Credits

Supervisor: mw. prof. dr. J. E. Rispens

(2)

1

How do we segment words from a stream of speech heard in an unknown language? Which cues are available to us as humans when trying to learn a new language, and could we still learn language if we were deprived of all but one of these cues? Recent studies have provided evidence that both adults and infants as young as 8 months old are able to perform statistical calculations on an acoustic stream consisting of an unknown language in order to segment words from it. The following study was carried out to investigate statistical learning in a group of 26 adults, who were exposed to an artificial language containing no cues to word boundaries except for transitional probabilities. Their performance was measured using a click detection task and a 2-alternative forced choice test, so that both online and offline learning could be taken into account. The data gathered from this study suggests that the subjects were not capable of learning the artificial language using statistical cues alone, thus providing evidence against the use of such learning mechanisms for language acquisition. It also discusses the potential of online measures to investigate statistical learning, highlighting the shortcomings of the click detection task, used in this study, as a measure for online learning.

Keywords: statistical learning, online learning, word segmentation, click detection, transitional probabilities,

distributional cues

1. Introduction

Research into statistical learning has shown that humans, from the earliest stages of life, are sensitive to various statistical aspects of their environments. Statistical learning mechanisms facilitate the detection of structure, allowing language learners to extract patterns from a sequence of stimuli and use these for further processing. Connectionist models as well as other computational analyses of linguistic corpora have demonstrated that there are a variety of statistical patterns available in language that can aid in language acquisition (Saffran & Kirkham, 2018). Statistical learning can be defined as the act of extracting the statistical properties of sensory input in time and space (Frost, Armstrong, Siegelman, & Christiansen, 2015), or the exploitation of statistical patterns in the natural world (Saffran & Kirkham, 2018). Misyak & Christiansen (2011) define it as the discovery of structure by way of statistical properties of the input. Statistical information sharpens predictions and by tuning predictions, learners can reduce errors to better anticipate outcomes (Saffran & Kirkham, 2018). It is generally understood that statistical learning is a type of implicit learning; both adults and children have demonstrated predictive knowledge of the relationships between stimuli following brief and passive exposure to syllable combinations in an artificial language. This ability has been demonstrated in various areas within the domain of language, including speech segmentation (Saffran, Aslin & Newport, 1996; Saffran, Newport & Aslin, 1996), phonotactic patterns (Dell, Reed, Adams, & Meyer, 2000), phonetic categories (Maye, Werker, & Gerken, 2002), and orthographic regularities (Pacton, Perruchet, Fayol, & Cleeremans, 2001). However, more recently, this ability has also been demonstrated in subjects presented with tones (Abla et al., 2008), musical tones (Creel, Newport, & Aslin, 2004; Saffran, Johnson, Aslin, & Newport, 1999), and visual shapes (Abla & Okanoya, 2009); and it has also been witnessed in non-human primates (Hauser, Newport & Aslin, 2001; Newport, Hauser, Spaepen, & Aslin, 2004). These more recent studies suggest that the mechanisms responsible for this type of language learning are not language-specific. As part of this study, we will examine the process of word segmentation and how this form of statistical learning may play a crucial (though not exclusive) role in the early stages of language acquisition.

1.1. Early Word Learning

There are a number of cues available to infants learning their first words in a new language, however, these can vary between different languages. In earlier days, it was believed that children learned words

(4)

3

primarily from isolated occurrences; Pinker (1984) proposed that words learned in isolation could help children segment multi-word utterances. However, later studies have argued against this theory. Recent research suggests that a significant amount of word segmentation occurs due to the infant’s ability to perform statistical calculations on the speech stream. When infants first encounter language, in order to derive meaning from a stream of sounds, they must first attempt to segment, or chunk, the stream into smaller meaningful parts. Speakers do not mark word boundaries with pauses, so the infant must determine where one word ends and the next begins. Take, for example, if an infant hears the words ‘red panda’. They must learn that red and panda are words, but that edpan (spanning the word boundary between red and panda) is not a word. We have seen evidence for an awareness of statistical regularities in infants as young as eight months (Saffran et al., 1996), where, following exposure to an artificial speech stream for just two minutes, they showed significant test-trial discrimination between words and non-words (p. 1927), suggesting an ability to rapidly segment words in a newly encountered language using only the statistical information provided by transitional probabilities (TP’s). In this study, we will focus on word segmentation based on these transitional probabilities as a form of distributional learning.

2. Transitional Probabilities as Distributional Cues

In 1996, Saffran, Newport & Aslin proposed a theory that, in order to segment (and to learn) words, infants could calculate, implicitly, the transitional probability (the probability of one event given the occurrence of another event) between syllables and, upon encountering certain combinations/structures more frequently, they could extract words from a stream of speech. The logic of this measure is applicable to syllables, phonemes, features and other types of sublexical units (p. 610); we will focus on its application to the combination of syllables as this is most relevant to the current study. Say we take the word panda and divide it into two syllables, pan and da. We want to consider the frequency of each syllable occurring on their own in English, which in this case is fairly high for both, but what percent of the time does da occur after pan in English? Pan can be proceeded by various other syllables, as is the case with words such as pan.cake, pan.sy, and pan.el. Each of these syllables will follow pan with some level of probability; we must then consider how this probability is calculated. There are two types of syllable pairings here: word-internal, which consists of syllable pairs that sit within a word as we saw in the above example; and word-external pairings, for syllables that span across word boundaries. The transitional probability between word-external pairs is less likely than it is between word-internal pairs. For example, pan#for, pan#ref and pan#under are all possible occurrences in English, in fact, pan followed by just about any syllable is possible, as long as it is part of a separate word (p. 610). Thus, we can conclude that word-external pairs generally occur far less frequently than word-internal pairs, and in order to calculate this frequency, we can use the following formula, as proposed by Saffran et al. (1996).

The transitional probability of Y (da) given X (pan) is:

1. frequency of pair 𝑋𝑌

frequency of 𝑋 → 2.

frequency of 𝑝𝑎𝑛.𝑑𝑎 frequency of 𝑝𝑎𝑛

High transitional probability means that the presence of X makes Y highly likely to occur, while low transitional probability means the presence of X only weakly predicts that Y may occur. If we look at example 3 below, the denominator is the same as that in example 2, but the numerators will be different because word-external pairs occur far less frequently (p. 610).

3. frequency of 𝑝𝑎𝑛#𝑓𝑜𝑟

frequency of 𝑝𝑎𝑛

Following Saffran et al’s. (1996) formalization of the transitional probabilities (TPs) theory, various studies confirmed that TPs between sounds can be tracked as a way of discovering word boundaries

(5)

4

(e.g. Saffran, Aslin & Newport, 1996; Saffran, Newport & Aslin, 1996; Pelucchi, Hay, & Saffran, 2009). However, this is just one type of learning that could be occurring within the given environment. Though most studies aim to eliminate any other factors that could play a role in facilitating the segmentation of words, such as stressed syllables, phonotactic cues, and prosodic cues, it is difficult to say if they have been eliminated entirely. If any potential cues do remain in the artificial language it is important that they be given the warranted attention, as this could mean that different learning mechanisms are at play.

3. Alternative Distributional Cues and their Interactions with Statistical Cues

Due to the extensive variation between different languages, infants must equip themselves with highly adaptive learning strategies (Thiessen & Saffran, 2003). Though an increased familiarity to the statistical structure of the language in an infant’s environments seems to provide one strategy that could be used for word learning across the majority of languages (Saffran, 2003), there are many more cues available.

The theory suggesting that word learning could occur through exposure to words spoken in isolation was disproved in the 90’s (for example, Aslin et al., 1996), but a more recent study by Brent and Siskend (2001) who chose to revisit this topic using speech recordings from mothers and infants outside of the lab, in their natural home environment (as opposed to the controlled lab environment used for Aslin et al.’s (1996) study), found that the frequency with which mothers spoke a given word in isolation was a significant predictor of whether their child would be able to use that word later on. Some earlier research, such as the study by Huttenlocher, Haight, Bryk, Seltzer, and Lyons (1991), which found that, between the ages of 16 and 24 months, the amount a mother speaks to her child is correlated with the child’s rate of vocabulary growth, seem to support this theory, given that more maternal speech would usually equate to a greater number of isolated words. This theory supports a model in which young children typically acquire a small initial vocabulary from exposure to isolated words, however it does not imply that exposure to isolated words is essential for native-language acquisition. Brent and Siskind (2001) note that laboratory studies of infant speech segmentation suggest that isolated words are probably not essential. However, it is likely that once infants have acquired a small initial vocabulary, they could use that to segment new words out of multi-word utterances by recognizing adjacent familiar words (p. 42).

Outside of isolated words, what are children learning from exposure to speech in its more natural form, as expressed in series of connected utterances? One theory is that they are aided by language-specific prosodic cues correlated with word boundaries. In English, the acoustic cue believed to have the most widespread and powerful influence on word segmentation is lexical stress (Thiessen & Saffran, 2003). Mehler et al. (1988) showed that infants could use prosodic patterns to differentiate between their own language and a foreign language by just 4 days of age. Jusczyk, Houston, and Newsome (1999) suggested that infants use a metrical segmentation strategy, where they attend to strong/weak syllable sequences and extract them from fluent speech. Based on their findings, where infants incorrectly segmented words to create the preferred ‘strong/weak’ trochaic pattern, they argue that rhythm is among the first cues to word boundaries available to infants. However, this produces the question of how infants discover that words in English tend to have this stress pattern if they do not yet know any words. Isolated words are unlikely to have any role here as they are not usually experienced until much later. Thiessen and Saffran (2003) propose three possible solutions to this question. The first possibility is that trochaic bias could be innate, meaning they do not need to be familiar with any words to discover this pattern. However, they note that this is unlikely due to the lack of trochaic word stress in many languages. Their second suggestion is that it is through encounters with isolated words, but as discussed above, this seems unlikely, as infants as young as 4 days have shown sensitivity to stress patterns whereas infant-directed speech does not usually occur until much later. Another problem with the words in isolation strategy is that it presupposes that infants will treat the utterance as a single word, when indeed a multi-syllabic word could easily be mistaken for two words or more. If infants were not treating such words as isolated units, it is unlikely that they would learn about the predominant stress pattern in English. A similar problem occurs with the fact that many utterances in English will not begin

(6)

5

with a stressed syllable, as would be the case with sentences beginning with a determiner, so the infant must learn to differentiate between stress patterns for isolated words, which tend to carry stress on the first syllable, against utterances (sentences), which often do not. Thus, having other cues at their disposal, may help the infant avoid this problem. The third possibility suggested by Thiessen and Saffran is that infants use a different strategy, such as statistical cues, to segment their first words, and then rely more on stress once a basic vocabulary foundation has been established. If this third suggestion is correct, and infants only rely on statistical cues until they become accustomed to the language’s stress pattern, then statistical cues should be ranked with a secondary role, being used only when stress cues prove unreliable, as is the case when listening to an unfamiliar language with a different stress pattern (p. 707). The study by Thiessen and Saffran (2003), set out to establish the age at which infants change their preference for distributional cues; they aimed to specify up to what age infants would favour statistical cues over stress cues. Their results supported the hypothesis that there is a developmental progression between the time when infants are first aware of stress cues for word segmentation and the time when they favour trochaic stress as a cue to word boundaries in English (p. 714). They suggest that 7-month-old infants are less reliant on stress cues for word segmentation than 9-month-old infants tend to be, propounding that the segmentation of words from fluent speech using stress cues may be an adaptive strategy, one that would require less encounters with the word before it can be segmented, as the infant only really needs to hear a syllable once to decide whether it is stressed or not.

“… hearing one instance of a trochaically stressed word is, in theory, all an infant needs to isolate it from fluent speech. In contrast, an infant needs to hear several instances of a word to learn that its syllables display some statistical coherence. A few exposures are not sufficient to build up a reliable statistical representation of a word.” (p. 715)

Infants may use statistical cues initially, turning to stress cues once they have had sufficient exposure to the language to recognize its typical stress patterns. The main developmental advantage of this process being that they learn to take advantage of multiple, partly redundant cues as opposed to focusing entirely on one. From studies such as this one by Thiessen and Saffran (2003) and an earlier study by Johnson and Jusczyk (2001), it could be concluded that older infants have learned that stress cues are not always reliable, and that they must make use of multiple cues for word boundaries to be discovered successfully. Furthermore, though statistical cues are less favoured by older infants due to their somewhat less efficient nature when compared with stress cues, they may be providing infants with their first insight into the acoustic regularities of a language, potentially giving these cues a very significant role in the course of infant language acquisition. Outside of these prosodic and statistical cues, there are further strategies suggested to play a role in word learning: Golinkoff & Alioto (1995), proposed that word learning is facilitated by utterance-final positioning of the new word (see also Slobin, 1973); Echols (1993) has suggested that word segmentation problems may first be approached by a word extraction mechanism based on perceptually salient units, where stressed and certain acoustically distinctive final syllables might be extracted and identified as initial words. Perceptual predispositions, such as a tendency to attend to stressed and word-final syllables may assist in the identification of word-level units by making particular syllables especially salient to children (Echols & Newport, 1992). Due to their salience, certain syllables may have a higher tendency to be extracted from the speech stream and, consequently, form the basis for first words (Echols, 1993).

“Thus, children would identify words, at least initially, by extractingparticular salient syllables from the speech stream rather than by segmenting an entire sequence of speech into word-level units.” (Echols, 1993, p.247)

Another potential cue for word segmentation may occur through sentence final learning. Fernald and Mazzie report that nouns, particularly “new” nouns, are particularly likely to occur in final position in infant-directed speech. Units in sentence- or phrase-final position may be more easily broken off, retained and stored in a representation by the child than units in other positions (Echols, 1993). Saffran, Newport & Aslin (1996) set up an experiment to investigate the role of distributional cues, exposing adults to an artificial language consisting of six trisyllabic words, where all potential cues except transitional probability were removed. Subjects were instructed to figure out where the words in the

(7)

6

speech stream began and where they ended. The results of their first experiment showed that subjects chose part-words containing the first and second syllables of words significantly less often than they chose part-words containing the second and third syllables of words. These words containing word-end syllables were selected 19% more often, giving the impression that subjects were learning the ends of words first, in accordance with the theories laid out by Slobin (1973), which will be discussed in more detail below, and Echols (1993).

Slobin (1973) developed a set of operating principles that address the universals of language acquisition in children and noted as his first operating principle (A) that, across languages, children pay attention to the ends of words. Early acquisition of inflections is observed in languages where they are realized as noun suffixes, compared with late acquisition of inflections in languages where they are realized as prenominal articles. As a general development universal, he proposes that:

“Universal A1: For any given semantic notion, grammatical realizations in the form of suffixes or postpositions will be acquired earlier than realizations in the form of prefixes or prepositions.” (p.192) He also notes the existence of syllable lengthening in many languages and the idea that this final-syllable lengthening could provide a cue to listeners that a linguistic unit has terminated (as suggested by Ernest Haden, [1962]).

There are various other segmentation cues and factors that may influence how the speech stream is perceived and processed. Echols (1993), describes a strategy for word segmentation, where children break off linguistic chunks that are larger than individual words (these larger linguistic chunks are known as amalgams). These may consist of sequences of words, often a short phrase from the adult language, which are treated as a single unit by the child. Many cues have been proposed as the cause for such segmentation include stress, rhythmic pattern (Cutler & Noris, 1988), duration (Nakatani, O’Connor, & Aston, 1981), and consistent morpho-syntactic frames (Peters, 1983). When determining which cues may have aided in word segmentation, it is important to remember that not all cues are treated as equal by all individuals, and that individual differences, particularly in children, may determine cue preference. Some children will give more attention to stressed syllables, these tend to be children that are more capable of, or interested in, producing segments of speech correctly (Fee & Ingram, 1982). Whereas, there are other children that show more interest in replicating the rhythmic pattern of an adult word; these will often use ‘filler’ syllables in their speech (Echols, 1993). Another aspect that some children show a preference for is the prosodic component of speech; these children will attend less to the specific phonological material in a sequence (Peters, 1977). In many languages, properties such as stress pattern or tone pattern are far more important than in English, so it makes sense that children would be able to attend to aspects of language other than only stressed and final syllables (Echols, 1993); they should be prepared to attend to any variety of different language properties, depending on which of the world’s languages they are raised with.

Distributional cues have been tested in various statistical learning studies using artificial languages, but the results of such studies are not always entirely applicable to real-world language learning. Pelucchi, Hay, and Saffran (2009), set out to check the relevance of the conclusions of such studies for natural language by testing English-learning infants using fluent Italian speech. Their aim was to investigate whether infants were able to track transitional probabilities when faced with input that had the complexity of natural language. Infants were tested using the Head Turn Preference Procedure as adapted by Saffran et al. (1996) using 3 different sets of experiments, first testing discrimination between familiar and novel words from an Italian speech stream, then testing more specifically, to check if the infants were sensitive to syllable sequences (as opposed to full words), and finally, to determine whether infants tracked more subtle statistical regularities when listening to the Italian passages. According to their results, infants were successful in discriminating between words and non-words in all 3 cases, providing evidence that infants as young as 8 months old are already performing statistical calculations in order to segment words in unfamiliar natural language, and confirming that this learning strategy is not exclusive to artificial languages.

(8)

7 4. Online Measures of Statistical Learning

Many previous experiments have investigated statistical learning using offline measures, for example the study of Saffran, Newport & Aslin (1996) discussed previously, which tested subjects’ performance using a two-alternative forced choice test (for more examples, see Pelucci, Hay & Saffran, 2003; Misyak & Christiansen, 2012; Poulin-Charronnat, Perruchet, Tillmann & Peereman, 2016), but few have looked at this through online measures, i.e. while the learning is taking place. Measuring online learning allows us to focus on the trajectory of learning, as opposed to a participants post-learning ‘knowledge’, derived from the basic alternative forced choice tests usually employed for offline learning tasks. With the use of offline measures there is also the issue of accounting for additional cognitive processes that are unrelated to statistical learning, such as memory capacity and decision-making bias, two abilities that are put to the test every time the subject has to recall and decide which patterns occurred during the exposure phase (Siegelman, Bogaerts, Kronenfeld, & Frost, 2018). Online measures allow us to exclude such additional elements and to better understand the operationalization of the theoretical construct that is statistical learning (p. 693).

Typically, to measure a participants’ statistical learning abilities, they are tested in a lab environment and given a narrow set of tasks consisting of a relatively brief exposure phase to some kind of auditory or visual stimuli and then, once the learning process has ended, a learning assessment involving two-alternative-forced-choice questions. Where a participants’ mean performance is significantly above chance, learning is said to have taken place, whereas participants with mean scores at chance level or slightly below chance are interpreted as poor performers and the score is taken as an indicator that learning did not take place. This basic ‘expose and test’ strategy undoubtedly served to provide some valuable insight into the learning process in the early stages of statistical learning research, for example, if learning has occurred, it proves to be a useful tool for quantifying the overall extent of learning that has taken place, it is also useful for comparing the extent of statistical learning between different populations and between different learning conditions (Siegelman et al., 2018), with studies such as Poulin-Charronnat, Perruchet, Tillmann & Peereman’s (2016) using it to differentiate between performance of groups that had been familiarized with a language versus those that had not. However, with advanced developments in technology for psychological research, so too should come developments in test methods. One of these developments comes with the advancement to online testing. Statistical learning is taken to be a process of continuously assimilating the regularities of the environment, where behaviour changes incrementally over time (p. 695).

“From a theoretical perspective, knowing what statistical information is picked up at a given point in time and at what rate is an important step towards a mechanistic understanding of SL... If similar offline performance following familiarization is consistently achieved through different learning trajectories, then this must tell us something important about the mechanisms of learning statistical regularities… In the same vein, if two populations with similar success rates in an offline task have different learning trajectories building up to this overall performance, then these two populations should not be considered as having identical SL abilities.” (p. 696)

Siegelman et al. (2018) also note the importance of online measures to pick up on individual learning differences, as individuals can differ not only in their overall learning magnitude, but also in their speed of learning, and these two operational measures may hold distinct predictive power. Siegelman and his colleagues thus conducted three experiments to investigate the benefits of online testing as a measure of visual statistical learning. Their first experiment, designed in such a way that participants would advance through the stimuli at their own pace, provided various kinds of insight into the learning process. Firstly, that the group learning trajectory could be well described by a logarithmic function, with a steep curve at the onset of learning. Secondly, it revealed that significant learning had already taken place after relatively little exposure time. This type of information can be valuable to expose the level of difficulty of a language learning task, providing insight into how exposure times should be adjusted for more efficient testing in subsequent experiments. Thirdly, and most importantly, they

(9)

8

found that there was a strong correlation between online results (reaction times) and offline test results using this self-paced testing method. A repeat experiment performed on the same participants at a later date showed that reaction time performance for this type of testing was a stable characteristic of individuals, RT’s provided a consistent ‘signature’ of a given participant consistent across both experiments, also proving that this type of measure had high re-test reliability.

In their second and third experiments, they investigate more complex settings where multiple statistical structures have to be learned. They found that once structures in a sequence changed (as newly arranged stimuli were introduced half way through the familiarization phase), a period of re-learning was required, but that participants did eventually assimilate the new structural properties of the input (p.719). Their results showed that online and offline measures provided non-overlapping information regarding learning, with the online measure picking up on certain effects that the offline measure has no way of detecting.

A better understanding of which mechanisms are at play in statistical language learning would allow us to obtain a richer picture of the word segmentation process. Many studies show that the mechanisms responsible for linguistic processing are also responsible for non-linguistic tasks; for example, studies into how humans process facial activity (Tranel, Damasio and Damasio (1988) showed the same mechanisms were at play as when processing vocal activity (Van Lancker, Cummings, Kreiman, & Dobkin, 1988). Two studies investigating the areas of the brain that are responsible for online processing during statistical learning were carried out by Abla and Okanoya (2008), and Abla, Katahire & Okanoya (2008). In the former study, Abla and Okanoya (2008) measure changes in blood oxygenation using multichannel near-infrared spectroscopy while participants are exposed to auditory sequences containing tone-words to identify which regions of the brain are engaged during statistical segmentation of tone sequences. They familiarized participants with certain tone-words by exposing them to these stimuli in a training phase before they were exposed to the continuous auditory sequence. When they were presented with sequences of familiar tone words and random (non-familiar) tone words, they saw a large increase in the oxygenated haemoglobin in the left inferior frontal cortex during the familiar statistical sequence condition, but no change was observed during the random sequence condition, suggesting that the left inferior frontal cortex plays an important role in statistical segmentation of tone sequences where subjects deploy knowledge obtained in a training phase to analyse data in a subsequent continuous sequence. This part of the brain has also been linked to musical sequence processing (Koelsch, Fritz, Schulze, Alsop, & Schlaug, 2005), working memory for linguistic information (Friederici, 2002) and musical information (Zatorre, Evans, & Meyer, 1994), abstract rule learning (Opitz & Friederici, 2003), statistical learning (McNealy, Mazziotta, & Dapretto, 2006), and sequential learning (Conway & Christiansen, 2001) suggesting that language and sequential learning overlap not just in the processing of sequential structures, but also in neural mechanisms and suggests that the inferior frontal cortex may provide a common neural source for the learning and processing of both linguistic and non-linguistic sequential structures (Abla and Okanoya (2008), p. 2792). Upon understanding the mechanisms shared for such processes, we might gain insight into which cues to use depending on a subjects’ strengths and abilities in other areas. For example, auditory statistical cues and prosodic cues, may be preferred by subjects with high musical sequence processing skills (Vasuki Sharma, Demuth, & Arciuli (2016) found a correlation between musical skills and auditory statistical learning, but not visual statistical learning), while those with low abilities in this area may favour other learning strategies, such as learning of words in isolation or through utterance-final positioning.

In order to further examine the neural processes responsible for statistical language learning (and whether these are also responsible for non-linguistic statistical learning tasks), Abla, Katahire and Okanoya carried out a study with scalp-recorded event-related potentials (ERPs) used as a direct, online measurement of brain activity. Previous ERP studies had shown that the N400 component reflected an aspect of learning nonsense words, but they did not establish whether this component reflected the process of learning or the results of learning (p. 953). Another area that remained to be investigated was whether the N400 reflected processing during non-linguistic segmentation tasks. In their study, Abla et al. (2008) exposed participants to a stream of tones instead of syllables. The statistical structure of the tone-words used was identical to that of the words used by Saffran et al. (1999). For their analysis, they divided learners into three groups, high, middle, and low learners, depending on their mean behavioural performance and standard deviation. They found that in high learners, the N400 amplitude was larger in the early learning session; in middle learners they saw this effect happen in later sessions; while in

(10)

9

low learners, they saw no effect. They found that N400 amplitude was correlated with transitional probabilities early in the session for high learners and later in the session for middle learners. The results of this study suggests that learners readily group sequences of tones in the same way they group words from artificial grammars.

A study by Turke-Browne, Junge, and Scholl (2005) suggests that these same mechanisms are also responsible for visual statistical learning, providing further evidence for the existence of a domain-general statistical learning device, as opposed to an innate language learning faculty. The study of Turke-Browne et al. also investigates the automaticity of visual statistical learning. Previous research had described statistical learning as a low-level implicit process, thought to occur “automatically” or as a by-product of mere exposure. However, their study reveals that statistical learning, at least in non-linguistic cases using visual objects, occurs for only some information in our local environment, and that attention plays a significant role in terms of how this information is selected. This necessity of attention for such learning seems less surprising when we consider the abundance of stimuli in our natural environment, and the fact that any statistical calculations must operate over some specified population. We are perceptually confronted with too much sensory input, so it seems unlikely that visual statistical learning would operate over every possible stimulus we encounter (p. 563). It seems more likely that such calculations would operate over populations that are important to us, and for this, calculations would not occur in an automatic, data-driven fashion, but in more of a selective application of attention. However, their study also provides evidence that some aspects of statistical learning do occur in an implicit manner; certain statistical operations appeared to be taking place despite the fact that observers had no intent to extract the hidden structure (they were engaged in a competing task at the time), and despite the fact that this processing did not result in subjects becoming consciously aware of the structures that were being learned (p. 563). Their findings raise questions about which features are and are not encoded in visual statistical learning (in other words, what aspects are being learned). ‘Such selection is important for determining the contents of our conscious experience, but also for determining the stimuli over which other lower-level processes will operate’ (p. 563). A deeper understanding of what controls this ‘selectiveness’ for learning of certain features, may be applied to other areas of statistical learning, potentially leading to a better understanding of the mechanisms governing statistical language learning.

One case where a deeper understanding of these mechanisms could prove useful for language acquisition would be in the ability to trigger different mechanisms as needed, for example Pena, Bonatti, Nespor, & Mehler (2002), and Perruchet, Tyler, Galland, & Peereman (2004) have gathered evidence to suggest that statistical computations in pseudo-speech are in some cases triggered by a lack of acoustic segmentation cues. By gaining a deeper understanding of the ‘hierarchy’ of language cues, we might better structure language teaching methods to engage specific learning mechanisms through deprivation of certain stimuli. An example of how this could be useful is in the modification of language teaching methods for children with William Syndrome, who have shown a stronger reliance on prosodic cues than their TP peers (Nazzi & Ramus, 2003). This deeper understanding of cue hierarchies would allow us to ‘filter out’ certain less relevant cues, allowing for a more bespoke form of learning, focusing on the cues that are not only most beneficial for the aspects of language being learned, but also making better use of those cues that are more easily processed by learners with specific language impairments.

5. Click detection – Online Measure or Interference?

A more recent study that set out to investigate statistical learning using online measures is that of Gomez, Bion & Mehler (2010); they used a click detection task to measure the segmentation process in adults. The click detection task works by recording a participants’ reaction times to a large number of stimuli and is based on the statistical learning theory that predictable elements should induce faster responses than unpredictable elements.

“If, while a series of reaction times is being taken, the stimulus is thrown in when the person is not expecting it, one naturally finds a marked prolongation of the reaction time, or the response may be omitted altogether.” (Milroy, 1909).

(11)

10

This effect has been documented in various implicit learning studies, including investigations into the serial reaction time task by Cleeremans & McClelland (1991) and Schvaneveldt & Gomez, (1998). During Gomez et al.’s (2010) study, they had their subjects listen to an artificial speech stream consisting of trisyllabic nonsense words for four minutes, this speech stream had ‘clicks’ superimposed on it at different intervals and they could be presented either within or between consecutive words. Subjects were asked to respond each time they heard a click and the hypothesis was that with more exposure time subjects would begin learning words and reacting faster to clicks placed between words, because the clicks that were placed within words would be less easy to predict. After two minutes of exposure to the speech stream, participants were slower to detect clicks located within words than clicks located between words. These results would suggest that the click detection task was sensitive to the online statistical computations taking place. In the past, adults were generally given more exposure to the speech stream than infants (with adults receiving in the realm of 10 minutes exposure, while infants were given as little as 2 minutes), but the results obtained from the 4-minute task used in Gomez et al.’s study would suggest that adults do not require more exposure than infants for online statistical learning mechanisms to become activated.

When considering the reliability of click detection as a measure of statistical learning, there is a second click detection study by Franco, Gaillard, Cleeremans and Destrebecqz’s (2014) that must be taken into account. Franco et al. (2014) carried out an experiment similar to Gomez et al. (2010), adding a two-alternative forced choice (2AFC) test at the end, to asses learning offline, following exposure to the speech stream. In testing the hypothesis that, over the length of exposure, reaction times would increase more for within-word than between-word clicks, they also wanted to examine the potentially distracting effect of click detection during the word segmentation task. They proposed that, even though statistical learning occurs without an intention to learn, it nevertheless requires attentional resources. Therefore, it could be affected by a concurrent task such as click detection (Franco et al. 2014). They note that there is evidence that dividing attention during statistical learning leads to poor performance in offline tests (Turk-Browne et al., 2005) and even more so when the stream of input in the statistical learning task and the concurrent task share the same modality (Toro et al., 2005). The first experiment yielded results showing no reliable association between reaction times to clicks and forced choice performance. Misyak, Christiansen, & Tomblim (2010b) conducted a similar study using click detection where participants were exposed to an artificial language sequence, and were simultaneously asked to click on the corresponding written words presented on screen. The reaction times showed that words in predictable locations were recognized faster than those in unpredictable locations. However, these reaction times did not correlate with the offline results either. This leads to us to question whether an increased response to stimuli in predictable locations really reflects stable learning. There is very little evidence to prove that this is actually the case, and Misyak, Christiansen & Tomblin (2010a), suggest that this may be due to the fact that online and offline measures tap different sub-components of statistical learning.

Returning to the inconclusive results of Franco et al.’s study (2014), they raise the question of whether the click detection task might negatively influence correct word extraction due to the attention it requires, and, as a consequence, result in poor performance in the 2AFC test. It also led them to question how RT’s might be affected in this case. To answer these questions, they performed a second experiment, containing two different types of speech streams. One in which participants were exposed to the same speech stream as in experiment 1 -it was still superimposed with clicks, but participants were not instructed to respond to the clicks- and a second, in which they were exposed to the same speech stream without the clicks. Mean performance for these tasks was significantly above chance at 68.52%, with 72.87% mean performance for no clicks and 64.17% mean performance for passive clicks. This result suggested that even just the presence of clicks impaired performance in the 2AFC task. They suggested that this could be due to the position of the clicks affecting segmentation, as subjects may have used them as benchmarks to chunk the speech stream, in which case clicks placed between words would lead to successful segmentation of the artificial language, while clicks placed within words would

(12)

11

lead them to assume incorrect boundaries, thus resulting in poor performance in the 2AFC task (p. 1398). To alleviate this issue, they performed a third experiment, where they compared one condition in which all clicks were placed between words, and another in which all clicks were placed within words. Results for this last experiment showed the mean reaction times (RTs) were faster in the Between condition (338 ms) than in the Within condition (371 ms), but only slightly. Mean performance in the 2AFC task was above chance (60.60%). Using the Bayes factor Analysis (BF), which resulted in a BF of 0.32, they were able to exclude the hypothesis that clicks placed between words constituted an aid to speech segmentation. The results of their third experiment also showed that the location of the clicks had no impact on performance in the 2AFC task. Performance was, in fact, lower than in the no-click condition in experiment 2. Furthermore, the RTs in their first experiment were significantly faster than those in the within condition of experiment 3, so there is clearly a need for further tests to be performed in this area before any strong conclusions can be drawn. When considering the implicit nature of the study, it could be interesting to note the difference between asking participants to try to identify words in the speech stream, as was done in Franco et al. (2014) or telling them to simply listen to the speech stream, as will be done in our study. “Indeed, whereas our results clearly show a negative impact of the click detection task on the offline measure, it is not clear whether the presence of the clicks within the speech stream or the additional detection instructions is what provides an additional impairment.” (Franco, et al. (2014), p. 1401)

Franco et al. also suggest that when participants were asked to ignore clicks it is possible that this created an attention suppression situation that could affect learning, though this seemed not to be the case when performance was similar in the 2AFC tasks for both conditions. An aspect of the results that was consistent for both Franco et al. and Gomez et al.’s studies was a gradual increase in reaction times. What can explain the progressive global increase of RTs? They exclude fatigue effects as the task is less than 5 minutes. One interpretation was that the emergence of word candidates might interfere with the click detection task, however Franco et al.’s experiment would appear to disprove that; thus, the overall increase in RTs remains an issue for further investigation.

In finding that their results contrasted with those of Gomez et al. (2010), Franco, et al. (2014) propose that the click detection task may impair statistical learning and that it should be used as a tool for online statistical learning only if its detrimental impact can be fully measured. However, they also note that the discrepancy between their results and those of Gomez et al. could be due to a smaller proportion of participants successfully extracting the words in their study, but that neither of the studies can prove this point, hence a third study is due.

6. Current Study

As discussed in the above studies, the mechanisms responsible for online processing differ from those used later in the offline test phase (e.g. Abla & Okanoya, 2009), and evidence of learning taking place has been found several minutes before performance in offline tests can be assessed (e.g. Gomez, et al., 2010). At the end of the familiarization stage, subjects will have formed stable representations of the patterns they encountered. Offline tests target such representations, but unlike online testing, they are blind to the dynamic formation of the extracted patterns. Online measures provide insight into how exactly representations update online given exposure to a continuous sensory input (Siegelman et al., 2018). We will use an online measure of learning in our experiments to gain better insight into the learning process as a whole, and, in order to further investigate the click detection task’s efficiency as a measure of on-line statistical learning, we will also include an offline task following exposure to the speech stream. For this we will use the Two Alternative Forced Choice (2AFC) Test, as was used by Franco, Gaillard, Cleeremans and Destrebecqz’s (2014), which will allow us to check for correlations between online and offline results, providing further evidence to help validate (or invalidate) the click detection method as a measure of statistical learning.

Similar to what Gomez et al. (2010) proposed in their study, our hypotheses are based around the assumption that reaction times to clicks located between consecutive words and within words would become increasingly different (one mode showing increasing RT’s and the other showing decreasing

(13)

12

RT’s) as statistical computations are carried out on the speech stream. This pattern should occur as subjects form predictions over time. The exposure time for participants will be double that of Gomez et al.’s study, with 8 minutes of speech stream presented, which we hope will provide some stronger correlations/trends between participants in the later minutes of the study. The study will be carried out on Dutch native speakers, using Dutch sounding pseudo-words, making it, to the best of our knowledge, the first study to test statistical learning in this language. Germanic languages typically display consistent stress patterns that can facilitate word learning, however, in our study, using artificial bi-syllabic words, the syllables have been positioned in such a way that stress is evenly distributed between the first and second syllables, thus eliminating any stress-related cues that may facilitate word segmentation.

Given that we are asking all participants to fill in an ‘exit-interview’ questionnaire, we will examine some of the individual differences which occur between subjects and the effect these may have had on our results. Though variables that we have not included for testing, such as verbal working memory, short term memory, vocabulary, cognitive motivation, and fluid intelligence tend to be some of the more insightful aspects into a subject’s performance (see Misyak and Christiansen [2012], who’s findings suggest strong inter-relationships between verbal working memory, language comprehension, and statistical learning), it may still be interesting to see if other variables such as gender and age, show any significant effects. Woods, Wyma, Yund, Herron and Reed (2015) found a significant correlation between age and SRT latencies in both their studies, “SRTs increased with age at a rate of ∼0.5 ms/year” (Wyma, Yund, Herron, and Reed, 2015, p. 10). However, they note that the increasing reaction times level off and remain relatively stable across adulthood, suggesting that the age-related slowing of SRTs primarily reflected slowed motor output.

Our main research goal is to establish the efficacy (or lack thereof) of click detection as a measure of statistical learning. This will be broken down into a series of smaller investigation, where we set out to establish (1) whether subjects show evidence of having learned (or segmented) words correctly through the 2AFC (offline) task; (2) if test version had any effect on performance; (3) whether reaction time evolution changes depending on mode as hypothesised, with reaction times going down over time for clicks between words and up over time for clicks within words; (4) we will check if there is a global increase in overall reaction times after 5 minutes of exposure, as was seen in Gomez, Bion, & Mehler’s (2011) and Franco, Gaillard, Cleeremans & Destrebecqz’s (2014) studies; (5) finally, we will check if task order had an effect on reaction times, to ensure that fatigue/mental strain did not significantly affect the results.

7. Methodology

7.1. Participants

A total of 26 native Dutch speakers (9 male, 17 female), between the ages of 18 and 35 (mean age of 28 years) participated, the original number of participants was 28 but two were removed from the study due to equipment malfunctions during testing. None had any background in linguistics and none reported any language or hearing impairments. They were tested in sound laboratories at the University of Amsterdam and were paid for their participation.

7.2. Stimuli

Two artificial speech streams were generated by recording a female speaker with a sampling frequency of 250 Hz. Syllables were recorded in isolation and then connected using Praat software (Boersma, 2001). Each of the speech streams contained four bi-syllabic nonsense words. Speech stream A contained the pseudo-words kiba, moti, dalu, gido; stream B contained the pseudo-words bamo, tida, lugi, doki. Participants were randomly assigned to one of the two streams. Each of the syllables lasted 278 ms (each consonant 118 ms, each vowel 160 ms), the syllable rate was 216 p/minute and there were no pauses between consecutive words. Each word was presented 216 times in a randomized order, with no word ever occurring twice in succession. The total exposure time was 8 minutes and 30 seconds, this consisted of a 30 second exposure phase first, followed by an 8-minute test phase.

(14)

13

A set of clicks were inserted into each speech stream using the Praat software (Boersma, 2001). Clicks could occur either between words or within words. The clicks were dispersed differently throughout the different ‘blocks’ of the speech stream. The initial 30 seconds (Block 0) is a training block, containing just 5-6 clicks. It functions as a way of getting participants accustomed to the click detection task. Block 1 is of 2 minutes duration and contains 10 sparsely distributed clicks. Block 2 (the middle 4 minutes of the speech stream) contains 72 clicks. Block 3, the final 2 minutes of exposure, contains 10 clicks. A Praat script was created to add the click sounds to the speech stream. These were inserted at different, random intervals for every participant. In order to distribute these evenly, the total number of syllables within each of the three blocks was divided by the number of syllables that should get a click sound. For example, block 2 contains 864 syllables, of which 72 syllables should be followed by a click. This means that on average, a click sound should occur every 12 syllables. So, within this block, 72 chunks of 12 syllables were created. In each of these 12 syllable chunks, a random syllable was chosen to be followed by a click. To ensure that clicks never occurred too close together, we limited the placement of clicks to syllables 3-10 within the 12-syllable chunks. This way there were never less than 4 syllables between each click. Immediately after the 8-minute exposure phase, participants were given a 2AFC task in which they were presented with two bi-syllabic words, one from the artificial language and one non-word (made up of the same syllables but with null transitional probability between them), for each trial. Participants were instructed to select the word that sounded like a word they had encountered in the speech stream. Each word was paired with each non-word once (every word and every non-word occurred 4 times) resulting in a total of 16 trials.

The experiment was run on a laptop to which an Edirol response box was attached, converting every button-press into a 10 ms sound pulse. An external recording device was also attached. The recordings were later divided into 2 sound tracks to facilitate analysis, the first track containing the speech stream that the participant was exposed to and the second track containing the sound pulses that were generated via the response box.

The final stage of the procedure involved a post-experiment questionnaire used to gauge feedback that would extend on the results of the offline trials. There were four versions of this questionnaire (see Appendices), depending on the order by which tasks were performed. As this ‘exit interview’ was always performed last, regardless of task order, it meant that 50% of the time, when it followed the word-referent mapping task, a version of the questionnaire was used in which most questions were tailored for that experiment making it less relevant to the current study. In any case, it allowed us to gather basic personal information that could help us to decipher the cause of any unusual variations in the results that might be attributed to individual differences such as age, sex, or level of education.

7.3. Procedure

Subjects participated in two separate experiments, the word segmentation task discussed in this paper and a second, word-referent mapping task (which is outside of the scope of this study). These

experiments were performed alongside a PhD student 1_{, who assisted throughout the different testing}

phases. For half of the participants, word-segmentation was tested first and word-referent mapping tasks second. This order was reversed for the other half of the participants to control for effects of fatigue. Subjects were tested individually in a quiet university lab and listened to the material through headphones. Each subject was instructed to listen to an alien language and to press the button-box mouse as fast as possible anytime they heard a click sound. The data collected from this exercise was extracted using E-Prime software (Schneider, Eschman, & Zuccolotto, 2002) and converted into an Excel document which could then be analysed using R (R and RStudio version 1.1.453). The final phase of the experiment was an ‘exit interview’ questionnaire, during which a series of questions were answered investigating whether participants felt that more implicit or explicit learning had taken place during the trials. The duration of each experiment ranged from 35-40 minutes. Participants were tested over the course of 10 weeks.

1._{This experiment, which formed part of a larger PhD study, was carried out with the help and supervision of a PhD student}

(15)

14 7.4. Statistical Analysis

After the data was extracted using E-prime, the next stage was pre-processing. Missed clicks and extra responses were removed from the analysis; these accounted for 1.64% of the data, consisting of 25 extra clicks and 13 missed clicks. The data was then logarithmically rescaled for statistical analysis. All analyses were carried out using R software.

8. Results

Figure 1. Mean accuracy score for each participant in the two-alternative forced choice test.

8.1. Offline Results of the Word Segmentation Task

A number of different tests were carried out on the data. One first aim was to establish whether word segmentation had taken place. The data containing the results of the offline tests were analysed using a One Sample t-test to calculate the overall performance of the group. Their results were evaluated against a performance of 50%. The mean accuracy score for the groups taking test version A and test version B combined was 0.44. Only 38% of the group scored more than 0.5 (above chance) in terms of accuracy, so there is no significant evidence to show that word learning took place, t(25) = -1.73, p = 0.95. The results of the 2AFC test are presented in figure 1.

Our next test was to compare performance for the two groups taking different test versions. For this, a Welch’s independent samples t-test was used. Participants that took Version A had a lower mean accu racy (M = 0.38, SD = 0.19) than those that took test Version B (M = 0.5, SD = 0.14), as can be seen in figure 2. This effect was not significant, t(20.67) = -1.78, p = 0.09.

(16)

15

Figure 2. Mean accuracy scores of each participant for Version A and Version B in the offline test.

Looking at figure 3, we see a difference in performance depending on whether participants took the word segmentation task first or second. Mean accuracy for those taking the word segmentation task first (WS1) was 0.51 (SD = 0.18), whereas those taking the task second (WS2) show a lower mean accuracy of 0.38 (SD = 0.14). The order in which tasks were taken had a significant effect on performance, F(1, 22) = 4.562, p = 0.04. The effect of the interaction between task order and version was not significant, F(1, 22) = 0.285, p = 0.59.

Figure 3. Mean accuracy scores for each participant taking word segmentation as the first task, vs. scores for those taking it as the second task in the offline test.

(17)

16

One more aspect of the offline results that stood out was that certain words were learned better than others. The main effect of target word on accuracy was significant, F(7, 22) = 2.195, p = 0.005. A post-hoc t-test was performed comparing the accuracy for each of the words. Subjects performed best in learning the words tida (.59), doki (.57), and dalu (.50), while accuracy for learning the word kiba was just .29.

8.2. Online Results of Click Detection task

The online data was analysed using a repeated measure analysis of variance (ANOVA). We computed the mean reaction times for each minute of exposure, for the different versions of the speech stream, for the mode (clicks placed between or within words), and for the order in which the task was carried out (whether this was before or after the word-referent mapping task). All graphics for the online data show the values converted back to normal scale.

Mode

The main effect of mode, that is, whether clicks were placed between words or within words, was not significant overall, F(1, 22) = 0.319, p = 0.57. We will now investigate how this condition interacted with the remaining conditions, i.e. how reaction times to different modes was affected by different versions of the tests, the effect of different exposure times on mode, and how administering the tasks in different orders may have had an effect on this condition.

Version

The evolution of reaction times to clicks between and within words for the different test versions can b e seen in figure 6. The pooled mean reaction time (RT) for version A and version B for clicks between words was 284.48 ms (SD = 91.48). For clicks within words, the mean RT for the two versions was a v ery similar 286.65 ms, (SD = 95.47). In order to test the effect of version on mean reaction time for bo th modes combined (i.e. without taking mode into account), we ran an ANOVA. The effect of version here was highly significant, F(1, 22) = 10.947, p = 0.00095, however, once this was analysed with res pect to each mode, the effect was not significant for either version, F(1, 25) = -0.572, p = 0.55. For ver sion A we saw slightly faster RT’s for clicks between words (M = 287.77 ms, SD = 96.05) than within words (M = 296.92 ms, SD = 102.17), but we saw the opposite trend for version B, with a mean reacti on time of 281.61 ms (SD = 87.28) for clicks between words and mean reaction time of 277.61 (SD = 88.27) for clicks within words.

Time (minutes)

Our next analysis was to test the effect of time in minutes on reaction times to clicks placed in the different modes (conditions). For the effect of time on mode, we did not see a significant difference, t(25) = -0.689, p = 0.478. We did, however, find a significant effect of time on reaction times overall, with a mean RT of 272.76 (SD = 63.25) for the first minute of the test, and a much slower mean RT of 307.99 (SD = 73.17) by the end of the test. t(25) = 1.969, p = 0.037. The normalized data is displayed in figure 4, below. As we can see from this graph, reaction times in the first minute for both clicks between and within words are relatively fast, likely due to the high alertness of participants during first exposure to the stimuli. In the second minute we see a slower mean reaction time, but this is where clicks were introduced with much higher frequency, and the trend that follows this second minute shows a gradual and consistent decline in reaction times for clicks between words until the fifth minute. The reaction time for clicks within words is less consistent, but also reaches its lowest point at the fifth minute. After the sixth minute, as clicks become widely dispersed, we see reaction times rise sharply.

(18)

17

Figure 4. Mean Reaction Times (RT’s) to clicks within words and clicks between words per minute in the online test.

Version x Time

The effect of interaction between time and version was F(7, 22) = 1.474, p = 0.17, which was not statistically significant. When we break this up to look at version A and B separately, we can see in figure 5, that once again the initial minute of exposure delivers a fast reaction time for both clicks between and clicks within words. In fact, the initial minute shows the lowest reaction time for clicks between words in version A, which goes quite against our hypothesis.

Figure 5. Mean Reaction Times (RT’s) to clicks within words and clicks between words per minute for version A and version B during online testing.

(19)

18

Task order

The last analysis carried out was to check how task order affected performance in this task. The combined mean RT for clicks between and within words for participants taking the word segmentation task first was 285.26 ms (SD = 90.5) and for those taking it second, the mean RT was 285.89 ms (SD = 96.13). These almost identical mean RT’s would suggest that task order makes no significant difference to reaction time, a paired samples t-test showed that this was indeed the case, no significant effect was found, t(25) = 0.642, p = 0.66. However, when we look at the modes separately, we can see some stronger differences between learning patterns/trends for participants taking the task first, and those taking it second.

Figure 6. Mean Reaction Times (RT’s) to clicks within words and clicks between words depending on task order.

If we look at figure 6, for the group who took the word segmentation task first (WS1), we can see the reaction times for clicks between words lessen after the second minute up until the sixth minute (M = 279.82, SD = 84.34). This trend sits in agreement with our hypothesis, however the RT’s for clicks within words for this group follows a similar pattern (M = 290.7, SD = 96.68).

For the participants taking this task after the word-referent mapping task (WS2), the RT’s begin at a similar rate with the mean for clicks between words gradually reducing after the second minute, but it is in the RT for clicks within words that we see a strong drop in reaction time after five minutes of exposure. The mean reaction time for clicks between words was 289.26 ms (SD = 98.12) whereas for clicks within words it was a faster 282.52 ms (SD = 94.13).The interaction effect between mode and task order was significant, F(1, 22) = 5.240, p = 0.02.

Exit interview

Finally, the questionnaire yielded some interesting responses, reflecting the level of automaticity of statistical learning present in these experiments. Of the 12 participants taking the relevant questionnaire, 10 said that their main point of focus during exposure to the speech stream was on the clicks. One participant said that they were focused on the clicks, but also on deciphering the pattern, and one other said that they were most focused on words occurring in the speech stream. In a subsequent question, they were asked directly if they noticed any pattern. Five participants said they did not notice any pattern, four noticed a pattern during the second half of exposure time and three participants said they noticed patterns almost from the start. When asked if they had noticed any words, nine participants said that they had. Two said they had not noticed any words, and the final participant said that they had

An Investigation into Statistical Language Learning Using Click Detection

MA Linguistics

Language & Society