Recalibration of human colour perception: what why and
how adjustments are made
Literature Thesis Alex Swartz 11616288
Supervisor: Jenny Bosten, Senior Lecturer, University of Sussex
Coassessor: Steven Scholte, Associate Professor, University of Amsterdam MSc in Brain and Cognitive Sciences, University of Amsterdam
1
Abstract
The input to the colour perception system of an individual varies over multiple timescales, from milliseconds to decades. Recalibration in response to perturbations in input maintains useful responses to colour stimuli. This review focusses on what why and how colour perception recalibrations are made within individual lifetimes. Evidence is found for recalibration to global perturbations of seasonal fluctuations, age-related thickening of the eye lens, and simulated visual environment differences as well as feature-selective
recalibrations in response to changes in language use in addition to colour-pattern and action-colour contingencies. Two major biological functions are shown to be realised by these recalibrations of maintaining both a consistent and an efficient input-response mapping. I conclude that a better understanding of recalibration will come from future investigations that consider which aspects of recalibration take precedence when conflicts arise between the different functions, mechanisms or features that recalibration is driven by, or contingent on.
Introduction
Colour processing depends greatly on the individual and their environment. Input to the colour perception system of a given observer changes over multiple timescales, ranging from milliseconds to decades. By adjusting to perturbations in input through recalibration, the colour perception system can maintain useful responses to relevant stimuli, avoiding being perturbed by task-irrelevant changes in input. The mechanisms behind these recalibrations may be common throughout the visual and other perceptual systems and understanding them would therefore have far-reaching implications for the study of sensation and perception.
In the external environment, colour information varies between locations with stark contrasts between arid desert and lush jungle ecosystems and within locations through daylight and seasonal cycles (Webster, Mizokami & Webster, 2007). Individual internal environments are also dynamic. As an individual ages, the lens over the eye thickens progressively, altering the wavelengths of light that pass through to the retina (Pokorny, Smith & Lutze, 1987). Further, linguistic processing may affect colour perception. While the extent to which language affects cognition is a contentious ongoing debate (Lucy, 2016), if there exists convincing evidence that language plays a role in the perceptual processing of colour, then perturbations in linguistic input, particularly in learning and using a new language may provide a drive for the recalibration of colour perception.
2
Interactions between the observer and their external environments may also provide input to the colour perception system as sensorimotor theories of perception posit an intimate relation between actions and perception (O’Regan & Noë, 2001). Perturbations in the relation between eye movements and colour input also may provide a drive for recalibration (Bompas & O’Regan, 2006a).
Several approaches have been developed to study what recalibrations are made by the colour perception system. The approaches range from highly reductive settings in which recalibration is studied in visual aftereffects or through the use of computational models, to viewing natural environments through coloured lenses for extended periods and to further naturalistic settings in which seasonal variations are measured experimentally in otherwise freely behaving individuals.
The range of approaches applied to this question has led to the development of theories of the biological functions of colour perception recalibration. Such functional theories propose the maintenance of a consistent input-response mapping (Clifford et al., 2007; Werner, 2014), and efficiently producing internal representations of the external world (Wark, Lundstrom and Fairhall, 2007). These two functions have been theorised to be achieved through proposed candidate recalibration methods of associative learning (Murch, 1976), adjusting the responsiveness of single cells in colour processing (Faruq, McOwan & Chittka, 2013), or the action of an error-correction device consisting of a neural circuit with a flexible transform function that is applied to early stages of colour perceptual processing (Dodwell & Humphrey, 1990).
Considering any change in input to the colour perception system that causes a systematic change in function lasting more than a few minutes as recalibration, the present paper reviews what how and why recalibrations in the colour perception system are made. I first review what perturbations in visual input have been shown to result in a recalibration of colour perception. The outcome shows that both global changes in input following seasonal cycles or age-related thickening of the lens and feature-contingent changes such as those linked to patterns, eye movements or language, all effect colour perception recalibration. Second, I review proposed theories of the biological functions of these recalibrations showing that these recalibrations can serve the purpose of maintaining a consistent mapping between the internal and external environments in an efficient way. Thirdly, I consider the
3
recalibrative mechanisms at both the levels of single cells and of neural circuits. Synthesising these three aspects of recalibration, I conclude by suggesting that recalibration needs to be considered in more complex environments and that mechanisms achieving the functions of recalibrations need to be considered together with attention paid to conflicts between each of what why and how recalibrations are made.
What Recalibrations Can be Made?
Recalibration of the colour perception system has been investigated from several different angles. One way has been to study recalibration in a reductive approach through studying colour aftereffects or using filtered lenses to control the type of information available. Other studies follow a more natural approach taking advantage of age-related differences or seasonal variations with longitudinal designs. The scope of recalibration is critical for understanding colour perception in natural contexts in which multiple sources may pull recalibration processes in different directions. In this section, I review evidence for colour perception adaptation and the methods they provide that can inform our evaluation of the biological function and mechanisms in the subsequent sections.
Changes in the External Environment
In order to study the recalibration of colour perception in response to aspects of the visual environment that greatly vary over time and space (see figure 1) some studies have taken an approach of using filters through the application of contact lenses, glasses or by placing filters over light sources. These filters alter the wavelengths of light that reach the eye, simulating differences in illuminations of different visual environments.
Eisner and Enoch (1982) conducted a study in which subjects wore a long wavelength pass filter over one eye for a week, resulting in subjects only receiving light in typical red and orange ranges. They found that their subjects ended up with reduced sensitivity, as
demonstrated by an increase in radiance needed in order for input to be detected, to the long-wavelength light after wearing lenses for a week. Another study conducted a series of experiments wherein subjects viewed the world through either red or green coloured lenses, either binocularly or monocularly or spent time in a room in which light sources were covered with filters (Neitz, Carroll, Yamauchi, Neitz & Williams, 2002). They found that wearing the red lenses for a prolonged period leads to the world appearing greener and the green lenses make the world appear more red upon removal.
4
Both of the above studies also tracked any changes in the composition of light that produce the four unique hues of red, green, yellow and blue that are uniquely pure
perceptions of colours in that they are not experienced as a mixture of other colours (for a review, see Kuehni, 2014). They both found that unique yellow shifted to longer wavelengths with red lenses. Additionally, Neitz and colleagues (2002) found that the unique yellow hue shifted to shorter wavelengths when green lenses were used and that these effects took several days of exposure to stabilise and the effects of this recalibration continued for at least hours after returning to a normal chromatic environment.
These studies convincingly show an ability of the colour perception system to recalibrate according to global changes in the colour input from the external world. The results could be interpreted as showing adjustment for changes in an ecosystem when long-term changes in the surrounding environment lead to an average change in red or green due to seasons or following migration. However, such generalisations to natural settings must be very tentative due to issues with completely eliminating the fluctuations in colour visual information and the other colours from vision not being representative of real environmental variations. Webster and colleagues (2007) argue that there should be different recalibrations specific to the parts of the retina that the light falls on because in real natural scenes, the top of the retina is more exposed to sky and the lower part to earth, which could implicate different adaptive states for different areas, the studies described above leave this question unaddressed. Such region-specific recalibration has been shown in pattern-contingent recalibrations (see below). More naturalistic approaches are required to allow generalisation beyond the highly controlled conditions in the two studies described above.
By using a longitudinal design and investigating colour perception over seasons, some studies have shown recalibration in more natural contexts. The first report of a seasonal effect on colour perception, by Richter (1948 as cited in Jordan & Mollon (1993)), was criticised for inadequate controlling of lab temperature which could have affected the equipment and accounted for their findings of a seasonal variation in colour perception. Jordan and Mollon (1993) were able to replicate the original findings but accounted for them by the equipment used being sensitive to ambient temperature. This was then followed up by Welbourne, Morland and Wade, (2015) who also measured seasonal changes in colour perception and controlled for lab temperatures using more sophisticated equipment. They found that only the unique yellow hue shifted over seasons, consistent with findings using lenses as described above, however, the effect was much smaller in that the wavelengths were moved to a lesser
5
extent which demonstrates the problem with using extreme methods involving the total removal of certain inputs through the application of long- or short-wavelength pass filters.
Figure 1. Differences in visual information between ecosystems and within ecosystems over different seasons. The upper images show a region in the Western Ghats in India during the monsoon season (left) and winter season (right). The lower pair shows Dog Valley in the Sierra mountain range in the United States, in spring (left) and autumn (right). Image adapted from Webster and colleagues, 2007.
Human environments are much more complex than can be captured by seasonal variations and or coloured lenses. Kompaniez, Abbey, Boone and Webster (2013) give the example of a radiographer who in their work looks for prolonged periods at medical images that have their own characteristic statistics and environmental differences such as this cause their own long-term perturbations to visual input and may provide further opportunities to study the recalibration of colour perception.
These studies therefore show that the colour perception system can recalibrate in response to changes in the overall environment but that in future research, care should be taken to ensure that recalibrations studied are representative of those that occur in real situations.
6
Aging Related Recalibration
Through typical aging, the lens of the human eye becomes more dense and opaque (Pokorny et al., 1987). This process leads to a steady increase of light absorption, particularly affecting the shorter wavelengths of light that reach the retina and therefore all later stages of colour perception (Weale, 1988). Despite this gradual perturbation in visual information reaching the retina, the physical characteristics of light evoking a given colour or that result in achromatic judgements are not associated with age (Schefrin & Werner, 1990, Werner & Steele, 1988, Werner & Schefrin, 1993, Wuerger, 2013). This dissociation itself is indicative of compensatory recalibration of the colour perception system. Two studies have developed independent methods of studying this long-term recalibration using coloured lenses or the recovery from surgical correction for extreme aging in the development of cataracts.
The first study to investigate recalibration in response to a progressive increase in lens density investigated colour perception at different timepoints following surgery to remove cataracts (Delahunt, Webster, Ma & Werner, 2004). Participants in this study were asked to make judgements of light settings that were achromatic. In their four elderly subjects, prior to surgery, the achromatic settings were in ranges that would be blue for typical observers. Immediately after surgery, achromatic judgements fell into wavelengths that would typically be yellow. They continued measurements for several months and found that after about three months, the achromatic settings settled at almost typical white points. These results therefore show recalibration to the extra short-wavelength light received following the removal of cataracts. Since the settings prior to surgery were slightly in the blue range, this suggests that compensation was not perfect, similarly for the returning to only almost within typical ranges for achromacy. This could be taken to show a limit of what recalibration processes can achieve, revealed through extreme demands in the pathological development of cataracts. It may also be explained by the old age itself. The four subjects had a minimum age of 63 and a mean age of 75, it may be that age-related declines in plasticity (Burke & Barnes, 2006), are behind this incomplete recalibration.
Tregillus, Werner and Webster (2016) used a different method to study recalibration in response to the age-related thickening of the eye lens. In their experiment, participants wore glasses in which the lenses had yellow filters designed to simulate the typical aging of a 70-year-old. Participants wore the glasses for eight hours each day for five consecutive days. When examining achromatic settings, shifts towards blue settings showed a partial
7
compensation for the lenses. However, when the lenses were removed a rapid
renormalization period was shown. This could show that the long-term build up is important for the stability of the recalibration. Alternatively, the slow and incomplete normalization following cataract removal may be explained by poorer plasticity in older adults.
Pattern-Contingent Recalibration
In what has become one of the most studied examples of long-term colour recalibration, McCollough (1965) discovered a phenomenon in which a few minutes of exposure to alternating vertical and horizontal gratings with oppositely coloured backgrounds (e.g. red and green) leads to the subsequent perception of horizontal and vertical gratings on achromatic backgrounds as opposite to the colour that they first presented with (see figure 2). The McCollough effect as an example of pattern-contingent recalibration suggests that the colour perception system can recalibrate features of visual scenes independently.
Figure 2. The McCollough Effect. Image adapted from Humphrey, Goodale, Corbetta and Aglioti (1995).
Originally reported as lasting an hour or more (McCollough, 1965), later replications and extensions of the original discovery showed that the McCollough effect is experimentally demonstrable for very long periods of several days (Riggs, White & Eimas, 1974), months (Jones & Holding, 1975), or even permanently (Vul, Krizay & MacLeod, 2008). Variations of the original effect have also been shown to work with plaids, i.e. the simultaneous
presentation of the two induction stimuli (Robinson & MacLeod, 2011). However, the stimuli used in these studies lack ecological relevance and it is therefore difficult to understand how such pattern-contingent recalibration affects colour perception in natural scenes. Some have suggested that the highly unnatural conditions under which effects like the McCollough effect
8
occur are irrelevant to natural vision (Gibson, 1986 as cited in Webster, 2015) and therefore recalibration in natural scenes.
Evidence has been mixed when examining whether pattern-contingent colour
perception recalibration can be shown with more complex stimuli that better represent vision in natural contexts. The first study to investigate such pattern-contingent colour perception recalibration presented single English real-word and English orthographically conforming, but non-word, letter strings coloured in either green or magenta. Results from this study suggested that only real words presented subsequently to a colour pairing are perceived oppositely to the induction colour (Allan, Siegel, Collins & MacQueen, 1989). This could suggest a semantic-contingent recalibration of colour perception and that simple arbitrary patterns from previous findings cannot be extended to text. However, results from a follow-up study using the same paradigm suggested that this evidence for semantic-contingent recalibration is not found when eye movements are controlled for (Humphrey, Skowbo, Symons, Herbert & Grant, 1994). However, while it has been shown that the McCollough effect is dependent on spatial location on the retina (Stromeyer, 1972; Stromeyer & Dawson, 1978), there is no strong reason to assume that more eye movements are made for non-words compared to real words. Therefore, the relationship between the meaning and form of text on colour perception recalibration remain uncertain. Further studies would be warranted to confirm the recalibration propensities for text.
Using more ecologically relevant stimulus pairings, a later study considered whether faces of a single individual with contracted or expanded features paired with either green or red could produce a McCollough effect (Yamashita, Hardy, De Valois & Webster, 2005). When subsequently viewing an achromatic contracted or expanded face, there was no
evidence for colour perception recalibration. One possible explanation for this finding is that pattern-contingent colour perception recalibrations are limited to simple patterns in isolated contexts. The identity of the face may have prevented recalibration as this visual input information was not perturbed and facial identity recognition may take precedence over pattern-contingent recalibration when these conflict. This could implicate a hierarchy of importance for selecting when recalibrations are made when faced with stable and varying features that could be adapted to. This type of question could be addressed by further research using the same individual (Yamashita et al., 2005).
9
Thus, while the McCollough effect is shown in reductive settings, it shows that colour perception recalibration is not always global but that feature-selective recalibrations can be made.
Action-Contingent Recalibration
Sensorimotor theories of perception (O’Regan & Noë, 2001) posit an intimate link between actions and perception. Proponents of this theory have used coloured lenses similarly to some of the studies above to demonstrate sensorimotor contingent colour perception recalibrations.
Bompas and O’Regan (2006a) conducted a series of experiments in which they had participants wear glasses with coloured lenses. The colours of the lenses ranged from short-wavelengths at the left visual field and faded to long short-wavelengths in the right side of the visual field. The application of these lenses thus produced a contingency between saccadic eye movements and colour input. After forty minutes of exposure with these glasses, saccade contingent recalibration was demonstrated such that white patches are perceived as yellower following saccades in the left direction and more blue when eyes moved in the right direction over a white background. Further, they showed that forcing subjects to perform many eye movements increased the strength of the recalibration.
In a series of follow-up experiments, Bompas and O’Regan (2006b) then used a computer screen instead of lenses and provided more evidence for action-contingent colour perception recalibration with new subjects. This follow-up study presented red shapes on one side of the screen and green shapes on another side of the screen successively and asked subjects to judge whether they were the same shape or different shapes, directing subjects to look directly at the shapes when making judgements. After training subjects in this way for forty minutes, two shapes were presented successively, one on each side of the screen and subjects were asked to judge if the second shape was more red or more green than the previous one. When the shapes were both neutral yellow subjects reported that the second targets on what was the green side during training were more red. The same effect was not found in a control experiment where both eyes were kept fixed in position during the presentation of the red and green coloured shapes. The action-contingent recalibration was shown to still be present after ten minutes post-training in some subjects.
10
A limitation of these studies is that they did not test the full timescale over which the recalibration occurred or lasted. Further, any relation between eye movements and colour in natural situations is unclear. However, it is clear that the colour perception system can recalibrate to action-specific perturbations in input and this recalibration can provide a method for future studies examining the mechanisms behind and functions of colour perception recalibration.
Language-Contingent Recalibration
By studying any preattentive influence of language in colour perception, one can consider language as an input to the colour perception system. The role of linguistic
categories in colour perception has been fraught with contention with many studies debating whether the effect of language on colour vision is pre- or post-perceptual i.e. whether differences in language used between individuals equate to differences in perceptual processing. This distinction is important because if language has a role in the perceptual processing of colour then perturbations in input to the colour perception system may lead to a recalibration to adapt to the new language used. Colour categories have alternatively been considered to affect post-perceptual stages of processing, but the underlying perceptual representation of colour and early stages of colour processing may remain unaffected.
Early behavioural studies showed linguistic category difference effects in colour discrimination abilities, claiming that language affects perceptual processing (Drivonikou et al., 2007; Gilbert, Regier, Kay & Ivry, 2006). But crucial in determining if language plays a pre- or post-perceptual role in colour vision is the timing of effects. If effects of language can be seen in very early processing stages then they may be attributable to perceptual processing stages (in the range of 100ms post stimulus), however if they occur later than this then they may be more associated with attentional or other later cognitive processes (Anllo-Vento, Luck and Hillyard, 1998). To study this effect, many studies took advantage of the high temporal resolution of electroencephalography (EEG) and examined at the modulation of event-related potentials (ERPs) that are known to index perceptual or post-perceptual processing stages caused by differences in linguistic categories.
Several studies based experiments on this logic of identifying linguistic modulation of early ERPs to address the timing of language-contingent effects on perception using an oddball paradigm (Athanasopoulos, Dering, Wiggett, Kuipers & Thierry, 2010; Clifford, Holmes, Davies & Franklin, 2010; Holmes, Franklin, Clifford & Davies, 2009; Fonteneau &
11
Davidoff, 2007; Mo, Xu, Kay & Tan, 2011; Thierry, Athanasopoulos, Wiggett, Dering & Kuipers, 2009; Xia, Xu & Mo, 2019) visual search tasks (Liu et al., 2009) or judgements about whether colours presented were the same or different (Liu et al., 2010). All of these studies claimed to have found evidence of pre-attentive perceptual processing roles of language. But as pointed out by Forder, He and Franklin (2017) many of them have a problem with same and different colour categories being equated in colour metrics that are not perceptually uniform and can affect early ERP components, with the notable exception of two, which do not face this problem due to their cross-linguistic approach (Athanasopolous et al., 2010; Thierry et al., 2009)
The first study to convincingly experimentally demonstrate a role for language as a pre-conscious input to the colour perception system was an electroencephalography study investigating deviants in an oddball paradigm. Thierry et al., (2009) used a paradigm in which participants observe a sequence of coloured shapes (circles and squares) the colours of these shapes were two different shades of blue in one condition and two different shades of green in another condition. Subjects were asked to respond whenever the infrequently presented coloured square was presented but the critical target stimulus was an infrequently presented change in colour of the circle stimuli which the subjects were given no instruction about. Subjects in the study were native Greek speakers and native English speakers.
Critically the blue oddball crossed Greek linguistic categories, with one stimulus being
ghalazio and the other stimulus being ble. Both stimuli cases fell into the blue English
linguistic category. Both of the different stimuli for the other colour fell into a single linguistic category in both Greek and English languages of green in English and prazino in Greek. By measuring a larger difference in visual mismatch negativity (vMMN) evoked by the deviants in Greek compared to English subjects, it is arguable that this study
demonstrated preconscious visual processing differences that are part of the early stages of colour perception. Further strengthening this interpretation, the vMMN was comparable across the green targets in both English and Greek speakers.
A problem pointed out with the interpretation of colour perception processing between Greek and English speakers is that the ERP waveforms suggested that the English speakers attended more to the colour differences than Greek participants because there is an attention-related P3 component for the English but not the Greek native speakers. Therefore the vMMN as measured by Thierry and colleagues (2009) may have not been a vMMN but rather an attention-related ERP (Clifford et al., 2010).
12
A later study trained subjects on new colour categories and found evidence only for late stage >300ms ERP differences (Clifford et al., 2012). He, Witzel, Forder, Clifford and Franklin (2014) also used an oddball task and found only evidence for post-perceptual differences when controlling for the discriminability of colour stimuli. However, a
subsequent study by Forder, He and Franklin (2017) showed, using a very similar paradigm that colour categories are important for early sensory stages of colour perception processing. The critical difference between these two studies is whether or not attention was directed to the stimuli differences. This suggested that linguistic categories may only affect colour perception processing when attention is directed away from the stimulus (Forder, He & Franklin, 2017).
In a follow-up analysis of the same dataset used by Thierry and colleagues (2009), the Greek participants were split into those that had been in the UK for a long time and those that had been in the English speaking country of the UK for a short period of time (Athanasopolos et al., 2010). They found that the vMMN evoked by the Greek category-violating blue
colours were virtually abolished in the individuals who had been in the UK and using English for a long period of time. This excludes the possibility of a role of attention in affecting the visual mismatch negativity in Greek speakers only. The reason why trained categories were unable to produce such an effect can be explained by there not being enough time and the importance of using the language for the effect to occur (Athanasopolos et al., 2010). The argument of more attention being paid by the Greek participants in the study cannot therefore explain the difference in vMMN in this follow-up study comparing short- and long-since emigrated Greek speakers.
Thus, it has been convincingly shown that linguistic categories play a role in the early stages of visual processing and that learning a new language can cause a recalibration in colour perception as shown by a loss of neural response to category effects that are no longer relevant to the individual.
Functions of Recalibration
Understanding the role that these recalibrations is not just of fundamental importance but may help uncover further aspects of visual information that are recalibrated too. It also provides opportunities to test mechanisms by questioning whether they are sufficient to account for recalibration. In this section we review two major theories of the functions of
13
recalibrative processes of maintaining a consistent mapping between the external environment and our internal representations of it and efficient perceptual coding.
Consistent Input-Response Mapping
One group of theories about why the colour perception system must adapt is to maintain a consistent match between inputs to and internal representations produced by the colour perception system. This is important to provide colour constancy, the stability of colour perception of the same visual scenes over time despite the changes, described above, in physical characteristics of the light reflected from surfaces over time and space (Foster, 2011). The importance of maintaining colour constancy through recalibration is clear when considering the role of colour in visual tasks and demonstrations of how these tasks rely on consistent perception of the colour of objects in the external environment (Werner, 2014).
Many visual tasks involve object detection and recognition and this process is
mediated by colour perception. The identification of ripe fruits against a background of dark leaves has been shown to be an important driving force for the evolutionary development of red-green colour vision in primates (Osorio & Vorobyev, 1996; Regan et al., 1998; Sumner & Mollon, 2000). Experiments with humans have shown that reaction times are faster for
coloured compared to achromatic food items and this was independent of measures of how diagnostic the colour was to the object (Wurm, Legge, Isenberg & Luebker, 1993). Extending the role of colour in recognition to more ecologically relevant situations, a later study showed that performance in a match to sample task using natural scenes improves when colour information is provided compared to achromatic scenes (Gegenfurtner & Rieger, 2000).
While the above studies show that colour information itself is relevant to object identification and recognition, they do not explicitly show that holding colours constant over time is important for this role. It is possible that these visual tasks could be performed by using the relative colour information despite perturbations in input over time that have not been corrected for by recalibration. Faruq and colleagues (2013) excluded this possibility using a computational modelling approach through simulations comparing the ability of bees to recognise and select flowers with better rewards that were marked by colour. They
compared models of bees with perfect colour constancy, no colour constancy, colour blind bees, those with trichromatic colour vision and bees with an implementation of colour constancy algorithm. They found that colour vision without colour constancy were slightly better than colour blind bees but that the added biologically realistic colour constancy
14
implementation led to better performance but only up to half of that of perfect colour
constancy. While this study is highly reductive, using a computational model of a biological model system, it shows that colour constancy is important for the role of colour in object recognition. Thus, holding colour perception constant in the face of perturbations in input through recalibrative mechanisms that maintain colour constancy allow for biologically relevant visual tasks to be achieved.
Beyond holding colours constant within an individual over time, holding colour perception constant between observers has also been suggested to have a biological function through anchoring individuals in a common environment (Clifford et al., 2007). Changes in linguistic input to the colour perception system could be a route to standardising colour perception between individuals, improving the consistency of the internal representation of the external environment between them. This draws support from the notion that using language in the environment is important for the removal of the large mismatch negativity seen in English-using but not necessarily English learning Greek native speakers in the study described above by Athanasopolous and colleagues (2010). A more detailed description of why such a consistent mapping is in itself relevant is lacking (but see discussion for a suggestion).
It can therefore be seen that maintaining a consistent input-response mapping within an individual over time is important for object recognition which can be seen as critical in contemporary tasks such as those involved in cosmetics and textiles (Foster, 2003). Further work is needed to understand why and whether colour perception needs to be held constant between individuals.
Efficient Input-Response Mapping
Another proposed function of colour perception recalibration is that it allows for efficient production of internal representations of the external world. This efficient coding hypothesis was first proposed by Barlow (1961 in Wark et al., 2007). Since neural circuits have a finite capacity in the distribution of sensory information that they can process and metabolic constraints do not allow for the simultaneous encoding of all possible sensory possibilities, it would be more efficient to recalibrate the limited inputs that can be registered based on what is typically encountered (Wark, Lundstrom & Fairhall, 2007). In this way by recalibrating a dynamic response range, a metabolically limited colour perception system can be maximally informative about the external environment (MacLeod & von der Twer, 2003).
15
Coding for colour perception through predictions about the external world is a further way in which recalibration has been considered to allow for an efficient input-response mapping (Gardner et al., 2005; Srinivasan, Laughlin & Dubs, 1982; Chopin & Mamassian, 2012). It may be that recalibrations center colour responses on what is expected of the external world and colour responses relate to deviations from those predictions (Webster, 2015). This is an efficient way to do code colour perception by reducing the average firing rate of neurons (Gardner et al., 2005).
Therefore efficiency is a convincing biological function of recalibration, both in terms of making predictions about the world and in only coding for what is typically encountered.
Mechanisms of Recalibration
Understanding the mechanisms of how the colour perception system recalibrates is necessary for understanding the function of the visual system. Many approaches describe a high-level mechanism that can be applied to some if not all instances of recalibration. It has also been questioned whether recalibrative processes occur at a single cell instead of a circuit level. Here we discuss three mechanisms by which recalibration has been considered to occur: associative learning, the action of an error-correcting device and recalibration at the level of single cells.
Associative learning
One of the earliest mechanisms proposed to explain recalibration in the McCollough effect was the form of associative learning that is classical conditioning (Murch, 1976). On this view, adaptation occurs through learning the association between the oriented grating as the conditioned stimulus and the paired colour as the unconditioned stimulus. However, this view has been challenged multiple times (Dodwell & Humphrey, 1990; Sheth & Shimojo, 2008; Skowbo & Forster, 1983). One example of why classical conditioning cannot account for the pattern-contingent recalibration in the McCollough effect is an experiment showing that presenting the purported unconditioned stimulus of colour alone intermitted with a paired presentation of colour with horizontal/vertical gratings does not lead to a reduced/absent effect (Skowbo and Forster, 1983). Since when presenting the colour alone, there is at least less association with the grating, a reduced or absent effect would be expected if classical conditioning was behind the pattern-contingent recalibration in the McCollough effect.
16
Attempts to explain recalibraton through associative learning are also more recent such as in explaining the shift with bilingualism shown by Athanasopolous et al (2010). As proposed by Casasanto (2008), learning a new language involves repeated simultaneous activation of conceptual categories with words and grammar. Using the new language involves strengthening these associations and because languages partition the world differently, this will result in a different mental map of the world (Athanasopolous et al., 2010). This seems especially possible given the importance of cultural immersion for the language-contingent calibration, by using the new language, new associations could be reinforced where other associations dependent on the original language would still be active without the cultural immersion and usage of the language.
Thus in full, associative learning mechanisms may be behind some forms of adaptation but they might not be able to account for others.
A Device for Error Correction
Dodwell and Humphrey (1990) argue that a combination of Helson’s (1948)
adaptation level principle and Andrews’ (1964) error correcting mechanism can describe how the McCollough pattern-contingent aftereffect occurs through a recalibration mechanism that in principle be considered a general-purpose and could underlie other colour perception, and indeed other sensory system recalibrations.
Helson’s (1948) adaptation level principle states that perceptual judgements are made with respect to a neutral point that is based on previously encountered stimuli. This neutral point would be the average of colour information that has previously been encountered and would correspond to the physical characteristics of light that is perceived as white; the average hue in scenes is colourless. By shifting this neutral point, the colour perception system can recalibrate colour perception so that the neutral point is that which corresponds to the dominant wavelength of the visual information, that is then perceived without any hue.
The error correcting mechanism from Andrews (1964) involves the updating of a transfer function applied to the information reaching the eye in order to generate an internal representation of the visual environment. Because aspects of the environment that remain static can be relied upon, changes in visual scene statistics can cue recalibration. When things that are usually stable change, this is registered as an error by the error correction device, and
17
the transfer function is updated to cancel this change, accounting for the error by re-establishing the statistical criteria known to be true of the external world.
Dodwell and Humphrey (1990) combine the adaptation level principle and the error-correcting mechanism to account for recalibration in the McCollough effect by describing that the lack of any correlation in the real world between orientation and colour is violated and this is registered as an error by an error-correction device. The registration of such an error is then used as a basis for updating a transfer function applied to input to later stages of visual processing. This update of the transfer function leads to the subsequently presented achromatic horizontal or vertical grates being corrected for resulting in the after-effects typical of McCollough effects (figure 2).
By doing so, the visual system can reorganise itself to take account of known
statistical properties of the visual environment and renormalize colour perception and account for changes in the mapping between the external and internal environments allowing for reliability of colour perception that is important for object identification. The explanation of recalibration through the action of an error-correcting device thus provides strong evidence for a mechanism through which recalibration can occur.
Recalibration of Single Cells
Recalibration of single cells involved at any stage of colour perception through a mechanism similar to von Kries photoreceptor adaptation (Von Kries, 1905 as cited in Foster, 2011) could underpin many of the recalibrations reviewed in the first section of this paper. The concept behind such single cell recalibration involves a similar principle to that of the error-correcting device described above in that it can describes how a neutral point is re-established when colour processing is perturbed in a given direction. However, it differs on the critical point that it acts on a single cell level rather than at the level of neural circuits.
The principle of von Kries adaptation was originally developed to describe how the receptors can avoid saturation by reducing their responses when overall light levels are high and increasing it when overall light levels are low (Chittka, Faruq, Skorupski & Weerner, 2014). But if the three cone subtypes of retinal photoreceptors can be changed independently of one another then a perturbation of a particular colour can be cancelled out by the
dampening down of other ones (Laughlin, 1989). This is in line with efficient coding as it only requires the sensitivity changes of single cells.
18
Photoreceptor subtype independent recalibration has been shown to be biologically possible, especially with the real demonstration that the UV receptor in bees has been empirically shown to be up to 16 times more sensitive than a green receptor under some conditions (von Helversen, 1972, in Chittka et al., 2014). Further, the modelling study described above showing that colour constancy helps with object identification and recognition in bees (Faruq et al., 2013) used von Kries adaptation for their adaptation modelling in their model bees. This study only examined colour constancy over a short period but the same principles can be applied to maintaining colour constancy over longer periods such as seasons or even over decades in response to age-related physiological changes, as discussed above.
A problem with von Kries photoreceptor adaptation as an account for all recalibration is that there is a fair amount of evidence for cortical roles in many recalibration processes, for example in the McCollough effect, (Humphrey, James, Gati, Menon & Goodale, 1999). To explain this, it is possible that the same principle of reducing/increasing the gain in single cells further upstream of the retina in a similar way to von Kries adaptation in photoreceptor cells is possible. However, it has been been questioned how long von Kries style adaptations can last and whether this is long enough for recalibrations lasting several minutes or more (Bompas & O’Reagan, 2006b). It is therefore clear that single cell adaptation as a mechanism is sufficient for at least some recalibrations but may not underlie all recalibrations.
Discussion
This review of what why and how recalibrations of the human colour perception system occur showed that recalibrations are made in response to perturbations in input at various levels. Recalibrations are made in response to changes to the entire visual
environment over locations, seasons, days or due to age-related developments. We also showed that recalibrations can be made in response to more limited perturbations in other inputs to the colour processing of patterns contingent on colour, actions in the form of eye movements and the language used. These recalibrations were shown to fulfil two major biological functions of efficiently producing an internal representation of the external world and consistently mapping the internal representation to the external world. The review also showed that at least three mechanisms potentially mediate at least some of these
recalibrations through associative learning, the action of a neural circuit error correcting device or recalibration at the level of single cells.
19
The different timescales over which recalibration occurs may reflect the action of mechanisms differentially tuned to the time required to understand the nature of the perturbation (Webster, 2015). Some of the effects described in this review, such as the McCollough effect, show recalibration after minutes of input perturbation whereas others require a longer period, for example seasonal colour changes or even age-related
developments. If the same mechanisms mediate recalibrations over several timescales then it remains unclear how the neural system tracks such changes (Wark, et al., 2007) and how the mechanisms resolve situations that demand recalibrations in differing directions (Webster, 2015). However, if different mechanisms exist for different timescales, questions arise regarding the differences in their selective nature. The relation between recalibrations in response to perturbations over different timescales can be addressed by designing future experiments that pit demands against one another (Webster, 2015).
Conflict between functions could also occur. The most metabolically efficient way to represent an object that is always the same colour may be to perform a recalibration that is specific to that object (Gardner et al., 2005), representing it as achromatic. However, this conflicts with the function of colour constancy. In some cases, such as with an example of a banana, this recalibration is not performed despite only being encountered in yellow. While it may be that colour constancy is a function that takes precedence over efficiency, it could also be because there are limits to what objects or object classes to which colour perception can selectively recalibrate.
The different aspects of colour processing inputs that can be recalibrated further suggest the possibility of conflicts. Findings of recalibration contingent on independent features of visual scenes suggest that in real scenes there can be multiple perturbations that could cause recalibrations and when these cannot be matched, it is unclear which would take precedence. The failure to demonstrate a McCollough effect with orientations embedded in faces by Yamashita and colleagues (2005) may suggest that face perceptual processes overule simple pattern-contingent colour recalibration but future research using faces from different individuals within the same induction period would be required to confirm this.
An explanation for suggestions that maintaining consistent input-response mapping of colour perception between individuals (Clifford et al., 2007) is lacking. A tentative proposal drawn from the findings of the importance of using language in everyday life for the
20
plays a social role through facilitating joint attention. It has been suggested that the pre-attentive role of language in colour perception is to direct attention (Thierry et al., 2009). By calibrating the colour perception system to make the same predictions about the external world (Webster, 2015) consistently across people using the same language, the salience of features in the surrounding world may be made more consistent (Barlow, 1961 in Wark et al., 2007). This direction of attention in the same way may facilitate more joint attention, which has been shown to be sufficient for social bonding (Wolf, Lunay and Dunbar, 2016). Before concluding that language-contingent colour perception recalibrations have a social role to play, future research using similar methods to Wolf, Lunay and Dunbar (2016) and comparing different language speakers, or those at different stages of learning a new
language or moving to a different language-speaking country as in previous studies (Thierry et al., 2009; Athanasopolous et al., 2010) are required.
The most important direction for the field to move with further research is therefore to investigate how conflicts within and between recalibration functions, mechanisms, and
feature-contingent colour perception recalibrations are resolved. A tractable goal for this approach to colour perception research would be to understand which mechanisms, functions and features take precedence in multi-faceted and highly dynamic natural scenes. This may involve designing experiments to pit mechanisms and functions against one another. An improvement on approaches using lenses would be to use virtual reality approaches in which a greater control over features in the visual environment is possible.
Recalibration in other visual domains and sensory modalities is beyond the scope of this review but synthesising these findings and examining whether common mechanisms and functions are involved, as well as investigating cross-modal conflicts should be a long-term goal in the study of recalibration of sensory systems. An important even longer-term goal would be to integrate sensory recalibration with other forms of recalibrations, such as the vestibulo-ocular reflex (see Pogson et al., 2019), to investigate the existence of large-scale recalibration principles.
Recalibrations after perturbations in different types of input to the colour perception system are common, widespread and varied. The recalibrations take place over different timescales, likely involve different mechanisms and serve multiple functions. Understanding these recalibrations in real-world settings remains obscured by likely conflicts between and
21
within aspects of what why and how recalibrations are made and further research is needed to address which of these take precedence in the event of conflicts.
22
References
Allan, L. G., Siegel, S., Collins, J. C., & MacQueen, G. M. (1989). Color aftereffect contingent on text. Perception & Psychophysics, 46(2), 105-113.
Andrews, D. P. (1964). Error-correcting perceptual mechanisms. Quarterly Journal of
Experimental Psychology, 16(2), 104-115.
Anllo‐Vento, L., Luck, S. J., & Hillyard, S. A. (1998). Spatio‐temporal dynamics of attention to color: Evidence from human electrophysiology. Human brain mapping, 6(4), 216-238.
Athanasopoulos, P., Dering, B., Wiggett, A., Kuipers, J. R., & Thierry, G. (2010). Perceptual shift in bilingualism: Brain potentials reveal plasticity in pre-attentive colour
perception. Cognition, 116(3), 437-443.
Bompas, A., & O'Regan, J. K. (2006a). Evidence for a role of action in colour perception. Perception, 35(1), 65-78.
Bompas, A., & O'Regan, J. K. (2006b). More evidence for sensorimotor adaptation in color perception. Journal of Vision, 6(2), 5-5.
Burke, S. N., & Barnes, C. A. (2006). Neural plasticity in the ageing brain. Nature reviews
neuroscience, 7(1), 30.
Casasanto, D. (2008). Who's afraid of the big bad Whorf? Crosslinguistic differences in temporal language and thought. Language learning, 58, 63-79.
Chittka, L., Faruq, S., Skorupski, P., & Werner, A. (2014). Colour constancy in insects. Journal of Comparative Physiology A, 200(6), 435-448.
Chopin, A., & Mamassian, P. (2012). Predictive properties of visual adaptation. Current
biology, 22(7), 622-626.
Clifford, A., Franklin, A., Holmes, A., Drivonikou, V. G., Özgen, E., & Davies, I. R. (2012). Neural correlates of acquired color category effects. Brain and cognition, 80(1), 126-143.
Clifford, A., Holmes, A., Davies, I. R., & Franklin, A. (2010). Color categories affect pre-attentive color perception. Biological psychology, 85(2), 275-282.
23
Clifford, C. W., Webster, M. A., Stanley, G. B., Stocker, A. A., Kohn, A., Sharpee, T. O., & Schwartz, O. (2007). Visual adaptation: Neural, psychological and computational aspects. Vision research, 47(25), 3125-3131.
Delahunt, P. B., Webster, M. A., Ma, L., & Werner, J. S. (2004). Long-term renormalization of chromatic mechanisms following cataract surgery. Visual neuroscience, 21(3), 301-307.
Dodwell, P. C., & Humphrey, G. K. (1990). A functional theory of the McCollough effect. Psychological review, 97(1), 78.
Drivonikou, G. V., Kay, P., Regier, T., Ivry, R. B., Gilbert, A. L., Franklin, A., & Davies, I. R. (2007). Further evidence that Whorfian effects are stronger in the right visual field than the left. Proceedings of the National Academy of Sciences, 104(3), 1097-1102.
Eisner, A., & Enoch, J. M. (1982). Some effects of 1 week’s monocular exposure to long-wavelength stimuli. Perception & psychophysics, 31(2), 169-174.
Faruq, S., McOwan, P. W., & Chittka, L. (2013). The biological significance of color constancy: An agent-based model with bees foraging from flowers under varied illumination. Journal of Vision, 13(10), 10-10.
Fonteneau, E., & Davidoff, J. (2007). Neural correlates of colour categories. Neuroreport,
18(13), 1323-1327.
Forder, L., He, X., & Franklin, A. (2017). Colour categories are reflected in sensory stages of colour perception when stimulus issues are resolved. PloS one, 12(5), e0178097.
Foster, D. H. (2003). Does colour constancy exist?. Trends in cognitive sciences, 7(10), 439-443.
Foster, D. H. (2011). Color constancy. Vision research, 51(7), 674-700.
Gardner, J. L., Sun, P., Waggoner, R. A., Ueno, K., Tanaka, K., & Cheng, K. (2005). Contrast adaptation and representation in human early visual cortex. Neuron, 47(4), 607-620.
Gegenfurtner, K. R., & Rieger, J. (2000). Sensory and cognitive contributions of color to the recognition of natural scenes. Current Biology, 10(13), 805-808.
24
Gilbert, A. L., Regier, T., Kay, P., & Ivry, R. B. (2006). Whorf hypothesis is supported in the right visual field but not the left. Proceedings of the National Academy of
Sciences, 103(2), 489-494.
He, X., Witzel, C., Forder, L., Clifford, A., & Franklin, A. (2014). Color categories only affect post-perceptual processes when same-and different-category colors are equally discriminable. JOSA A, 31(4), A322-A331.
Helson, H. (1948). Adaptation-level as a basis for a quantitative theory of frames of reference. Psychological review, 55(6), 297.
Holmes, A., Franklin, A., Clifford, A., & Davies, I. (2009). Neurophysiological evidence for categorical perception of color. Brain and cognition, 69(2), 426-434.
Humphrey, G. K., Goodale, M. A., Corbetta, M., & Aglioti, S. (1995). The McCollough effect reveals orientation discrimination in a case of cortical blindness. Current
Biology, 5(5), 545-551.
Humphrey, G. K., James, T. W., Gati, J. S., Menon, R. S., & Goodale, M. A. (1999). Perception of the McCollough effect correlates with activity in extrastriate cortex: A functional magnetic resonance imaging study. Psychological Science, 10(5), 444-448.
Humphrey, G. K., Skowbo, D., Symons, L. A., Herbert, A. M., & Grant, C. L. (1994). Text-contingent color aftereffects: A reexamination. Perception & psychophysics, 56(4), 405-413.
Jones, P. D., & Holding, D. H. (1975). Extremely long-term persistence of the McCollough effect. Journal of Experimental Psychology: Human Perception and Performance, 1(4), 323.
Jordan, G., & Mollon, J. D. (1993). The Nagel anomaloscope and seasonal variation of colour vision. Nature, 363(6429), 546.
Kompaniez, E., Abbey, C. K., Boone, J. M., & Webster, M. A. (2013). Adaptation aftereffects in the perception of radiological images. PloS one, 8(10), e76175.
Kuehni, R. G. (2014). Unique hues and their stimuli-state of the art. Color Research &
25
Laughlin, S. B. (1989). The role of sensory adaptation in the retina. Journal of Experimental
Biology, 146(1), 39-62.
Liu, Q., Li, H., Campos, J. L., Wang, Q., Zhang, Y., Qiu, J., ... & Sun, H. J. (2009). The N2pc component in ERP and the lateralization effect of language on color
perception. Neuroscience letters, 454(1), 58-61.
Liu, Q., Li, H., Campos, J. L., Teeter, C., Tao, W., Zhang, Q., & Sun, H. J. (2010). Language suppression effects on the categorical perception of colour as evidenced through ERPs. Biological Psychology, 85(1), 45-52.
Lucy, J. A. (2016). Recent advances in the study of linguistic relativity in historical context: A critical assessment. Language Learning, 66(3), 487-515.
MacLeod, D. I., & von der Twer, T. (2003). The pleistochrome: optimal opponent codes for natural colours. Colour perception: Mind and the physical world, 155-184.
McCollough, C. (1965). Color adaptation of edge-detectors in the human visual system. Science, 149(3688), 1115-1116.
Mo, L., Xu, G., Kay, P., & Tan, L. H. (2011). Electrophysiological evidence for the left-lateralized effect of language on preattentive categorical perception of
color. Proceedings of the National Academy of Sciences, 108(34), 14026-14030.
Murch, G. M. (1976). Classical conditioning of the McCollough effect: Temporal parameters. Vision Research, 16(6), 615-619.
Neitz, J., Carroll, J., Yamauchi, Y., Neitz, M., & Williams, D. R. (2002). Color perception is mediated by a plastic neural mechanism that is adjustable in adults. Neuron, 35(4), 783-792.
O'Regan, J. K., & Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and brain sciences, 24(5), 939-973.
Osorio, D., & Vorobyev, M. (2008). A review of the evolution of animal colour vision and visual communication signals. Vision research, 48(20), 2042-2051.
Pogson, J. M., Taylor, R. L., Bradshaw, A. P., McGarvie, L., D’Souza, M., Halmagyi, G. M., & Welgampola, M. S. (2019). The human vestibulo-ocular reflex and saccades: normal subjects and the effect of age. Journal of neurophysiology, 122(1), 336-349.
26
Pokorny, J., Smith, V. C., & Lutze, M. (1987). Aging of the human lens. Applied
optics, 26(8), 1437-1440.
Regan, B. C., Julliot, C., Simmen, B., Viénot, F., Charles–Dominique, P., & Mollon, J. D. (2001). Fruits, foliage and the evolution of primate colour vision. Philosophical
Transactions of the Royal Society of London. Series B: Biological Sciences, 356(1407),
229-283.
Riggs, L. A., White, K. D., & Eimas, P. D. (1974). Establishment and decay of orientation-contingent aftereffects of color. Perception & Psychophysics, 16(3), 535-542.
Robinson, A., & MacLeod, D. (2011). The McCollough effect with plaids and gratings: Evidence for a plaid-selective visual mechanism. Journal of vision, 11(1), 26-26.
Schefrin, B. E., & Werner, J. S. (1990). Loci of spectral unique hues throughout the life span. JOSA A, 7(2), 305-311.
Sheth, B. R., & Shimojo, S. (2008). Adapting to an aftereffect. Journal of Vision, 8(3), 29-29.
Skowbo, D., & Forster, T. (1983). Further evidence against the classical conditioning model of McCollough effects. Perception & psychophysics, 34(6), 552-554.
Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982). Predictive coding: a fresh view of inhibition in the retina. Proceedings of the Royal Society of London. Series B.
Biological Sciences, 216(1205), 427-459.
Stromeyer III, C. F. (1972). Contour-contingent color aftereffects: Retinal area specificity. The American journal of psychology, 227-235.
Stromeyer III, C. F., & Dawson, B. M. (1978). Form-colour aftereffects: Selectivity to local luminance contrast. Perception, 7(4), 407-415.
Sumner, P., & Mollon, J. D. (2000). Catarrhine photopigments are optimized for detecting targets against a foliage background. Journal of Experimental Biology, 203(13), 1963-1986.
Thierry, G., Athanasopoulos, P., Wiggett, A., Dering, B., & Kuipers, J. R. (2009). Unconscious effects of language-specific terminology on preattentive color
27
Tregillus, K. E., Werner, J. S., & Webster, M. A. (2016). Adjusting to a sudden “aging” of the lens. JOSA A, 33(3), A129-A136.
Vul, E., Krizay, E., & MacLeod, D. I. (2008). The McCollough effect reflects permanent and transient adaptation in early visual cortex. Journal of vision, 8(12), 4-4.
Wark, B., Lundstrom, B. N., & Fairhall, A. (2007). Sensory adaptation. Current opinion in
neurobiology, 17(4), 423-429.
Weale, R. A. (1988). Age and the transmittance of the human crystalline lens. The Journal of
physiology, 395(1), 577-587.
Webster, M. A. (2015). Visual adaptation. Annual review of vision science, 1, 547-567.
Webster, M. A., Mizokami, Y., & Webster, S. M. (2007). Seasonal variations in the color statistics of natural images. Network: Computation in neural systems, 18(3), 213-233.
Welbourne, L. E., Morland, A. B., & Wade, A. R. (2015). Human colour perception changes between seasons. Current Biology, 25(15), R646-R647.
Werner, A. (2014). Spatial and temporal aspects of chromatic adaptation and their functional significance for colour constancy. Vision research, 104, 80-89.
Werner, J. S., & Schefrin, B. E. (1993). Loci of achromatic points throughout the life span. Josa a, 10(7), 1509-1516.
Werner, J. S., & Steele, V. G. (1988). Sensitivity of human foveal color mechanisms throughout the life span. JOSA A, 5(12), 2122-2130.
Wolf, W., Launay, J., & Dunbar, R. I. (2016). Joint attention, shared goals, and social bonding. British Journal of Psychology, 107(2), 322-337.
Wuerger, S. (2013). Colour constancy across the life span: evidence for compensatory mechanisms. PloS one, 8(5), e63921.
Wurm, L. H., Legge, G. E., Isenberg, L. M., & Luebker, A. (1993). Color improves object recognition in normal and low vision. Journal of Experimental Psychology: Human
perception and performance, 19(4), 899.
Xia, T., Xu, G., & Mo, L. (2019). Bi-lateralized Whorfian effect in color perception: Evidence from Chinese Sign Language. Journal of Neurolinguistics, 49, 189-201.
28
Yamashita, J. A., Hardy, J. L., De Valois, K. K., & Webster, M. A. (2005). Stimulus
selectivity of figural aftereffects for faces. Journal of Experimental Psychology: Human
