Can pictures say no or not?: Negation and denial in the visual mode

(1)

Tilburg University

Can pictures say no or not?

Oversteegen, E.; Schilperoord, J.

Published in:

Journal of Pragmatics

DOI:

10.1016/j.pragma.2014.03.009

Publication date:

2014

Document Version

Publisher's PDF, also known as Version of record

Link to publication in Tilburg University Research Portal

Citation for published version (APA):

Oversteegen, E., & Schilperoord, J. (2014). Can pictures say no or not? Negation and denial in the visual mode.

Journal of Pragmatics, 67, 89-106. https://doi.org/10.1016/j.pragma.2014.03.009

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

(2)

Can

pictures

say

no

or

not?

Negation

and

denial

in

the

visual

mode

§

Eleonore

Oversteegen

*

,

Joost

Schilperoord

1 TilburgUniversity,Postbox90153,5000LETilburg,TheNetherlands

Received26September2013;receivedinrevisedform24March2014;accepted30March2014

Abstract

Inprinciple,verbalandimagelanguageshavedifferentwaysofcodingconceptualcontent.Moreover,thereisnoreasontobelieve thatbothmodes,thelinguisticandthevisual,canconveyidenticalcontents,andindeed,linguistshaveclaimedthatimagesarenotsuited forexpressingthemeaningofcertainlinguisticcategories,likenegation.Asthelinguistic literaturearguesconvincingly,innatural languageadistinctionbetweennegationanddenialisjustified.Employinginsightsinvisualcommunicationandcognitionscience,this paperexploresthepossibilitiesforvisuallyexpressingnegationand/ordenial.Atthehandofboththeanalysisandanempiricalpilotstudy ofasetofadvertisements,wecomeupwithapositiveanswertothetitlequestion:yes,picturescansay‘no’.

Keywords: Negation;Denial;NLsemantics;Images;Erasure;Experiment

1. Introduction

Thetopicofthispapercanbephrasedas:canyousayXisnotYwithapicture?OrcanapicturesaynotX?Ifitcomes toexpressivepotential,thereisconsensusinthelinguisticliterature:thevisualmodeisbyfarinferiortonaturallanguage. Forexample,thevisualmodelacksthekindofexpressivedevicespermittingtype/tokendistinctions,suchasmodifiers and determiners; neithercanit expresscausality, optatives,conditionals, or spatiotemporal references. Ona more generallevel,thereceivedviewisthatthevisualmodelacksvocabularyandgrammar,andthereforecannotexpress assertions,i.e.expressionsthatcanbeassignedatruevalue.Thepossibilityofnegationcanbeseenasalitmustest here:onlyifacertainmodalityiscapableofexpressingatruepropositionp,isitcapableofreversingitstruevalueby expressingnot p.AclearstanceonthesubjectistakenbyWorthinhisillustriousstatement‘Picturescan’tsayain’t’ (Worth,1981:162).Inthesamevein,Jackendoff(2007,105ff)arguesthatonlynaturallanguageallowsonetoattendto whatisnot,andtoexpressthecomplexconceptualstructuresthatnegationsupposes.Morerecently,Khemlanietal. statethat‘Muchoflanguagecannothaveaperceptualrepresentation,suchasavisualimage,andnegationisoneofthe mostimportantofsuchconcepts’(Khemlanietal.,2012:546).Caseclosed,soitwouldappear.

However,Gioraetal.(2009,inthisissue)takeadifferent,lesslogocentricstanceonthematterofpictorialnegation. Addressingthequestionwhetherthevisualmodalityisequippedwithdevicesequivalenttolexical/morphologicalmarkers www.elsevier.com/locate/pragma Availableonlineatwww.sciencedirect.com

ScienceDirect

JournalofPragmatics67(2014)89--106

§_The_authors_are_grateful_to_Jung_von_Matt,_Alster_Werbeagentur_GmbH,_the_{International}_Service_for_Human_Rights,_Sony_Benelux_and_Sony

EuropeLimited,Saatchi&Saatchi,theLoveMarksCompany,forgrantingpermissiontoreproduceinourpaperimagesofwhichtheyarecopy rightsholders.

*Correspondingauthor.TilburgUniversity,Postbox90153,5000LETilburg,TheNetherlands.Tel.:+31134662691. E-mailaddress:E.Oversteegen@uvt.nl(E.Oversteegen).

1

Tel.:+31134669111.

(3)

suchasno,notorun-,Gioraetal.(2009)callattentiontoconventionalvisualmarkerslike(red)crossesandstripesused, for example, in traffic signs and price tags announcing discounts. Such markers establish negation by being superimposeduponanimage(oraword),theirscopebeingdeterminedbytheirsize.Inamessagelikediscriminationthe scopeofthestrikethroughislimitedtodiscrimi,whiletheremainingnationsuggestssomeoppositemeaningthatresults fromnegatingthefirstpart:nodiscriminationproducesnation.Theauthorsconcludethatthosesigns‘(...)allowapicture, takenasawhole,tovisuallysay‘‘no’’(...)’(Gioraetal.,2009:2224).

Markerslikestripesandcrossesaredecidedlynon-verbal.Atthesametime,however,theyarenotanintegralpartof thesemanticsofthepictorialmodalitybutinsteadconstituteaseparate,extra-diageticcodingsystemrelativetothevisual mode.Thegoalofthispaperistoexplorepossibilitiesforvisualnegation.However,ratherthanfocusingonthekindof extra-diageticmarkersstudiedbyGioraetal.,weintendtoexplorevisualformsofnegationthatcanbeconsideredintegral tothevisualmode.Thequestionaddressedhereiswhetheritispossibleforastillpicturetoattendviewerstowhatisnot, usingdevicesthatbelongtothesystemofvisualsignsanditsprinciplesofassemblage.Bytakingissuewiththesematters westrivetodeepenourunderstandingofthestructuralfactorsandexpressivepotentialofvisualexpressions--inshort, ourunderstandingofvisualcommunication.

Thepaperisorganizedinfivesections.Section1islogocentric:itdiscussesviewsonlinguisticnegationinorderto identifysomeofitsformsandprerequisites.Theconclusionisthatnegationcomesindifferenttypes,thatitissensitiveto focus,andthatitmayapplytovariouspartsandaspectsofanutterance.Section2addressesingeneraltermssomeofthe expressiveproblemsthatvisualcommunicationisfacedwith.Sections3and4presentthemainresultsofourstudy.In section3,weisolate andexemplifytwopictorialtemplatescapableofputtingacertainentityintheviewer’sfocusof attention,andsubsequentlynullifythatentity.Section4reportsonanempiricalstudyintotheway‘real’viewersrespondto imagesthatinstantiatethetemplatesoutlinedinsection3.Section5concludesthepaper.

2. Typesofnegationinnaturallanguage

Innaturallanguagesemantics,adistinctionisdrawnbetweennegationanddenial.Awarenessofthisdistinctionmay enhanceourrecognitionofthekindofnegationeffectspicturesmightbringabout.Negationisasententialoperatorwhich changesthetruthvalueofthesentence.Althoughnegationmarkerssuchas‘not’or‘no’canoccupyvariouspositionsina sentence,atthelevel offormalrepresentationtheplaceofthenegatorisrelatively fixedandallowsonlyforminimal variation,suchasscopevariationwithrespecttootheroperatorslikequantifiersormodaloperators.2Intheexamples(1), takenfromKhemlanietal.(2012),(1a)exemplifiesbroadscopeorsententialnegation,whereas(1b)exemplifiesnarrow scopeorVP-negation.

(1) a. Noartistsarebeekeepers b. Someartistsarenotbeekeepers

Thecommunisopiniois,however,thatthedifferencebetweensententialandVPnegationcanbeneglected.Horn(2001)

treatsbothcasesasVPnegation,whileGiannakidou(2004)arguesthattheanalysisofnegationasaVP-operatorisnotat oddswiththepropositionalanalysis,butratheravariantofit(Giannakidou,2004;seealsoNapoli,2006:247).

DenialisaconceptofSpeechAct theory.Itsfunctionistoobjecttoapreviousutterance,thatis,‘(...)toremove previouslyintroducedmaterialfromthecommonground’(vanderSandt,1991:1).Inordertoappreciatethedifference betweennegationanddenial,comparethefollowingexamples.

(2) a. S1 Peterlosthiswife.Maryisnothappy. b. S1 Maryishappy

S2 Maryisnothappy

Sentencepair(2a)containsanexampleofnegation.Thetwosentencesenumerate(sad)situations,ortogetherforman argumentforintroducingPeterandMarytoeachother.Inthiscase,statingthatMaryisnothappyisnotmeanttorefute someoneelse’sclaimtothe contrary,i.e.thatMaryishappy.Thedialogin(2b)exemplifiesdenial.S2’scontribution producessomeinconsistencywithregardtotheassertionmadebyS1,totheeffectofremovingitscontent(i.e.Maryis happy)fromthecommonground.In(2a),ontheotherhand,nosuchinconsistencyisatstake.Inotherwords,denials negateacertainutteranceXthatispartoftheimmediatediscoursecontextbyexpressingnot(X),whereasnegations

2_Cf._Moscati₍₂₀₀₆₎_:_‘The_overt_realization_of_sentential_negation_is_subject_to_broad_{cross-linguistic}_variation,_while_the_logical_{representation}_of

(4)

expressapredicate(notX)withoutindicatingthatXispartofthecontext.Hence,adistinctivefeatureofdenialappearsto bethesuppositionofthecontraryinthecommonground.Denialisthekindofnullificationthatgoestogetherwiththis supposition.3

Unlikenegation,whosescopealwaysconcernstheentireutterance(ortheVP)inwhichtheovertmarkerappears,the scopeofdenialcanvary.Denialmay bemetalinguistic(Horn,1985),while otherformsarefocussensitive(vander AuweraanddeVogelaer,2005).Metalinguisticdenialconstitutesa‘(...)deviceforobjectingtoapreviousutteranceon anyground whatever’(Horn, 1985, 121).Apartfrom aprevious utterance’scontent(see2b),such groundsmay be incompleteness,wrongregister,falseimplicatures,falsepresuppositions,andsoforth.Theexpressionsin(3),takenfrom

Lee(2005),exemplifysomepossibilitiesofmetalinguisticdenial. (3) a. S1 Johnstoppedsmoking

S2 Johndidnotstopsmoking;heneversmoked b. S1 Somemenarechauvinists

S2 Somemenaren’tchauvinists;allmenarechauvinists c. S1 Grannyisfeelinglousy

S2 Grannyisn’tfeelinglousy,Johnny,sheisbadlyindisposed!

S2’sutterancein(3a)deniesthepresuppositionofS1’sutterance.IfJohnhasquittedsmoking,itispresupposedthathe musthavebeenasmoker,whichisthereforepartofthecommonground.Onlythispartoftheconceptualcontentofthe utteranceisactuallydenied;i.e.Johnneversmoked.S2’sutterancein(3b)deniestheimplicatureofS1’sassertion.If somemenarechauvinist,theremustbemenwhoarenot.Iftheimplicatureisconsideredpartoftheconceptualstructure, thenthatpartofthecontentisdenied:allmenarechauvinists.Finally,inexample(3c)thedenialdoesnotaffectthe conceptual content of the proposition but only its form: Granny is feeling lousy. Apparently, S2 opposesto using disrespectfullanguage.

Focussensitivedenialisillustratedbyexamples(4a--d),whereitalicsmarkstress. (4) S1 MaryintroducedBilltoSue

S2 a.MarydidnotintroduceBilltoSue(butJanedid) b.MarydidnotintroduceBilltoSue(butJohnwas) c.MarydidnotintroduceBilltoSue(buttoJill)

d.MarydidnotintroduceBilltoSue(shementionedhimtoher)

S2’scontributionsa--daregenuinecasesofdenialinthatallobjecttosomeclaimtothecontrary.However,inthesecases, onlythestressedpartofeachsentenceiscontrastedwithS1’sutterance;stressservingasanovertfocusmarker.In(4a), itconcernstheagentMary,whereasin(4b)and(4c)denialappliestothepatientBillorthebeneficiarySue.In4(d),finally, the actionofintroducingisdenied,but not thearguments involved.In fact,focussensitivedenialmay applytoany constituent as long as it is somehow put in focus through stress or other types of marking like the (pseudo)cleft construction(cf.ItwasnotBillwhomMaryintroducedtoSue).

TheeffectoffocusincasesliketheseisdescribedindetailinRooth(1985,1992).Inhisview,focushastheeffectof addinganadditionalsemanticvaluetoasentence.

‘Atanintuitivelevel,wethinkofthefocussemanticvalueofasentenceasasetofalternativesfromwhichthe ordinarysemanticvalueisdrawn,orasetofpropositionswhichpotentiallycontrastwiththe ordinarysemantic value’(Rooth,1992,76).

Thesetofalternativesiscomparabletoanequivalenceclass.ThefocussemanticvaluethusresemblesaSaussurian paradigm astheequivalenceclassindicatesa setofalternatives,eachofwhichmayreplacetheactualexpression. Hence,in(4c),thefocussemanticvalueconsistsofsomecontextuallygivensetofwomenwhichincludesSue.In(4)b,it consistsofasetofmales,includingBill,andin(4)dthefocussemanticvalueconsistsofsomealternativeactionswhich MarycouldhaveperformedwithrespecttoBillandSue,suchasmention,handover,andsoon.

AlthoughRooth(1992)doesnotexplicitlydiscussthecaseofanegativemodifier--heisconcernedwithadverbssuch asonly--thenotionoffocussemanticvaluedoesexplainhowtheadverbnotinteractswithfocus.Forexample,in(4b)not selectstheonlymemberofthesetthatthesentenceactuallyexpresses:Bill,whichisthe‘ordinary’semanticvalue.Asa

3_A_parallel_distinction_is_the_one_between_retention_versus_suppression_of_negated_concepts₍_Giora_et_al.,₂₀₀₉_)._If_not_X_is_intended_to_negate_X,

(5)

consequence,referenceisestablishedtoacontextuallyevokedsetofalternativesforthefocusedexpression.Ineffect, focuseddenialcreatesasetandreferstotheabsenceofsomememberfromthe setwithrespecttotheproposition expressed.

Thecriterionwhich wephrasedfor denial--removingpreviouslyintroduced materialfrom thecommon ground --appliesto all focusedcasesin (4).Eachofthe examples wouldbeunacceptable wereitnot for the presenceof a suppositioninthecontextconcerningthepositivevariantofthesentence.Inthesecases,however,onlypartoftheentire utteranceisobjectedto:thefocusedpart.Ifanentireutteranceisdenied,asin(2b),itishardtospeakaboutfocusorfocus sensitivity. In the words of Khemlani et al. (2012):‘‘Intonational focus in a denial acts as a way to associate the interpretationofthefocusedelementofasentencewithnegation,andalmostalwaysservestoreducethescopeofthe negation’’(Khemlanietal.,2012:545).

Toconclude,fourtypesofnegationcanbedistinguished.First,instandardnegationthenegatorhasscopeovereither VPorSandreversesthetruthvalueoftheproposition.Second,inbasiccasesofdenial,thenegatoralsoappliestothe entire utterance, but additionallytakes along the assumption that its contrary was uttered in the context. Third, in metalinguisticdenial,theobjectionisexpressedwithrespecttotheassumedcontraryonawidechoiceofgroundsother thanthecontent.Andfourth,infocussensitivedenial,anequivalenceclassisevoked,asetofalternativesforthereferent oftheexpressiondenied.Theselinguisticallymotivateddistinctionssuggestasetofexpressiverequirements,or‘building instructions’forthementalrepresentation,whichareinstrumentalinbringingaboutnegationordenialinanymodality.An attempttophrasethesebuildinginstructionsispresentedas(5),(6)and(7).Instructions(5)and(6)definestandard negationanddenialrespectively,while(7)definesfocussensitivedenial.4_In₍₅₎_and_(6),_‘p’_stands_for_proposition,_and_in

(7),‘x’standsforentity. Negation:

(5) If thegoalistoexpress‘notp’

Then updatethediscoursemodelbyintroducingp And updatethediscoursemodeladdinganegatortop Denial:

(6) If thegoalistoexpress‘notp’ And pispartofthediscoursemodel

Then updatethediscoursemodelbyerasingp (7) If thegoalistoexpress‘notx’

And somefocussetF:x2Fispartofthediscoursemodel Then updatethediscoursemodelbyerasingx

Notethattheconsequentinstructionsin(6)and(7)arestatedintermsoferasingratherthanremoving,asinvanderSandt (1991).TheconceptoferasurewasadoptedfromGioraetal.(2009)andisintendedtoexpressthefactthatxdoesnot completelydisappearfromthementalmodel.Or,inotherwords,xwillberetainedratherthansuppressed(seeforthis processingdistinctionGiora,2003,2007).

Thebuildinginstructions(5)to(7)allowustospecifythequestionthispaperaddressesasfollows:whatformcanthe buildinginstructions(5--7)takeoninthevisualmode,eitherfornegatingordenyinganassertivesentenceasin(5)and(6) respectively,orfordenyingpartofit,asin(7)?Forthatpurpose,insection2,wewillfirstdiscusstheexpressivepotential ofthevisualmodeingeneral.

3. Canpicturesexpressassertions?

Negationanddenialoperateon(partsof)assertions.Consequently,thefirstquestionrequiringananswerseemstobe: canpicturesexpressassertions?This sectiondiscussessomeoftheargumentsput forth againstorin supportof a positiveresponse.

Thecommonviewonthesideoftheantagonistsisthat,picturescannotbetrueorfalseinthemselves,sincetheyare capableofpresentingonly(cf.Gombrich,1960;Bennett,1974;Eco,1976;Jackendoff,2007;Kennedy,2008;Messaris, 2009).Ifthisistrue,thesearchfornegationordenialinpictureswouldbefutile.Ifnotruthvaluecanbeattributedto

4_Given_the_complex_character_of_{metalinguistic}_denial,_pertaining_to_{presuppositions,}_register,_etc.,_we_will_for_now_ignore_the_possibility_of_visual

(6)

pictures, no truth value can be changed --ergo: there is no pictorial negation. If picturescannot state that some assumptionmadepreviouslydoesnothold,thereisnopictorialdenial.

Theinabilityofpicturestomaketrueorfalsestatementshasbeenarguedonthegroundsofcontextualincompleteness (cf.Gombrich,1960:58--59;Noth,1996).Onlyinconjunctionwithaverbalmessageisitpossibleforapicturetomake assertions.Hence,picturesmaybepartoftrueorfalsestatements,buttheycannotmakethemontheirown.Inthesame vein, ithas been argued that picturescan only functionas predicates (Bennett,1974:263), or as arguments of a proposition(Muckenhaupt,1984:88),i.e.aspartofalargerwhole.

Other argumentsagainst visual negationpertain tothe non-existence ofa visual vocabularyand syntax. Hence picturesdonotallowsemanticdemarcationanddecoding,bothbeingnecessaryrequirementstoexpresspropositions and to establish reference unambiguously (cf. among others Fleming, 1996; Johnson, 2003). Whereas natural languageisaconventionalsignsystem,wherewordscanbecoinedforanythingandeverythingwemightwish,thevisual mode,beingiconicratherthanarbitraryandconventional,hasalessproductivesign-meaningrelation.Fornegation, theresimplyisnovisualsign.Burkegivesusanexplanationforthisomission:‘therearenonegativesinnature(i.e.the visibleworld,eo&js),whereeverythingsimplyiswhatitisandasitis(...)forthenegativeisanidea;therecanbeno imageofit’(Burke,1966citedinLakeandPickering,1998:81).Inaccordance,Khemlanietal.(2012)claimthatnegation cannothaveaperceptualrepresentation--hencecannotbevisualized.Accordingtoallscientistsmentionedhere,there cannotbeacertain(partofan)imageinherentlyexpressingnegation(ordenial).

Argumentstothecontrary,however,canalsobefound.Noth(1996)providestwoargumentsagainstthecontextual incompletenessstancewithrespecttopictures.Asoneconvincingcounterexample,Nothmentionspolicephotoswhichare usedtorepresentanactualstateofaffairs--and‘(...)onlybecausetheyassert[thisstateofaffairs,eo&js]cantheyserveas legalorscientificdocumentsoftruth’(Noth,1996:12).Inaddition,hearguesthatthedifferencebetweenverbalandpictorial messagesisnotoneofcompletenessversusincompletenesssincebothtypesofmessageareinfactincomplete.Ingeneral, naturallanguageexpressionsneedcontextualizationaswelltobecomeassertions,sothisisnottypicalofpictures:‘The differencebetweenverbalandpictorialassertionsisthatthecontextualindicatorsofanassertioninthemediumoflanguage canbeexpressedinthesamemedium,whilethoseofpictorialmessagescannot’(Noth,1996:12).Inthisview,theway picturesmayfunctionincertaincontextsiscomparabletohownaturallanguagepropositionsmayfunctionincontext,andas such theycanbe interpretedasassertions iftheyappear in a propercontext(cf. Blair,2012; Groarke,2009).Asa consequence,thenon-existenceofavisualsignfornegationdoesnotnecessarilyimplythatthereisnovisualization; negationcouldsimplybeestablishedinanotherway,forexamplebysomeoperationthatisuniquelyvisual.

Lookingatthenaturallanguagebuildingrules(5)to(7),theonefornegationin(5)actuallyimpliestheadditionofsome negator.Thismarkerbeingabsentinthevisualmode,weturnto(6)and(7),buildingrulesfortherepresentationofnatural language denials. There, we find that the effect of denial consists in erasing a certain entity from the mental representation.However,incontrastto(6),thefocussensitivedenialin(7)doesnotapplytoanentireassertionandit bringsalongsomecontextuallydefinedset.Forthesereasons,itseemssensibletostartlookingforavisualpendantof focussensitivedenial,obeyingthebuildinginstructionin(7).

For visually expressing this type of denial, there seem to be two important general prerequisites. First, some encompassingwholeFmustbeexpressedinthepicture.Secondly,somesalientpartxofFmustbenoticeablyabsent, butatthesametimereconstructable.Translatingintothevisualdomain,theseprerequisiteswouldbemetbyerasinga certainhighlysalientattributeorentityfromadepictedobjectorscene.Asweshallargue,thereseemstobenoreason alongthelinesofGombrich(1960)orBurke(1966)toprohibitthepossibilityofanerasureoperationinpictures,aslongas itcanbe‘observed’somehowbytheinterpreterandaslongasthereisvisualcontext.Thenextsectiondemonstrateshow visualdenialmaybeeffectedbyapplyingthegraphicoperationerasexonimages.

4. Visualdenial:theerasureoperation

Currentresearchonvisualcommunicationsuggeststhatstillimagescanconveycomplexconceptualstructureslike categorization,analogy,causalityandeventemporalintervals(cf.Forceville,1997;McQuarrieandMick,1999;Maesand Schilperoord,2008).InSchilperoord(2013)itisarguedthatsuchconceptualeffectscanbebroughtaboutbydeliberate invocationofcertaintypesofvisualanomaliesinotherwise‘realistic’dispositionsofobjectsorscenes.Suchanomalous depictmentsarecalledvisualincongruities.ConsiderFig.1--anadvertisementforabrandofcartires.

(7)

Fourtypesofgraphicoperationstocreatevisualanomalieshavebeendiscerned:insertion,substitution,distortionand erasure(cf. Schilperoord, 2013).5Because buildinginstruction(7)as definedfor focuseddenialstipulatesthe main functionofdenialtobetoerasepreviouslyintroducedmaterialfromthecommonground,theerasingoperationseemsa promisingcandidateforestablishingvisualdenial.

Thissectionisolatestwogeneraltemplateswhichmayservetovisuallyexpressdenial.Thedistinctionadherestothe generaldistinctionbetweenschematicandcategoricalorganizationofinformationincognitivetheoriesofmemoryand informationretrieval (cf. HudsonandFivush,1983; Mandler,1978;Shen, 1999).While section 4.4discussesthe possibilityofcategoricallyorganizedimages,sections4.1--4.3specifythreetypesofschemasthatmayservetoprovide therequiredcontextforsomeomittedpart.Thesethreekindsofschemascorrespondtothreebasicunitsofhuman experience:objects,scenesandevents.Inordertoinvestigatetheuseofinstruction(7)forthevisualizationsofthese unitsofexperience,weneedtoslightlyamendit.Objects,scenesandeventsarenotcommonlythoughtofassetsof entities.Usually,therelationbetweenanobjectlikeahumanfaceanditsconstituents:eyes,eyebrows,mouth,nose, hair,etc.,isanalyzedasapart-wholerelation.Thisperspectivecanbearguedinmereologicalterms.AsstatedinVarzi (1996)‘‘Asaformaltheory,mereologyissimplyanattempttosetoutthegeneralprinciplesunderlyingtherelationships betweenawholeanditsconstituentparts,justlike settheoryisanattempttosetouttheprinciplesunderlyingthe relationshipsbetweenaclassanditsconstituentmembers’’(Varzi,1996:260).Since,indeed,entitieslikeeyes,mouth andnoseareconstituentsratherthan(similar)elementsofaface;wewillhenceforthrefertoFasa‘whole’,whichserves asalocalcontexttodetectsomemissingpartX.Hence,buildinginstruction(7)canberevisedas(8),where‘‘’’stands fortherelation‘‘inclusion’’or‘‘properpartof’’.

(8) If thegoalistoexpress‘notX’

And somewholeW:XWispartofthediscoursemodel Then updatethediscoursemodelbyerasingX

4.1. Schematicallyorganizedimages:object-baseddenial

ConsiderFig.2.Itshowsanexample of therhetorical operationeraseapplied toan imagedepictinga certain object:Incongruityisestablishedbytheerasingofamouthinthewoman’sface.Theaccompanyingcaptionreads: Don’tbewithoutatopic.Readbooks.Theadvertisementaimsatraisingtheaudience’sawarenessoftheimportance ofliteracy.

IfFig.2isapproachedasifresultingfrombuildinginstruction(8),accordingtothesecondcondition,thereshouldbe somewholeW:XW,apartofthediscoursemodel,for(8)tobeapplicable.Thisconditionismetifthefaceisinterpreted asthewholeWandthemouthasitsproperpartX.However,(8)doesnotsufficeasarulefordenial.ThegirlinFig.2does notwearglasses --doesthatmeanthatwemightaswellinterpretthe glassesasbeingvisuallynegated?Withouta contextmakingglassesrelevant,thisisnotthecase.Consequently,foranobjecttofunctionasW,ithastoobeycertain conditions.Firstly,theobjectinquestionmusthaveidentifiableparts.Inaddition,theabsenceofsomeidentifiablepartX

Fig.1. Insertion.

5

(8)

willhavetoattracttheattentionoftheviewerbecausethispartisexpectedor,inotherwords,predictable.Therefore,an objectimagemaybesaidtofit(8)asWif(atleastsomeof)itspartsarepredictable.

Todetermineinwhatcasepartsarepredictablefromwholes,weturntothefieldofvisualrecognition.Inthisfield,the conceptofdiagnosticityhadbeencoinedtosingleoutthekindofattributesthatarecrucialforrecognizingacertainobject (seeSkowronskiandCarlston,1986;Deane,1992;OlivaandSchijns,1997).Ingeneral,anentityXisconsideredapartof someobjectOifXisanattributeofO,butnot(necessarily)predictablebymeansofO.Ontheotherhand,Xisconsidered a diagnosticpart if its presenceis actually predicted byO. Consequently, if X isa diagnostic part of O, X will be experiencedasmissingifitisomittedinavisualrepresentationofO(providedasuitableviewingpointisoffered).The representationofahuman,mouthlessfaceinFig.2meetsthecriterion:X,themouth,ispredictablefromW,theface, whereasapairofglasseswouldnotbe.

Consequently,toproduceobject-basedpictorialdenial,(8)isspecifiedas(9),whereOmeansObjectand‘‘p’’stands for‘‘predictableproperpartof’’.6

(9) If thegoalistoexpress‘no/notX’

And someObject:XpOispartofthediscoursemodel Then updatethediscoursemodelbyerasingX

IsFig.2actuallyintendedtoexpressdenial?Inotherwords,theconditionsinthesecondandthirdlinesof(9)arerealized byFig.2--butisthefirstaswell?Istherereasontoqualifythefigureasanintendedvisualdenial?Therecertainlyis:ifwe donotinterpretthefigureasexpressing‘‘thisgirlhasnomouth’’,thereisnowaytoobtaintheintended,metaphorically derivedinterpretation:‘‘thisgirlhasnothingtosay’’--orasthecaptionputsit:‘‘(thisgirlis)withoutatopic’’.7Consequently, the denialis aprerequisite for the interpretation of the advertisementas a whole. Inthis case,as in theexamples discussedinfollowingsections,aviewerwillinterprettheimageasexpressingdenial,buts/hewillonlybeabletoexplain the functionofthedeniedentityinterms ofthe messageasawhole. Weconcludebyphrasingtheexpectation that subjectsconfrontedwithanimagelikeFig.2willinterpretitintermsofdenial.8

Fig.2. Object-baseddenial.

6_The_second_and_third_conditions_in₍₉₎_presuppose_that_a_mental_{representation}_of_some_object_O_contains_its_predictable_parts_(cf._Deane,

1992).

7_The_image_in_Fig.₂_could_just_as_well_serve_a_campaign_to_promote_freedom_of_speech_or_democratic_elections._In_such_cases,_the_image

servesasanargument.

8

(9)

4.2. Schematicallyorganizedimages:scene-baseddenial

Asecondtypeofschema-basedvisualdenialoperatesonknowledgeofnaturalscenes.Scenerecognitionisoneof themainconcernsoftheoriesofvisualperception.HendersonandFerreira(2004)offerthefollowingdefinition.

‘[Anaturalsceneis]asemanticallycoherent(andoftennameable)human-scaledviewofareal-worldenvironment comprisingbackgroundelementsandmultiplediscreteobjectsarrangedinaspatiallylicensedmanner’(Henderson andFerreira,2004:5)

Biederman(1982)hassubmittedamodelthataccountsforscenecoherencyintermsof(only)fivetypesofrelations betweenanoverallsceneontheonehand--i.e.thewhole--anditsattributesandcharacteristicsontheother--theparts. Theserelationsinvolvesupport;mostobjectscannotfloat,orplace;certainobjectsoftenoccupypredictablelocations withinascene.AnotherrelationdistinguishedbyBiederman,probability.referstothe(un)likelinessthatacertainobjectis partofacertainscene,andcanthusbeconsideredthesceniccounterpartofdiagnosticobjectattributes.Thisrelationis especiallyrelevantforourpurposes:whereastheoperationofinsertionhastheeffectofplacinganobjectinattentional focusbecauseofitslowprobability(seethedraininthecountryroad,Fig.1),erasureoperatesintheoppositedirection: theerasedobjectisrenderedsalientduetoitshighlyprobablepresence.ConsiderFig.3,showinganadclaimingthatno waterisneededifoneusestherecommendedrinsefreehandcleansinggel.

Thepictureexpressesthismessagebyhavingthehighlyprobablefaucetremovedfromthewashbowl.Thegeneral formatofscene-basedpictorialdenialcanbephrasedas(10),whereSstandsforscene,andXpSindicatesthatX constitutesapredictablepartofS.Althoughprobabilitymaybeslightlyweakerthanpredictability,‘highlyprobable’becomes predictable.

And somesceneS:XpSispartofthediscoursemodel Then updatethediscoursemodelbyerasingX

4.3. Schematicallyorganizedimages:event-baseddenial

Event-baseddenialoperatesonknowledgeofevents.Unlikeobjectsandscenes,eventsaredynamicbynature,which entailsthatperceivingacertainactionallowsviewerstoconstructeventstructurespecifyingacausal-temporalsequence of actions -- the whole -- of which the perceived action is a part. Event schemas can be specified as complex representationsthatcaptureactors(whoareinvolvedintheevent?);objects(whatkindsofobjectsareused?);causaland hierarchicalrelationsamongsubevents(whatisdonetowhatend?);temporalsequences(whatstagescanbediscerned, whatcomesbeforewhat?);andspatiotemporalaspects(wheredoestheeventusuallytakeplace,howmuchtimewillit take?--seeforexampleZacksandTversky,2001;KurbyandZacks,2008;RadvanskyandZacks,2011).Apicturemay shownomorethanasliceofaneventbutduetotheevokedeventschema,thevieweriscapableofinferringawhole

(10)

sequenceofactions,actors,andtheirconceptualandtemporalrelations.Fig.4presentsanadforaGymschool.The claimedmuscularstrengthonegainsbyvisitingthegymisillustratedbyamancarryingacouchonhisown.

AccordingtoZacksandcolleagues,thedynamicsofeventknowledgecanbecapturedintermsofourabilitytopredict bothupcomingevents,andthepresenceofactorsandobjectsfromtheperceptualcuesasprovided.Imaginethepicture in(4)showntoaviewer,whilebeingcoveredbyasheetthatismovedslowlytotheright(seeFig.4a--c).

Initially,scenicknowledgeofasuburbangardenwillbeactivated(4a).Subsequently,however,theperceptionofaman carryingacouchurgestheviewertoactivateschematicknowledgeoftheeventof‘carryingacouch’(4b),whichrendersthe presenceofasecondactorcarryingtheothersideofthecouchhighlypredictable.Predictionsthatarenotborneout(thereis nosecondactor)causethecognitivesystemtomonitoranevent-basedincongruity(4c).Ratherthanadjustingtheschema (‘apparentlyit’spossiblethatonepersoncarriessuchacouch’),theviewerwilldecidethatsomethingisostensiblyabsent --or:omitted.Consequently,presentedwithFig.4c,theviewerwillstartoutbyregisteringnosecondman.

Theintendedmeaningofthepicturein(4c)willremainobscure.Onlyiftheviewerisconfrontedwiththeentiread(4c), includingthebrandmark,themeaningoftheabsentsecondmanmaybecomeclear:heisnotrequiredforcarryingthe couch,sincethemanthatispicturedhasgrownmusclesintheGym.

Likescene-baseddenial,event-baseddenialcanbebroughtaboutbyomittinganactororobjectwhosepresenceis highlypredictableonthebasisoftheactivatedeventschema.Thegeneralbuildinginstructionsforeventbasedvisual denialcanbephrasedas(11),whereEstandsforevent.

And someschemeeventE:XpEispartofthediscoursemodel Then eraseXfromthediscoursemodel

Summarizing,visualincongruitiesexperiencedasomissions,canbeenvisagedasbeingcreatedbyerasingsomepartx fromapictureofanobject,asceneoranevent.Thexisrenderedsalientifitconstitutesapredictablepartofan object-schema,ascene-schemaoranevent-schema.Theseomissionsinvokebuildinginstruction(9)to(11),containingan instructiontoerasesomething,andhencetheyaresupposedtobringaboutdenial.

4.4. Categoricallyorganizedimages:juxtapositionasabasisfordenial

Anotherpotentialapplicationoferasureresultinginanoticeablyabsent,oromitteditemcomesclosertotheoriginal rule(7),asitseemstoinvolveatruefocussetF.ThisapplicationisexemplifiedinFig.5whichshowsanadvertisement thatwaspublishedshortlybeforethe2008OlympicGamesinChinainordertoraiseawarenessoftheviolationofhuman rightsofthehostingcountry.

(11)

TheupperrowdepictsacoupleoftypicalWesternconsumerproducts,eachofwhichformsacolumnwithaChinese copyofit,exceptfortherightmostitem.Thisitemisthereforeexperiencedasmissing:thereisnocopyofthemanifestof humanrights.9_The_intended_{interpretation}_hence_involves_denial._In_this_section_we_argue_that_this_template_for_visual

denialcomesclosetoRooth’snotionoffocussemanticvalue.Thebuildinginstructionforthistypeofvisualdenialisthe unadjustedversionof(7),repeatedhereas(12).

(12) If thegoalistoexpress‘notX’

And somefocussetF:x2Fispartofthediscoursemodel Then updatethediscoursemodelbyerasingx

TosubstantiatetheclaimthatFig.5instantiates(12),wefirstfocusonthefocussetF,and,secondly,onthewayinwhich theawarenessofaremoveditemisbroughtaboutinthiscase.

ThepictorialdesigntemplateemployedinFig.5isreferredtoassymmetricobjectalignment(seeSchilperoordetal., 2009;TengandSun,2002).Thetemplateemploysobjectjuxtaposition,i.e.,thedetachmentofasetofobjectsfromtheir familiarenvironmentandbyshowingthemsidebyside,withinasingle,mostlyneutralpictorialplane.Alignmentcanbe createdbyemployingtheobjects’sizeorshape,thespatialorientationofdepictingthem,equaldistancestowardeach other,projectionalonganaxisandthelike.InFig.5,alignmentisperceptuallycreatedbyshowingtheobjectsassimilarly sized,depictingthematequaldistances,andbyusingsimilarcolors(red,silver).Despitetheirobviousdifferencesthe perceptualcuesprovidedbythevisualtemplatesuggestssomekindofobjectsimilarity-basedontheprimaryincentive closeness is similarity (cf. the clumpiness principle, Casasanto, 2009). This, in turn, might encourage viewers to conceptuallygrouptheobjectsforexamplebycreatingtheadhoccategoryproductsofacivilizedculture(cf.Barsalou, 1983;vanWeeldenetal.,2011,2012).ThisthenmightexplainthewayobjectalignmentinFig.5managestoevokea setF.

InFig.5,groupingfunctionsbothhorizontally and vertically.Theupper rowin the imagehorizontally evokes the (adhoc)equivalenceclassproductsofacivilizedculture.Theverticalcolumnsrepresentasecondequivalenceclass,of almostidenticalitems:theproductsascopiedbyChina.Asaconsequence,theemptyslotintherightmostpositionisnot ‘justempty’:thebookletisinfocusandnoticeablyabsentfromthelowerequivalentclass,whereitisexperiencedasbeing omitted.Insum,thealignmenttemplatesuggeststheomittedentitytobeanelementofanequivalenceclass--andat thesametimedeniesitsexistenceasfarasChinaisconcerned.Thefocussemanticvalueinthiscaseconsistsofthe contextuallygivensetofcommoditiesincludingahumanrightsdocument.10

4.5. Summary

Wehaveexploredsomepossibilitiesofvisualdenialstartingfromthethird‘buildinginstruction’forfocussensitive linguisticnegation.Linguisticnegationmarkerslikeno ornot canbeinterpreted asinstructionsfor erasingfrom the discoursemodelanentityxbeingpartofanequivalenceclassF.Twogeneralvisualtemplateswhichseemtoqualifyas

Fig.5. UnfortunatelyChinadoesnotcopyeverything.

9_This_{interpretation}_is_further_{substantiated}_by_the_caption:_‘China_doesn_’t_copy_everything_’.

10_Obviously,_human_rights_are_not_commodities,_or_an_industrial_product._But_at_the_appropriate_level_of_abstraction,_the_analogy_becomes_clear:

(12)

visualpendantsofthelinguisticcasehavebeenidentified:schema-baseddenial,broughtaboutbyomittingobjectsthat areexpectedtobepresentaspartsofawhole(anobject,asceneoranevent),andcategoricaldenial,broughtaboutby omittinganelementofanequivalenceclass,establishedbyjuxtaposition.Next,weturntoempiricalmatters. 5. Empiricalstudy

5.1. Introduction

Theaimoftheexperimentthissectionreportsonistoprovideafirstandinformaltestforthe hypothesisthatthe operationerasureappliedtoimagesliketheonesinFigs.2--5producesanegationinterpretationasdiscussedinthe formersections.Thematerialsusedtoanalyzepictorialnegationcomefromalargecorpusofpicturesfromadvertising messages.Suchimagesoftenservetoexpresscomplexmessagesthatviewershavetointerpretratherthanrecognize. Designersofthoseimagesoperateattheforefrontofexploringthepossibilitiesofvisualcommunication.However,our claimsprobefurtherandarenotlimitedtothisparticulargenreofvisualcommunication.Theyextendtoothervisual genreslikecartooning,comics,visualarts,filmorinstruction.

Threequestionswereaddressedintheexperiment:

(i)aretheimagesresultingfromerasureinterpretedintermsofnegation? (ii)doespragmaticcontextaffecttheinterpretation?

(iii)doestheoperationsubstitutionhaveasimilareffectoninterpretation?

Inthispaper,wefocusonthefirsttwoquestions.Togetsomepreliminaryanswers,weshowedsubjectsseveralofthe kindofimagesdiscussedintheformersections,andaskedthemtointerprettheimageswhilethinkingoutloud(cf.Ericson andSimon, 1984).Thedataobtainedinathinking-out-loudexperimentrequiresystematicinterpretation,whichmust proceedaccordingtoapre-establishedprotocol (seesection4.2).Byconsequence,theresultingdataprovideafirst validationofpredictedmentaloperationsandtheiroutcomes--buttheydonotallowtoformallytesthypotheses.Tothat end,theresultscanbeusedtosetupsystematicexperiments.

The images thatwere usedeither did ordid not containsigns of theirpragmatic function,i.e. expressingsome advertisingclaim.Infact,halfoftheparticipantsweredeniedtheknowledgethattheimageswerepartofadvertising messagesbyremovingallsignsofcommunicativeintentionorsender(textcaptions,brandnamesandsoon).Theother halfsawtheoriginalImages--i.e.aspartofacertainadvertisement.Asaconsequenceofouranalysesinsection3,we predictedthatallsubjectswouldinterprettheincongruitiesintermsofnegation,and,moreover,thatonlythesecondgroup ofsubjectswouldbeabletoexplainhowthenegationfunctionedwithintheoverallmessage.Inwhatfollows,the images-as-imageswillbecalled‘nonad-versions’,whiletheimages-as-advertisementwillbecalled‘ad-versions’.

5.2. Method

Thedatasamplingmethodemployedisknownasthinking-out-loud(EricsonandSimon,1984).Themethodrequires subjectstoverbalize,asspontaneouslyaspossible,everythoughtthatpopsupintheirmindswhileperformingonsome cognitively demandingtask, such as interpretinga certain image.Thereports provided are transcribed, resultingin protocolswhicharebelievedtoreflectrelevanttracesoftheonlinementalprocessesofpeoplewhileperformingonthe taskathand(cf.EricsonandSimon,1984).

Subjects(n=24, meanage22.2,7males)wererecruitedfromthe studentsubjectpooloftheTilburgInstitutefor CommunicationandCognition.Theyreceivedstudypointsascredit.

Materials. Eachsubject sawnine images:six of thesecontaining schematically organized omissions and three categoricallyorganizedomissions.11Theywereallliketheoriginals,butshowneitherwithout(non-ad-versions)orwith (ad-versions)verbalcaptionsorbrandnames.OneofthesewastheGymad(Fig.4).Thevariantsusedintheexperiment areshowninFig.6aandb.

Procedure.Eachsubjectwastestedindividuallyinaquietroom,andinthecompanyofoneexperimenter.Priorto testing,subjectswereinformedthattheyweregoingtoseeasetofpicturesthattheyhadtointerpret.Tofamiliarizethem withvisualincongruities,theywereshownanexampleofavisualincongruitynotbasedontheerasureoperation.After

11_The_latter_three_images_were_meant_to_address_research_question_(iii)_above,_and_were_shown_either_in_the_original_version_(for_example_the

(13)

that,theyreceivedtheinstruction(inDutch):‘Inamoment,youwillbeshownseveralpicturesliketheexampleyoujust saw.Explainwhatyouseeinthatpictureanddescribewhat,inyouropinion,thispictureistryingtotellyou.Whatdoesit mean?’Inaddition,theinstructionurgedthesubjectstoverbalizeeverythoughttheyhadwhiletryingtomakesenseofthe images.Next,theywerebrieflytrainedinthethinking-out-loudtechnique,usingtheso-called‘Tower-of-Hanoi’problem (seeAnderson, 1983).12 _After_this _training_session,_the _experiment _started. _The_test _images _were_presented _to _the

subjectsforaslongastheywanted,usingapowerpointpresentationandanormalcomputerscreen.Duringtheentire sessions,recordingsweremadeoftheirthoughtprocesses.Theexperimenter’sonlyrolewastoencouragesubjectsto keepontalkingwhileprocessingthepictures.13Therecordingswerelatertranscribedverbatimandsubjectedtofurther analysis.

5.3. Dataanalysis

Therecordingswereanalyzedaccordingtothefollowinganalyticalprotocol.First,theyweresegmentedintosimple clauses,consideredtobethegrammaticalandpragmaticconcordanceofasingleunitofthought(cf.Schilperoordand Verhagen,1998).Theclauseswerefurthergroupedintoclustersreflectingthefollowinginterpretationstages: (13) i. registering(describingtheimage)

ii. diagnosing(describingtheincongruity)

iii. integrating(makingsenseoftheimageandtheincongruity)

Subsequently,allsegmentswerecodedforthepresenceof(Dutch)negationmarkers,suchasthelexicalmarkers‘geen’ (no),‘niet’(not),andthemorphologicalmarker‘on-’(un-).Inaddition,segmentswerecodedforthepresenceofverbs indicatingabsence,suchas‘ontbreken’(lack),‘missen’(miss)or‘verwijderen’(remove,erase).

5.4. Results

TheresultswillbediscussedwithreferencetooftheGymadvertisement(seeFig.6aforthenon-adversionandFig.6b forthead-version).

5.4.1. Nonad-version

Consider protocol (14).The first column containslinenumbers, the secondone stagecoding, the third one the presence(1)orabsence(0)oflinguisticnegationmarkers,andthefourthonetheprotocolsegments.Negationmarkers (andothercluesthatweconsiderrelevant)areboldfaced.

Fig.6. (a)Withoutbrandname,and(b)withbrandname.

(14) 1 i 0 Iseesomeonecarryingacouchintothehouse 2 ii 0 Butheiscarryingitbyhimself

3 ii 0 Ontheothersidethereoughttobeanotherperson

12_Detailed_descriptions_of_this_procedure_and_{justification}_of_its_various_stages_{(instruction,}_practice)_are_provided_in_van_Someren_et_al.₍₁₉₉₄₎_. 13

(14)

Inthefirstfourlines,protocol(14)representsthestages(i)and(ii),ofdescribingboththeimageandtheincongruity, i.e.themissingperson, usinganegationmarker(line4).Thelines 5to9convey thesubjectsinabilitytoarriveata coherentinterpretation.Protocol(15)hasapatternsimilartotheonein(14).

Apartfroma markerofabsence(‘missing’),theprotocolcontainsanalternativeclueindicatingthatthe participantis monitoringthe anomalous absenceof thesecond actor.Inline4,herefersto theabsentperson, usingthe definite determiner‘de’(the).Asecondexampleoccursinline9.Definitedeterminersareconsideredmarkersofhighaccessibility of the intended referent(cf. Ariel, 1988). In a given stretch of discourse, the useof a definite determiner usually accompaniesasecondorthirdreferencetotheentitydenotedbythenoun.Becauseline4marksafirstreferencetothe ‘missing’actor,theuseofthedefinitedeterminerseemstosignalacrucialpropertyofdenial:itsfocus-sensitivity.

Sometimes,subjectsevenusepronounstorefertothemissingsecondactorwithouthavingintroducedthempriorto thisreference.Seeprotocol(16),line6.

Notethatthepronouninlines1,2,and5referstothepersonthatcanactuallybeseen,whereasthepronouninline 6referstothe‘absent’person--i.c.theomission.Despitetheinherentlyambiguousnatureofpronouns,thisparticipant referstoadistinctactorusingthesamepronounashedoesinlines1to3.Again,thistestifiestothehighlysalientcognitive statusofthereferredentities.Thehighaccessibilityofthereferentmayalsobeinferredfromtheuseoftheordinalnumeral (15) 1 i 0 Abungalow

2 i 0 Ahousewithamoverwhoiscarryingacouchinside 3 ii 1 Butheiscarryingitonhisown

4 ii 1 Andthesecondmanismissing 5 iii 0 It’sprobablyanadformanpower 6 iii 0 Thebrand...Iwouldn’tknowit

7 i 0 Thatmanjustcarriesthecouchonhisown 8 i 0 Thatmanliftsthecouchonhisown

9 ii 1 Anditlookslikethesecondpersonhasbeenremoved 4 ii 1 Butheisnotthere

5 iii 0 Thereisnoproductnameorsomething 6 iii 0 SoIdon’tknowwhatthisisabout 7 iii 0 Perhapsitisaboutthecouch 8 iii 0 Becauseyouseeitinthecenter

9 iii 0 Oraboutmoverswhoaresogoodthattheydothejobalone

(16) 1 i 0 Hewalksalongthegardenpath 2 i 0 Andhecarriesahugecouch

3 ii 1 ...whichnormallyhecannotcarryonhisown 4 ii 1 Well,that’snotgoingtohappen

(15)

‘tweede’(thesecondman)incombinationwiththedefinitedeterminer:seeprotocol(15),line4.Iftheeventschemaof carryingindeedallowsforthepredictionoftwoactors,theordinalnumeralonceagainsignalsthesaliencyofthatconcept. Thefactthatsubjectsnoticethatthesecondactorisremovedfromthesceneratherthanjustbeingabsent,isfurthermore suggested,inprotocolsnotcitedhere,bytheuseofverbslike‘toleaveout’,‘erased’,‘to‘‘out-Photoshop’’’andsoon. Inthenon-adcondition,noparticipantcanmakesenseoftheincongruity(c.f.(13)stage(iii)),justaspredicted.Inthe protocols(14),(15),and(16),thereisevidenceoftheparticipants’inabilitytoexplainwhythereisno‘secondman’.In protocol(14)line5,andinprotocol(15),line6,theabsenceofabrandnameismentioned,aswellastheconsequential problemwithexplainingtheanomaly.Otherwaysofmarkingthefailuretounderstandtheimagesare‘Idon’tknow’;‘Ihave noideawhatthisimageisabout’andsoon.Notethatallmarkerssignalingtheinabilitytoexplain(stage(iii)in(13))were producedaftermonitoringtheabsenceofasecondperson(stage(ii)in(13)).

Someparticipantsmadeattemptstoresolvetheincongruitythattheynoticedbyexpressingcertainexpectationswith regardto thecouch, ratherthantothe secondactormissing.Again,negationmarkers areusedtoarrive atsuch a resolution(see17).

Similarpatternswerefoundfordifferentads. 5.4.2. Ad-version

Resultsfromthead-versionfurthertestifytotheclaimarguedinthispaper:allsubjectswouldinterprettheincongruities intermsofnegation,butonlysubjectsintheadconditionareabletoexplainhowthenegationfunctionswithintheoverall message.Considertheprotocolin(18).

Inthisrecording,negationoccursasearlyasinline1andisrepeatedinline3,whereasthebrandisnotreferredtobefore line4.Assoonasthebrandnameisregistered,thesubjectisabletostartinterpretingtheadvertisement(lines5,6and7). This pattern, regularly occurring in the protocols resulting from the ad version, suggests the predictedprocessing sequencein(13):(i)monitoringtheanomaly;(ii)interpretingitintermsofnegation;and(iii)markingtheinabilitytoexplain thenegation(non-adcondition)orintegratingthenegatedentitywithintheoverallmessage(adcondition).Evidently,this sequencerequiresfurtherresearch.

6. Conclusion

Theoutcomesofthisstudysuggestthatvisualincongruitiesbasedonremoveevokethekindofprocessingreflected bythebuildinginstruction(8).Mostparticipantsinterpretedtheimagesasintendingtocommunicatetheabsenceofsome saliententity(clauses1and2in8)andtheyregularlyreflectedbeingawareoftheomission(clause3).Inthenon-ad condition,themajorityofthesubjects(foreachad,10to11outof12)didinterprettheimageasnegatingsomeentity;they (18) 1 Ii 1 Someoneislacking

2 i 0 Iseesomeonecarryingacouch 3 ii 1 Butonepersonisnotthere 4 ii 0 GoldGymIt’saGym

5 iii 0 Andthatmanisamemberofthatgym

6 iii 0 Andheissostrongthathecancarrythatcouchonhisown 7 iii 1 Heissostrongthatthesecondpersonisnolongerneeded (17) 1 i 0 Yes,Iseeahouseandalawn

2 i 0 Amancarryingacouchononeside 3 iii 1 WellIguessthatcouchisnotthatheavy

(16)

usedovertnegationmarkerstoreflectthisinterpretation.However,theywerenotabletosolvetheincongruity.Onthe other hand, subjects processingthe ad-versionswere actually ableto explain the negated element in terms ofthe advertisement’sclaim.Still,thegroupofsubjectsallowedtoseethebrandnamealwaysstartedwithdescribingand diagnosingtheimageitself,usingnegationjustlikethesubjectsinthenon-adconditiondid.

7. Discussion

We confined ourselves to the visual pendant of focus sensitive, narrow scope denial, as captured by building instruction(7).Is itpossibletoset thepicturesdiscussedin thepaper apartfrom ‘pure’ casesof negation(seethe examplesin1)?Thedistinctivefeatureofdenialwasdescribedinsection1as:thesuppositionofthecontraryofwhatis deniedincommonground.Thelinguisticcases(2b)and(4)necessitatesomepreviousutteranceofthesuppositiontothe contrary.Itis,forexample,communicativelyawkwardtosay‘MarydidnotintroduceBilltoSue(butJanedid)’ifnospeaker hasmadetheclaimthatMarydid.Ifweweretoapplythisrequirementtovisualexpressions,visualdenialswouldnotbe possible.Evenifweconsidersequentialvisuallanguage(comics)withanimagedepictinganentityxandasecondonein whichxismissing,viewerswouldnotinterpretthesecondimageasdenyingthefirstone.Amorelikelyreadingwouldbe ‘xhasgone’of‘xhasbeenremoved’.Inotherwords,suchpairsofimagesdonotproduceincompatible‘statements’. However,ifwereplacethe‘suppositionofthecontrary’bythesuppositionthatanabsententityXisapredictablepartof somewholeY,and,consequently,thatXisexpectedtobepresentwheneverYis,thenalladvertisementsdiscussedin thepaperarecasesofvisualdenial.

Ontheotherhand,thefactthatthereisnopreviousassertioninthevisualmodecanbetakentoindicatethatourcases of‘picturessayingno’arevisualinstancesofnegation.Theobservationthatanegationinvokesitspositivependanthas beenmadeforthe linguisticmodeaswell,witnessthepolyphonyapproachdevelopedbyOswaldDucrotand Jean-ClaudeAnscombre.Insum,wecanseenogroundfordecidingwhetherthecasesdiscussedinthispapershouldbe consideredvisualnegationorvisualdenial.

ContrarytoWorth’slament,theanswertothequestioncanpicturessay‘‘no’’?seemstobe‘Yes,inspecificcircumstances picturescansay‘‘no’’’.Thepresentpaperhaspresentedtwotypesofpictorialassemblagecapableofexpressingnegation/ denial:schematicallyorganizedimagesandcategoricallyorganizedimages.Wearguedthatthepresenceofacertainentity canbevisuallydeniedbyapplyingtheoperationerase.Acrucialprerequisiteisthattheimagemustallowviewerstonoticean omission.Oncethisconditionismet,thetemplatesactuallyinstantiatethebuildinginstructionsin(7)and(8).

Generalizing,wecanconcludethattheessenceofvisualnegation/denialresides,somewhatparadoxically,inrendering anobjectorattributehighlysalientbynotshowingit.Thatis,to‘mention’itwithoutmentioningit.Apparently,Khemlanietal. wereslightlytoopessimisticwhenstatingthatnegation‘‘(...)cannothaveaperceptualrepresentation,suchasavisual image(...)’’(Khemlanietal.,2012:546).Thefactthatnegationcanbeperceptuallyrepresented(i.e.asanimage)couldbe furthersubstantiatedifwewouldbeabletodemonstratethatKhemlanietal.’sdefinitionofnegationappliesequallytocases ofvisualnegationdistinguishedinthispaper.Theirdefinitiontakesnegationtoconstitutea‘(...)functionthattakesasingle argument,determinedbyscope,whichreferstoasetofmodels.Thecoremeaningofnegationisafunctionthatreturnsthe complementoftheset.’(Khemlanietal.,2012:545).14_The_argument_taken_by_a_sentential_negation_function_is,_obviously,_the

non-negatedsentence,and themodelsreferredtoarethoseinwhichthe non-negatedsentenceholds.Applyingthis definitionofnegationtotheChinaadvertisementinFig.5,theargumentofthenegationfunctionwouldbethesetofmodels containingtherelevantentityF0.F0holdsinmodelsliketheonecontaining:A,B,C,D,E,F,A0,B0,C0,D0,E0,F0,aspicturedin (19a),butalsoinmodelslike(19b)and(c),whereA,BandA0,B0,etc.representthedepictedobjects.

(19) a A B C D E F A0 B0 C0 D0 E0 F0 b C D E F C0 D0 E0 F0 c F0

14_The_negated_expression_can_be_of_several_types,_witness_a_different_formulation_of_the_definition:_‘‘the_core_meaning_of_negation_refers_to_the

(17)

Thesetofmodelsresultingfromthenegationfunctionformsthecomplementofthissetofmodels.Eachofthesemodels maycontainanyoftheitemsexcept,ofcourse,F0.Oneofthesemodelswouldinfactbe(20).

(20) A B C D E F A0 B0 C0 D0 E0

Since(20)isaschematizedversionoftheactualadvertisingimage,theconclusionhastobethatitispartofthesetof modelsresulting from the negation functionand so the image should,according to Khemlaniet al.’s definition,be interpretedasnegation.Andbecausenolanguageisusedhere,butonlyvisualentities,itshouldbeinterpretedasvisual negation.

Thereremainsthequestionconcerningscope:isthevisualmodesuitableforsmallscopenegationonly?Considerthe quiteintriguingimageinFig.7.

Whereasallearlierexamplescouldbeanalyzedasinstantiatingthebuildinginstructionsforfocussensitivedenial,and allconcernedanentitythatwaspartofacertainobject,sceneorevent,thisimageseemstodenyafull-fledgedevent structure[PLAY[BRASSBAND,INSTREET]].Itdoessobyindexicalmeans:shadowsaretheeffectscausedbyentities.We concludebynotingthatmuchremainstobediscoveredaboutlimitationsandpossibilitiesofthevisualmode.Inaddition, wehopetohavedemonstratedthe fruitfulnessofsuchresearchbeingguidedbylinguistictheoriesof meaning.Not becausethevisualandtheverbalmodesareidentical--butbecausetheyhaveafascinatingthingincommon:expressive creativity.

References

Anderson,JohnR.,1983.TheArchitectureofCognition.MITPress. Ariel,Myra,1988.Referringandaccessibility.J.Linguist.24(1),65--87. Barsalou,LawrenceW.,1983.Adhoccategories.Mem.Cogn.11(3),211--227. Bennett,JohnG.,1974.Depictionandconvention.Monist58,255--268.

Biederman,Ian,1982.Onthesemanticsofaglanceatascene.In:Kubovy,M.,Pomerantz,J.R.(Eds.),PerceptualOrganization.Lawrence Erlbaum,Hillsdale,NJ, pp.213--263.

Blair,Anthony,2012.Thepossibilityandactualityofvisualarguments.In:Blair,J.A.(Ed.),GroundworkintheTheoryofArgumentation.Springer Science+BusinessMedia.

Burke,Kenneth,1966.LanguageasSymbolicAction.UniversityofCaliforniaPress,Berkeley.

Casasanto,Daniel,2009.Whenisalinguisticmetaphoraconceptualmetaphor? In:Evans,V.,Pourcel,S.(Eds.),NewDirectionsinCognitive Linguistics.Amsterdam,Benjamins.

Deane,PaulD.,1992.GrammarinMindandBrain.ExplorationsinCognitiveSyntax.MoutondeGruyter,Berlin. Eco,Umberto,1976.ATheoryofSemiotics.IndianaUniversityPress,Bloomington.

Ericson,K.Anders,Simon,HerbertA.,1984.ProtocolAnalysis:VerbalReportsasData.MITPress,Cambridge,MA. Fleming,David,1996.Cantherebevisualarguments? Argument.Advoc.33,1--20.

Forceville,Charles,1997.PictorialMetaphorinAdvertising.Routledge,London.

Giannakidou,Aanastasia,2004.LaurenceHornAnaturalhistoryofnegationreviewedbyAnastasiaGiannakidou.J.Linguist.40,426--433. Giora,Rachel,2003.OnOurMind:Salience,ContextandFigurativeLanguage.OxfordUniversityPress,NewYork.

Giora,Rachel,2007.‘‘AgoodArabisnotadeadArab--aracistincitement’’:ontheaccessibilityofnegatedconcepts.In:Kecske’,I.,Horn,L.R. (Eds.),ExplorationsinPragmatics:Linguistic,CognitiveandInterculturalAspects.MoutondeGruyter,Berlin, pp.129--162.

(18)

Giora,Rachel,Heruti,V.,Metuki,N.,Fein,O.,2009.‘‘Whenwesaynowemeanno’’(Decker2007)(emphasisadded):interpretingnegationin visionandlanguage.J.Pragmatics41,2222--2234.

Gombrich,E.H.,1960.ArtandIllusion.Phaidon,London.

Grice,Paul,1975.Logicandconversation.In:Cole,P.,Morgan,J.L.(Eds.), SyntaxandSemantics:SpeechActs,vol.3.AcademicPress, NewYork.

Groarke,Leo,2009.Fivethesesontoulminandvisualargument.In:VanEemeren,FransH.,Garssen,Bart(Eds.), PonderingonProblemsof Argumentation,ArgumentationLibrary,vol.14.pp.229--239.

Henderson,JohnM.,Ferreira,F.,2004.Sceneperceptionforpsycholinguists.In:Henderson,J.M.,Ferreira,F.(Eds.),TheInterfaceofLanguage, Vision,andAction.PsychologyPress,NewYork.

Horn,LawrenceR.,1985.Metalinguisticnegationandpragmaticambiguity.Language61,121--174. Horn,LawrenceR.,2001.ANaturalHistoryofNegation.CSLIPublications,Stanford,CA.

Hudson,JudithA.,Fivush,Robyn,1983.Categoricalandschematicorganizationandthedevelopmentofretrievalstrategies.J.Exp.Child Psychol.36,32--42.

Jackendoff,Ray,2007.Language,Consciousness,Culture:EssaysonMentalStructure.MITPress,Cambridge,MA.

Johnson, RalphH.,2003.Why ‘visualarguments’ aren’t arguments.Retrieved from:http://web2.uwindows.car/faculty/arts/philosophy/ILat/ edited_johnson.doc.

Kennedy,JohnM.,2008.Metaphorandart.In:Gibbs,W.R.(Ed.),CambridgeHandbookofMetaphorandThought.CambridgeUniversityPress, NewYork, pp.447--461.

Khemlani,Sangeet,Isabel,Orenes,Johnson-Laird,P.N.,2012.Negation:atheoryofitsmeaning,representation,anduse.J.Cogn.Psychol.24 (5),541--559.

Kurby,ChristopherA.,Zacks,JeffreyM.,2008.Segmentationintheperceptionandmemoryofevents.TrendsCogn.Sci.vol12,72--79. Lake,RandellA.,Pickering,BarbaraA.,1998.Argumentation,thevisual,andthepossibilityofrefutation:anexploration.Argumentation12,79--93. Lee,Hye-Kyung,2005.Presuppositionandimplicatureundernegation.J.Pragmatics37,595--609.

Maes,Alfons,Schilperoord,Joost,2008.Classifyingvisualrhetoric:theinterplayofconceptualstructuralandrhetoricalaspectsofvisualrhetoricin advertising.In:Phillips,J.,McQuarrie,E.F.(Eds.),GoFigure!NewDirectionsinAdvertisingRhetoricM.E.Sharpe,Arnounk,NY,pp.227--257. Mandler,Jean,1978.Categoricalandschematicorganizationinmemory.In:Puff,C.R.(Ed.),Memory,Organization,andStructure.Academic

Press,NewYork.

McQuarrie,EdwardF.,Mick,David G.,1999.Visual rhetoricinadvertising:text-interpretive,experimental andreader-response analyses. J.Consum.Res.26,37--54.

Messaris,Paul,2009.What’svisualabout‘visualrhetoric’? Q.J.Speech95(2),210--223. Moscati,Vincenzo,2006.TheScopeofNegation.Diss.UniversityofSiena.

Muckenhaupt,Manfred,1984.TextundBild.TübingerBeitragezurLinguistik. Napoli,E.,2006.Negation.GrazerPhilosophischeStudien72,233--252. Noth,Winfred,1996.Canpictureslie? SemioticRev.Books6,10--12.

Oliva,Aude,Schijns,PhilippeG.,1997.CoarseblobsorFineedges?Evidencethatinformationdiagnosticitychangestheperceptionofcomplex visualstimuli. Cogn.Psychol.34,72--107.

Radvansky,GabrielA.,Zacks,RoseT.,2011.Eventperception.Cogn.Sci.2(6),608--620. Rooth,Mats,1985.AssociationwithFocus.(PhD)GLSA,UniversityofMassachusetts. Rooth,Mats,1992.Atheoryoffocusinterpretation.Nat.Lang.Semantics1,75--116.

Schilperoord,Joost,2013.HowImagesMean.StructureandContentofVisualIncongruitiesinAdvertisingImages.MsUniv.ofTilburg. Schilperoord,Joost,Verhagen,Arie,1998.Conceptualdependencyandtheclausalstructureofdiscourse.In:Koenig,J.(Ed.),Discourseand

Cognition.BridgingtheGap.CSLIPublications,Stanford,CA, pp.141--163.

Schilperoord,Joost,Alfons,Maes,Ferdinandusse,H.,2009.Perceptualandconceptualvisualrhetoric:thecaseofsymmetricobjectalignment. MetaphorSymbol24,155--174.

Shen,Yeshayahu,1999.Principlesofmetaphorinterpretationandthenotionof_{‘domain’:}aproposalforahybridmodel.J.Pragmatics31, 1631--1653.

Skowronski,JohnJ.,Carlston,DonaldE.,1986.Traitmemoryandbehaviormemory:theeffectsofalternativepathwaysonimpressionjudgment responsetimes.J.Pers.Soc.Psychol.50(1),5--13.

Teng,NormanY.,Sun,Sewen,2002.Grouping,simile,andoxymoroninpictures:adesign-basedcognitiveapproach.MetaphorSymbol17, 295--316.

vandenBergh,Loeki,Kuypers,Chantal,2010.Kunnenbeeldenneezeggen?(Canpicturessayno?).(UnpublishedMasterthesis)Universityof Tilburg.

vanderAuwera,Johan,deVogelaer,G.,2005.NegationandQuantification.Webh01.ua.ac.be. vanderSandt,Rob,1991.Denial.In:CLSIIPapersfromtheParasessiononNegation.

vanSomeren,MaartenW.,Barnard,YvonneF.,Sandberg,JacobijnA.C.,1994.TheThinkAloudmethod:APracticalGuidetoModelingCognitive Processes.DepartmentofSocialScienceInformatics,UniversityofAmsterdam,AcademicPress,London.

vanWeelden,Lisanne,Maes,Alfons,Schilperoord,Joost,Cozijn,Rein,2011.Theroleofshapeincomparingobjects:Howperceptualsimilarity mayaffectvisualmetaphorprocessing.MetaphorSymbol26,272--298.

vanWeelden,Lisanne,Maes,Alfons,Schilperoord,Joost,Swerts,Marc,2012.Howobjectshapeaffectsvisualmetaphorprocessing.Exp. Psychol.59,364--371.

Varzi,Achille,1996.Parts,wholes,andpart-wholerelations:theprospectsofmereology.DataKnowl.Eng.20(3),259--286. Verhagen,Arie,2005.ConstructionsofIntersubjectivity:Discourse,SyntaxandCognition.OxfordUniversityPress,Oxford.

Worth,Sol,1981.StudyingVisualCommunication.UniversityofPennsylvaniaPress,Philadelphia.Retrievedfrom:http://astro.temple.edu/_ruby/ wava/worth/svscom.html.

(19)

EleonoreOversteegenstudiedlinguisticsandlogic.In1989,shewroteaPhDontemporalsemantics.Sheteachesdiscoursetheory,semantics andargumentationasassociateprofessorattheUniversityofTilburg.Hermainresearchinterestareinthefieldofsemanticsandpragmaticsof discourse.