SMELs: A Data-driven Middleware for Smart Miscellaneous Electrical Load Management in Buildings

60(/V$'DWDGULYHQ0LGGOHZDUHIRU6PDUW

0LVFHOODQHRXV(OHFWULFDO/RDG0DQDJHPHQWLQ

%XLOGLQJV

%DODML.DOOXUL

)XWXUH&LWLHV/DERUDWRU\ 6LQJDSRUH(7+&HQWUH 6LQJDSRUH NDOOXUL#DUFKHWK]FK

6HNKDU.RQGHSXGL7KDP.ZRN

:DL.XD+DUQ:HL

1DWLRQDO8QLYHUVLW\RI6LQJDSRUH 6LQJDSRUH EGJVQN#QXVHGXVJ EGJWNZ#QXVHGXVJ EGJNXDKZ#QXVHGXVJ

$QGUHDV.DPLODULV

'HSDUWPHQWRI&RPSXWHU6FLHQFH 8QLYHUVLW\RI7ZHQWH 7KH1HWKHUODQGV DNDPLODULV#XWZHQWHQO

Abstract² URZWK LQ ,QIRUPDWLRQ DQG &RPPXQLFDWLRQ 7HFKQRORJ\,&7KDVWULJJHG DQ XQSUHFHGHQWHGSUROLIHUDWLRQRI DSSOLDQFHV DND 0LVFHOODQHRXV (OHFWULFDO /RDGV 0(/V LQ EXLOGLQJV 7LOO QRZ PDQDJLQJ 0(/V HQHUJ\ FRQVXPSWLRQ LQ DQ RSWLPXP FRVWHIIHFWLYH DQG LQWHOOLJHQW PDQQHU LQ EXLOGLQJV UHPDLQ DQ RSHQFKDOOHQJH 7KLV DUWLFOH LQWURGXFHV D QHZ VXSHUYLVHG GDWDGULYHQ PLGGOHZDUH WRZDUGV 6PDUW 0LVFHOODQHRXV (OHFWULFDO /RDG PDQDJHPHQW LQ EXLOGLQJV 60(/V ,W FDQ SHUIRUP DXWRPDWLF H[WUDFWLRQ PRGHOLQJ DQG FODVVLILFDWLRQ RI WKH VHPDQWLFV RI RIILFH DSSOLDQFHV E\ DQDO\]LQJ DJJUHJDWHG HOHFWULFDO ORDG VLJQDWXUHV IURP VHYHUDO HOHFWULFDO RXWOHWV LQ WKH ZRUNSODFH HQYLURQPHQW 7KH UHVXOWV RI DQDO\]LQJ PRUH WKDQ  HOHFWULFDO ORDG VLJQDWXUHV IURP RIILFH ZRUNVWDWLRQV SUHVHQW FODVVLILFDWLRQ SHUIRUPDQFH UDQJLQJ IURP  XSWR  7KH SUHOLPLQDU\ ILQGLQJV IURP WKH VWXG\ GHPRQVWUDWHWKHSRWHQWLDORI60(/VDVDPLGGOHZDUHWHFKQRORJ\ LQ,QWHUQHWRI7KLQJV,R7HQDEOHGVPDUWEXLOGLQJV7KHQRYHOW\ RI WKH SURSRVHG DSSURDFK OLHV LQ FRPELQLQJ WKH XVH RI RSWLPXP VHQVRUV DQG H[LVWLQJ GDWDGULYHQ WHFKQLTXHV WR H[WUDFW GHWDLOHG LQVLJKWVDERXWDSSOLDQFHVRSHUDWLRQLQUHDOEXLOGLQJV



² ; ;

; ; ;

, ,1752'8&7,21

7KH JURZWK LQ ,&7 RYHU WKH \HDUV KDV WUDQVIRUPHG WKH QDWXUH RI ZRUN DFURVV ZRUNSODFH HQYLURQPHQWV 7KH SUROLIHUDWLRQ RI 0(/V VXFK DV SHUVRQDO FRPSXWHUV PRQLWRUV 0XOWL)XQFWLRQDO 'HYLFHV 0)'V DQG 9R,3 SKRQHV LQ EXLOGLQJVRQRQHKDQGLPSURYHGSURGXFWLYLW\EXWRQWKHRWKHU KDQG DIIHFWHG WKH RYHUDOO EXLOGLQJ HQHUJ\ WUHQGV (YLGHQFHV

DUH VWLOO RSHQ LQ SUDFWLFH >@ 0RVW RIWHQ EXLOGLQJ RFFXSDQWV ODFN WKH NQRZOHGJH DERXW RSHUDWLQJ DSSOLDQFHV UDWLRQDOO\ >@ 7RGD\,R7DQGSHUYDVLYHFRPSXWLQJSDUDGLJPKDVPXOWLSOH[HG WKH UROH RI 0(/V DV QRW RQO\ EXLOGLQJ ORDGV EXW DOVR DV FDWDO\VWV IRU HQHUJ\ FRQVHUYDWLRQ LQ EXLOGLQJV (PEHGGLQJ DSSURSULDWH PLGGOHZDUH WHFKQRORJ\ LQWR H[LVWLQJ %XLOGLQJ 0DQDJHPHQW 6\VWHP %06 HQDEOH PDQDJLQJ EXLOGLQJV HYHQ PRUH HIILFLHQWO\ 7KHUH DUH QR HYLGHQFHV IRU PLGGOHZDUH WKDW VXSSRUWVVPDUWHUPDQDJHPHQWRI0(/VLQEXLOGLQJV$UHYLHZ RQUHOHYDQWSHUYDVLYHFRPSXWLQJPLGGOHZDUHDSSURDFKHVIURP OLWHUDWXUHV VXFK DV 6HQVRU$FW >@ &DVVRZDU\ >@ SUHVHQWV D YRLGLQWKLVUHJDUG

,Q WKH FXUUHQW FRQWH[W WKH PLGGOHZDUH LV D VRIWZDUH OD\HU WKDW EULGJHV HQGXVH DSSOLFDWLRQV HJ HPDLO WR UHPLQG DQ HPSOR\HH DERXW UDWLRQDO DSSOLDQFH XVDJH WLSV JRDOV DQG FRPSHWLWLRQ ZLWK LQIRUPDWLRQ DERXW 0(/V RSHUDWLQJ PRGHV FI )LJ  7KH LQWHUDFWLRQ EHWZHHQ RFFXSDQWV DQG 0(/V LQ RIILFHEXLOGLQJVFRQVXPHHQHUJ\7KHLQIRUPDWLRQSHUWDLQLQJWR SDWWHUQ RI XVDJH RI 0(/V DUH FROOHFWHG DV WHPSRUDO ORDG VLJQDWXUHV E\ PXOWLSOH ,R7HQDEOHG VHQVRUV HJ VPDUW SOXJV RFFXSDQF\ GHWHFWRUV GHSOR\HG LQ WKH EXLOGLQJV IRU WKLV SXUSRVH 6XFK LQIRUPDWLRQ LV SDVVHG RQWR WKH SURSRVHG PLGGOHZDUH 60(/V ZKLFK SURFHVVHV WKH GDWD IURP PXOWLSOH ZRUNVWDWLRQV 7KH JRDO RI 60(/V LV WR IDFLOLWDWH HQGXVH DSSOLFDWLRQVLQIXWXUH,R7GHSOR\PHQWVLQVPDUWEXLOGLQJVZLWK LQIRUPDWLRQDERXWUHDOWLPHXVDJHSDWWHUQVRIPXOWLSOHRIILFH DSSOLDQFHVE\RFFXSDQWV7KXVHQDEOHLGHQWLI\LQJRSSRUWXQLWLHV DQG GHYHORSLQJ LQWHOOLJHQW LQWHUYHQWLRQV WR FRQVHUYH HQHUJ\ ZDVWHV HVSHFLDOO\ DW WKH KXPDQPDFKLQH LQWHUIDFH XVLQJ OLPLWHGKDUGZDUHVHQVRUV

7KHSXUSRVHRIWKLVDUWLFOHLVWRLQWURGXFHDQHZVXSHUYLVHG

2018 IEEE 16th Int. Conf. on Dependable, Autonomic & Secure Comp., 16th Int. Conf. on Pervasive Intelligence &

 )LJ $SSOLFDWLRQRI60(/VPLGGOHZDUHLQ,QWHOOLJHQW%XLOGLQJ

0DQDJHPHQW

7KLV SDSHU LV RUJDQL]HG DV IROORZV VHFWLRQ ,, SUHVHQWV WKH EDFNJURXQG WR WKH SURSRVHG DSSURDFK WR LPSOHPHQW WKH PLGGOHZDUHVHFWLRQ,,,GHVFULEHVWKHDFWXDOLPSOHPHQWDWLRQRI WKH PLGGOHZDUH VHFWLRQ ,9 SUHVHQWV NH\ ILQGLQJV IURP FDVH VWXG\RI60(/VDSSOLHGWR2IILFH3OXJ/RDG'DWDVHW23/' >@ DQG VHFWLRQ 9 SUHVHQWV WKH FRQFOXVLRQ OLPLWDWLRQ DQG SRVVLEOHQH[WVWHSVIRUIXWXUHUHVHDUFKLQWKLVGLUHFWLRQ

,,%$&.*5281'$1'35,25:25.6 A. Challenges in energy management of MELs in office

buildings

0RQLWRULQJHYHU\DSSOLDQFHRQ ZRUNVWDWLRQV0(/VLQDQ RSHUDWLRQDO RIILFH EXLOGLQJ XVLQJ LQGLYLGXDO ,R7 HQDEOHG PHWHUVFRQWLQXRXVO\IRUORQJGXUDWLRQLVQRWRQO\FKDOOHQJLQJ EXW DOVR H[SHQVLYH DQG LPSUDFWLFDO >@ $GGLWLRQDO FKDOOHQJH OLHV LQ LGHQWLI\LQJ WKH VWDWHRIRSHUDWLRQ RI 0(/V EH\RQG VLPSOH RQRII PRGH UHFRJQLWLRQ XVLQJ OLPLWHG PHWHUV 7R DOOHYLDWH WKLV SUREOHP UHVHDUFK HIIRUWV DLPHG DW LGHQWLI\LQJ LQGLYLGXDODSSOLDQFHVDQGWKHLUHQHUJ\FRQVXPSWLRQEUHDNGRZQ IURPVLQJOHSRLQWPHDVXUHPHQWV DUHHYROYLQJVLQFHV >@ 7KLV FODVVLFDO DSSURDFK LV WHUPHG DV Non­Intrusive Load Monitoring RU RWKHUZLVH FDOOHG load disaggregation 7KH VXUYH\ LQ OLWHUDWXUH VKRZV VHYHUDODSSURDFKHV WR QRQLQWUXVLYH DSSOLDQFH PRGH LGHQWLILFDWLRQ IRU UHVLGHQFHV >@ EXW OLPLWHG HYLGHQFHVLQZRUNSODFHRIILFHHQYLURQPHQWV>@

B. Temporal Subsequence Mining applied to MELs

7KH FRQWLQXRXV PRQLWRULQJ RI 0(/V UHVXOW LQ ORDG VLJQDWXUHV WKDW DUH IXQGDPHQWDOO\ WHPSRUDO LQ QDWXUH 0RUHRYHUFKDUDFWHULVWLFVLJQDWXUHVRIDSSOLDQFHRSHUDWLRQVDUH UHODWLYHO\ VKRUWHU LQ GXUDWLRQ DQG KHQFH OLH KLGGHQ ZLWKLQ VXEVHTXHQFHV 7LPH 6HULHV 6XEVHTXHQFH 766 UHIHUV WR VKRUW WHPSRUDO SDWWHUQV ZLWKLQ WLPHVHULHV GDWD 0LQLQJ VXFK 766 SDWWHUQV ZLWKLQ WHPSRUDO GDWDVHWV DFURVV GLYHUVH ILHOGV FDQ EH IRXQG LQ OLWHUDWXUH >@ +RZHYHU WKHUH DUH OLPLWHG HYLGHQFHV RILWVDSSOLFDWLRQWRHQHUJ\DQGEXLOWHQYLURQPHQWLQSDUWLFXODU 7KH DSSOLFDELOLW\ RI 766 PLQLQJ WR FKDUDFWHUL]H DQG FODVVLI\ ORDG VLJQDWXUHV RI D FRPPRQ RIILFH 0(/V LH 0)' LV ILUVW LQWURGXFHGLQRXUSUHYLRXVZRUN>@,WGHVFULEHVVHYHUDOVWHSV WRZDUGV PDQXDO IHDWXUH H[WUDFWLRQ FKDUDFWHUL]DWLRQ DQG

VXSHUYLVHG FODVVLILFDWLRQ RI WHPSRUDO ORDG VLJQDWXUHV XVLQJ *UDPPDU9L] >@ 7KH ILQGLQJV LQGLFDWH WKH SRWHQWLDO RI 6\PEROLF $JJUHJDWH DSSUR;LPDWLRQ 6$; >@ WR PRGHO WHPSRUDO DSSOLDQFH IHDWXUHV DQG IXUWKHU FODVVLI\ JURXQGWUXWK RSHUDWLRQV RI 0(/V IURP DJJUHJDWHG ORDG VLJQDWXUHV +RZHYHUWKHDSSURDFKSUHVHQWHGLQ>@LVQRWVFDODEOHDFURVV ODUJH 0(/V GDWDVHWV GXH WR PDQXDOO\ VXSHUYLVHG IHDWXUH H[WUDFWLRQ 2XU VXEVHTXHQW UHVHDUFK ZRUN VXFFHVVIXOO\ H[WHQGHG WKH DSSOLFDWLRQ RI 6$;960 >@ IUDPHZRUN WR FODVVLI\ 0(/V RSHUDWLRQV IURP D ODUJH GDWDVHW >@ 7KLV IUDPHZRUN LQWHOOLJHQWO\ FRPELQHG XQVXSHUYLVHG IHDWXUH H[WUDFWLRQ SDUDPHWHU RSWLPL]DWLRQ 6$; GLVFUHWL]DWLRQ DQG VXSHUYLVHG FODVVLILFDWLRQ 7KH UHVXOWV VKRZHG JRRG DFFXUDF\ aLQLGHQWLI\LQJJURXQGWUXWKRSHUDWLRQVRI0)'IURPD ODUJH FROOHFWLRQ ! RI DJJUHJDWHG ORDG VLJQDWXUHV 7KH DSSURDFKWRLPSOHPHQWLQJPLGGOHZDUHSUHVHQWHGLQ6HFWLRQ,,, VKDOO KHOS XQGHUVWDQG KRZ 6$; HQDEOHV H[WUDFWLQJ PRGHOLQJ FKDUDFWHULVWLFV VXEVHTXHQFHV DQG XVLQJ WKHP LQWHOOLJHQWO\ WRZDUGV EHQFKPDUN FODVVLILFDWLRQ 2WKHU YDULDQWV RI 766 PLQLQJ VXFK DV 3RZHU6$; >@ DQG VKDSHOHWV >@ KDYH DOVR UHSRUWHGVXFFHVVLQUHVLGHQWLDOVPDUWPHWHULQJDSSOLFDWLRQVVXFK DVFXVWRPHUVHJPHQWDWLRQDQGHQHUJ\FRQVXPSWLRQIRUHFDVWLQJ

,,,352326('0,''/(:$5(

7KH DSSURDFK WR LPSOHPHQW WKH SURSRVHG PLGGOHZDUH 60(/V IRU DSSOLDQFH PRGH LGHQWLILFDWLRQ LQ ZRUNSODFH HQYLURQPHQWLVGLVFXVVHGLQWKLVVHFWLRQ7KHVFKHPDWLFLQ)LJ GHVFULEHV WKHGDWDGULYHQSURFHVVWKURXJKYDULRXVIXQFWLRQDO EORFNVDFURVVPXOWLSOHVWDJHV

A. Intelligent temporal feature engineering

7KH VHPDQWLFV RI 0(/V DUH REWDLQHG E\ FROOHFWLYHO\ PHDVXULQJ WKH DJJUHJDWHG ORDG DW VHYHUDO HOHFWULFDO RXWOHWV LQ ZRUNSODFH HQYLURQPHQW 7KH 23/' LV D ODUJH UHSRVLWRU\ RI ODEHOOHGWHPSRUDOORDGVLJQDWXUHVRIPXOWLSOHRIILFH0(/VIURP VHYHUDOLQGLYLGXDOZRUNVWDWLRQV6XFKDGDWDVHWIRUPVWKHLQSXW VWDJH  WR WKH PLGGOHZDUH SUHVHQWHG LQ )LJ  6SHFLILF FKDUDFWHULVWLFV RI WKH GDWDVHW DQDO\]HG LQ WKLV DUWLFOH LV SUHVHQWHGLQ6HFWLRQ,9



                     7KLVLVIROORZHGE\WKHSURFHVVRIWUDQVIRUPLQJWHPSRUDOORDG VLJQDWXUHV RI DSSOLDQFHV LQWR LQWHOOLJHQW IHDWXUHV IRU LPSURYHG DSSOLDQFH PRGH LGHQWLILFDWLRQ 7KH LQWHOOLJHQW IHDWXUH HQJLQHHULQJ SURFHVV LQ 60(/V LV EDVHG RQ DQ H[LVWLQJ 766 PLQLQJWHFKQLTXHFDOOHG6$;>@,WIROORZVWKUHHPDLQVWHSV DV LQWXLWLYHO\ GHVFULEHG LQ )LJ D QDPHO\ WHPSRUDO GLVFUHWL]DWLRQ V\PEROL]DWLRQ DQG JUDPPDUUXOH LQGXFWLRQ $ WHPSRUDOO\DJJUHJDWHGORDGVLJQDWXUHVHFZKHQDGHVNWRS 3&DPRQLWRUDQGD0)'RSHUDWHFRQFXUUHQWO\DFURVVGLIIHUHQW PRGHVRIRSHUDWLRQLVSUHVHQWHGDVDQH[DPSOH7KHKLJKOLJKWHG VXEVHTXHQFHVZLWKLQWKHLQSXWHSLVRGHSUHVHQWVLPLODULW\LQWKH JURXQGWUXWK DSSOLDQFH RSHUDWLRQV HJ 0)' LQ Copy RSHUDWLRQ H[KLELWHG DV DSSUR[LPDWH WHPSRUDO VLJQDWXUH VLPLODULW\ 7KH SXUSRVH RI IHDWXUH HQJLQHHULQJ VWHS LV WR GLVFRYHU DQG PRGHO VXFK VLPLODULWLHV IURP D ODUJH ODEHOOHG DSSOLDQFH ORDG VLJQDWXUH UHSRVLWRU\ 7KLV LV GRQH E\ VDPSOLQJ WHPSRUDO ORDG VLJQDWXUHV LQWR UHODWLYHO\ VPDOOHU VXEVHTXHQFHV

XVLQJ VOLGLQJ ZLQGRZ WHFKQLTXH ZLWK ZLQGRZ VL]H Z  6XEVHTXHQWO\ WKHVH FRQWLQXRXVWLPH VXEVHTXHQFHV DUH GLVFUHWL]HG XVLQJ 3LHFHZLVH $JJUHJDWH $SSUR[LPDWLRQ 3$$ WHFKQLTXH ZLWK 3$$ VL]H S  7KLV LV GHVFULEHG DV GLVFUHWL]DWLRQ VWHS LQ )LJ D 7KH UHVXOWLQJ GLVFUHWL]HG VXEVHTXHQFHV LQ WKH QH[W VWHS DUH V\PEROL]HG ZLWK DOSKDEHW VL]H D  DQG WKXV WUDQVIRUPHG LQWR &RQWH[W)UHH*UDPPDU &)* UXOHV HJ baabccbc WKURXJK V\PEROL]DWLRQ 7KLV LV IROORZHG E\ GLVFRYHULQJ H[WUDFWLQJ DQG PRGHOLQJ VHYHUDO DSSUR[LPDWHO\ VLPLODU VXEVHTXHQFHV IURP LQSXW GDWDVHW EDVHG RQPDWFKLQJ&)*UXOHV,QWKHH[DPSOHQLQHVLPLODU&)*UXOHV DUH GLVFRYHUHG IURP WKH H[DPSOH LQSXW DJJUHJDWH ORDG VLJQDWXUH 7KLV VWHS LV WHUPHG DV JUDPPDUUXOH LQGXFWLRQ 7RJHWKHU DOO WKHVH WKUHH VWHSV UHSUHVHQW 6$; IHDWXUH HQJLQHHULQJ,WLVLPSRUWDQWWRQRWHWKDW6$;LVDSDUDPHWHUL]HG WHFKQLTXH DQG GHWHUPLQLQJ WKH RSWLPXP SDUDPHWHUV Z S D IROORZV QIROG FURVV YDOLGDWLRQ DQG ',5(&7 RSWLPL]DWLRQ

D F E )LJ D,OOXVWUDWLRQRIIHDWXUHHQJLQHHULQJLQ60(/VXVLQJ6$;LVSUHVHQWHGLQVWHSVXVLQJDQDJJUHJDWHORDGVLJQDWXUHDVLQSXWIURP23/' ZLWKKLJKOLJKWHGHSLVRGHVUHSUHVHQWLQJ0)'LQCopyRSHUDWLRQWREHH[WUDFWHGE$QH[DPSOHRIWHUPIUHTXHQF\LQYHUVHGRFXPHQW IUHTXHQF\DQGORQJHVWFRPPRQVXEVHTXHQFHFRPSXWDWLRQXVLQJ%DJRI:RUGVF$YHFWRUVSDFHUHSUHVHQWDWLRQRIWHPSRUDOORDGVLJQDWXUHV DQGFRVLQHVLPLODULW\PHDVXUHWRFODVVLI\WKHPXVLQJ9HFWRU6SDFH0RGHO

DOJRULWKP DV LPSOHPHQWHG LQ >@ &ROOHFWLYHO\ WKHVH VWHSV UHSUHVHQW VWDJH  LQ 60(/V DQG LW UHVXOWV LQ LQWHOOLJHQW DSSOLDQFHIHDWXUHVFDOOHG&)*UXOHV

$ ODUJH UHSRVLWRU\ RI LQSXW WHPSRUDO ORDG VLJQDWXUHV LV LQWHOOLJHQWO\ SURFHVVHG XVLQJ 6$;_{ UHVXOWLQJ LQ D KXJH} FROOHFWLRQ RI ODEHOOHG &)* UXOHV ZLWK DSSOLDQFH PRGHV DV ODEHOV &ROOHFWLYHO\ WKH\ IRUP %DJRI5XOHV %R5 GLFWLRQDU\ IRU23/'DVVKRZQLQ)LJE7KLVSURFHVVLVODEHOOHGVWDJH LQ)LJ7\SLFDOO\%R5FRQVLVWRIPXOWLSOHEDJVFRPSRVHG RIVHYHUDOLQVWDQFHVRI&)*UXOHVWRJHWKHUZLWKWKHLUIUHTXHQF\ FRXQW (DFK EDJ FRUUHVSRQGV WR LQGLYLGXDO JURXQGWUXWK DSSOLDQFHRSHUDWLRQ PRGHV7KHVH&)*UXOHVDUHHPSOR\HGDV ³LQWHOOLJHQW´IHDWXUHVLQWKHVXEVHTXHQWVWDJHV6SHFLILFGHWDLOV RQ HPSOR\LQJ WKHVH IHDWXUHV WRZDUGV LPSURYHG DSSOLDQFH PRGHVUHFRJQLWLRQLVGLVFXVVHGLQWKHIROORZLQJVHFWLRQ B. Hidden appliance mode recognition via classification

,QWHOOLJHQW HQHUJ\ PDQDJHPHQW RI 0(/V LQ RIILFH ZRUNVWDWLRQV UHTXLUHV GLVDPELJXDWLQJFODVVLI\LQJ LQGLYLGXDO KLGGHQ DSSOLDQFH RSHUDWLRQ PRGHV IURP DJJUHJDWH ORDG VLJQDWXUHV 7KLV LV WKH QH[W VWHS LQ WKH GDWDGULYHQ SURFHVV VWDJH  LQ )LJ  &ODVVLI\LQJ XQODEHOHG DJJUHJDWH ORDG VLJQDWXUHVFRQWDLQLQJFKDUDFWHULVWLFHSLVRGHVRIDSSOLDQFHVDQG WKHLUJURXQGWUXWKRSHUDWLRQVUHTXLUHVRSKLVWLFDWHGWHFKQLTXHV ,Q 60(/V VHYHUDO WHFKQLTXHV VXFK DV /RQJHVW &RPPRQ 6XEVHTXHQFH/&67HUP)UHTXHQF\7),QYHUVH'RFXPHQW )UHTXHQF\ ,') 9HFWRU 6SDFH 0RGHO 960 DQG 0D[LPXP 9RWLQJ 09 DUH HPSOR\HG IRU WKLV SXUSRVH 7KH LPSOHPHQWDWLRQ GLIIHUV IURP RXU SUHYLRXV ZRUN >@ E\ LQFOXGLQJ PXOWLSOH DSSURDFKHV EDVHG RQ VXEVWULQJ VLPLODULW\ WRZDUGV LPSURYHG FODVVLILFDWLRQ DQG SHUIRUPDQFH EHQFKPDUNLQJ 7KUHH EHQFKPDUN FODVVLILFDWLRQ DSSURDFKHV LQWURGXFHGLQWKHFODVVLILFDWLRQVWHSLHVWDJHDUHDVIROORZV x $SSURDFK  )HDWXUH ZHLJKWLQJ EDVHG RQ FRPELQHG /&6 DQG SURGXFW RI 7) DQG ,') EXW FODVVLILFDWLRQ EDVHGRQ09

x $SSURDFK)HDWXUHZHLJKWLQJEDVHGRQSURGXFWRI7) DQG,')EXWFODVVLILFDWLRQEDVHGRQ960

x $SSURDFK  )HDWXUH ZHLJKWLQJ EDVHG RQ FRPELQHG ZHLJKWHG /&6 DQG SURGXFW RI 7) DQG ,') EXW FODVVLILFDWLRQEDVHGRQ960

7KH7)UHSUHVHQWVQXPEHURIWLPHVDVSHFLILFUXOHDSSHDUV LQDQDSSOLDQFHPRGHFODVVRUDEDJ2QWKHRWKHUKDQG,') UHSUHVHQWVWKHVSHFLILFLW\RIDUXOHWRDQDSSOLDQFHPRGHFODVV RU D EDJ 7KH SURGXFW RI 7) DQG ,') IRU HDFK UXOH JLYHV LWV UHODWLYH VLJQLILFDQFH WR D SDUWLFXODU DSSOLDQFH PRGH FODVV +RZHYHU /&6 GLIIHUV IURP WKH IRUPHU E\ FRQVLGHULQJ WKH VXEVWULQJ VLPLODULW\ EHWZHHQ WZR &)* UXOHV $OVR LW HQFDSVXODWHVWKHOHQJWKRIWKHORQJHVWFRPPRQVXEVWULQJ)LJ E SUHVHQWV WKH GHILQLWLRQ RI DOO WKUHH IHDWXUH ZHLJKWLQJ PHDVXUHVZLWKH[DPSOHRI&)*UXOHVIURP%R57KHQH[WVWHS LQ 60(/V LV XVLQJ WKH ODEHOOHG IHDWXUHV IURP %R5 WRZDUGV VXSHUYLVHGFODVVLILFDWLRQRIWKHXQODEHOHGWHVWGDWD7KHVWDJHV   DQG  GHVFULEHG DERYH KHOS WUDQVIRUP WKH WHPSRUDO ORDG VLJQDWXUHV LQWR IHDWXUHV VXLWDEOH IRU UHSUHVHQWLQJ DV YHFWRUV

 _{,PSOHPHQWDWLRQLQKHULWHGIURPKWWSVJLWKXEFRPM0RWLIMPRWLI5}

7KLVIDFLOLWDWHVDSSO\LQJRQHRIWKHPRVWHIIHFWLYHFODVVLILFDWLRQ WHFKQLTXHVIURP,QIRUPDWLRQ5HWULHYDONQRZQDV960>@,W KHOSVUHSUHVHQWVHYHUDOWHPSRUDOORDGVLJQDWXUHVLQWKHGDWDVHW DV YHFWRUV LQ D PXOWLGLPHQVLRQDO YHFWRU VSDFH DQG FODVVLI\ WKHP$VLPSOLILHGWZRGLPHQVLRQDOUHSUHVHQWDWLRQRIDPXOWL GLPHQVLRQDOYHFWRUVSDFHFRQVLVWLQJRIERWKODEHOOHGWUDLQDQG XQODEHOHG WHVW YHFWRUV LV SUHVHQWHG LQ )LJ F 7KH YHFWRUV ODEHOOHG YG YG DQG YGGHQRWH ODEHOOHG YHFWRUV ZKRVH DSSOLDQFH PRGH FODVV LV NQRZQ DSULRUL ZKLOH YHFWRU ODEHOOHG YTGHQRWHXQODEHOHGYHFWRUZKRVHDSSOLDQFHPRGHFODVVLVWR EH GHWHUPLQHG 7KH FRVLQH VLPLODULW\ ZKLFK LV D PHDVXUH RI GLVWDQFHEHWZHHQYHFWRUVLQWKHPXOWLGLPHQVLRQDOYHFWRUVSDFH LV XVHG LQ FODVVLI\LQJ WKH XQODEHOHG YHFWRU 7KH PDWKHPDWLFDO H[SUHVVLRQRIFRVLQHVLPLODULW\PHDVXUHLQYROYHVWKHSURGXFWRI 7),')ZHLJKWVRILQGLYLGXDOIHDWXUHVFRQVWLWXWLQJDYHFWRUDV VKRZQLQ)LJF2QWKHRWKHUKDQG60(/VDOVRLQWHJUDWHV VLPSOH 09EDVHGFODVVLILFDWLRQ DSSURDFK LQRUGHU WR IDFLOLWDWH FODVVLI\LQJIHDWXUHVWKDWPDWFKSDUWLDOO\EHWZHHQWHVWDQGWUDLQ YHFWRUV ,W FRPSXWHV DQ DJJUHJDWH VFRUH EDVHG RQ ERWK IXOO\ PDWFKHG IHDWXUHV XVLQJ 7),') DQG SDUWLDOO\ PDWFKHG IHDWXUHV XVLQJ /&6 DQG FODVVLILHV WKH WHVW GDWD EDVHG RQ PD[LPXP VLPLODULW\ VFRUH YRWHV )LQDOO\ WKH FODVVLILFDWLRQ HUURUIURPPXOWLSOHVDSSURDFKHVDUHDQDO\]HGWREHQFKPDUNWKH SHUIRUPDQFHVWDJHLQ)LJ2QHRIWKHNH\FRQWULEXWLRQVRI 60(/VLVHQDEOLQJEHQFKPDUNWKHSHUIRUPDQFH RIFODVVLI\LQJ JURXQGWUXWKDSSOLDQFHPRGHVIURPDJJUHJDWHGORDGVLJQDWXUHV XVLQJ PXOWLSOH 766 DSSURDFKHV 7KH VRIWZDUH LPSOHPHQWDWLRQ RIWKHPLGGOHZDUHLVPDGHLQ-DYDDQGLVEDVHGRQRSHQVRXUFH 6$;960 IUDPHZRUN >@ 7\SLFDOO\ UXQQLQJ 60(/V RQ D SHUVRQDOFRPSXWHUWRWUDLQDQGFODVVLI\DGDWDVHWZLWKORDG VLJQDWXUHVWDNHVIHZPLQXWHVWRFRPSOHWH

,95(68/76$1'',6&866,21

7KHDJJUHJDWHORDGVLJQDWXUHVIURP23/'ZKHQDGHVNWRS RU ODSWRS 3& D PRQLWRU DQG DQ 0)' RSHUDWH FRQFXUUHQWO\ LQ RIILFH ZRUNVWDWLRQ LV FRQVLGHUHG IRU DQDO\VLV 7KH DQDO\VLV RI SHUIRUPDQFH RI 60(/V SUHVHQWHG LQ WKLV VHFWLRQ FRQVLGHUV GLVDPELJXDWLQJ RU FODVVLI\LQJ PXOWLSOH KLGGHQ RSHUDWLRQDO PRGHV RI DQ LQGLYLGXDO RIILFH DSSOLDQFH LH 0)' IURP WKH DJJUHJDWHGGDWDVHW7DEOH,SUHVHQWVYDULRXVRSHUDWLRQDOPRGHV RIVHYHUDORIILFHDSSOLDQFHFDWHJRULHVFRQVLGHUHGIRUDQDO\VLV,W VKRZV WKUHHRSHUDWLRQDO PRGHV IRU GHVNWRS RU ODSWRS3&YL] 2Q 6OHHS 2II WKUHH RSHUDWLRQDO PRGHV IRU PRQLWRU YL]  %ULJKWQHVV  %ULJKWQHVV 2II DQG ILYH RSHUDWLRQDO PRGHV IRU 0)' YL] ,GOH &RS\ 6FDQ 3ULQW 2II )RU WKH SXUSRVH RI LOOXVWUDWLRQ 7DEOH , DOVR VKRZV  OHJLWLPDWH XVH FDVH VFHQDULRV ZKHQ GHVNWRS 3& PRQLWRU DQG 0)' FRQFXUUHQWO\ RSHUDWH +RZHYHU WKH DQDO\VLV SUHVHQWHG LQ WKLV VWXG\ FRQVLGHUV WKUHH FRPPRQ DSSOLDQFH FRPELQDWLRQV FI 7DEOH,,IRXQGLQDW\SLFDORIILFHZRUNVWDWLRQQDPHO\

x 'HVNWRS 3& ZLWK 0RQLWRU DQG 0)' ODEHOOHG DV '03

x /DSWRS3&ZLWK0RQLWRUDQG0)'ODEHOOHGDV/03 x /DSWRS3&ZLWK0)'ODEHOOHGDV/3

 7KHDQDO\VLVRIFODVVLI\LQJJURXQGWUXWKRSHUDWLRQVRIWKUHH GLIIHUHQW LQVWDQFHV RI 0)' YL] 3 3 DQG 3 IURP DERYH FRQFXUUHQW DSSOLDQFH RSHUDWLRQV DUH DQDO\]HG LQ

7$%/(,86(&$6(&20%,1$7,216:,7+'(6.7233&021,725 $1'0)',123(5$7,21

 WKLV VHFWLRQ &ROOHFWLYHO\ PRUH WKDQ  DJJUHJDWH ORDG VLJQDWXUHV DFURVV DOO FRPELQDWLRQV DUH DQDO\]HG XVLQJ 60(/V DQGWKHUHVXOWVDUHVXPPDUL]HGLQ7DEOH,,(DFKGLVDJJUHJDWLRQ XVHFDVH LQ WKH WDEOH KDYH IHZ KXQGUHGVRI ODEHOOHG DJJUHJDWH ORDG VLJQDWXUHV    HWF IRU DQDO\VLV (DFK VLJQDWXUH LQ WKHP FRQVLVW RI VHYHUDO VXEVHTXHQFHV HDFK UHSUHVHQWLQJ D FRPELQDWLRQ RI PXOWLSOH DSSOLDQFH RSHUDWLRQV FI 7DEOH , 7KH JRDO LV WR UHFRJQL]H KLGGHQ JURXQGWUXWK RSHUDWLRQDO PRGHV RI 0)' IURP WKH XQODEHOHG WHVW GDWDVHW )RU H[DPSOH FODVVLI\LQJ  RSHUDWLRQDO PRGHV LH   RI3IURPWHVWVLJQDWXUHVEDVHGRQWUDLQHGIHDWXUHV

LH IURPWUDLQLQJVLJQDWXUHVLQXVHFDVHLQ 7DEOH,,7KHDQDO\VLVSUHVHQWHGLQWKHWDEOHLVFDWHJRUL]HG EDVHGRQWZRDSSURDFKHVWRVSOLWWKHHQWLUHLQSXWGDWDVHWD WUDLQ DQGWHVW EWUDLQ DQGWHVW 7KH FROXPQ ODEHOOHG classification measures UHSUHVHQW GLVWLQFW 6$; SDUDPHWHUV IRU HDFK XVHFDVH LQ >Z S D@ IRUPDW WRJHWKHUZLWKWKHEUHDNGRZQRIVLJQDWXUHFRXQWIRUWUDLQLQJ DQG WHVWLQJ 7KHVH 6$; SDUDPHWHUV DUH REWDLQHG XVLQJ RSWLPL]DWLRQ HQJLQH DV GHVFULEHG LQ WKH SUHYLRXV VHFWLRQ 7KH SHUFHQWDJH RI XQODEHOHG WHVW VLJQDWXUHV FODVVLILHG SRVLWLYHO\ DUH DOVR SUHVHQWHG DV UHVXOWV LQ WKH WDEOH 7KH IROORZLQJDUHVRPHREVHUYDWLRQVDQGILQGLQJVGUDZQ x 'HSHQGLQJRQWKHQDWXUHRIVSOLWWLQJWKHGDWDVHW6$;

SDUDPHWHUV YDU\ )RU H[DPSOH XVHFDVH  SUHVHQWV HLWKHU >@ RU >@ FRUUHVSRQGLQJ WR  RU VSOLWLQWKHLQSXWGDWDVHW

x 'HFLSKHULQJ RSHUDWLRQDO PRGHV RI 0)'V XVLQJ DSSURDFK  PDUJLQDOO\ RXWSHUIRUPV DSSURDFK  ZLWK  VSOLW LQ WKH GDWDVHW ZKHQ FRPSDUHG WR  VSOLW 7KLV VXJJHVWV WKDW LQWHJUDWHG IHDWXUH VHOHFWLRQ EDVHGRQERWKFRPSOHWHDQGSDUWLDOVXEVWULQJVLPLODULW\ PDUJLQDOO\ RXWSHUIRUPV IHDWXUH VHOHFWLRQ EDVHG RQ FRPSOHWH VXEVWULQJ VLPLODULW\ HVSHFLDOO\ ZKHQ FODVVLILFDWLRQLVEDVHGRQ960+RZHYHU WKHVDPHLV QRW WUXH ZKHQ FODVVLILFDWLRQ LV EDVHG RQ 09 7KLV VXJJHVWVWKHSRWHQWLDORILQWHOOLJHQWIHDWXUHHQJLQHHULQJ DSSURDFKWKDWFRPELQHV/&6DQG7),')PHDVXUHV x &OHDUO\PRVWXVHFDVHVXQGHUERWKDSSURDFKHVDQG

SHUIRUPV EHWWHU ZLWK SHUIRUPDQFH H[FHHGLQJ  +RZHYHULQWKHFDVHRIDSSURDFKRQO\IHZXVHFDVHV SUHVHQW SHUIRUPDQFH H[FHHGLQJ  7KLV LQGLFDWHV WKDW960EDVHGDSSOLDQFHPRGHFODVVLILFDWLRQLVEHWWHU

7$%/(,,6800$5<2)5(68/76&/$66,)<,1*08/7,3/(23(5$7,21$/02'(62)0)'6)520$/$5*($**5(*$7('$7$6(7

WKDQ09ZKHQXVLQJWHPSRUDOIHDWXUHV

x 2YHUDOO WKH SUHOLPLQDU\ EHQFKPDUN DQDO\VLV RI PRUH WKDQ  DJJUHJDWH ORDG VLJQDWXUHV IURP PXOWLSOH RIILFHZRUNVWDWLRQVXVLQJ60(/VSUHVHQWFODVVLILFDWLRQ SHUIRUPDQFHUDQJLQJ EHWZHHQ DQG7KLV VKRZV WKH VXFFHVV RI WKH SURSRVHG PLGGOHZDUH LQ GLVDPELJXDWLQJ WKH KLGGHQ DSSOLDQFH RSHUDWLRQV LQ DJJUHJDWHGORDGVLJQDWXUHV

9&21&/86,21/,0,7$7,21$1')8785(:25. $QDSSURDFKWRHOXFLGDWHWKHVHPDQWLFVRIRIILFHDSSOLDQFHV KLGGHQLQDJJUHJDWHORDGVLJQDWXUHVIURPPXOWLSOHZRUNVWDWLRQV LV SUHVHQWHG XVLQJ 60(/V 7KH QRYHOW\ OLHV LQ GHYHORSLQJ DQ DSSURDFKWRLPSOHPHQWDPLGGOHZDUHFRPELQLQJRSWLPXP,R7 VHQVRUV DQG H[LVWLQJ GDWDGULYHQ WHFKQLTXHV WR IDFLOLWDWH LPSURYHG DSSOLDQFHV PRGH PRQLWRULQJ LQ RIILFH EXLOGLQJV 7KHSUHOLPLQDU\UHVXOWVGHPRQVWUDWHWKHVXFFHVVRIGHFLSKHULQJ YDULRXVRSHUDWLRQDOPRGHVRIDFRPPRQRIILFHDSSOLDQFHIURP D KXJH FROOHFWLRQ RI DJJUHJDWH ORDG VLJQDWXUHV FROOHFWHG IURP RIILFH ZRUNVWDWLRQV23/' ,Q SDUWLFXODU IHDWXUHHQJLQHHULQJ DSSURDFKFRPELQLQJ/&6DQG7),')LVIRXQGWREHWWHUPRGHO FKDUDFWHULVWLF ORDG VLJQDWXUHV IRU VXSHUYLVHG FODVVLILFDWLRQ $GGLWLRQDOO\ FODVVLILFDWLRQ EDVHG RQ 960 SUHVHQWHG EHWWHU SHUIRUPDQFHRYHUVLPSOH09DSSURDFKHVSHFLDOO\LQDQDO\]LQJ ODUJHWHPSRUDOGDWDVHWV

,Q WKH UHDOP RI SHUYDVLYH FRPSXWLQJ DQG ,R7 HPEHGGLQJ VXFK PLGGOHZDUH LQWR WKH IDEULF RI %06 VKDOO IDFLOLWDWH LQWHOOLJHQW DSSOLFDWLRQV WKDW UHTXLUH XQGHUVWDQGLQJ 0(/V RSHUDWLRQV LQ UHDOWLPH DW SHUVRQDOL]HG RFFXSDQWOHYHO :H HQYLVLRQ WKH IXWXUH RI VPDUW EXLOGLQJV ZLWK FRRSHUDWLYH DQG LQWHOOLJHQW HQHUJ\ PDQDJHPHQW RI 0(/V LQ ZRUNSODFHV 7KLV FRPELQHV XVH RI OLPLWHG VHQVRUV FORXGEDVHG LPSOHPHQWDWLRQ RI60(/VPLGGOHZDUHUHDOWLPHDSSOLDQFHPRGHLGHQWLILFDWLRQ DQGLPSURYHGHFRIHHGEDFNPHVVDJLQJVHUYLFHV

+RZHYHUWKHSURSRVHGPLGGOHZDUHLVVWLOOYHU\QDVFHQWDQG SRVH FHUWDLQ OLPLWDWLRQV IRU LWV UHDOZRUOG DSSOLFDWLRQ )LUVWO\ UHVXOWV SUHVHQWHG LQ WKLV DUWLFOH LV OLPLWHG WR LGHQWLI\LQJ 0)' RSHUDWLRQDO PRGHV DORQH ZKLFK H[KLELWV FKDUDFWHULVWLF VLJQDWXUH SDWWHUQV ([WHQGLQJ WKH DSSOLFDWLRQ WR RWKHU RIILFH DSSOLDQFHV HJ GHVNWRSV ODSWRSV PRQLWRUV WKDW H[KLELW UHODWLYHO\ VWHDG\ ORDG VLJQDWXUHV LV D VXEVHTXHQW VWHS LQ GHYHORSLQJ WKH PLGGOHZDUH $ SRWHQWLDO DSSURDFK LQ WKLV GLUHFWLRQ LV DQ HQVHPEOH RI PXOWLSOH FODVVLILFDWLRQ DSSURDFKHV EDVHG RQ LQWHOOLJHQW IHDWXUHV REWDLQHG )RU H[DPSOH WKH RSHUDWLRQDOPRGHVRI0)'DUHUHODWLYHO\VKRUWOLYHGDQGVKDOO EH FODVVLILHG EDVHG RQ VLPLODULWLHV EHWZHHQ VXEVHTXHQFHV 2Q WKHRWKHUKDQGWKHRSHUDWLRQDOPRGHVRIRWKHUDSSOLDQFHVVXFK DV GHVNWRSV ODSWRSV DQG PRQLWRUV DUH UHODWLYHO\ VWHDG\ DQG ORQJOLYHGDQGVKDOOEHFODVVLILHGEDVHGRQVLPLODULWLHVEHWZHHQ FRPSOHWHWHPSRUDOVHTXHQFHVLQORDGVLJQDWXUHV6HFRQGO\WKH SURSRVHGDSSURDFKLVVXSHUYLVHGXVLQJODEHOOHGORDGVLJQDWXUHV IURP KDQGIXO RI DSSOLDQFHV DQG WKHLU FRPELQDWLRQV SUHVHQW LQ 23/' 7KXV WKH SHUIRUPDQFH RI WKH SURSRVHG PLGGOHZDUH EH\RQG WKH DSSOLDQFH LQVWDQFHV LQ 23/' LV QRW YDOLGDWHG &ROODERUDWLYHO\ XSVFDOLQJ 23/' WKURXJK FURZG DQG RSHQ VRXUFLQJ ZLOO DGGUHVV WKLV JDS )LQDOO\ LQ WKH ,R7 SDUDGLJP IXWXUH UHVHDUFK VKDOO YDOLGDWH 60(/V LQ UHDO ZRUOG WKURXJK FORXGEDVHGLPSOHPHQWDWLRQ

5()(5(1&(6

>@ . 0F.HQQH\ 0  *XHUQVH\ 5 3RQRXP DQG - 5RVHQIHOG ³&RPPHUFLDO PLVFHOODQHRXV HOHFWULF ORDGV (QHUJ\ FRQVXPSWLRQ FKDUDFWHUL]DWLRQDQGVDYLQJVSRWHQWLDOLQE\EXLOGLQJW\SH´7,$; //&/H[LQJWRQ0$7HFK5HS'SS

>@ $ .DPLODULV '7+ 1JDQ $ 3DQWD]DUDV %  .DOOXUL 6 .RQGHSXGL DQG 7 .ZRN :DL ³*RRG SUDFWLFHV LQ WKH XVH RI ,&7 HTXLSPHQW IRU HOHFWULFLW\ VDYLQJV DW D XQLYHUVLW\ FDPSXV´ ,Q 3URFHHGLQJV RI *UHHQ &RPSXWLQJ&RQIHUHQFHSS'DOODV7H[DV86$,((( >@ $.DPLODULV%.DOOXUL6.RQGHSXGLDQG7.ZRN:DL³$OLWHUDWXUH

VXUYH\ RQ PHDVXULQJ HQHUJ\ XVDJH IRU PLVFHOODQHRXV HOHFWULF ORDGV LQ RIILFHV DQG FRPPHUFLDO EXLOGLQJV´ -RXUQDO IRU 5HQHZDEOH DQG 6XVWDLQDEOH(QHUJ\5HYLHZVYROSS

>@ 3$UMXQDQ1%DWUD+&KRL+6LQJK36LQJKDQG0%6ULYDVWDYD ³6HQVRUDFW D SULYDF\ DQG VHFXULW\ DZDUH IHGHUDWHG PLGGOHZDUH IRU EXLOGLQJPDQDJHPHQW´,Q3URFHHGLQJVRIWKHIRXUWK$&0ZRUNVKRSRQ (PEHGGHG 6HQVLQJ 6\VWHPV IRU (QHUJ\(IILFLHQF\ LQ %XLOGLQJV 7RURQRWR2QWDULR&DQDGDSS$&0

>@ 3 .DWKLUDYHOX / 6KDULIL DQG / 9HLJD ³&DVVRZDU\ 0LGGOHZDUH SODWIRUPIRUFRQWH[WDZDUHVPDUWEXLOGLQJVZLWKVRIWZDUHGHILQHGVHQVRU QHWZRUNV´ ,Q3URFHHGLQJVRIWKHVHFRQGZRUNVKRSRQ0LGGOHZDUHIRU &RQWH[W$ZDUH$SSOLFDWLRQVLQWKH,R79DQFRXYHU%&&DQDGD SS$&0

>@ %.DOOXUL6.RQGHSXGL+:.XD7.ZRN:DLDQG$.DPLODULV ³23/' 7RZDUGV LPSURYHG QRQLQWUXVLYH RIILFH SOXJ ORDG GLVDJJUHJDWLRQ´ ,Q 3URFHHGLQJV RI ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ %XLOGLQJ(IILFLHQF\DQG6XVWDLQDEOH7HFKQRORJLHV6LQJDSRUHSS ,(((

>@ *:+DUW³1RQLQWUXVLYHDSSOLDQFHORDGPRQLWRULQJ´,Q3URFHHGLQJV RIWKH,(((YROSS,(((

>@ $ =RKD $ *OXKDN 0 $ ,PUDQ DQG 6 5DMDVHJDUDU ³1RQLQWUXVLYH ORDG PRQLWRULQJ DSSURDFKHV IRU GLVDJJUHJDWHG HQHUJ\ VHQVLQJ $ VXUYH\´6HQVRUVYROSS

>@ $ 5RJULJXH] 6 7 6PLWK $ .LII DQG % 3RWWHU ³6PDOO SRZHU ORDG GLVDJJUHJDWLRQ LQ RIILFH EXLOGLQJV EDVHG RQ HOHFWULFDO VLJQDWXUH FODVVLILFDWLRQ´ ,Q 3URFHHGLQJV RI ,QWHUQDWLRQDO (QHUJ\ &RQIHUHQFH (1(5*<&21/HXYHQ%HOJLXPSS,(((

>@ 7 F )X $ UHYLHZ RQ WLPH VHULHV GDWD PLQLQJ (QJLQHHULQJ $SSOLFDWLRQVRI$UWLILFLDO,QWHOOLJHQFH, YROSS >@ %.DOOXUL$.DPLODULV6.RQGHSXGL+:.XDDQG7.ZRN:DL

³$SSOLFDELOLW\ RI XVLQJ WLPH VHULHV VXEVHTXHQFHV WR VWXG\ RIILFH SOXJ ORDGDSSOLDQFHV´-RXUQDOIRU(QHUJ\DQG%XLOGLQJVYROSS 

>@ 3 6HQLQ - /LQ ; :DQJ 7 2DWHV 6 *DQGKL $ 3 %RHGLKDUGMR & &KHQ 6 )UDQNHQVWHLQ DQG 0 /HUQHU ³*UDPPDUYL]  D WRRO IRU JUDPPDUEDVHG SDWWHUQ GLVFRYHU\ LQ WLPH VHULHV´ ,Q-RLQW (XURSHDQ &RQIHUHQFH RQ 0DFKLQH /HDUQLQJ DQG .QRZOHGJH 'LVFRYHU\ LQ 'DWDEDVHV1DQF\)UDQFHYROSS$&0

>@ -/LQ(.HRJK/:HLDQG6/RQDUGL³([SHULHQFLQJ6$;DQRYHO V\PEROLF UHSUHVHQWDWLRQ RI WLPH VHULHV´ -RXUQDO IRU 'DWD 0LQLQJ DQG .QRZOHGJH'LVFRYHU\YROSS

>@ 3 6HQLQ DQG 6 0DOLQFKLN ³6$;960 ,QWHUSUHWDEOH WLPH VHULHV FODVVLILFDWLRQ XVLQJ VD[ DQG YHFWRU VSDFH PRGHO´ ,Q3URFHHGLQJV RI ,((( FRQIHUHQFH LQ 'DWD 0LQLQJ 'DOODV 7; 86$  SS  ,(((

>@ %.DOOXUL$.DPLODULV6.RQGHSXGL+:.XDDQG7.ZRN:DL ³&ODVVLI\LQJ2IILFH3OXJ/RDG$SSOLDQFH(YHQWVLQWKHFRQWH[WRI1,/0 XVLQJ 7LPHVHULHV 'DWD 0LQLQJ´ ,Q 3URFHHGLQJV RI WKH WK 5(+9$ :RUOG&RQJUHVVDQG&/,0$$DOERUJ'HQPDUNYRO >@ $ 5HLQKDUGW DQG 6 .RHVVOHU ³3RZHU6$; )DVW PRWLI PDWFKLQJ LQ

GLVWULEXWHG SRZHU PHWHU GDWD XVLQJ V\PEROLF UHSUHVHQWDWLRQV´ ,Q 3URFHHGLQJV RI ,((( WK &RQIHUHQFH RQ /RFDO &RPSXWHU 1HWZRUNV :RUNVKRSV(GPRQWRQ$%&DQDGDSS,((( >@ 2 3 3DWUL $ 9 3DQDQJDGDQ & &KHOPLV DQG 9 . 3UDVDQQD

³([WUDFWLQJ GLVFULPLQDWLYH IHDWXUHV IRU HYHQWEDVHG HOHFWULFLW\ GLVDJJUHJDWLRQ´ ,Q 3URFHHGLQJV RI ,((( 7HFKQRORJLHV IRU 6XVWDLQDELOLW\6XV7HFK3RUWODQG2586$SS,(((

>@ &'0DQQLQJ35DJKDYDQDQG+6FKXW]H³6FRULQJWHUPZHLJKWLQJ DQGWKHYHFWRUVSDFHPRGHO´,Q ,QWURGXFWLRQWR,QIRUPDWLRQ5HWULHYDO SS