A System for High-Volume Acquisition and Matching of Fresco Fragments

A System for

High-Volume Acquisition and Matching of Fresco Fragments

Reassembling Theran Wall Paintings

Benedict Brown^1,2, Corey Toler-Franklin¹, Diego Nehab^1,3, Michael Burns¹, Andreas Vlachopoulos⁴, Christos Doumas^4,5,

David Dobkin¹, Szymon Rusinkiewicz¹, Tim Weyrich^1,6

1Princeton University

3Microsoft Research

5National University of Athens

2Katholieke Universiteit Leuven

4Akrotiri Excavations, Thera

6University College, London

Bronze Age Thera

• Modern day Santorini

• Aegean civilization: c. 1700 BC

• Traded with other

Mediterranean civilizations

• Evidence of fishing, agriculture, and livestock

• Volcanic eruption c. 1650 BC

NASA Visible Earth

Akrotiri

• Major archaeological excavation since 1967

• Well-preserved by ash

• Most significant find:

plaster wall paintings

–

Pigments excellently preserved

Thera Foundation

Akrotiri

• Major archaeological excavation since 1967

• Well-preserved by ash

• Most significant find:

plaster wall paintings

–

Pigments excellently preserved

–

But shattered in pieces by earthquake

The Akrotiri Jigsaw

• Current assembly

process is laborious

The Akrotiri Jigsaw

• Current assembly process is laborious

• Enough work for

another century

Fragment Characteristics

• Conservators consider:

–

size, thickness

–

level of erosion

–

discoloration and fading

–

set of pigments

–

curvature / flatness

–

texture of the back

–

string impressions

Constrained 3-D Acquisition Protocol

• Automatic turntable control

• Acquire scans at 45°

• Two 360° scan sequences

–

Face-down: front face at known plane

–

Face-up: front face visible

Color and Normals: 2-D Acquisition

• Custom scan software

–

One-click acquisition

–

Preview scan locates fragment

• _{Five scans}

–

Four front orientations (photometric normals)

–

One back orientation

Scan Alignment with Multi-Way ICP

• Align fragments scanned on turntable

–

Axis of rotation gives initial guess

–

Standard algorithm to improve alignments:

Iterative Closest Points [BESL 1992], [CHEN 1992]

• Flat front surfaces lead to instability

• Improved algorithm: Multi-way ICP

–

Constrain all scan-to-scan

transformations to be identical

–

Equivalent to solving for a single rotation axis

Front/Back Alignment

• Flipping fragment is uncalibrated

• Little overlap between front and back scans

• Front/back alignment is

vertically unstable

Front/Back Alignment

• Use front face to determine vertical alignment

–

Visible in front scans

–

On (calibrated) turntable surface in back scans

• Initial guess and ICP for

within-plane alignment

2-D/3-D Alignment

• Flatbed scanner has superior color

• Can’t use calibration

[LEVOY 2000]

, reliable silhouette

[LENSCH 2000]

, or features

[LIU 2006] [CHEN 2007]

• Use image alignment: PCA + downhill simplex

Projected 3-D Color Flatbed Scan

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Ribbon Matching

• Try all possible alignments

• Update alignment incrementally

• Regular edge

parameterization:

similar to image

correlation

Fragment Matching

ICP Matching

– Nearest neighbor

correspondence search

– Iterate to find matches

– 45 seconds per fragment pair

Ribbon Matching

– Regular edge sampling for correspondences

– Exhaustive search with incremental update

– 2 seconds per pair

Original (irregular) mesh Resampled ribbon

Erosion Detection

• Erosion causes incorrect alignments

• Detected on ribbons with normal constraint

Fragment Back Fragment Front

No Erosion Detection

Erosion Detection

• Erosion causes incorrect alignments

• Detected on ribbons with normal constraint

Fragment Back Fragment Front

No Erosion Detection With Erosion Detection

Outline

• System design

• Processing pipeline

• _Matching

• _Results

Ribbon Matching Results

Synthetic Fresco

25 mm strip width 12.5 mm strip width 50 mm strip width

Future Work (Matching)

• Multi-cue matching

• Improved ribbon matching/

Handling gaps

–

Dynamic programming can probably handle gaps

–

Record all possible alignments instead of only best candidates to do saliency analysis

• Global matching

–

Fuse matched fragments and re-match

–

Do global consistency checks on networks of matches

Future Work (Scanners)

We want to scan:

• large fragments

• assembled edges?

• edge and back normals

Approach:

• Hand-held scanner

• Two cameras and a projector/fixed pattern

• Alignment similar to in-hand scanner

• Should be able to get normals from mutiple views

Future Work (Scanners)

We want to scan:

• large fragments

• assembled edges?

• edge and back normals

Approach:

• Hand-held scanner

• Two cameras and a projector/fixed pattern

• Alignment similar to in-hand scanner

• Should be able to get normals from mutiple views

Acknowledgments

• Princeton University: Tom Funkhouser, Dimitris Gondicas, Matt Plough, Phil Shilane, Xiaojuan Ma

• Akrotiri Excavation, Laboratory of Wall Paintings:

Manolis Hamaoui, Litsa Kalambouki, Marina Papapetrou, Panagiotis Vlachos, Alexandros Zokos, Iakovos Michailidis, Fragoula Georma, Niki Spanou

• Special thanks to David Koller (University of Viriginia),

Misha Kazhdan (Johns Hopkins University), and Peter Nomikos Jr.

• Funding: Thera Foundation, Kress Foundation,

Seeger Foundation, Cotsen Family Foundation, and NSF Grants CCF-0347427 and CCF-0702580