How to transform your images into 3D models

(1)

ARC 3D Webservice ARC 3D Webservice

How to transform your images into 3D models

Maarten Vergauwen Maarten Vergauwen

info@arc3d.be info@arc3d.be

Overview Overview

►►

What is it? What is it?

►

How does it work? How does it work?

►►

How do you use it? How do you use it?

►

Results Results

What does the 3D Webservice do?

images © Maurizio Forte, CNR Rome

The ARC 3D Webservice allows you to transform your images into 3D models automatically. You only need a digital camera, a PC and access to the Internet.

What are the different steps?

►

►Step 1: Record a sequence of Step 1: Record a sequence of images of a scene or object images of a scene or object

►

►Step 2: Upload the images to Step 2: Upload the images to the ARC server

the ARC server

►

►Step 3: The server computes Step 3: The server computes the 3D reconstruction the 3D reconstruction

►

►Step 4: Download the results Step 4: Download the results from the ARC website from the ARC website

►

►Step 5: Visualize the resultsStep 5: Visualize the results

Overview Overview

►►

What is it? What is it?

►

How does it work? How does it work?

►

How do you use it? How do you use it?

►►

Results Results

How does it work?

►►

The 3D Webservice only uses images as The 3D Webservice only uses images as input.

input.

►

Computer vision techniques are used to Computer vision techniques are used to compute the results.

compute the results.

Automatic matching of images Automatic matching of images

Automatic computation of camera calibration Automatic computation of camera calibration

Automatic dense matching for depth map Automatic dense matching for depth map generation

generation

(2)

Image Matching Image Matching

►►

The entire process is based on finding matches The entire process is based on finding matches between images.

between images.

Record your pictures not too far apart, so the Record your pictures not too far apart, so the computer can match them easily!

computer can match them easily!

Camera Calibration Camera Calibration

►

No prior knowledge on camera calibration is No prior knowledge on camera calibration is available, so all information must be recovered available, so all information must be recovered from the images

from the images

►►

It is therefore important that enough information It is therefore important that enough information is present in the images!

is present in the images!

General motion of the cameraGeneral motion of the camera

General structure in the sceneGeneral structure in the scene

Enough overlap (only points that are visible in at least 3 Enough overlap (only points that are visible in at least 3 images are useful)

images are useful)

See

See tips and tricks

tips and tricks

for more info for more info

Dense Matching Dense Matching

►

After recovery of the After recovery of the camera calibration, camera calibration, dense depth maps are dense depth maps are computed

computed

►

These contain the These contain the depth of every pixel depth of every pixel and a quality measure and a quality measure (how confident we are (how confident we are of each particular of each particular pixel)

pixel)

Overview Overview

►►

What is it? What is it?

►

How does it work? How does it work?

►►

How do you use it? How do you use it?

►

Results Results

How do you use it?

►

Obtain an account from the website Obtain an account from the website

►

Download and install the programs Download and install the programs

►

Use the upload program Use the upload program

Load your images Load your images

Upload them to the server Upload them to the server

►

Have a coffee Have a coffee

►

Download the results from the server Download the results from the server

►►

Visualize the results Visualize the results

Obtain an account Obtain an account

Surf to www.arc3d.beand navigate to Request Login

(3)

Obtain an account (2) Obtain an account (2)

Fill in name, email address and the captcha form

Obtain an account (3) Obtain an account (3)

Your login information will be sent to your email address

Download and install Download and install

Install as Administrator in the default place or as normal user, specifying a writeable folder

Upload Client Upload Client

Upload Client (2) Upload Client (2)

►

►Images selected by userImages selected by user

►

►Subsample to smaller size Subsample to smaller size for faster upload and for faster upload and computation computation

►

►Authentication with server Authentication with server

@ ARC

►

►Organization into scenes Organization into scenes and sequences and sequences

Select or create a new sceneSelect or create a new scene

Supply a sequence labelSupply a sequence label

Add academic reference if Add academic reference if desired*

desired*

* If ARC decides to use your images in a publication, it will add this reference. See conditions on the ARC homepage.

Have a coffee

(4)

Download the results Download the results

►►

The ARC service reports its result to you by The ARC service reports its result to you by email

email

►

If the reconstruction has succeeded, a URL If the reconstruction has succeeded, a URL is specified from where you can download a is specified from where you can download a zip file containing the results.

zip file containing the results.

►►

If the reconstruction failed, some more If the reconstruction failed, some more information is given, including tips on how information is given, including tips on how to improve the result.

to improve the result.

Visualize the results Visualize the results

►

Two tools are available Two tools are available for inspecting the for inspecting the results

results

►►

The ARC team The ARC team recommends

recommends MeshLab

MeshLab,

, a tool developed by a tool developed by CNR CNR- -ISTI in Pisa, Italy ISTI in Pisa, Italy

►►

A simple model viewer A simple model viewer comes with the ARC comes with the ARC installer

installer

Visualize with MeshLab Visualize with MeshLab

Download MeshLab from meshlab.sf.netand read the documentation

Visualize with Model Viewer Visualize with Model Viewer

►

►Download results from serverDownload results from server

►

►Unzip file with images, cameras, Unzip file with images, cameras, depth and count maps depth and count maps

►►Select one image on the left. Select one image on the left.

Reconstruct 3D model for this Reconstruct 3D model for this specific depth map specific depth map

►►Parameters to tune:Parameters to tune:

Minimum quality (count)Minimum quality (count)

Subsample for smaller modelsSubsample for smaller models

Minimum angle w.r.t. viewing Minimum angle w.r.t. viewing direction. This ensures that direction. This ensures that shearing angles are removed shearing angles are removed

Median filter size: larger filter Median filter size: larger filter for smoother (less noisy) for smoother (less noisy) models

models

Visualize with Model Viewer (2) Visualize with Model Viewer (2)

►►

Select the model tab Select the model tab and view the result in and view the result in a 3D widget

a 3D widget

►

Texture comes from Texture comes from the original image the original image

►►

How to transform your images into 3D models

ARC 3D Webservice ARC 3D Webservice