Mesh optimisation

Reconstruction in the roundwith Photometric Normals

George Vogiatzis,Carlos Hernandez Esteban, Roberto Cipolla

Uniform albedo Lambertian objects (with highlights)

• Challenging objects

• Lack of features makes correspondences hard

• Silhouetteand shadingare only available cues

Photometric stereo

• Single Viewpoint

• Move light-source for each image

• Same pixel always corresponds to same surface point

• With known light directions can estimate n

• Integrate normals to get depth map

Photometric stereo

• To get more than depth-maps, we need multiple-viewpoints…

• … and in that case pixels are no longer automatically in correspondence

• However, if some correspondence is given, photometric stereo can proceed as usual

• Our strategy:

1.Estimate light directions & intensities

2.Evolve a surface using photometric stereo with approximate correspondences from the current surface (starting from visual hull)

1. Light estimation

• Three surface points with known surface normals and their image intensities are enough to estimate a directional light source

• But where do you get these three points ?

1. Light estimation

• Answer: From the visual hull

1. Light estimation

Light estimation

• Recover generators by random sampling

Accuracy of light estimation

1. Light estimation

2. Multi-view photometric stereo

• Mesh with vertices x1,…,xM

• And faces f=1,…,F

• Define photometric normals v1,…,vF

• Minimise sum of two energies

• Emwith respect to x1,…,xM

• Evwith respect to v1,…,vF

2. Multi-view photometric stereo

• evolution

Full algorithm for uniform objects

Results

Results

Results

Results

Results –Fitzwilliam

Museum

Results –Fitzwilliam

Museum

Results –Fitzwilliam museum

Results –Fitzwilliam

Museum

Results –Fitzwilliam

Museum

Multi-view Dense Stereo [Hernandez et al 2004]

Multi-view Photometric Stereo

Buddha

• For multi-albedo objects the technique  can improve the results of state-of-the-art correspondence based techniques e.g. [Hernandez et al 2004]

Results –Fitzwilliam

museumBuddha

Multi-view Photometric StereoMulti-view Dense Stereo [Hernandez et al 2004]

• For multi-albedo objects the technique  can improve the results of state-of-the-art correspondence based techniques e.g. [Hernandez et al 2004]

Other sources:

• Mesh opimisation
• Towards flattenable mesh surfaces 
• A CL surface deformation approach [PDF]