ABSTRACT
Bone
reconstruction using endoscopy is important for computer
aided minimally invasive orthopedic surgery. During
surgery an endoscope consisting of a camera and one
or more light sources is inserted through a small incision into
the body and the acquired images are analyzed. Since
bone surface is featureless, shading is the primary cue
for shape perception. However, due to the small field of
view of the endoscope, only a small part of the bone and its
occluding contour are visible in any single image. Therefore even
human perception of bone shape from such images can
be hard. We present a novel technique to reconstruct the surface of
the bone by applying shape-from-shading to a sequence of
endoscopic images, with partial boundary in each image. We
first perform geometric and the photometric calibration for
the endoscope. We then extend the classical shape-from-shading algorithm
to include a near point light source that is
not optically co-located with the camera. By tracking the
endoscope we are able to align partial shapes obtained from
different images in the global (world) coordinates. An ICP
algorithm is then used to improve the matching, resulting in
a complete occluding boundary of the bone. Finally, a
complete and consistent shape is obtained by simultaneously re-growing
surface normals and depths in all views. We
demonstrate the accuracy of our technique using simulations and
experiments with artificial bones.
|
PUBLICATIONS
“A Multi-image
Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes”
Chenyu Wu, Srinivasa G. Narasimhan, Branislav Jaramaz,
International Journal of Computer
Vision (IJCV), Feb. 2009
[PDF]
“Shape
Reconstruction from Endoscopic Images”
Chenyu Wu, Srinivasa G. Narasimhan, Branislav Jaramaz
8th Annual
Meeting of the Int. Society for Computer Assisted Orthopaedic
Surgery (CAOS’08), June 4-7, Hong
Kong, China,
2008
[PDF]
“Shape-from-Shading under Near
Point Lighting and Partial views for Orthopedic Endoscopy”
Chenyu Wu, Srinivasa G. Narasimhan, Branislav Jaramaz,
Workshop on Photometric Analysis For Computer Vision (PACV’07), in conjunction with ICCV’07
[PDF] [Adobe Best Paper, PACV 07]
“Endoscope
Calibration and Derivation for Shape from Shading”
Chenyu Wu, Srinivasa G. Narasimhan, Branislav Jaramaz
Tech. Report
CMU-RI-TR-07, Robotics Institute, Carnegie Mellon
University, Dec, 2007
[PDF]
|
PICTURES (click on thumbnails to enlarge images)
|
Perspective projection
model for endoscope imaging system with two near point light sources: ~O
is the camera projection center. ~s1 and ~s2 are two
light sources. We assume the plane consisting of ~O, ~s1 and ~s2
is parallel to the image plane. The camera coordinate system (X - Y -
Z) is centered at ~O and Z-axis is parallel to the optical axis and
pointing toward the image plane. X-axis and Y-axis are parallel to the
image plane. F is the focal length. a
and b are two parameters related to the position of the light
sources. Given a scene point ~P = (x; y; z), the projected
image pixel is ~p = (~x; ~y; F), where (~x; ~y)
are image coordinates. Assuming a Lambertian
surface, the surface illumination therefore depends on the surface albedo, light source intensity and fall-off, and the
angle between the normal and light rays.
|
|
This shows the results
of shape from shading from a single image. (a) Input image. (b) Shape from
shading. (1) - (5) are captured from different viewpoints.
|
|
Simulation results of
shape from shading from multiple views. (a)-(d)
Synthesized images of different parts of a sphere. (e) Reconstructed
sphere.
|
|
Illustration
of the problems about directly merging the individual shapes in the world
coordinates. 18 images are captured by moving the endoscope horizontally
(only translation). Four of them are shown as an illustration. (a) After
removing the distortion and illumination effects, the boundaries in each
image are labeled by hand, and the initial (p; q) are computed
automatically on the boundaries. (b) Shape from each single image is
reconstructed. (c) Unaligned shapes in the world coordinates. (d) Unaligned
3D contours in the world coordinates.
|
|
Illustration of the multi-image
shape-from-shading algorithm. (a) Aligned shape in the world coordinates. (b)
Aligned 3D contours in the world coordinates. (c) Projection of the global
constraints onto each image. (d) Final shape are
reconstructed.
|
|
Different views of the reconstructed surface (yellow) against the
ground truth (red). (a) View from the top (b) View
from the bottom (c) View from the left side (d) View from the right side.
|
|
(a) Real shape captured by a regular camera. (b) Reconstructed shape
under orthographic projection. (c) Reconstructed shape under perspective
projection.
|
|
