Image and Model based 3D Bone Reconstruction

Results of 3D reconstruction given the global atlas constraint and local image constraints. In the top row from left to right: S₂, and final shape S_K from SFS, registered atlas S_K^a, refined shape S_F^a by maximizing the likelihood. In the bottom row from left to right: S₅ from SFS, S_K compared with laser scanned ground truth, S_K^a compared with the ground truth, S_F^a compared with the ground truth. The hausdorff distance is displayed using the HSV color map.

ABSTRACT

During minimally invasive orthopedic surgeries, knowing the 3D shape of bone surfaces in real time will greatly enhance the understanding of interior anatomy. We have proposed a multi-image bone reconstruction algorithm from endoscopic images to achieve this goal. However, this method relies on the surface shading alone and attempts to solve the well-known but under-constrained shape-from-shading problem leading to over-smoothness of shapes. In order to deal with this problem and partial occlusions in the surgical environment, we introduce the statistical shape atlas as a prior to constraint the multi-image shape-from-shading (MISFS) of endoscopic images of the spine. We first generate the best representation of the MISFS reconstruction from the atlas. We apply the surface normals from this generated shape as soft constraints for the MISFS algorithm and reconstruct the shape under the new constraints. Since the reconstruction may differ under the new constraints, the corresponding representation from the atlas may change as well. We repeat above two steps until the new atlas generation is close enough to the one in the previous step. This atlas representation is considered the best reconstruction given the bottom-up MISFS method. To furthermore improve the result we propose a top-down refine step. We synthesize a series of 2D images from the atlas representation given the same poses where we captured the original images. By maximizing the likelihood of the image gradients, we are able to refine the atlas coefficients locally. Finally combining bottom-up MISFS and top-down refine procedures the best 3D shape of the spine can be obtained.

PUBLICATIONS

“Spine Vertebra Reconstruction from Multiple Endoscopic Images under Atlas Constraints”

Chenyu Wu, Srinivasa G. Narasimhan, Branislav Jaramaz

Submitted to IEEE Transactions on Medical Imaging, 2009

[PDF]

VDIEOS

	Capture Endoscopic Images (WMV)
	Compare SK (Bottom Up Method without Atlas Constraint) to Ground Truth (AVI)
	Compare SK ^a (Bottom Up Method with Atlas Constraint) to Ground Truth (AVI)
	Compare SF ^a (Bottom Up Method with Atlas Constraint + Top Down Method) to Ground Truth (AVI)

PICTURES (click on thumbnails to enlarge images)

	Setup of Capturing Endoscopic Images
	Illustration of the bottom up method: The artificial vertebra is showed in the middle. Surrounding images are captured by an oblique endoscope from different poses involving rotations and translations, which are illustrated by colored circles. All the images are geometric rectified and normalized for illumination.
	Illustration of the top down method: (a) OpenGL rendering SK ^a. (b) Corresponding endoscopic image. (c) Synthesized endoscopic image from the same camera pose. (d) Normalized endoscopic image. (e) Gradient of synthesized image Ga. (f) Gradient of normalized image Go. We generate a series of images at different poses and compare them with the original endoscopic images. Since the imaging modalities are different between the synthesized images from the atlas shape and the original endoscopic images, the image appearances have a lot of difference. We thus minimize the difference on image gradients which are most robust to lighting variations, by manipulating the coefficients of the atlas shape. With this method we refine SK ^a to get the final shape SF ^a.
	Results of 3D reconstruction given the global atlas constraint and local image constraints: In the top row from left to right: S2, and final shape SK from SFS, registered atlas SK^a, refined shape SF^a by maximizing the likelihood. In the bottom row from left to right: S5 from SFS, SK compared with laser scanned ground truth, SK^a compared with the ground truth, SF^a compared with the ground truth. The hausdorff distance is displayed using the HSV color map.

Last Update: September 2009