Image Fusion – 3D and Quantitative Imaging Laboratory

Medical imaging offers a variety of scan types that are used to view specific functions or anatomy. One scan type may provide a completely different set of information from another scan type when imaging the same portion of the body. Medical image fusion is the technique of combining these different scan types into one aligned set of data, providing the information of both scans while maintaining their anatomical relationships. Each modality has strengths and limitations, which is why combining them can be so beneficial.

How is Image Fusion Performed?

The process of medical image fusion begins with the collection of images from various sources. These images are refined for quality and aligned using the registration technique. Learn more about image registration in our article here.

After the images are aligned an algorithm is used to combine the two scans. The complexity of these algorithms varies, from basic pixel analysis to intricate methods like wavelet transformation, which breaks down an image by various details of its shapes. The use of Artificial Intelligence (AI) and machine learning is increasingly popular in this field. AI is skilled at identifying patterns in large amounts of data, which helps to accelerate and enhance the accuracy of the image fusion process.

A radiology technologist verifies the accuracy of the image fusion before creating any further imaging or measurements.

Figure A: A CT scan of the head. Electrodes can be seen implanted into the brain by passing through the skull. CT imaging is particularly useful for displaying solid structures such as bone and metal.

Figure B: MRI scan of the same patient as Figure A, displaying the details of the brain surface and internal structures such as the ventricles. MRI scans are exceptional at displaying the soft tissue structures that CT scans cannot but are less useful for bone structures and cannot display metal.

Figure C: Fused images from both Figure A and B. The positioning of the electrodes from the CT scan are visualized with the brain structures of the MRI scan. This fusion was additionally edited to remove the skull and other non-important structures.

Image Fusion in the 3DQ Lab

The 3DQ Lab uses image fusion to create visuals for various neurological interventions. Before discussing examples of fusion, it’s important to understand the individual imaging used below.

CT (Computed Tomography): Best for viewing bones and blood vessels. It uses X-rays to create cross-sectional images of the body.
Functional MRI (fMRI) Measures and maps brain activity by detecting changes in blood flow, important for treatment and diagnosis of neurological conditions.
Structural MRI (Magnetic Resonance Imaging): Excellent for viewing soft tissues like organs and muscles. It uses magnetic fields and radio waves to create detailed images.
Diffusion Tensor Imaging (DTI): A type of MRI that maps the pathways of nerve tracts within the brain.

Functional MRI

Figure D: Structural MRI is displayed as grayscale, and the processed functional MRI data is represented in red and blue. These colorized areas highlight regions of the brain that are utilized in motor function.

Diffusion Tensor Imaging (DTI) reveals details about the location and strength of neural pathways, offering more than just anatomical information. When fused with structural MRI, the resulting fMRI images can pinpoint critical areas for language and motor functions in the brain. This is useful in understanding the impact of a stroke on these functions and in planning surgical procedures to remove affected tissue while preserving motor capabilities.

Fiber Tractography

Figure E: Fiber Tractography combines MRI and DTI technologies similar to Figure D. The structural MRI is displayed as grayscale, and the processed fiber tracts are multi-color.

Fiber Tractography maps the pathways of nerve fibers, with colors in the images indicating the fibers’ directions, not their functions. The images are used for visualizing the spatial relationship between white matter tracts and any brain lesions, in addition to displaying areas to avoid during brain resection.

Fusion Grid

Figure F: Seen in Figure C above, a Fusion Grid is created from the combination of MRI and CT scans. The structural MRI is grayscale and the CT electrodes are white.

A Fusion Grid is used to place electrodes that monitor brain activity and help pinpoint the origin of seizures. In certain cases, these implants can also deliver targeted electrical stimulation to interrupt or lessen the severity of a seizure. The data gathered from this monitoring is important for a neurological team, informing decisions on medication adjustments or surgical interventions to manage and treat epilepsy.

Tumor Quantification

Figure G: Fusion of structural MRI data in grayscale, and multi-colored tumor perfusion data.

Tumor quantification is used for accurate cancer staging, treatment planning, and monitoring therapy effectiveness. This is achieved by combining a structural MRI with a scan that displays the varying blood flow rates of the tumor, with varying rates displayed as distinct colors to differentiate their intensity. These colored perfusion rates are then fused with the structural MRI to display their location in relation to nearby anatomy.

3D Printing

Figure H: A 3D reconstruction of the spine and spinal cord. Bone CT is displayed as yellow, MRI bone is pink, and MRI spinal cord is green.

Fusion is also utilized in 3D printing to combine the strengths of different scanning modalities. During the segmentation process, image masks, or 3d representations of selected anatomy, can be combined with different scans of the same patient. This allows healthcare professionals to hand-pick structures from multiple scans and combine them into a single 3D scene for printing.