Fusion of different modalities has great potential in modern medical imaging if we can extract and combine the advantages of all of them. In this work, we develop a diagnostic tool using the electroencephalogram (EEG) and magnetic resonance imaging (MRI) data to locate anatomical origins of brain abnormalities. First, we use a novel and computationally efficient direct modeling approach (DMA) for constructing an accurate model of the head in 3D using MRI data. Then, we embed the EEG dipole source location into the head coordinate system using the independent component analysis ( ICA) and the downhill simplex algorithm on the activation map of each independent source. Finally, a transformation matrix is derived for projecting the dipole location from the conventional head coordinates onto the 3D head model.

The basis of DMA is that a black background surrounds the pixels that correspond to the head. First, the entire MRI scan data is converted into a bitmap file format. Each slice is then processed to extract pixels that represent the head. To increase processing efficiency, pixels from the centre of the head, which would not affect the final boundary of the head, are eliminated. These slices are stacked up in order to form the 3D mesh model in OpenGL. Thus, the head could be modelled in 3D without the conventional segmentation process, thus greatly reducing the computational complexity and the processing and rendering time, while ensuring accuracy.

Source localisation from EEG is an ill-posed inverse problem. The solution to this problem is not unique as different configurations of sources can generate identical scalp potential maps. To solve this inverse problem, ICA can be applied to the EEG data. ICA decomposes the EEG scalp recordings into several independent sources, each of which produces unique activation maps when projected onto the scalp. Each of these maps is then used to localise the respective dipoles using the downhill simplex method. In this way, the dipole locations can be estimated from the EEG recordings.

The above methods give the dipole source location in the head coordinate system. This location is then converted into the MRI coordinate system to superimpose the EEG source location on the MRI data.

An OpenGL rendering of the modelled head is shown in Figure 1. The model can be rotated and translated to increase viewing dimensions. A slider moves along the height of the model to display the MRI slice used for modeling the layer corresponding to the slider position. Using this model, clinicians can study the anatomical origins of brain signals and their relation with the surrounding tissues. Using the dipole locations embedded on the MRI slices, doctors can study how the brain signals have been affected due to various disorders. Results from this project show that DMA can be used as a useful tool in understanding the mysteries of the most complex organ, the brain.