How can researchers visualize an epileptic seizure within the brain? What does the flow of electrical current emanating from the heart look like as it propagates through the body? Using computer simulation and visualization, biomedical researchers can examine increasingly complex systems posing problems that were unapproachable only a few years ago. Advances in computer hardware and algorithms have made it possible to create three-dimensional computer visualizations of such physiologic phenomena. In particular, some recent applications of computer graphics and visualization to problems in cardiology and neuroscience have provided researchers and physicians with powerful tools for understanding and addressing important medical conditions.

Introduction

From Leonardo Da Vinci's anatomical drawings to Wilhelm Roentgen's first X-ray of the human hand to today's use of computer graphics and virtual reality to ``fly through'' three-dimensional reconstructions of magnetic resonance imaging (MRI) data, researchers and physicians have for centuries used visualization in their quest to understand human physiology.

The goals of medical simulation and visualization are multifaceted. While some simulations and visualizations facilitate diagnosis, others help physicians plan surgery, therapy, and other forms of treatment. Still other simulation and visualization techniques are used for medical training and to acquire a better understanding of human physiology. Among the most exciting and pressing problems facing computer scientists today are those involving the use of computers in medical applications. With the help of computers, the medical community now verges on important breakthroughs in diagnosing, controlling, treating, and even curing numerous life-threatening conditions, including heart disease and a variety of cancers.

Over the past two decades, the techniques of computer simulation and visualization have had a substantial impact on the field of medicine, just as they have on other areas of engineering. The reasons for this are numerous. First, the increasing sophistication of computers, particularly increases in memory capacity, CPU speed, and graphics hardware, has produced a parallel increase in the size and complexity of medical data that can be realistically visualized. At the same time, computer simulation allows biomedical researchers to subject these increasingly high resolution models and visualizations to increasingly sophisticated quantitative and qualitative examination.

Still, the complexity of the human body far outstrips the capabilities of even the largest computational systems, and will do so for some time to come. However, computers are making it possible to graphically display images of both the anatomy and function of the internal organs, control robotic surgical devices, and simulate physiologic responses in ways that are becoming ever more crucial to the effective practice of medicine. In addition, they have been used successfully to suggest physiologically and clinically important scenarios and results in a number of areas. For instance, researchers at the University of Utah and elsewhere are using scientific computing and visualization to address clinical problems in a variety of areas including cardiology and neuroscience.

Cardiology and Neuroscience

Many times each second, the brain sends electrical impulses racing through the body's web of nerve cells to the motor neurons, where they initiate the electrochemical reactions that cause muscles to contract. Several decades ago, scientists recognized that these excitation currents produce an electrical field that can be detected as small voltages on the scalp. The heart also produces electric currents that flow through the thorax and generate voltage on the surface if the chest, the ECG. By measuring changes in the patterns of the body's electrical activity, scientists and physicians can detect some forms of heart disease and neurological disorders. In 1887 British physiologist Augustus D. Waller published the first human electrocardiogram. A few years later, Willem Einthoven repeated these measurements and distinguished five peaks, which he chose to call the P, Q, R, S, and T waves, thinking that more peaks and waveforms might be discovered and would need their own letters (to date, the U wave is the only additional peak). This first ECG machines were large and heavy compared to the small, light, and portable versions commonly used today. The first machines did not use single electrodes attached to the body, but rather patients inserted both arms and one leg into buckets of salt water (yielding the so-called limb leads). In 1924, German psychiatrist Hans Berger recorded the first electroencelphogram (EEG). The EEG electrodes measures the small electrical activity from the brain and do not have a regular sequence of waves like the ECG. Instead, the EEG contains continuous trains of activity and information is encoded more subtly in the frequency of the signals or their statistical features.

While the modern technologies of electrode design and electronic recording apparatus differ significantly from their predecessors, the ECG and EEG waveforms are essentially the same as those recorded by Einthoven and Berger. Even with the substantial advances in ECG and EEG technology, most of the machines in clinical use today provide relatively coarse descriptions of the overall electrical activity of the heart or brain. This limitation in resolution is primarily due to the fact that standard ECG and EEG measurements represent the cumulative electrical activity of the heart or brain as a very small number of simple point sources of bioelectricity. Physicians uses these glimpses to help spot disorders by comparing the patient's ECGs/EEGs with an atlas of waveforms that correspond to particular disease states. Compressing all this information into a small number of features is very efficient, but can lack the sensitivity and spatial resolution required for diagnosing many illnesses.

In some difficult cases, doctors turn to other techniques that are more invasive, costly, and painful and in rare cases, to exploratory surgery. In some cases of epilepsy, for example, physicians must establish whether the source of this abnormal electrical activity is well localized, and hence operable. At present, this diagnosis may require the application of electrodes directly to the surface of the brain. Similarly, in clinical cardiology there are forms of abnormal heart rhythm (arrhythmias) that do not respond to drug treatment or to artificial stimulation and must be treated surgically. Identifying the target site for this intervention, however, is seldom possible with standard ECGs and requires either roving catheters in the veins and the heart itself, or even more direct access via open-chest surgery. All these procedures improve diagnostic accuracy but at considerably increase in the risk, discomfort, and cost to the patient compared to an approach based on the ECG.

Now, using computer simulation and visualization, an interdisciplinary team of scientists at the University of Utah¹ is developing diagnostic tools that may reduce the need for these cases of preoperative surgery. Using computers, the researchers are simulating and visualizing the electrical fields emanating from the heart and brain. Using large-scale, three-dimensional computer models of the thorax and head, the researchers can produce more detailed visual representations of the electrical activity within the heart and brain than the currently used heart and brain snapshots from standard ECGs and EEGs, respectively. A primary goal of the researchers is to develop these techniques based on painless, risk-free measurements from the body surface and gain information that is now only available through highly invasive diagnostic procedures.

As an example of such a technique, to ``see'' the electrical activity on the surface of the heart technicians first attach electrodes to a patient's skin at optimally selected sites across the body. As in standard ECGs, these electrodes detect the changes in voltage that accompany each heart beat. Rather than using the 3 to 10 electrodes of a conventional ECG, however, this process, known as ``body surface potential mapping,'' or BSPM, utilizes 16 to 192 electrodes. BSPM produces a far more detailed image of electrical activity on the surface of the body than the conventional ECG.

While these body surface maps reveal more about the underlying electrical activity of the heart than the standard ECG, the signals are somewhat smoothed and lack definition. So that researchers can estimate the electrical activity right at the heart, the electrical readings are then incorporated into a large-scale computer model of the human thorax, constructed from MRI scans of the patient. Because the shape and constitution of the thorax affects how the heart activity is transmitted to the skin, the researchers include all relevant tissues and organs into the computer model. The level of complexity of the resulting model represents a profound improvement over the thorax models commonly used by scientists even a decade ago.

Measurement, modeling, and simulation of brain activity follow a similar course. Researchers place an array of electrodes, ranging from the standard 32 to up to 128, onto the scalp. These electrodes record the electrical activity of the brain (the EEG) from the surface of the head. Researchers use these EEG recordings, along with an individualized computer model made from MRI scans of a specific patient's head, to localize abnormal electrical activity.

Before electrical imaging procedures become common tools in clinics, researchers have more work to do. It is one thing to visualize geometry and anatomy; it is another thing to enable physicians in the clinical setting to see what is occurring within an anatomical system in a way that is meaningful to their work with patients. Computer modeling and the process of allowing researchers and physicians to ``see'' back to the electrical source (an inverse problem) involve enormous computations that now sometimes require supercomputers or distributed clusters of high-end workstations. While these problems depend on highly complex mathematics, the solutions must be provided invisibly, then visualized quickly for models to be clinically useful. Researchers have not yet accomplished the levels of accuracy and speed required, but early results from actual clinical uses of their methods suggest exciting promise for the possibilities in the near future.

Another use for this technology is in neurosurgery. Our research team is currently collaborating with the Surgical Planning Laboratory and Department of Neurosurgery at Brigham's and Women's Hospital/Harvard Medical School and radiologists and neurosurgeons at the University of Utah to develop an interactive software environment for neurosurgery and surgical planning. New ``open magnet'' MRI imaging systems now exist in which the patient, surgeon, and surgical assistant are all within the imaging field. This allows the surgeons to view 2D MRI scans as they are performing the surgery (with special polymer surgical instruments). What the surgeons want, however, is to view full 3D volume rendered images from the MRI output. Furthermore, they want to be able to manipulate these 3D images and to overlay/register other visualizations from simulations or other imaging modalities on/within the 3D MRI images. They also wish to perform new simulations (of the electrical activity from a temporal lobe epilepsy, for example) during the surgery. All of these goals require interactive visual supercomputing of large-scale data, within a time-critical environment. This is a very challenging task, which will require significant advances in both computer hardware and software.

Cardiac Defibrillation Device Design

Over 500,000 people in the US die each year due to ventricular fibrillation resulting from coronary heart disease and myocardial infarction. Danger strikes when the heart alters its normal, steady beat and slips into a condition called fibrillation. Fibrillation is a state in which the electrical activity throughout the heart is scrambled, resulting in the inefficient pumping of blood. Unless the fibrillation is reversed using an applied electrical shock, the condition leads to death. In many cases, because the patient goes from apparent good health to death so quickly, this condition is called "sudden cardiac death".

While external defibrillation systems that are applied to treat fibrillation have long been in use, researchers have only rather recently developed ``implantable'' defibrillation units that automatically detect and regulate an arrythmic heartbeat. Implanted within the patient's chest with electrodes placed near the heart (at a rate of over 30,000 per year in the US today), these devices first detect the earliest manifestations of abnormal electrical activity that are the hallmark of fibrillation. The device then attempts to restore normal rhythm by pacing the heart, but if this fails, the device can apply a jolt of electricity in order to return the heart to its normal beat. Such a defibrillation shock is, indeed, frightening and sometimes painful to the patient, but studies have shown that the use of defibrillators can significantly reduce mortality.

Concurrent with our research into computer modeling for medical diagnostic purposes, our research team is also using computer visualization and simulation to help bioengineers design the next generation of implantable cardiac defibrillation. A few years ago, software created by Utah researchers was used to evaluate a prototype internal cardiac defibrillator designed by engineers at Pacesetter, Inc. This software was capable of generating a computer model suitable for simulating electrode configurations and voltage distributions on the human thorax. To help design defibrillators, researchers have used our computer models of the thorax to simulate various configurations of electrodes and stimulation pulses and visualize the results with computer graphics using the SCIRun computational steering system. With our software, engineers are able to imagine improvements to a device, and place them directly into the computer model. They are thereby able to test new dfibrillator designs using the computer prior to animal and/or human trials, thus increasing the safety of the device. The use of interactive scientific visualization has been pervasive throughout this project (as in other aspects of the group's work) and is opening up new ways of perceiving and investigating the complexities of physiologic systems. Someday, implanted electrodes may be used routinely to manage electrical abnormalities in the heart.

There remain many apsects of difibrillator design still to be investigated and optimized. The device should deliver the proper electric shock to the heart, yet should not damage other internal organs. To complicate matters, many overweight individuals have an insulating layer of fat over their heart, reducing the effectiveness of the electrical shock. Additional important design questions include: Where are the best electrode locations? How many should there be? Does one design apply equally well to patients of different shape and size? It is clear that effective design of new generation defibrillators will only occur through the extensive use of computer simulation and visualization. The Utah team will continue to provide such specialized tools in collaboration with researchers in other institutions and industries.

Medical Imaging

While the first x-ray by the German physicist Roentgen occurred in 1895, the first full head scan from an x-ray computed tomography (x-ray CT) machine used for medical diagnosis was in 1974. An even more recent imaging, and now pervasive, technology is magnetic resonance imaging (MRI). The first MRI of a head occurred at Thorn-EMI Laboratories and Nottingham University in England, circa 1980. Both CT and MRI technologies have become a standard part of modern hospitals worldwide. Medical imaging provides pictures of organisms that can reveal anatomical, structural, and even functional information, as, for example, in computed tomography from X-ray images or captured reflections from ultrasound scanners. Other forms of imaging seek to describe the distribution in the body of electric potential or current or electric and magnetic fields; or it may describe the distribution of a particular substance, which may in turn reflects altered absorption or metabolic activity in some part of the body. At present, each of these measurement methods has its own computerized control and support, but seldom are the results from different methods merged in any quantitative way.

An interdisciplinary research effort at the University of Utah is to merge measurements, analysis, and visualization paradigms for a range of medical imaging techniques and couple them with associated simulation methods. Measurement techniques include electrical and magnetic imaging of the brain and heart (ECG, EEG, MCG, and MEG), magnetic resonance imaging (MRI), functional MRI and radionuclide imaging, X-ray computed tomography (CT), and ultrasound. Furthermore, we are working together to create new simulation algorithms and technologies for the coupling of imaging and simulation. The measurements will drive simulations of underlying function such as the electrophysiologic excitation and recovery of heart cells or the firing of different regions of the brain in response to external stimuli. Collaborations among measurement, analysis, visualization, and simulation specialists will provide insights into critical problems that would be unapproachable using each modality separately. The application of this paradigm will begin with medicine and specifically bioelectric fields from the heart and brain, but its potential utility extends to any domain in which one wishes to link measurements with functional descriptions of complex systems to gain new insights into system behavior.

Advances in medical imaging technology have been accompanied by substantial increases in the size of the data sets produced by these technologies. For instance, the first CT and MRI images had fairly course resolution (64 by 64 pixels such that a volume at this resolution would be 512,000 bytes). With the desire for more detail, the resolution of scanners has continued to increase. The resolution of today's images are 512 by 512 pixels such that a volume at this resolution would be 260 million bytes or 260 Mb). We will soon see increases in resolution to 1024 by 1024 pixels such that volumes of images will be over one billion bytes (1 Gb). Such high resolution images are a challenge to interactively visualize. As such, another aspect of our on-going research is to create new ways to visualize large-scale, high resolution data so that radiologists, surgeons, and researchers can interact with the data in intuitive and meaningful ways.

Computer Science

The applications mentioned here are only the beginning of a long list of potential uses for computer simulation and visualization in medicine. More and more researchers in fields outside computer science are depending on computer models to help them see and understand the complex environments in which they work. These are researchers for whom the computer is an important tool---one that must work quickly, easily, and with a large degree of flexibility. To reduce the time such researchers spend in the computer modeling, simulation, and visualization process, and to provide a powerful, flexible tool for exploration of computational engineering problems, we have developed a scientific problem solving environment (PSE) software system called SCIRun. Pronounced ``ski-run.''.<sup>2</sup> The SCIRun software system provides a ``computational workbench'' from which scientists and engineers can choose various visualization techniques to design and modify simulations interactively. Engineers are able to imagine improvements to a device, place the improvements directly into the computer model, then visualize the results interactively. The system is especially useful in enabling researchers to test new devices using the computer prior to animal and/or human trials, thus reducing the numbers and risks of such trials.

This progress will build on foundations that are centuries old. Researchers have investigated the electrical properties of the human body for many decades. Techniques such as finite element analysis, regularization methods, and linear algebra theory provide a strong basis for the mathematics of these materials and have enjoyed dramatic growth because of modern computers. Coupling modern advances in these methods with expanded knowledge of quantitative physiology and improvements in computing technology has made possible this form of scientific investigation.

The computational advances that have had the most profound effect are:
* CPU processing power
* Parallel computing
* High density memory (RAM)
* High density, inexpensive disk storage
* High bandwidth communication networks
* Interactive 3D graphics technology
* Modern high-level programming languages
* Modern operating systems
* High resolution display technology
* Data acquisition devices

However, even though these technologies have enabled our simulations, they often remain the limiting factors in achieving accurate answers in a reasonable amount of processing time. Further advances in these areas will be required in order to continue the advances in computational medicine. Fortunately, these advances are likely to continue for the foreseeable future.

Our research would not have been possible without the advent of powerful computational systems that can both quickly compute simulation results and perform fast 3D graphics operations. This combination of fast processing speed tightly coupled with fast graphics processing has recently been dubbed ``visual supercomputing.'' We see such tight coupling of CPU and graphics hardware to be critical for advancing the state-of-the-art in computational engineering, science, and biomedical applications. As has been the tradition in high-performance computing, the advances that we make on high-performance computing systems today will be possible on the desktop computers of tomorrow. In the same vein, the high-performance computing systems of tomorrow will enable a whole new range of exciting biomedical applications.

The Future

In my opinion, medical simulation and visualization will play an ever greater role in medical research. Already, researchers and physicians at the cutting edge are moving toward using highly interactive virtual and enhanced reality systems for diagnosis, treatment, surgical planning and surgery itself on a regular basis. Furthermore, it will be only a short time before medical researchers will use computer visualization and virtual/enhanced reality to work collaboratively over large distances. While such medical visualization systems are not yet in place, systems are now beginning to be used on a research basis at many university medical centers. We are approaching a revolution in medical visualization as computer scientists and medical researchers collaborate to create state-of-the-art visualization tools for the medical profession. These tools will improve diagnosis and treatment while effectively decreasing medical costs and risk. Let me offer a prediction of a more exciting scenario that is based upon conversations with my School of Medicine colleagues at Utah, Professors Robert MacLeod and William Orrison:

Upon arrival to the hospital, the patient will press her finger to a sensor that will identify her, extract a DNA sample for analysis, and allow entry to a short corridor that leads to the waiting room (yes, there will probably be waiting rooms in our future...). As the patient walks down the short corridor, she will be scanned by a number of imaging devices. Together with her physician, they both look at a fully, registered, multi-modal, high resolution, interactive, three-dimensional visualization of the patient. Already highlighted will be possible abnormalities in structure and function. The physician will be able to manipulate and further analyze various suspect regions and then simulate possible treatments, from drug and gene therapies to minimally invasive surgical reconstructions. In the far future, one could imagine ``at home'' imaging and analysis systems that would communicate remotely with the physician to prepare the diagnosis and treatment.

No matter precisely when the above scenario plays out, elements of it already exist, I believe we are fast approaching a revolution in medical imaging and visualization as computer science and medical researchers collaborate to create state-of-the-art visualization and integrated analysis tools for the medical profession. These tools will improve diagnosis and treatment while effectively decreasing medical costs.

However, as rapidly as these new medical imaging and visualization methods emerge, the extent of their effectiveness and impact will rely on the availability of a new kind of scientist. This new scientist will combine expertise in anatomy and physiology with a specific set of skills in physics, mathematics, bioengineering, and computer science. The result will be a person who is qualified in \ital{computational bioimaging}. In my opinion, educational training will become the largest obstacle to such a revolution as our universities tend to lag far behind advances in technology, especially in multidisciplinary application areas. However, if we can gain the commitment of our legislative and educational bodies to train researchers and physicians to use the latest technological resources, a revolution in diagnosis and treatment will soon be upon us and we can seize this opportunity for enormous benefit to society.

About the Author

Dr. Christopher R. Johnson, directs the Scientific Computing and Imaging Institute at the University of Utah. His research is focused on scientific computing, particularly in inverse and imaging problems, adaptive methods for partial differential equations, problem-solving environments, large-scale computational problems in medicine, and scientific visualization. Professor Johnson received his bachelors degree from Wright State University, and his masters and doctoral degrees from the University of Utah. Currently, Dr. Johnson holds faculty appointments in the School of Computing and the Departments of Physics and Bioengineering at the University of Utah. Dr. Johnson also directs the NIH Center for Bioelectric Field Modeling, Simulation, and Visualization; co-directs the Computational Engineering and Science Program; is co-founder of Visual Influence Inc.; and is the founder and director of the University of Utah Engineering Undergraduate Scholars Program.

In 1992, Professor Johnson was awarded a Young Investigator's (FIRST) Award from the National Institute of Health. He received the National Science Foundation's National Young Investigator Award in 1994, and the NSF Presidential Faculty Fellow from President Clinton in 1995. In 1997 he received the Presidential Teaching Scholar Award from the University of Utah, and in 1999, Professor Johnson was awarded the Utah Governor's Medal for Science and Technology.

______________________________________________________________
¹ The Utah team is made up of Chris Johnson, Chuck Hansen, and Steve Parker from the School of Computing, Robert MacLeod from the Department of Bioengineering and the Cardiovascular Research and Training Institute; William Orrison, Grant Gullberg, and Dennis Parker from the Department of Radiology, and Peter Heilbrun from the Department of Neurosurgery.
 
² SCIRunderives its name from the Scientific Computing and Imaging (SCI) nstitute at the University of Utah.