This course presents current techniques in 3D medical visualization. We
describe the problem as a pipeline from acquisition to display, examining
new data acquisition technologies, outlining algorithms and optimization
strategies, and pointing out inherent problems along the way. Case studies
will be presented periodically throughout the course, illuminating the
motivations, benefits, and potential pitfalls of computer graphics research
in medicine.
| Course Organizer |
|
|
Terry S. Yoo |
|
* Indicates required proposal item
This course presents current techniques for 3D medical visualization. We describe the problem as a pipeline from acquisition to display, examining new data acquisition technologies, outlining algorithms and optimization strategies, and pointing out inherent problems along the way. Case studies will be presented periodically throughout the course, illuminating the motivations, benefits, and potential pitfalls of computer graphics research in medicine.
The growing healthcare industry is providing new opportunities for applied research in computer graphics. Although volume rendering and mesh generation techniques will be briefly covered, the course will concentrate on areas of the visualization pipeline not traditionally covered: acquisition and medical evaluation. This course will present not only how to approach 3D visualization in medicine, but also, through case studies, will discuss the motivations and limitations of such methods. Participants interested in getting started in this area will learn about the sources of volume medical data, and clinicians will present their view of visualization and what their requirements are for effective and safe applications of computer graphics in their field.
Duration: Full Day
Format: Lectures, Case Studies, and a final half-hour panel session on future research directions.
The proposed course revisits and updates material presented at SIGGRAPH '93 and SIGGRAPH '94. This material has not been covered in four years, and while the basic techniques have not changed substantially, there have been important advances in data acquisition and the growth of interactive graphics in the clinic. This proposal emphasizes the advances in interactive volume ultrasound, interventional MRI, and image guided therapy. Case studies will be presented showing where and why computer visualization has been effective with the hope of motivating members of the audience to participate in this growing field. Slides, video, and perhaps interactive demonstrations of visualization systems will be used to present the material.
Basic knowledge of 3D computer graphics and an understanding of the basic principles of image processing. Some familiarity with medical terminology or experience working on a clinical project would be useful, but not necessary.
Participants will gain insights into what makes an effective medical visualization and the processes by which they are created. Other topics include the sources of 3D clinical data and their characteristics, the future of advanced displays (virtual worlds) in medical settings, and the uses of interactive computer graphics for surgery.
We are designing the course around the presentation of a simplified medical visualization pipeline. Like a graphics pipeline, there are steps throughout the procedure that are familiar; however, the beginning is image acquisition rather than geometry/modeling. The segmentation/classification stages are new to graphics people, but not to people familiar with image processing techniques. The later half of the pipeline will cover rendering techniques commonly used in medical visualization, though perhaps in less detail than normally covered in either the introductory course or the advanced course in volume visualization. A section on display systems will follow, including the utilization of head-mounted display technology in medical applications.
The remainder of the course will be dedicated to familiarizing attendees with current directions of computer graphics in medicine, including current and developing applications for this technology, and existing problems regarding accuracy, robustness, and interaction with the medical community.
At all times during the course, through case studies we will emphasize the role that visualization plays in diagnosis and treatment. The presentation will be driven toward application. Analysis of error or noise will be discussed relative to its impact upon the medical task to be performed. Examples of rendering techniques will typically be presented using medical data, to continue to familiarize the computer graphics portion of the audience with the medical aspect, and to keep the medical attendees involved in the process.
It is our intention to address those members of the graphics community who are interested in expanding their research directions toward medicine. It is essential that computer scientists who apply their knowledge to medicine understand the requirements that are particular to clinical applications, specifically: robustness, limiting artifactual error, and maximizing comprehension and retention (targeting clinicians).
We hope this course will be an enlightening presentation of this material for all concerned. Speakers from both the medical as well as the computer graphics community will be present to provide their particular view of the field. We have selected speakers familiar with each stage of the pipeline, giving particular insight into each of the areas where error or improvement can be introduced. The whole collection of proposed speakers represents one of the finest cross-sections of the use of computer graphics in medicine.
Although people working in the field of volume visualization regularly manage large volumetric datasets, they often do not have a full command of the imaging modality that creates the data. To provide for optimal visualization of information within medical volume data, it is essential that the process be suited to the modality used to create the data itself.
There are three imaging modalities that we will be considering in this course: X-ray computed tomography, nuclear medicine imaging (including both PET (positron emission tomography) and SPECT (single photon emission computed tomography), and magnetic resonance imaging. We intend to briefly cover the mechanisms underlying each of these modalities, their strengths and their weakness. A treatment of the noise properties, linear/geometric distortions, and contrast sensitivity of each of these modalities will be discussed, not only in light of their uses in medicine, but also regarding their impact on the visualization process.
An often serious weakness in volume visualization systems is the lack of a robust classifier. Segmentation is the division of an image into coherent regions. Classification is the labeling of those regions, often with the aid of a user. Classifiers can be based on many different mechanisms; however, the simplest are intensity windows, useful primarily with X-ray CT data.
More advanced mechanisms based upon the geometry of images or of statistical measurements made of images are available, though often underutilized in our community. There is a large body of literature covering statistical and structural pattern recognition techniques that is often overlooked by volume visualization specialists. We will present some approaches to segmentation and classification that are directed toward medicine. These methods are either automated or semi-automatic, requiring interaction with a medical expert (but not necessarily a computer expert).
An entire hour of the course will be dedicated to rendering. This may be redundant to people who are already familiar with display techniques for volume data; however, it is considered an essential element of the course. Moreover, even though much of this material may overlap with course materials presented in courses on volume visualization, we will retain our focus on specifically medical applications. For example, curvilinear grids do not often exist in medical data; on the contrary, regularly sampled rectilinear grids with an abundance of information usually flood visualization systems. The issue is not in how to display the data, but how best to extract pertinent information and present it for maximum effect.
Volume rendering topics to be covered will include raycasting volume renderers, splatting (hierarchical splatting, and attempts at interactive splatting), and Fourier rendering. The surface rendering discussion will cover both the marching cubes and the dividing cubes algorithms, as well as the recent work in automated mesh refinement.
Emerging technology in immersive display (virtual reality) is showing remarkable promise in medical applications. The challenges in overcoming the technological limitations are difficult, but the opportunities seem endless. We will present work in viewing ultrasound data through head-mounted displays as well as postulate some new directions for this technology in medicine.
Future direction for the field will be covered at the end with questions from the audience about any aspect of the course. The final session is expected to be a panel, with the audience participating, directing questions at any of the speakers and addressing the whole range of issues from acquisition to display.
It is our contention that the focus of this course is no more narrow than other sub-fields also presented as courses at SIGGRAPH. The growth of biomedical imaging conferences and symposia such as Virtual Reality in Medicine as well as a significant number of recent SIGGRAPH papers related to this topic seem to indicate growing interest in this area.
|
Terry S. Yoo Assistant Professor, Department of Radiology University of Mississippi Medical Center 2500 North State Street Jackson, MS 39216-4505, USA (601) 984-2521 fax: (601) 984-2542 yoo@fiona.umsmed.edu Research Assistant Professor University of Mississippi Department of Computer Science 302 Weir Hall University, MS 38677, USA (601) 232-7621 fax: (601) 232-5623 yoo@cs.olemiss.edu home phone: (601) 236-3769 |
Ron Kikinis, M.D. Associate Professor of Radiology Harvard Medical School Director, Surgical Planning Laboratory AMB II, L1-Room 0069, Radiology Brigham & Women's Hospital 75 Francis St. Boston, MA 02115, USA (617) 732-7692 fax: (617) 732-7963 kikinis@bwh.harvard.edu Bill Lorensen Graphics Engineer GE Corporate Research and Development 1 Research Circle Bldg KW, Rm C215 Niskayuna, NY 12309 (518) 387-6744 fax: (518) 387-6981 lorensen@crd.ge.com |
|
Henry Fuchs Federico Gil Professor Department of Computer Science The University of North Carolina at Chapel Hill Chapel Hill, NC 27599-3175, USA (919) 962-1911 fax: (919) 962-1799 fuchs@cs.unc.edu |
Michael W. Vannier, M.D. Professor and Chairman Department of Radiology University of Iowa College of Medicine 200 Hawkins Drive \ 3966A JCP Iowa City, IA 52242-1077, USA (319) 356-3372 fax: (319) 356-2220 michael-vannier@uiowa.edu |
Terry S. Yoo is an Assistant Professor of Radiology at the
University of Mississippi Medical Center and Research Assistant
Professor of Engineering at the University of Mississippi. His
research interests include visualization of 3D medical data,
interactive computer graphics and medical image processing. He
is currently developing a research program in Interventional
Magnetic Resonance Imaging. He worked for BBN Laboratories,
Incorporated, AT&T Technologies, and MCNC before beginning his
doctoral studies. During an interim in his doctoral research, he
served as site coordinator for the NSF/ARPA Science and Technology
Center for Computer Graphics and Scientific Visualization. In 1993,
he was co-chair of a SIGGRAPH Course on Three Dimensional
Visualization Using Medical Data.
Yoo received his A.B. in Biology from Harvard in 1985 and his M.S. and Ph.D.
in Computer Science from the University of North Carolina at Chapel Hill in
1990 and in 1996, respectively.
Henry Fuchs is Federico Gil Professor of Computer Science and
Adjunct Professor of Radiation Oncology at the University of North
Carolina at Chapel Hill. He received a BA in Information and Computer
Science from the University of California at Santa Cruz in 1970 and a
Ph.D. in computer science from the University of Utah in 1975. He has
been an associate editor of ACM Transactions on Graphics (1983-1988) and
the guest editor of its first issue (Jan. 1982). He was the chairman of
the first of the Symposia on Interactive 3D Graphics (1986), co-director
of the NATO Advanced Research Workshop on 3D Imaging in Medicine (1990),
and co-chair of the National Science Foundation Workshop on the Future
of Virtual Environments Research (1992). He received the 1992 Computer
Graphics Achievement Award from ACM/SIGGRAPH and the National Computer
Graphics Association Academic Award (1992). His current research interests
include the application of head-mounted display technologies to problems
in medicine.
Ron Kikinis is the Director of the Surgical Planning Laboratory
of the Department of Radiology, Brigham and Women's Hospital and
Harvard Medical School, Boston, MA, and an Associate Professor of
Radiology at Harvard Medical School, as well as an Adjoint Professor
of Biomedical Engineering at Boston University. His interests include
the development of clinical applications for image processing, computer
vision and interactive rendering methods. He is currently concentrating
on introducing interactive computer graphics into the operating room.
He is the author and co-author of more than 52 peer-reviewed articles.
Before joining Brigham and Women's Hospital in 1988, he worked as a
researcher at the ETH in Zurich and as a resident at the University
Hospital in Zurich, Switzerland. He received his M.D. from the
University of Zurich, Switzerland in 1982.
Bill Lorensen is a Graphics Engineer in the Electronic Systems
Laboratory at GE's Corporate Research and Development Center in
Schenectady, NY. He has over 25 years of experience in computer
graphics and software engineering. Bill is currently working on
algorithms for 3D medical graphics and scientific visualization.
He is a co-developer of the marching cubes and dividing cubes
surface extraction algorithms, two popular isosurface extraction
algorithms. Bill is one of the chief architects of LYMB, an
object-oriented software development environment written in C.
His other interests include computer animation, color graphics
systems for data presentation, and object-oriented software tools.
Bill is the author or co-author of over 60 technical articles on
topics ranging from finite element pre/postprocessing, 3D medical
imaging, computer animation and object-oriented design. He is a
co-author of "Object-Oriented Modeling and Design" published by
Prentice Hall, 1991. He is also co-author with Will Schroeder and
Ken Martin of the book "The Visualization Toolkit: An Object-Oriented
Approach to 3D Graphics" published by Prentice Hall in February 1996.
He gives frequent tutorials at the annual SIGGRAPH and IEEE Visualization
conferences.
Bill holds 24 US Patents on medical and visualization algorithms.
In 1991, he was named a Coolidge Fellow, the highest scientific
honor at GE's Corporate R&D.
Prior to joining GE in 1978, he was
a Mathematician at the US Army Benet Weapons Laboratory where he
worked on computer graphics software for structural analysis. He
has a BS in Mathematics and an MS in Computer Science from Rensselaer
Polytechnic Institute.
Michael W. Vannier is a Professor and Head of the Department of
Radiology at the University of Iowa College of Medicine. He
received a B.S.M.E in Mechanical Engineering from the University
of Kentucky in 1971, a B.S. in Engineering Sciences from Colorado
State University in 1971, and an M.D. in 1976 at the University
of Kentucky. He then did his residency in Radiology at the
Mallinckrodt Institute of Radiology, Washington University, St. Louis,
where he later stayed to become Full Professor of Radiology and
Affiliate Professor of System Science and Mathematics in the College
of Engineering. Before taking his current position, from 1994 through
1996, Dr. Vannier was Georgia Eminent Scholar in Medical Imaging,
Emory University School of Medicine.
Dr. Vannier is a Fellow of the American College of Radiology. He is
a recipient of the Lindberg Award from the American Institute of
Aeronautics and Astronautics. He is Editor-in-Chief of IEEE
Transactions on Medical Imaging, and serves on the editorial boards
of several other medical journals. His interests span the wide field
of computer graphics and image processing to improve the diagnostic
value of radiologic images.
Unlike the presentation strategy in previous years, applications
and case studies are not being held until the end of the course.
Rather, a greater emphasis is being placed on case studies, affording
them more time. The intent is to provide real examples for the lessons
being taught and to illuminate motivating and limiting factors while
teaching the techniques.
Course Syllabus:
A Preliminary Course Syllabus
1st morning session - (90 mins total)
INTRO.[Yoo] (15 mins)
why visualize?
medical image pipeline
topics to be covered
topics not to be covered
what we hope to accomplish today.
CASE STUDY: Computer Assisted Neurosurgery [Kikinis] (30 mins)
SEGMENTATION AND CLASSIFICATION [Kikinis] (30 mins)
Bayesian Statistical Segmentation
Deformable Surfaces/Volumes for Segmentation
Atlas based segmentation
MEDICAL IMAGE ACQUISITION (CT) [Yoo] (15 mins)
Intro: CT, MRI, PET/SPECT, Ultrasound
CT Physics -
Parameters: slice thickness, tissue window
Artifacts: partial voluming, motion
---BREAK---
2nd morning session - (105 Mins total)
MEDICAL IMAGE ACQUISITION (MRI) [Yoo] (15 mins)
MRI Physics -
Parameters: slice thickness, gap, noise
Artifacts: partial voluming, motion
geometric distortion, non-stationary intensity distortion
VOLUME RENDERING [Yoo] (30 mins)
raycasting, splatting, Fourier rendering
acceleration, parallel algorithms
integrated segmentation and rendering (e.g., Volume Seedlings)
hardware based volume rendering
CASE STUDY: Marching Through the Virtual Human [Lorensen] (30 mins)
SURFACE RENDERING [Lorensen] (30 mins)
Marching Cubes, Dividing Cubes
Textures and visualization
Polygon Decimation
Experiments in Smoothing
---LUNCH---
1st afternoon session - (90 Mins total)
CASE STUDY: Optical Surface Scanning in Medicine [Vannier] (30 mins)
CLINICAL PROBLEMS [Vannier] (30 mins)
Applications of 3D CT/MRI and Optical Surface Scanning
Craniofacial reconstruction for surgical planning
Prosthetic design for lower limbs
PROBLEMS SPECIFIC TO MEDICINE II [Kikinis] (30 mins)
Clinical performance: rapid, robust tool development
Interactive graphics in surgery: do no harm
Comparison: registration
Multiple Sclerosis
---BREAK---
2nd afternoon session - (105 Mins total)
CASE STUDY: Clinical Uses for Head Mounted Displays [Fuchs] (30 mins)
Volume ultrasound
DISPLAY [Fuchs] (30 mins)
Perceptual issues: stereo, kinetic depth, head motion parallax
Proprioception
HMD
video see through
optical see-through
challenges to head mounted display technology:
tracking, real-time rendering, registration.
PROBLEMS SPECIFIC TO MEDICINE [Vannier] (30 mins)
Quantitative vs. Qualitative Measurement
Noise: understanding the source of the data
Robustness (in medicine)
Volume and mass measurement
FUTURE/ RESEARCH DIRECTIONS [All speakers] (15 mins)
Open Floor Q&A