July 1, 2008: Optical Flow

July 1, 2008: Optical Flow

Gary Zientara from the Deparment of Radiology at Harvard Medical School is doing work similar to our DDDAS and has brought two papers to our attention.

Grant Number: 5R01CA086879-05
Project Title: Control System for MRI Monitored Thermal Therapies

PI Information: ZIENTARA, GARY P. zientara@bwh.harvard.edu ASSOCIATE PROFESSOR OF RADIOLOGY

Abstract: DESCRIPTION (Verbatim from Applicant’s Abstract): We propose to develop, implement, and validate use of a computerized control system for MR-monitored thermal therapies (interstitial laser thermal therapy and cryotherapy) that is to be attached to our 0.5T open configuration interventional MR scanner. The system will utilize 3D MRI information, providing audible and visual prior warning signals and an alarm signal at the therapy endpoint. We aim to directly answer the cancer treatment clinical need for accurate 3D spatial-temporal monitoring and control of tumor thermal therapies in heterogeneous tissue, currently beyond human capability, to guarantee patient safety while optimizing therapy effectiveness. Our proposed system will provide recognizable prior warning signals before the endpoint is reached, and an alarm signal at the instance when the thermal therapy endpoint occurs, as the thermal exposure level or therapy boundary meets or exceeds pre-defined 3D boundaries or safety limits. Specifically, our project consists of: (1) the integration of specially configured computer hardware into our interventional MR scanner electronics; (2) the development of a software package and user-interface for 3D monitoring and control of a thermal ablation therapy based on a physician-specified therapy volume boundary and exposure limit; (3) initial off-line system testing using gel phantoms and posttreatment human patient MRI data; and, (4) fully on-line real-time validation testing during tumor thermal therapies on our interventional MR scanner. Software will be developed or integrated into the system to provide state-of-the-art capability for monitoring and predicting evolving therapy margins and thermal exposure. In Software Module One, a monitoring and control system will be developed based on 3D optical flow, previously demonstrated as clinically useful in our hospital. In Software Module Two, a parallel predictive pathway will be created combining existing fast 3D data segmentation and heat conduction modeling software. In Trial Phase One, off-line (not in real-time) gel and ‘virtual’ patient tests will be performed and control case data acquired. Later, in Trial Phase Two, real-time validation trials will take place during clinical therapy cases, to study the accuracy, usefulness and timeliness of the system results when executing on the specific computer hardware chosen for implementation.

Public Health Relevance: This Public Health Relevance is not available.

Thesaurus Terms: computer assisted patient care, computer system design /evaluation, human therapy evaluation, neoplasm /cancer, neoplasm /cancer thermotherapy computer human interaction, computer program /software, computer system hardware, quality of life human data, magnetic resonance imaging


The paper, MRI monitoring of Laser Ablation Using Optical Flow,

  1. G. P. Zientara, P. Saiviroonporn, P. R. Morrison, M. P. Fried, S. G. Hushek, R. Kikinis, and F. A. Jolesz. MRI Monitoring of Laser Ablation Using Optical Flow. JOURNAL OF MAGNETIC RESONANCE IMAGING, 8(6):1306-1318, 1998. [ bib ]

“The optical flow method is used for visualizing and quantifying the dynamics of tissue changes observed by MRI during thermal ablations. An approach was implemented for parallel two-dimensional optical flow calculations including the replacement of spurious velocities. Velocity magnitude results were found to be accurate in low-noise cases in tests using series of synthetic images. Optical flow results are presented from thermal ablation experiments utilizing a homogeneous polyacrylamide gel phantom and heterogeneous rabbit liver tissue in vivo, exhibiting heating and cooling with the accompanying quantitative characterization of the dilation and contraction of the thermally affected region. Results demonstrate that optical flow is capable of noninvasive real-time monitoring and control of interstitial laser therapy (ILT).”

published in 1998 by Zientara et. al., provides a description of a system similiar to our DDDAS system. The paper, Image-guided Tumor Ablation: Proposal for Standardization of Terms and Reporting Criteria,

  1. S. Nahum Goldberg, J. William Charboneau, III Dodd, Gerald D., Damian E. Dupuy, Debra A. Gervais, Alice R. Gillams, Robert A. Kane, Jr Lee, Fred T., Tito Livraghi, John P. McGahan, Hyunchul Rhim, Stuart G. Silverman, Luigi Solbiati, Thomas J. Vogl, and Bradford J. Wood. Image-guided Tumor Ablation: Proposal for Standardization of Terms and Reporting Criteria. Radiology, 228(2):335-345, 2003. [ bibDOIhttp ]

“The field of image-guided tumor ablation requires standardization of terms and reporting criteria to facilitate effective communication of ideas and appropriate comparison between treatments with different technologies, such as chemical ablation (ethanol or acetic acid) and thermal therapies, such as radiofrequency, laser, microwave, ultrasound, and cryoablation. On the basis of this premise, a working committee was established with the goal of producing a proposal on such standardization. The intent of the Working Group is to provide a framework that will facilitate the clearest communication between investigators and will provide the greatest flexibility in comparisons between the many new, exciting, and emerging technologies. The members of the Working Group now propose a vehicle for reporting the various aspects of image-guided ablation therapy, including classifications of therapies and procedures, appropriate descriptors of image guidance, and terms to define imaging and pathologic findings. Methods for standardizing the reporting of follow-up findings and complications and other important aspects that require attention when reporting clinical results are addressed. It is the group’s hope and intention that adherence to the recommendations of this proposal will facilitate achievement of the group’s main objective: improved precision and communication in this field that lead to more accurate comparison of technologies and results and ultimately to improved patient outcomes. (C) RSNA, 2003″

is a special NIH report published in 2003 that provides collection of standardized terms accepted by the medical community. We should be using these terms when describing experiments preformed using our DDDAS to facilitate communication with the medical community.


Paper Review of MRI monitoring of Laser Ablation Using Optical Flow

This paper focuses on the same selling points as our DDDAS

  • Percutaneous1 tissue ablation via lasers
  • Interstitial laser therapy (ILT) as a promising treatment of focal disease
  • Minimally invasive using MRI guidance
  • Thermal dose depends on the tissue thermal history

During ILT absorbed optical energy is converted into heat, followed by heating of surrounding tissue.

Figure 1: (1) Coagulation zone – immediately adjacent to laser source, irreversible thermal damage occurs. (2) Marginal zone – partial cell survival. (3) Outer zone – reversible damage.

The optical flow method:

  • provides “quantitative spatially specific dynamic information characterizing tissue changes during thermal therapies and shows it capabilities for real-time monitoring and control of ILT”
  • “is used to visualizing and quantifying the dynamics of tissue changes observed by MRI during thermal ablations”

Tissue changes such as the “dilation of the necrosed core and marginal zone during ILT”. Optical flow may be used to predict the boundary of the marginal zone, Figure 1, during treatment.

What is Optical Flow?

Consider a time series of MR images with intensity values describes by the 4D function, Figure 2.

Figure 2: a specific time instance of a 4D MR field, E(x,y,z,t).

Whether the a Lagrangian or Eulerian description of the intensity field is most appropriate is debatable and would require a further study of imaging physics. From the Lagrangian point of view, the intensity field could be associated with a material point and thus a particular particle would have a particular intensity value associated with it. However, the intensity field represented within the MR image is assumed to take an Eulerian reference frame and optical flow is computed under the assumption:

  • each image region with a given intensity at an initial time may possibly change position or even deform, but will retain the same intensity value, i.e.

  • This is known as the brightness constancy assumption and it implies that an intensity change at a particular pixel is due to motion.

As in continuum mechanics, v, would represent the velocity of the material, or in this case, the tissue. More precisely, v, is the apparent motion of image regions of constant intensity. A coupled set of equations, solvable for the velocity may be obtained by differentiating (1.1) with respect to x, y, and z and assuming that

The equations are given as follows2.



In the presence of image noise at more stable optical flow solution is obtained by combining the brightness constancy assumption (1.1) with its derivative (1.2).


This leads to an over determined system3.

What Experiments were performed? Results?

A set of three results was presented:

  • Verfication problem using synthetic image that capture the dilation of a circle
  • Phantom Experiments
  • In-vivo rabbit liver

In the experiments, a time series of 2D images were acquired in the plane of the optical fiber, 11 second acquisition time. This was a proof of concept study and all computations were done after the fact.

The computer used was a Connection Machine 2 made by Thinking Machine Corp. A few hardware specifications:

  • 64,000 processors (5,200 cores)
  • 2.5 GFLOPS peak performance (55TFLOPS)
  • 512MB RAM (10.4 TB)
  • 300 Gbit/s Memory bandwidth (1333 MHz FSB, dual channels with 533MHz fully buffered DIMMS)

The verification problem involved essentially reproducing the velocities from the dilation of circle. The intensity profile fo the circle varied radially. The test case was constructed with series of images with a know velocity of the circle dilation. Results for the verification problem show good agreement within the central region of the circle (5-35 pixels).

The velocity vectors provide quantitative information about the motion and reshaping of the deforming regions, during heating and cooling. This information is unavailable in standard differences images, Figure 2B and 4B of the paper. However, numbers for the velocity magnitude were not provided.

The main goal of the paper was to demonstrate the potential of the optical flow method to automate the laser treatment process.

  • Assume that a pixel exists in the marginal region if the difference between the intensity value at time t is greater than 20% from a baseline image.
  • Using the optical flow velocity at time t-2Δt, project the pixel positions from t-2Δt to t.

The comparison of the optical flow prediction to the observed ≥ 20% regions provides accurate results.

NIH special report – Image-guided Tumor Ablation: Proposal for Standardization of Terms and Reporting Criteria

This provides a standardization of terms that should be used when communicating with the medical community. In particular, for our DDDAS project, the following points should be noted:

  1. Thermal ablation procedures include energy sources that destroy a tumor with either heat (RF, microwave,laser) or cold (cryoablation).
  2. For thermal therapies, energy is “applied”.
  3. “Operation” implies open surgery, the term “procedure” is preferred.
  4. a “procedure” refers to a single intervention episode that consists of one or more ablations performed on one or more tumors.
  5. a “treatment” consists of one or more “procedures”.
  6. the term “applicator” should be used to describe the energy applicator, as opposed to “needles”. More precisely:
    • RF applicators are electrodes.
    • Microwave applicators are antennas.
    • Laser applicators are fibers.
    • Cryoprobes are used in cryoablation.
  7. Control “To control an image-guided ablation procedure, the treatment should be monitorable, such that the operator can utilize the image-based information obtained during tmonitoring to control it. This may simply be repositioning of a therapy applicator on the basis of physician experience, imaging findings, and thermal feedback, or it could be as sophisticated as an automated system that automatically terminates the ablation at a critical point in the procedure.

Future Direction

  • Paper comparing the arrhenius damage model to the two-state damage model.
    • Use model parameters from Sepideh’s experiments.
    • Experiments and simulations would attempt to mimic the heating profile seen in the animal/phantom experiments.
  • Formulate Pennes model within uncertainty quantification framework and use the last canine experiment to test the methodology.
    • The calibration/optimal control sections of the canine experiment will be used for calibration/validation under the UQ framework.



percutaneous- during surgery, access to internal organs via needle


Notice that these equations could also be derived from a eulerian conservation law of the intensity.
Using standard arguements and mapping back to material coordinates
Using det F = det F div v, implies
Finally, div v = 0 gives (1.2).


Spurious velocities may be seen in pixels that violate the theory assumptions. The spurious velocities are replaced with a so-called multipoint-based-with-least-squares(MPLS) method. The overdetermined system is created by taking the brightness constancy assumption (1.1) multiple times using a different pixel neighborhood to approximate the derivatives.