Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle

We present an endomicroscope apparatus that exhibits out-of-focus background rejection based on wide-field illumination through a flexible imaging fiber bundle. Our technique, called HiLo microscopy, involves acquiring two images, one with grid-pattern illumination and another with standard uniform illumination. An evaluation of the image contrast with grid-pattern illumination provides an optically sectioned image with low resolution. This is complemented with high-resolution information from the uniform illumination image, leading to a full-resolution image that is optically sectioned. HiLo endomicroscope movies are presented of fluorescently labeled rat colonic mucosa. ©2009 Society of Photo-Optical Instrumentation Engineers

Contents

• BODY OF ARTICLE
• Acknowledgments
• REFERENCES
• FIGURES
• FOOTNOTES

The development of a simple, robust high-resolution fluorescence endomicroscope is driven by preclinical and clinical needs.¹ Standard wide-field techniques are hampered by their inability to to reject out-of-focus background, generally leading to low signal contrast. Strategies to reduce out-of-focus background have been based on confocal detection^2,3,4,5,6 or two-photon excitation,^7,8 both requiring some sort of scanning mechanism. Alternatively, out-of-focus background can be rejected in a nonscanning wide-field endoscope by use of structured illumination microscopy (SIM),⁹ which we have implemented with a flexible fiber bundle.¹⁰ While SIM is effective at optical sectioning, we have found that it is highly susceptible to sample motion, the difficulty being that high-resolution image information in SIM is distributed over a series of at least three raw images, meaning that any misregistration between the raw images leads to artifacts in the final processed SIM image. Recently, we have developed a novel hybrid-illumination technique to address this problem.¹¹ In this technique, two raw images are required, only one containing high-resolution information and the other containing low-resolution information. Hence, the name of our new imaging technique: HiLo microscopy.

The two raw images required for HiLo microscopy are based respectively on uniform and nonuniform (or structured) illumination. In our initial implementation of HiLo microscopy, the nonuniform illumination was obtained with laser speckle.¹¹ However, HiLo microscopy is more general than this and can be implemented with any type of nonuniform illumination. In particular, we demonstrate here the implementation of HiLo endomicroscopy with nonuniform illumination in the form of a grid pattern, of the same type as used in SIM. Indeed, our HiLo endomicroscope setup is identical to our previous SIM endomicroscope setup (see Fig. 1), except that for HiLo, the spatial light modulator now toggles between two illumination patterns, grid and uniform, whereas for SIM, it sequentially produced three grid patterns of incrementing phase.¹⁰

Figure 1.

The principle of HiLo microscopy was described in Ref. 11. In brief, a final optically sectioned HiLo image is constructed from the fusion of complementary in-focus high- and low-frequency image components. High-frequency components in the uniform illumination image are inherently in focus and are extracted with a high-pass filter. In-focus low-frequency components, on the other hand, must be extracted in a more complicated manner, since the simple application of a low-pass filter to the uniform illumination image does not reject out-of-focus background. To reject low-frequency out-of-focus background, we evaluate the local contrast in the fluorescence image obtained with nonuniform illumination. This local contrast is higher for in-focus image components than for out-of-focus components and hence is axially resolved. A multiplication of the local image contrast with the original uniform illumination image then provides an optically sectioned image, although at low resolution. A fusion of this in-focus low-resolution image with the complementary high-resolution image (inherently in focus) then leads to a full-resolution image that is axially resolved over all spatial frequencies within the microscope passband.

A key step in HiLo microscopy is the extraction of local image contrast from the nonuniform illumination image. In our previous implementation where the nonuniform illumination consisted of laser speckle, this local image contrast was evaluated by calculating the standard deviation of the nonuniform illumination image intensity over local, coarse-grained resolution areas. In our current implementation with a grid pattern, we will adopt a slightly different, although essentially equivalent, approach based on single-sideband demodulation. To understand this approach, let us phenomenologically decompose our uniform illumination image I_u() into in-focus and out-of-focus components. That is, we write

where ={x,y} are spatial coordinates in the image plane. Our final goal is to isolate I_in().

The nonuniform image can be decomposed similarly into

where the (imaged) grid illumination is modeled as a sinusoidal pattern of spatial frequency _g in the x direction, with arbitrary phase and (imaged) modulation contrast M. Note that only the in-focus image component appears to be modulated, whereas the out-of-focus component does not, precisely because the latter is out of focus.

I_n() and I_u() are the two raw images required for HiLo microscopy. The ratio R()=I_n()/I_u() of these two images leads to

where C()=I_in()/[I_in()+I_out()] is the local image contrast that we have set out to derive. The Fourier transform of R()—namely, ()—is illustrated in Fig. 2, where we observe that () [the Fourier transform of C()] and its complex conjugate reside in sidebands centered at ±_g. The more highly contrasted the image, the taller and narrower these sidebands become. While several techniques may be used to extract () from (), we adopt the standard technique of single-sideband demodulation. In the frequency domain, this corresponds to isolating only a single sideband ₊() by applying a one-sided high-pass filter to () (dashed line in Fig. 2), followed by an inverse Fourier transform to retrieve R₊(). Provided the sidebands are well separated from one another (i.e., they do not overlap), then the local image contrast is given by C()=[R₊()R()]^1/2/M. From C(), derived in this manner, and the uniform illumination image I_u(), we can then infer I_in()=C()I_u(). That is, we can extract I_in() from I_u(), thereby obtaining an optically sectioned image containing only in-focus contributions.

Figure 2.

There are two difficulties with the preceding procedure. First, the sidebands ₋() and ₊() may be so wide as to overlap. This problem can be alleviated by choosing a one-sided, high-pass filter cutoff profile that helps suppress the overlap region. The second problem is that, in general, M is not known a priori. We thus define a new parameter I_su()=[R₊()R()]^1/2I_u() that is independent of M. Moreover, we purposefully restrict I_su() to spatial frequencies smaller than _g by applying a low-pass filter to I_su(), with user-defined cutoff frequency _cg, obtaining I_lp()=LP[I_su()]. In practice, LP[I_su()] is performed by convolving I_su() with a square window of size 2/_g (or integral multiple thereof, to minimize the possibility of aliasing). In addition to confining I_su() to a well-defined bandwidth, such filtering helps suppress potential artifacts arising, for example, from a nonperfectly sinusoidal illumination pattern.

Finally, the low-resolution image I_lp() is combined with complementary high-resolution information I_hp() obtained by applying a high-pass filter directly to the uniform illumination image, such that I_hp()=HP[I_u()]=I_u()−LP[I_u()]. The final processed HiLo image is given by

where is a scaling factor introduced to ensure a seamless transition of the frequency content of I_hilo() across the cutoff frequency _c. [In effect, the introduction of compensates for the fact that M is unknown—see Ref. 11 for an explanation of how is derived from I_lp() and I_hp(), based on their complementary nature.] We emphasize that I_hilo() is axially resolved over all spatial frequencies, both high and low, within the endomicroscope passband and therefore constitutes an optically sectioned image. Moreover, it provides a faithful rendition of the sample since, to high accuracy, I_hilo() is directly proportional to the in-focus fluorophore concentration in the sample.

Figure 3 provides comparisons of standard wide-field and HiLo endomicroscope images through a fiber bundle. It should be noted that a longer grid period leads to a stronger imaged modulation depth M—however, at the expense of weaker HiLo optical sectioning capacity. A grid period of 30  µm was found to provide a reasonable compromise between grid pattern contrast and HiLo optical sectioning capacity. A rough measure of this sectioning capacity can be inferred from a measurement of the detected signal strength from a uniform fluorescent half-space whose interface is scanned through the focal plane. A comparison of the signal strengths acquired with wide-field and HiLo endomicroscopy is illustrated in Fig. 3(d), where we observe that the HiLo signal strength decays much more precipitously than the wide-field signal strength. The corresponding HiLo axial resolution for a laterally uniform plane is inferred from the derivative of the HiLo signal strength, and is found, in this case, to be about 30-µm FWHM. Note that for a laterally uniform sample, I_hp() vanishes, and the HiLo image is comprised solely of I_lp().

Figure 3.

Finally, a comparison of wide-field and HiLo endomicroscopic imaging of colon tissue in motion is illustrated in the multimedia movies (Video 1 and Video 2). Note the absence of a residual grid pattern or other motion-related artifacts. Our net HiLo imaging acquisition rate was about 2  Hz (i.e., 250  ms per raw image). This rate was software limited and will be improved in future versions of our apparatus. Our imaging resolution is about 2.6  µm, limited by the Nyquist frequency associated with the (magnified) fiber core quasiperiodicity (see Ref. 10).

Figure v1.

Figure v2.

In conclusion, we have successfully demonstrated the capacity of HiLo microscopy to provide optically sectioned imaging of fluorescently labeled colon tissue through a flexible optical fiber bundle. We anticipate that this will open new possibilities in high-resolution fluorescence endomicroscopy.

This work was partially funded by the NIH (R21 EB007338).

1. A. Hoffman, M. Goetz, M. Vieth, P. R. Galle, M. F. Neurath, and R. Kiesslich, “Confocal laser endomicroscopy: technical status and current indications,” Endoscopy 38, 1275–1283 (2006). [MEDLINE] first citation in article
   
2. A. R. Rouse and A. F. Gmitro, “Multispectral imaging with a confocal microendoscope,” Opt. Lett. 25, 1708–1710 (2000). [ISI] [MEDLINE] first citation in article
   
3. L. D. Swindle, S. G. Thomas, M. Freeman, and P. M. Delaney, “View of normal human skin in vivo as observed using fluorescent fiber-optics confocal microscopic imaging,” J. Invest. Dermatol. 121, 706–712 (2003). [MEDLINE] first citation in article
   
4. F. Jean, G. Bourg-Heckly, and B. Viellerobe, “Fibered confocal spectroscopy and multicolor imaging system for in vivo fluorescence analysis,” Opt. Express 15, 4008–4017 (2007). [MEDLINE] first citation in article
   
5. H.-J. Shin, M. C. Pierce, D. Lee, H. Ra, O. Solgaard, and R. Richards-Kortum, “Fiber-optic confocal microscope using MEMS scanner and miniature objective lens,” Opt. Express 15, 9113–9122 (2007). [MEDLINE] first citation in article
   
6. J. T. C. Liu, M. J. Mandella, H. Ra, L. K. Wong, O. Solgaard, G. S. Kino, W. Piyawattanametha, C. H. Contag, and T. D. Wang, “Miniature near-infrared dual-axes confocal microscope utilizing two-dimensional microelectromechanical systems scanner,” Opt. Lett. 32, 256–258 (2007). [MEDLINE] first citation in article
   
7. B. Flusberg, E. Cocker, W. Piyawattanametha, J. Jung, E. Cheung, and M. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods 2, 941–950 (2005). [MEDLINE] first citation in article
   
8. P. Kim, M. Puoris'haag, D. Coté, C. P. Lin, and S. H. Yun, “In vivo confocal and multiphoton microendoscopy,” J. Biomed. Opt. 13, 010501 (2008). [MEDLINE] first citation in article
   
9. D. Karadaglic, R. Juškaitis, and T. Wilson, “Confocal endoscopy via Structured Illumination,” Scanning 24, 301–304 (2002). [Inspec] [MEDLINE] first citation in article
   
10. N. Bozinovic, C. Ventalon, T. Ford, and J. Mertz, “Fluorescence endomicroscopy with structured illumination,” Opt. Express 16, 8016–8025 (2008). [MEDLINE] first citation in article
    
11. D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett. 33, 1819–1821 (2008). [MEDLINE] first citation in article
    

Full figure

Fig. 1. (a) HiLo endomicroscope setup: an expanded laser beam (Cobolt Calypso, =491  nm) is directed onto a spatial light modulator (Holoeye LC-R 768) to create grid and uniform illumination patterns, which are then projected into an imaging fiber bundle (600-µm useful diameter; 30,000 fibers; 1.9-µm core diameters; 3.3-µm core separations) equiped with a distal micro-objective (Mauna Kea Technologies: NA=0.8 water, working distance=60  µm, field of view=240  µm). The resultant fluorescence is isolated with a dichroic and emission filter (Chroma) and recorded with a CCD camera (QImaging Retiga). Raw images of fluorescently labeled lens-paper fibers with (b) grid and (c) uniform illumination. First citation in article

Full figure

Fig. 2. Schematic of (). The right sideband ₊() is isolated from the left sideband and from the zero-frequency delta function by applying a one-sided, high-pass filter to () (dashed line). Note: The exact cutoff response of the filter function can be chosen somewhat arbitrarily. First citation in article

Full figure

Fig. 3. Intermediate (a) I_lp() and (b) I_hp() images obtained from raw images in Fig. 1, and (c) final HiLo image. [Note: Panel (b) contains both positive and negative values.) (d) Demonstration of HiLo sectioning capacity using axially scanned fluorescent half-space (Chroma plastic slide—see inset.) First citation in article

Full figure

Fig. V1. Wide-field movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (MPG, 3.2  MPG) [URL: http://dx.doi.org/10.1117/1.3130266.1]. First citation in article

Full figure

Fig. V2. HiLo movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (acquired with same grid period as in Fig. 3) (MPG, 3.2  MG) [URL: http://dx.doi.org/10.1117/1.3130266.2]. First citation in article

^†Address all correspondence to E-mail: jmertz@bu.edu.

Up: Issue Table of Contents
Go to: Previous Article | Next Article
Other formats: HTML (smaller files) | PDF ( kB)