Showing 1–38 of 38 results for author: Waller, L

Inductively Scalable, Single-Step Neural Surrogates for Wave-Scattering Inverse Problems

Authors: Charles Dove, Laura Waller

Abstract: Neural network surrogates are an emerging alternative to traditional electromagnetic wave simulators like finite-difference time-domain (FDTD); their goal is to replace rigorous physical simulations with pre-trained neural networks that solve wave-scattering forward and inverse problems orders of magnitude faster. However, nonrecurrent, single-step surrogates have scaled only to a few tens of simu… ▽ More Neural network surrogates are an emerging alternative to traditional electromagnetic wave simulators like finite-difference time-domain (FDTD); their goal is to replace rigorous physical simulations with pre-trained neural networks that solve wave-scattering forward and inverse problems orders of magnitude faster. However, nonrecurrent, single-step surrogates have scaled only to a few tens of simulation variables. Here, we show that this barrier can be overcome by dynamically generating salient training examples during training, rather than randomly sampling the large space of possible examples. We introduce an algorithm that runs in parallel with surrogate training, using gradient ascent to search refractive-index and source configurations for cases where the surrogate disagrees with a full-wave ground-truth simulator. We also use source and ground-truth normalization with an evolving replay dataset to stabilize and accelerate learning. Using this approach, we train a fast, single-step surrogate for two-dimensional wave scattering with up to 41,772 controllable variables, including dense, freely configurable grids of refractive indices and complex-valued sources. The resulting neural surrogate is robustly accurate across diverse structured and unstructured examples and generalizes inductively to larger domains, reaching over 3 million controllable variables without retraining, a $73.8\times$ increase. We demonstrate the surrogate on large-scale forward simulations and inverse design of freeform beam splitters and gradient-index (GRIN) lenses up to 98 wavelengths wide, showing comparable or better performance than FDTD-based designs, with speedups from $1.29\times$ to $26.5\times$. These results demonstrate a practical path toward fast, robustly accurate, inductively scalable neural simulators for photonic inverse design and other wave-scattering inverse problems. △ Less

Submitted 18 August, 2026; originally announced August 2026.

Spectral DiffuserScope: a compact snapshot hyperspectral microscope

Authors: Neerja Aggarwal, Eric Markley, Kyung Chul Lee, Seung Ah Lee, Junghyun Bae, Nakkyu Baek, Wook Park, Min Sung Cho, Youngbin Lim, Polly Fordyce, Stephanie Eberly, Lydia L. Sohn, William D. Houck, Kristina Monakhova, Laura Waller

Abstract: Hyperspectral fluorescence microscopy enables important biological and clinical applications, but conventional systems are bulky or require scanning, limiting temporal resolution and throughput. We introduce a computational snapshot hyperspectral microscope that uses compressed sensing to achieve higher spatial-spectral resolution than traditional snapshot systems. Our device is compact (~15 cm x… ▽ More Hyperspectral fluorescence microscopy enables important biological and clinical applications, but conventional systems are bulky or require scanning, limiting temporal resolution and throughput. We introduce a computational snapshot hyperspectral microscope that uses compressed sensing to achieve higher spatial-spectral resolution than traditional snapshot systems. Our device is compact (~15 cm x 6 cm x 6 cm) and easily attaches to standard fluorescence microscopes. We benchmark our system against existing snapshot methods through simulations to evaluate its spatial and spectral performance. Experimental imaging of fluorescent beads, labeled cells, and lanthanide hydrogel beads demonstrates a practical, high-throughput solution for hyperspectral microscopy in biological and clinical applications. △ Less

Submitted 29 June, 2026; originally announced June 2026.

ConvRML: High-Quality Lensless Imaging with Random Multi-Focal Lenslets

Authors: Leyla A. Kabuli, Clara S. Hung, Vasilisa Ponomarenko, Eric Markley, Laura Waller

Abstract: Mask-based lensless imagers use simple optics and computational reconstruction to design compact form factor cameras with compressive imaging ability. However, these imagers generally suffer from poor reconstruction quality. Here, we describe several advances in both hardware and software that result in improved lensless imaging quality. First, we use a precision-manufactured random multi-focal le… ▽ More Mask-based lensless imagers use simple optics and computational reconstruction to design compact form factor cameras with compressive imaging ability. However, these imagers generally suffer from poor reconstruction quality. Here, we describe several advances in both hardware and software that result in improved lensless imaging quality. First, we use a precision-manufactured random multi-focal lenslet (RML) phase mask to produce improved measurements with reduced multiplexing. Next, we implement a ConvNeXt-based reconstruction architecture, which provides up to 6.68 dB improvement in peak signal-to-noise ratio over state-of-the-art attention-based architectures. Finally, we establish a parallel imaging setup that simultaneously images a scene with RML, diffuser and lens systems, with which we collect datasets with 100,000 measurements for each system, to be used for reconstruction model training and evaluation. Using this standardized system, we quantify the improved measurement quality of the RML compared to a diffuser using the modulation transfer function and mutual information. Our ConvRML system benefits from both the optical and the computational developments presented in this work, and our contributions establish resources to support continued development of high-quality, compact, and compressive lensless imagers. △ Less

Submitted 4 February, 2026; originally announced February 2026.

Comments: 28 pages, 11 figures

Correcting Fabrication-Induced Curvature in Micromirror-Based Spatial Light Modulators with a Microlens Array

Authors: Munkyu Kang, Elizabeth Murray, Leyla A. Kabuli, Rikky Muller, Laura Waller

Abstract: Computer generated holography requires high-speed spatial light modulators (SLMs) for dynamically patterning light in 3D. Piston-motion micromirror-based SLMs support high-speed ($\geq$ 10 kHz) phase modulation; however, fabricating micromirror arrays with sufficient fill factor necessary for high diffraction efficiency is challenging. In particular, the larger mirrors of high fill factor designs… ▽ More Computer generated holography requires high-speed spatial light modulators (SLMs) for dynamically patterning light in 3D. Piston-motion micromirror-based SLMs support high-speed ($\geq$ 10 kHz) phase modulation; however, fabricating micromirror arrays with sufficient fill factor necessary for high diffraction efficiency is challenging. In particular, the larger mirrors of high fill factor designs are susceptible to stress-induced curvature that significantly degrades optical performance. In this work, we introduce an optical compensation method using a pitch-matched microlens array (MLA) to focus light onto just the center of each mirror. Our approach thus avoids curvature-induced artifacts and improves optical fill factor to nearly 100$\%$, independent of the original mechanical fill factor. Through simulations and experiments on a fabricated micromirror array with bowed mirrors, we show that the Pearson correlation coefficient of the imparted phase profile is improved from 0.11 to 0.85 and the brightness of a holographically-generated single spot is enhanced by 8$\times$ with our microlens array in place. Our hybrid optical-electromechanical strategy thus provides a scalable path toward high-speed, high-fidelity wavefront control for applications such as adaptive optics, holographic displays, and optogenetics. △ Less

Submitted 26 May, 2026; v1 submitted 4 November, 2025; originally announced November 2025.

Comments: This paper has been published in Optics Express

Journal ref: Opt. Express 34(8), 15783-15796 (2026)

Large-scale compressive microscopy via diffractive multiplexing across a sensor array

Authors: Kevin C. Zhou, Chaoying Gu, Muneki Ikeda, Tina M. Hayward, Nicholas Antipa, Rajesh Menon, Roarke Horstmeyer, Saul Kato, Laura Waller

Abstract: Microscopes face a trade-off between spatial resolution, field-of-view, and frame rate -- improving one of these properties typically requires sacrificing the others, due to the limited spatiotemporal throughput of the sensor. To overcome this, we propose a new microscope that achieves snapshot gigapixel-scale imaging with a sensor array and a diffractive optical element (DOE). We improve the spat… ▽ More Microscopes face a trade-off between spatial resolution, field-of-view, and frame rate -- improving one of these properties typically requires sacrificing the others, due to the limited spatiotemporal throughput of the sensor. To overcome this, we propose a new microscope that achieves snapshot gigapixel-scale imaging with a sensor array and a diffractive optical element (DOE). We improve the spatiotemporal throughput in two ways. First, we capture data with an array of 48 sensors resulting in 48x more pixels than a single sensor. Second, we use point spread function (PSF) engineering and compressive sensing algorithms to fill in the missing information from the gaps surrounding the individual sensors in the array, further increasing the spatiotemporal throughput of the system by an additional >5.4x. The array of sensors is modeled as a single large-format "super-sensor," with erasures corresponding to the gaps between the individual sensors. The array is placed at the output of a (nearly) 4f imaging system, and we design a DOE for the Fourier plane that generates a distributed PSF that encodes information from the entire super-sensor area, including the gaps. We then computationally recover the large-scale image, assuming the object is sparse in some domain. Our calibration-free microscope can achieve ~3 μm resolution over >5.2 cm^2 FOVs at up to 120 fps, culminating in a total spatiotemporal throughput of 25.2 billion pixels per second. We demonstrate the versatility of our microscope in two different modes: structural imaging via darkfield contrast and functional fluorescence imaging of calcium dynamics across dozens of freely moving C. elegans simultaneously. △ Less

Submitted 18 July, 2025; originally announced July 2025.

Computationally Efficient Information-Driven Optical Design with Interchanging Optimization

Authors: Eric Markley, Henry Pinkard, Leyla Kabuli, Nalini Singh, Laura Waller

Abstract: Recent work has demonstrated that imaging systems can be evaluated through the information content of their measurements alone, enabling application-agnostic optical design that avoids computational decoding challenges. Information-Driven Encoder Analysis Learning (IDEAL) was proposed to automate this process through gradient-based optimization. In this work, we study IDEAL across diverse imaging… ▽ More Recent work has demonstrated that imaging systems can be evaluated through the information content of their measurements alone, enabling application-agnostic optical design that avoids computational decoding challenges. Information-Driven Encoder Analysis Learning (IDEAL) was proposed to automate this process through gradient-based optimization. In this work, we study IDEAL across diverse imaging systems and find that it suffers from high memory usage, long runtimes, and a potentially mismatched objective function due to end-to-end differentiability requirements. We introduce IDEAL with Interchanging Optimization (IDEAL-IO), a method that decouples density estimation from optical parameter optimization by alternating between fitting models to current measurements and updating optical parameters using fixed models for information estimation. This approach reduces runtime and memory usage by up to 6x while enabling more expressive density models that guide optimization toward superior designs. We validate our method on diffractive optics, lensless imaging, and snapshot 3D microscopy applications, establishing information-theoretic optimization as a practical, scalable strategy for real-world imaging system design. △ Less

Submitted 11 July, 2025; v1 submitted 10 July, 2025; originally announced July 2025.

Designing lensless imaging systems to maximize information capture

Authors: Leyla A. Kabuli, Henry Pinkard, Eric Markley, Clara S. Hung, Laura Waller

Abstract: Mask-based lensless imaging uses an optical encoder (e.g. a phase or amplitude mask) to capture measurements, then a computational decoding algorithm to reconstruct images. In this work, we evaluate and design lensless encoders based on the information content of their measurements using mutual information estimation. Our approach formalizes the object-dependent nature of lensless imaging and quan… ▽ More Mask-based lensless imaging uses an optical encoder (e.g. a phase or amplitude mask) to capture measurements, then a computational decoding algorithm to reconstruct images. In this work, we evaluate and design lensless encoders based on the information content of their measurements using mutual information estimation. Our approach formalizes the object-dependent nature of lensless imaging and quantifies the interdependence between object sparsity, encoder multiplexing, and noise. Our analysis reveals that optimal encoder designs should tailor encoder multiplexing to object sparsity for maximum information capture, and that all optimally-encoded measurements share the same level of sparsity. Using mutual information-based optimization, we design information-optimal encoders for compressive imaging of fixed object distributions. Our designs demonstrate improved downstream reconstruction performance for objects in the distribution, without requiring joint optimization with a specific reconstruction algorithm. We validate our approach experimentally by evaluating lensless imaging systems directly from captured measurements, without the need for image formation models, reconstruction algorithms, or ground truth data. Our comprehensive analysis establishes design and engineering principles for lensless imaging systems, and offers a model for the study of general multiplexing systems, especially those with object-dependent performance. △ Less

Submitted 11 February, 2026; v1 submitted 10 June, 2025; originally announced June 2025.

Comments: 27 pages, 12 figures

Journal ref: Optica 13(2) 227-235 (2026)

Sample Motion for Structured Illumination Fluorescence Microscopy

Authors: Ruiming Cao, Guanghan Meng, Laura Waller

Abstract: Structured illumination microscopy (SIM) uses a set of images captured with different illumination patterns to computationally reconstruct resolution beyond the diffraction limit. Here, we propose an alternative approach using a single speckle illumination pattern and relying on inherent sample motion to encode the super-resolved information in multiple raw images. From a set of raw fluorescence i… ▽ More Structured illumination microscopy (SIM) uses a set of images captured with different illumination patterns to computationally reconstruct resolution beyond the diffraction limit. Here, we propose an alternative approach using a single speckle illumination pattern and relying on inherent sample motion to encode the super-resolved information in multiple raw images. From a set of raw fluorescence images captured as the sample moves, we jointly estimate both the sample motion and the super-resolved image. We demonstrate the feasibility of the proposed method both in simulation and in experiment. △ Less

Submitted 27 February, 2025; v1 submitted 5 February, 2025; originally announced February 2025.

Journal ref: Opt. Lett. 50 (2025) 4074-4077

Estimation-theoretic analysis of lensless imaging

Authors: Leyla A. Kabuli, Nalini M. Singh, Laura Waller

Abstract: We analyze lensless imaging systems with estimation-theoretic techniques based on Fisher information. Our analysis evaluates multiple optical encoder designs on objects with varying sparsity, in the context of both Gaussian and Poisson noise models. Our simulations verify that lensless imaging system performance is object-dependent and highlight tradeoffs between encoder multiplexing and object sp… ▽ More We analyze lensless imaging systems with estimation-theoretic techniques based on Fisher information. Our analysis evaluates multiple optical encoder designs on objects with varying sparsity, in the context of both Gaussian and Poisson noise models. Our simulations verify that lensless imaging system performance is object-dependent and highlight tradeoffs between encoder multiplexing and object sparsity, showing quantitatively that sparse objects tolerate higher levels of multiplexing than dense objects. Insights from our analysis promise to inform and improve optical encoder designs for lensless imaging. △ Less

Submitted 24 January, 2025; originally announced January 2025.

Comments: 7 pages, 3 figures

Scalable dataset acquisition for data-driven lensless imaging

Authors: Clara S. Hung, Leyla A. Kabuli, Vasilisa Ponomarenko, Laura Waller

Abstract: Data-driven developments in lensless imaging, such as machine learning-based reconstruction algorithms, require large datasets. In this work, we introduce a data acquisition pipeline that can capture from multiple lensless imaging systems in parallel, under the same imaging conditions, and paired with computational ground truth registration. We provide an open-access 25,000 image dataset with two… ▽ More Data-driven developments in lensless imaging, such as machine learning-based reconstruction algorithms, require large datasets. In this work, we introduce a data acquisition pipeline that can capture from multiple lensless imaging systems in parallel, under the same imaging conditions, and paired with computational ground truth registration. We provide an open-access 25,000 image dataset with two lensless imagers, a reproducible hardware setup, and open-source camera synchronization code. Experimental datasets from our system can enable data-driven developments in lensless imaging, such as machine learning-based reconstruction algorithms and end-to-end system design. △ Less

Submitted 31 January, 2026; v1 submitted 22 January, 2025; originally announced January 2025.

Comments: 5 pages, 3 figures, to be published in SPIE Photonics West 2025 Proceedings

Perturbative Fourier Ptychographic Microscopy for Fast Quantitative Phase Imaging

Authors: Martin Zach, Kuan-Chen Shen, Ruiming Cao, Michael Unser, Laura Waller, Jonathan Dong

Abstract: In computational phase imaging with a microscope equipped with an array of light emitting diodes as illumination unit, conventional Fourier ptychographic microscopy achieves high resolution and wide-field reconstructions but is constrained by a lengthy acquisition time. Conversely, differential phase contrast (DPC) offers fast imaging but is limited in resolution. Here, we introduce perturbative F… ▽ More In computational phase imaging with a microscope equipped with an array of light emitting diodes as illumination unit, conventional Fourier ptychographic microscopy achieves high resolution and wide-field reconstructions but is constrained by a lengthy acquisition time. Conversely, differential phase contrast (DPC) offers fast imaging but is limited in resolution. Here, we introduce perturbative Fourier ptychographic microscopy (pFPM). pFPM is an extension of DPC that incorporates dark-field illumination to enable fast, high-resolution, wide-field quantitative phase imaging with few measurements. We interpret DPC as the initial iteration of a Gauss-Newton algorithm with quadratic regularization and generalize it to multiple iterations and more sophisticated regularizers. This broader framework is not restricted to bright-field measurements and allows us to overcome resolution limitations of DPC. We develop tailored annular dark-field illumination patterns that align with the perturbative interpretation and lead to an improvement in the quality of reconstruction with respect to other common illumination schemes. Consequently, our methodology combines an enhanced phase reconstruction algorithm with a specialized illumination strategy and offers significant advantages in both imaging speed and resolution. △ Less

Submitted 10 March, 2025; v1 submitted 13 January, 2025; originally announced January 2025.

Information-driven design of imaging systems

Authors: Henry Pinkard, Leyla Kabuli, Eric Markley, Tiffany Chien, Jiantao Jiao, Laura Waller

Abstract: Imaging systems have traditionally been designed to mimic the human eye and produce visually interpretable measurements. Modern imaging systems, however, process raw measurements computationally before or instead of human viewing. As a result, the information content of raw measurements matters more than their visual interpretability. Despite the importance of measurement information content, curr… ▽ More Imaging systems have traditionally been designed to mimic the human eye and produce visually interpretable measurements. Modern imaging systems, however, process raw measurements computationally before or instead of human viewing. As a result, the information content of raw measurements matters more than their visual interpretability. Despite the importance of measurement information content, current approaches for evaluating imaging system performance do not quantify it: they instead either use alternative metrics that assess specific aspects of measurement quality or assess measurements indirectly with performance on secondary tasks. We developed the theoretical foundations and a practical method to directly quantify mutual information between noisy measurements and unknown objects. By fitting probabilistic models to measurements and their noise characteristics, our method estimates information by upper bounding its true value. By applying gradient-based optimization to these estimates, we also developed a technique for designing imaging systems called Information-Driven Encoder Analysis Learning (IDEAL). Our information estimates accurately captured system performance differences across four imaging domains (color photography, radio astronomy, lensless imaging, and microscopy). Systems designed with IDEAL matched the performance of those designed with end-to-end optimization, the prevailing approach that jointly optimizes hardware and image processing algorithms. These results establish mutual information as a universal performance metric for imaging systems that enables both computationally efficient design optimization and evaluation in real-world conditions. A video summarizing this work can be found at: https://waller-lab.github.io/EncodingInformationWebsite/ △ Less

Submitted 6 November, 2025; v1 submitted 30 May, 2024; originally announced May 2024.

Non-Destructive, High-Resolution, Chemically Specific, 3D Nanostructure Characterization using Phase-Sensitive EUV Imaging Reflectometry

Authors: Michael Tanksalvala, Christina L. Porter, Yuka Esashi, Bin Wang, Nicholas W. Jenkins, Zhe Zhang, Galen P. Miley, Joshua L. Knobloch, Brendan McBennett, Naoto Horiguchi, Sadegh Yazdi, Jihan Zhou, Matthew N. Jacobs, Charles S. Bevis, Robert M. Karl Jr., Peter Johnsen, David Ren, Laura Waller, Daniel E. Adams, Seth L. Cousin, Chen-Ting Liao, Jianwei Miao, Michael Gerrity, Henry C. Kapteyn, Margaret M. Murnane

Abstract: Next-generation nano and quantum devices have increasingly complex 3D structure. As the dimensions of these devices shrink to the nanoscale, their performance is often governed by interface quality or precise chemical or dopant composition. Here we present the first phase-sensitive extreme ultraviolet imaging reflectometer. It combines the excellent phase stability of coherent high-harmonic source… ▽ More Next-generation nano and quantum devices have increasingly complex 3D structure. As the dimensions of these devices shrink to the nanoscale, their performance is often governed by interface quality or precise chemical or dopant composition. Here we present the first phase-sensitive extreme ultraviolet imaging reflectometer. It combines the excellent phase stability of coherent high-harmonic sources, the unique chemical- and phase-sensitivity of extreme ultraviolet reflectometry, and state-of-the-art ptychography imaging algorithms. This tabletop microscope can non-destructively probe surface topography, layer thicknesses, and interface quality, as well as dopant concentrations and profiles. High-fidelity imaging was achieved by implementing variable-angle ptychographic imaging, by using total variation regularization to mitigate noise and artifacts in the reconstructed image, and by using a high-brightness, high-harmonic source with excellent intensity and wavefront stability. We validate our measurements through multiscale, multimodal imaging to show that this technique has unique advantages compared with other techniques based on electron and scanning-probe microscopies. △ Less

Submitted 28 March, 2024; originally announced April 2024.

Comments: 47 pages, 16 figures (4 in main text, 12 supplement) 2 tables

Journal ref: Science Advances 7(5), eabd9667 (2021)

Unified, Verifiable Neural Simulators for Electromagnetic Wave Inverse Problems

Authors: Charles Dove, Jatearoon Boondicharern, Laura Waller

Abstract: Simulators based on neural networks offer a path to orders-of-magnitude faster electromagnetic wave simulations. Existing models, however, only address narrowly tailored classes of problems and only scale to systems of a few dozen degrees of freedom (DoFs). Here, we demonstrate a single, unified model capable of addressing scattering simulations with thousands of DoFs, of any wavelength, any illum… ▽ More Simulators based on neural networks offer a path to orders-of-magnitude faster electromagnetic wave simulations. Existing models, however, only address narrowly tailored classes of problems and only scale to systems of a few dozen degrees of freedom (DoFs). Here, we demonstrate a single, unified model capable of addressing scattering simulations with thousands of DoFs, of any wavelength, any illumination wavefront, and freeform materials, within broad configurable bounds. Based on an attentional multi-conditioning strategy, our method also allows non-recurrent supervision on and prediction of intermediate physical states, which provides improved generalization with no additional data-generation cost. Using this O(1)-time intermediate prediction capability, we propose and prove a rigorous, efficiently computable upper bound on prediction error, allowing accuracy guarantees at inference time for all predictions. After training solely on randomized systems, we demonstrate the unified model across a suite of challenging multi-disciplinary inverse problems, finding strong efficacy and speed improvements up to 96% for problems in optical tomography, beam shaping through volumetric random media, and freeform photonic inverse design, with no problem-specific training. Our findings demonstrate a path to universal, verifiably accurate neural surrogates for existing scattering simulators, and our conditioning and training methods are directly applicable to any PDE admitting a time-domain iterative solver. △ Less

Submitted 30 March, 2024; originally announced April 2024.

Wavefront Randomization Improves Deconvolution

Authors: Amit Kohli, Anastasios N. Angelopoulos, Laura Waller

Abstract: The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image. Digital correction techniques, such as deconvolution, have limited ability to correct the blur, since some spatial frequencies in the scene are not measured adequately (i.e., 'zeros' of the system transfer function). We prove that the addition of a random mask to an imaging system… ▽ More The performance of an imaging system is limited by optical aberrations, which cause blurriness in the resulting image. Digital correction techniques, such as deconvolution, have limited ability to correct the blur, since some spatial frequencies in the scene are not measured adequately (i.e., 'zeros' of the system transfer function). We prove that the addition of a random mask to an imaging system removes its dependence on aberrations, reducing the likelihood of zeros in the transfer function and consequently decreasing the sensitivity to noise during deconvolution. In simulation, we show that this strategy improves image quality over a range of aberration types, aberration strengths, and signal-to-noise ratios. △ Less

Submitted 12 February, 2024; v1 submitted 12 February, 2024; originally announced February 2024.

Comments: The first two authors contributed equally

ACM Class: I.4.4

Depth-multiplexing spectral domain OCT for full eye length imaging with a single modulation unit

Authors: Guanghan Meng, Xue Dong, Andrew Zhang, Fabio Feroldi, Austin Roorda, Laura Waller

Abstract: Clinical measurement of a patient's axial eye length is emerging as a crucial approach to track progression and monitor management of myopia. However, the preferred method for such measurements is swept-source OCT, whose cost prohibits broad use, especially in lower-income communities. Spectral domain (SD) OCT is a more affordable option, but it has limited imaging depth range, so is not suitable… ▽ More Clinical measurement of a patient's axial eye length is emerging as a crucial approach to track progression and monitor management of myopia. However, the preferred method for such measurements is swept-source OCT, whose cost prohibits broad use, especially in lower-income communities. Spectral domain (SD) OCT is a more affordable option, but it has limited imaging depth range, so is not suitable for full eye length measurement. Depth-multiplexing (DM) techniques for SD-OCT provide a workaround by capturing images at multiple depths within the eye. However, these methods typically require multiple light modulation units or detectors for simultaneous imaging across depths, adding complexity and cost. In response, we propose a novel DM-SD-OCT approach that utilizes a single light modulation unit for depth encoding. We capture images at multiple depths within the eye simultaneously with a single line scan camera, then computationally demix the contributions from different depths. Here, we demonstrate acquisition and demixing of signals from three distinct depths within the eye and validate experimentally in human subjects. Our method thus offers a cost-effective solution for comprehensive eye length measurement in clinical myopia research. △ Less

Submitted 13 February, 2026; v1 submitted 29 November, 2023; originally announced November 2023.

Fast non-iterative algorithm for 3D point-cloud holography

Authors: Nathan Tessema Ersaro, Cem Yalcin, Liz Murray, Leyla Kabuli, Laura Waller, Rikky Muller

Abstract: Recently developed iterative and deep learning-based approaches to computer-generated holography (CGH) have been shown to achieve high-quality photorealistic 3D images with spatial light modulators. However, such approaches remain overly cumbersome for patterning sparse collections of target points across a photoresponsive volume in applications including biological microscopy and material process… ▽ More Recently developed iterative and deep learning-based approaches to computer-generated holography (CGH) have been shown to achieve high-quality photorealistic 3D images with spatial light modulators. However, such approaches remain overly cumbersome for patterning sparse collections of target points across a photoresponsive volume in applications including biological microscopy and material processing. Specifically, in addition to requiring heavy computation that cannot accommodate real-time operation in mobile or hardware-light settings, existing sampling-dependent 3D CGH methods preclude the ability to place target points with arbitrary precision, limiting accessible depths to a handful of planes. Accordingly, we present a non-iterative point cloud holography algorithm that employs fast deterministic calculations in order to efficiently allocate patches of SLM pixels to different target points in the 3D volume and spread the patterning of all points across multiple time frames. Compared to a matched-performance implementation of the iterative Gerchberg-Saxton algorithm, our algorithm's relative computation speed advantage was found to increase with SLM pixel count, exceeding 100,000x at 512x512 array format. △ Less

Submitted 7 September, 2023; v1 submitted 21 June, 2023; originally announced June 2023.

Comments: 22 pages, 11 figures, manuscript and supplement

Roadmap on Deep Learning for Microscopy

Authors: Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C. D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu , et al. (50 additional authors not shown)

Abstract: Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the… ▽ More Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences. △ Less

Submitted 7 March, 2023; originally announced March 2023.

Dynamic Structured Illumination Microscopy with a Neural Space-time Model

Authors: Ruiming Cao, Fanglin Linda Liu, Li-Hao Yeh, Laura Waller

Abstract: Structured illumination microscopy (SIM) reconstructs a super-resolved image from multiple raw images captured with different illumination patterns; hence, acquisition speed is limited, making it unsuitable for dynamic scenes. We propose a new method, Speckle Flow SIM, that uses static patterned illumination with moving samples and models the sample motion during data capture in order to reconstru… ▽ More Structured illumination microscopy (SIM) reconstructs a super-resolved image from multiple raw images captured with different illumination patterns; hence, acquisition speed is limited, making it unsuitable for dynamic scenes. We propose a new method, Speckle Flow SIM, that uses static patterned illumination with moving samples and models the sample motion during data capture in order to reconstruct the dynamic scene with super-resolution. Speckle Flow SIM relies on sample motion to capture a sequence of raw images. The spatio-temporal relationship of the dynamic scene is modeled using a neural space-time model with coordinate-based multi-layer perceptrons (MLPs), and the motion dynamics and the super-resolved scene are jointly recovered. We validate Speckle Flow SIM for coherent imaging in simulation and build a simple, inexpensive experimental setup with off-the-shelf components. We demonstrate that Speckle Flow SIM can reconstruct a dynamic scene with deformable motion and 1.88x the diffraction-limited resolution in experiment. △ Less

Submitted 28 July, 2022; v1 submitted 3 June, 2022; originally announced June 2022.

Journal ref: IEEE International Conference on Computational Photography (ICCP) 2022

A MEMS-based optical scanning system for precise, high-speed neural interfacing

Authors: Cem Yalcin, Nathan Tessema Ersaro, M. Meraj Ghanbari, George Bocchetti, Sina Faraji Alamouti, Nick Antipa, Daniel Lopez, Nicolas C. Pégard, Laura Waller, Rikky Muller

Abstract: Optical scanning is a prevalent technique for optical neural interfaces where light delivery with high spatial and temporal precision is desired. However, due to the sequential nature of point-scanning techniques, the settling time of optical modulators is a major bottleneck for throughput and limits random-access targeting capabilities. While fast lateral scanners exist, commercially available va… ▽ More Optical scanning is a prevalent technique for optical neural interfaces where light delivery with high spatial and temporal precision is desired. However, due to the sequential nature of point-scanning techniques, the settling time of optical modulators is a major bottleneck for throughput and limits random-access targeting capabilities. While fast lateral scanners exist, commercially available varifocal elements are constrained to >3ms settling times, limiting the speed of the overall system to hundreds of Hz. Faster focusing methods exist but cannot combine fast operation and dwelling capability with electrical and optical efficiency. Here, we present a varifocal mirror comprised of an array of piston-motion MEMS micromirrors and a custom driver ASIC, offering fast operation with dwelling capability while maintaining high diffraction efficiency. The ASIC features a reconfigurable nonlinear DAC to simultaneously compensate for the built-in nonlinearity of electrostatic actuators and the global process variations in MEMS mirrors. Experimental results demonstrate a wide continuous sweeping range that spans 22 distinctly resolvable depth planes with refresh rates greater than 12 kHz. △ Less

Submitted 7 December, 2021; originally announced December 2021.

Comments: This work has been submitted to the IEEE for possible publication

Sparse deep computer-generated holography for optical microscopy

Authors: Alex Liu, Yi Xue, Laura Waller

Abstract: Computer-generated holography (CGH) has broad applications such as direct-view display, virtual and augmented reality, as well as optical microscopy. CGH usually utilizes a spatial light modulator that displays a computer-generated phase mask, modulating the phase of coherent light in order to generate customized patterns. The algorithm that computes the phase mask is the core of CGH and is usuall… ▽ More Computer-generated holography (CGH) has broad applications such as direct-view display, virtual and augmented reality, as well as optical microscopy. CGH usually utilizes a spatial light modulator that displays a computer-generated phase mask, modulating the phase of coherent light in order to generate customized patterns. The algorithm that computes the phase mask is the core of CGH and is usually tailored to meet different applications. CGH for optical microscopy usually requires 3D accessibility (i.e., generating overlapping patterns along the $z$-axis) and micron-scale spatial precision. Here, we propose a CGH algorithm using an unsupervised generative model designed for optical microscopy to synthesize 3D selected illumination. The algorithm, named sparse deep CGH, is able to generate sparsely distributed points in a large 3D volume with higher contrast than conventional CGH algorithms. △ Less

Submitted 12 December, 2021; v1 submitted 30 November, 2021; originally announced November 2021.

Comments: 5 pages, 4 figures, to be presented at NeurIPS 2021 Deep Learning and Inverse Problems workshop

Roadmap on Wavefront Shaping and deep imaging in complex media

Authors: Sylvain Gigan, Ori Katz, Hilton B. de Aguiar, Esben Ravn Andresen, Alexandre Aubry, Jacopo Bertolotti, Emmanuel Bossy, Dorian Bouchet, Joshua Brake, Sophie Brasselet, Yaron Bromberg, Hui Cao, Thomas Chaigne, Zhongtao Cheng, Wonshik Choi, Tomáš Čižmár, Meng Cui, Vincent R Curtis, Hugo Defienne, Matthias Hofer, Ryoichi Horisaki, Roarke Horstmeyer, Na Ji, Aaron K. LaViolette, Jerome Mertz , et al. (20 additional authors not shown)

Abstract: The last decade has seen the development of a wide set of tools, such as wavefront shaping, computational or fundamental methods, that allow to understand and control light propagation in a complex medium, such as biological tissues or multimode fibers. A vibrant and diverse community is now working on this field, that has revolutionized the prospect of diffraction-limited imaging at depth in tiss… ▽ More The last decade has seen the development of a wide set of tools, such as wavefront shaping, computational or fundamental methods, that allow to understand and control light propagation in a complex medium, such as biological tissues or multimode fibers. A vibrant and diverse community is now working on this field, that has revolutionized the prospect of diffraction-limited imaging at depth in tissues. This roadmap highlights several key aspects of this fast developing field, and some of the challenges and opportunities ahead. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: submitted to J.Phys Photonics (IOP), 116 pages, 23 sections

Distributed Reconstruction Algorithm for Electron Tomography with Multiple-scattering Samples

Authors: David Ren, Michael Whittaker, Colin Ophus, Laura Waller

Abstract: Three-dimensional electron tomography is used to understand the structure and properties of samples in chemistry, materials science, geoscience, and biology. With the recent development of high-resolution detectors and algorithms that can account for multiple-scattering events, thicker samples can be examined at finer resolution, resulting in larger reconstruction volumes than previously possible.… ▽ More Three-dimensional electron tomography is used to understand the structure and properties of samples in chemistry, materials science, geoscience, and biology. With the recent development of high-resolution detectors and algorithms that can account for multiple-scattering events, thicker samples can be examined at finer resolution, resulting in larger reconstruction volumes than previously possible. In this work, we propose a distributed computing framework that reconstructs large volumes by decomposing a projected tilt-series into smaller datasets such that sub-volumes can be simultaneously reconstructed on separate compute nodes using a cluster. We demonstrate our method by reconstructing a multiple-scattering layered clay (montmorillonite) sample at high resolution from a large field-of-view tilt-series phase contrast transmission electron microscopty dataset. △ Less

Submitted 15 October, 2021; originally announced October 2021.

Exceeding the limits of algorithmic self-calibration in super-resolution imaging

Authors: Eric Li, Stuart Sherwin, Gautam Gunjala, Laura Waller

Abstract: Fourier ptychographic microscopy is a computational imaging technique that provides quantitative phase information and high resolution over a large field-of-view. Although the technique presents numerous advantages over conventional microscopy, model mismatch due to unknown optical aberrations can significantly limit reconstruction quality. Many attempts to address this issue rely on embedding pup… ▽ More Fourier ptychographic microscopy is a computational imaging technique that provides quantitative phase information and high resolution over a large field-of-view. Although the technique presents numerous advantages over conventional microscopy, model mismatch due to unknown optical aberrations can significantly limit reconstruction quality. Many attempts to address this issue rely on embedding pupil recovery into the reconstruction algorithm. In this paper we demonstrate the limitations of a purely algorithmic approach and evaluate the merits of implementing a simple, dedicated calibration procedure. In simulations, we find that for a target sample reconstruction error, we can image without any aberration corrections up to a maximum aberration magnitude of $λ$/40. When we use algorithmic self-calibration, we can increase the aberration magnitude up to $λ$/10, and with our in situ speckle calibration technique, this working range is extended further to a maximum aberration magnitude of $λ$/3. Hence, one can trade-off complexity for accuracy by using a separate calibration process, which is particularly useful for larger aberrations. △ Less

Submitted 15 September, 2021; originally announced September 2021.

Untrained networks for compressive lensless photography

Authors: Kristina Monakhova, Vi Tran, Grace Kuo, Laura Waller

Abstract: Compressive lensless imagers enable novel applications in an extremely compact device, requiring only a phase or amplitude mask placed close to the sensor. They have been demonstrated for 2D and 3D microscopy, single-shot video, and single-shot hyperspectral imaging; in each of these cases, a compressive-sensing-based inverse problem is solved in order to recover a 3D data-cube from a 2D measureme… ▽ More Compressive lensless imagers enable novel applications in an extremely compact device, requiring only a phase or amplitude mask placed close to the sensor. They have been demonstrated for 2D and 3D microscopy, single-shot video, and single-shot hyperspectral imaging; in each of these cases, a compressive-sensing-based inverse problem is solved in order to recover a 3D data-cube from a 2D measurement. Typically, this is accomplished using convex optimization and hand-picked priors. Alternatively, deep learning-based reconstruction methods offer the promise of better priors, but require many thousands of ground truth training pairs, which can be difficult or impossible to acquire. In this work, we propose the use of untrained networks for compressive image recovery. Our approach does not require any labeled training data, but instead uses the measurement itself to update the network weights. We demonstrate our untrained approach on lensless compressive 2D imaging as well as single-shot high-speed video recovery using the camera's rolling shutter, and single-shot hyperspectral imaging. We provide simulation and experimental verification, showing that our method results in improved image quality over existing methods. △ Less

Submitted 21 June, 2021; v1 submitted 12 March, 2021; originally announced March 2021.

Comments: 17 pages, 8 figures

Journal ref: Optics Express Vol. 29, Issue 13, pp. 20913-20929 (2021)

Three-dimensional bi-functional refractive index and fluorescence microscopy (BRIEF)

Authors: Yi Xue, David Ren, Laura Waller

Abstract: Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) info… ▽ More Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) information or the label-free RI (structural) information, though it may be valuable to have both. For example, biological tissues have heterogeneous RI distributions, causing sample-induced scattering that degrades the fluorescence image quality. When both fluorescence and 3D RI are measured, one can use the RI information to digitally correct multiple-scattering effects in the fluorescence image. Here, we develop a new computational multi-modal imaging method based on epi-mode microscopy that reconstructs both 3D fluorescence and 3D RI from a single dataset. We acquire dozens of fluorescence images, each 'illuminated' by a single fluorophore, then solve an inverse problem with a multiple-scattering forward model. We experimentally demonstrate our method for epi-mode 3D RI imaging and digital correction of multiple-scattering effects in fluorescence images. △ Less

Submitted 25 January, 2022; v1 submitted 23 November, 2020; originally announced November 2020.

Comments: 21 pages, 4 figures

Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy

Authors: Kyrollos Yanny, Nick Antipa, William Liberti, Sam Dehaeck, Kristina Monakhova, Fanglin Linda Liu, Konlin Shen, Ren Ng, Laura Waller

Abstract: Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal pha… ▽ More Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal phase mask at the objective's aperture stop. Placing the phase mask at the aperture stop significantly reduces the size of the device, and varying the focal lengths enables a uniform resolution across a wide depth range. The phase mask encodes the 3D fluorescence intensity into a single 2D measurement, and the 3D volume is recovered by solving a sparsity-constrained inverse problem. We provide methods for designing and fabricating the phase mask and an efficient forward model that accounts for the field-varying aberrations in miniature objectives. We demonstrate a prototype that is 17 mm tall and weighs 2.5 grams, achieving 2.76 $μ$m lateral, and 15 $μ$m axial resolution across most of the 900x700x390 $μm^3$ volume at 40 volumes per second. The performance is validated experimentally on resolution targets, dynamic biological samples, and mouse brain tissue. Compared with existing miniature single-shot volume-capture implementations, our system is smaller and lighter and achieves a more than 2x better lateral and axial resolution throughout a 10x larger usable depth range. Our microscope design provides single-shot 3D imaging for applications where a compact platform matters, such as volumetric neural imaging in freely moving animals and 3D motion studies of dynamic samples in incubators and lab-on-a-chip devices. △ Less

Submitted 11 October, 2020; originally announced October 2020.

Comments: Published with Nature Springer in Light: Science and Applications

Journal ref: Light: Science & Applications 9.1 (2020): 1-13

Fourier DiffuserScope: Single-shot 3D Fourier light field microscopy with a diffuser

Authors: Fanglin Linda Liu, Grace Kuo, Nick Antipa, Kyrollos Yanny, Laura Waller

Abstract: Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provi… ▽ More Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we for the first time establish a theoretical framework and design guidelines, which are verified by numerical simulations, then build an experimental system that achieves $< 3$ um lateral and $4$ um axial resolution over a $1000 \times 1000 \times 280$ um$^3$ volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially. △ Less

Submitted 16 September, 2020; v1 submitted 29 June, 2020; originally announced June 2020.

Comments: 19 pages, 7 figures

Journal ref: Optics Express 28, 28969-28986 (2020)

Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array

Authors: Kristina Monakhova, Kyrollos Yanny, Neerja Aggarwal, Laura Waller

Abstract: Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensi… ▽ More Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensive computational camera for snapshot hyperspectral imaging. Our system consists of a tiled spectral filter array placed directly on the image sensor and a diffuser placed close to the sensor. Each point in the world maps to a unique pseudorandom pattern on the spectral filter array, which encodes multiplexed spatio-spectral information. By solving a sparsity-constrained inverse problem, we recover the hyperspectral volume with sub-super-pixel resolution. Our hyperspectral imaging framework is flexible and can be designed with contiguous or non-contiguous spectral filters that can be chosen for a given application. We provide theory for system design, demonstrate a prototype device, and present experimental results with high spatio-spectral resolution. △ Less

Submitted 28 September, 2020; v1 submitted 15 June, 2020; originally announced June 2020.

Comments: 10 pages, 10 figures, Optica

Journal ref: Optica 7, 1298-1307 (2020)

A micromirror array with annular partitioning for high-speed random-access axial focusing

Authors: Nathan Tessema Ersumo, Cem Yalcin, Nick Antipa, Nicolas Pegard, Laura Waller, Daniel Lopez, Rikky Muller

Abstract: Dynamic axial focusing functionality has recently experienced widespread incorporation in microscopy, augmented/virtual reality (AR/VR), adaptive optics, and material processing. However, the limitations of existing varifocal tools continue to beset the performance capabilities and operating overhead of the optical systems that mobilize such functionality. The varifocal tools that are the least bu… ▽ More Dynamic axial focusing functionality has recently experienced widespread incorporation in microscopy, augmented/virtual reality (AR/VR), adaptive optics, and material processing. However, the limitations of existing varifocal tools continue to beset the performance capabilities and operating overhead of the optical systems that mobilize such functionality. The varifocal tools that are the least burdensome to drive (ex: liquid crystal, elastomeric or optofluidic lenses) suffer from low (~ 100 Hz) refresh rates. Conversely, the fastest devices sacrifice either critical capabilities such as their dwelling capacity (ex: acoustic gradient lenses or monolithic micromechanical mirrors) or low operating overhead (e.g., deformable mirrors). Here, we present a general-purpose random-access axial focusing device that bridges these previously conflicting features of high speed, dwelling capacity and lightweight drive by employing low-rigidity micromirrors that exploit the robustness of defocusing phase profiles. Geometrically, the device consists of an 8.2 mm diameter array of piston-motion and 48 um-pitch micromirror pixels that provide 2pi phase shifting for wavelengths shorter than 1 100 nm with 10-90 % settling in 64.8 us (i.e., 15.44 kHz refresh rate). The pixels are electrically partitioned into 32 rings for a driving scheme that enables phase-wrapped operation with circular symmetry and requires less than 30 V per channel. Optical experiments demonstrated the array's wide focusing range with a measured ability to target 29 distinct, resolvable depth planes. Overall, the features of the proposed array offer the potential for compact, straightforward methods of tackling bottlenecked applications including high-throughput single-cell targeting in neurobiology and the delivery of dense 3D visual information in AR/VR. △ Less

Submitted 29 September, 2020; v1 submitted 13 June, 2020; originally announced June 2020.

Comments: 38 pages, 8 figures

Journal ref: Light: Science & Applications (2020) 9:183

Deep Phase Decoder: Self-calibrating phase microscopy with an untrained deep neural network

Authors: Emrah Bostan, Reinhard Heckel, Michael Chen, Michael Kellman, Laura Waller

Abstract: Deep neural networks have emerged as effective tools for computational imaging including quantitative phase microscopy of transparent samples. To reconstruct phase from intensity, current approaches rely on supervised learning with training examples; consequently, their performance is sensitive to a match of training and imaging settings. Here we propose a new approach to phase microscopy by using… ▽ More Deep neural networks have emerged as effective tools for computational imaging including quantitative phase microscopy of transparent samples. To reconstruct phase from intensity, current approaches rely on supervised learning with training examples; consequently, their performance is sensitive to a match of training and imaging settings. Here we propose a new approach to phase microscopy by using an untrained deep neural network for measurement formation, encapsulating the image prior and imaging physics. Our approach does not require any training data and simultaneously reconstructs the sought phase and pupil-plane aberrations by fitting the weights of the network to the captured images. To demonstrate experimentally, we reconstruct quantitative phase from through-focus images blindly (i.e. no explicit knowledge of the aberrations). △ Less

Submitted 24 January, 2020; originally announced January 2020.

Comments: 11 pages, 3 figures

High-resolution 3D refractive index microscopy of multiple-scattering samples from intensity images

Authors: Shwetadwip Chowdhury, Michael Chen, Regina Eckert, David Ren, Fan Wu, Nicole Repina, Laura Waller

Abstract: Optical diffraction tomography (ODT) reconstructs a samples volumetric refractive index (RI) to create high-contrast, quantitative 3D visualizations of biological samples. However, standard implementations of ODT use interferometric systems, and so are sensitive to phase instabilities, complex mechanical design, and coherent noise. Furthermore, their reconstruction framework is typically limited t… ▽ More Optical diffraction tomography (ODT) reconstructs a samples volumetric refractive index (RI) to create high-contrast, quantitative 3D visualizations of biological samples. However, standard implementations of ODT use interferometric systems, and so are sensitive to phase instabilities, complex mechanical design, and coherent noise. Furthermore, their reconstruction framework is typically limited to weakly-scattering samples, and thus excludes a whole class of multiple-scattering samples. Here, we implement a new 3D RI microscopy technique that utilizes a computational multi-slice beam propagation method to invert the optical scattering process and reconstruct high-resolution (NA>1.0) 3D RI distributions of multiple-scattering samples. The method acquires intensity-only measurements from different illumination angles, and then solves a non-linear optimization problem to recover the sample 3D RI distribution. We experimentally demonstrate reconstruction of samples with varying amounts of multiple scattering: a 3T3 fibroblast cell, a cluster of C. elegans embryos, and a whole C. elegans worm, with lateral and axial resolutions of 250 nm and 900 nm, respectively. △ Less

Submitted 9 September, 2019; v1 submitted 30 August, 2019; originally announced September 2019.

Comments: 24 pages, 8 figures

Computational structured illumination for high-content fluorescent and phase microscopy

Authors: Li-Hao Yeh, Shwetadwip Chowdhury, Laura Waller

Abstract: High-content biological microscopy targets high-resolution imaging across large fields-of-view (FOVs). Recent works have demonstrated that computational imaging can provide efficient solutions for high-content microscopy. Here, we use speckle structured illumination microscopy (SIM) as a robust and cost-effective solution for high-content fluorescence microscopy with simultaneous high-content quan… ▽ More High-content biological microscopy targets high-resolution imaging across large fields-of-view (FOVs). Recent works have demonstrated that computational imaging can provide efficient solutions for high-content microscopy. Here, we use speckle structured illumination microscopy (SIM) as a robust and cost-effective solution for high-content fluorescence microscopy with simultaneous high-content quantitative phase (QP). This multi-modal compatibility is essential for studies requiring cross-correlative biological analysis. Our method uses laterally-translated Scotch tape to generate high-resolution speckle illumination patterns across a large FOV. Custom optimization algorithms then jointly reconstruct the sample's super-resolution fluorescent (incoherent) and QP (coherent) distributions, while digitally correcting for system imperfections such as unknown speckle illumination patterns, system aberrations and pattern translations. Beyond previous linear SIM works, we achieve resolution gains of 4x the objective's diffraction-limited native resolution, resulting in 700 nm fluorescence and 1.2 um QP resolution, across a FOV of 2x2.7 mm^2, giving a space-bandwidth product (SBP) of 60 megapixels. △ Less

Submitted 27 March, 2019; v1 submitted 21 December, 2018; originally announced January 2019.

Journal ref: Biomed. Opt. Express 10, 1978-1998 (2019)

A Practical Reconstruction Method for Three-Dimensional Phase Contrast Atomic Electron Tomography

Authors: David Ren, Michael Chen, Laura Waller, Colin Ophus

Abstract: Electron tomography is a technique used in both materials science and structural biology to image features well below optical resolution limit. In this work, we present a new algorithm for reconstructing the three-dimensional(3D) electrostatic potential of a sample at atomic resolution from phase contrast imaging using high-resolution transmission electron microscopy. Our method accounts for dynam… ▽ More Electron tomography is a technique used in both materials science and structural biology to image features well below optical resolution limit. In this work, we present a new algorithm for reconstructing the three-dimensional(3D) electrostatic potential of a sample at atomic resolution from phase contrast imaging using high-resolution transmission electron microscopy. Our method accounts for dynamical and strong phase scattering, providing more accurate results with much lower electron doses than those current atomic electron tomography experiments. We test our algorithm using simulated images of a synthetic needle geometry dataset composed of an amorphous silicon dioxide shell around a silicon core. Our results show that, for a wide range of experimental parameters, we can accurately determine both atomic positions and species, and also identify vacancies even for light elements such as silicon and disordered materials such as amorphous silicon dioxide and also identify vacancies. △ Less

Submitted 10 July, 2018; originally announced July 2018.

Multiplexed phase-space imaging for 3D fluorescence microscopy

Authors: Hsiou-Yuan Liu, Jingshan Zhong, Laura Waller

Abstract: Optical phase-space functions describe spatial and angular information simultaneously; examples include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient a… ▽ More Optical phase-space functions describe spatial and angular information simultaneously; examples include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101-Megavoxel 3D volume ($1010\times510\times500μ$m with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture. △ Less

Submitted 30 May, 2017; v1 submitted 27 March, 2017; originally announced March 2017.

Structured illumination microscopy with unknown patterns and a statistical prior

Authors: Li-Hao Yeh, Lei Tian, Laura Waller

Abstract: Structured illumination microscopy (SIM) improves resolution by down-modulating high-frequency information of an object to fit within the passband of the optical system. Generally, the reconstruction process requires prior knowledge of the illumination patterns, which implies a well-calibrated and aberration-free system. Here, we propose a new \textit{algorithmic self-calibration} strategy for SIM… ▽ More Structured illumination microscopy (SIM) improves resolution by down-modulating high-frequency information of an object to fit within the passband of the optical system. Generally, the reconstruction process requires prior knowledge of the illumination patterns, which implies a well-calibrated and aberration-free system. Here, we propose a new \textit{algorithmic self-calibration} strategy for SIM that does not need to know the exact patterns {\it a priori}, but only their covariance. The algorithm, termed PE-SIMS, includes a Pattern-Estimation (PE) step requiring the uniformity of the sum of the illumination patterns and a SIM reconstruction procedure using a Statistical prior (SIMS). Additionally, we perform a pixel reassignment process (SIMS-PR) to enhance the reconstruction quality. We achieve 2$\times$ better resolution than a conventional widefield microscope, while remaining insensitive to aberration-induced pattern distortion and robust against parameter tuning. △ Less

Submitted 12 January, 2017; v1 submitted 26 October, 2016; originally announced November 2016.

Journal ref: Biomed. Opt. Express 8, 695-711 (2017)

Experimental robustness of Fourier Ptychography phase retrieval algorithms

Authors: Li-Hao Yeh, Jonathan Dong, Jingshan Zhong, Lei Tian, Michael Chen, Gongguo Tang, Mahdi Soltanolkotabi, Laura Waller

Abstract: Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, a nonlinear inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multipl… ▽ More Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, a nonlinear inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multiple proposed inverse algorithms in terms of experimental robustness. We find that the main sources of error are noise, aberrations and mis-calibration (i.e. model mis-match). Using simulations and experiments, we demonstrate that the choice of cost function plays a critical role, with amplitude-based cost functions performing better than intensity-based ones. The reason for this is that Fourier ptychography datasets consist of images from both brightfield and darkfield illumination, representing a large range of measured intensities. Both noise (e.g. Poisson noise) and model mis-match errors are shown to scale with intensity. Hence, algorithms that use an appropriate cost function will be more tolerant to both noise and model mis-match. Given these insights, we propose a global Newton's method algorithm which is robust and computationally efficient. Finally, we discuss the impact of procedures for algorithmic correction of aberrations and mis-calibration. △ Less

Submitted 18 December, 2015; v1 submitted 9 November, 2015; originally announced November 2015.

Journal ref: Opt. Express 23, 33214-33240 (2015)

Computational illumination for high-speed in vitro Fourier ptychographic microscopy

Authors: Lei Tian, Ziji Liu, Li-Hao Yeh, Michael Chen, Jingshan Zhong, Laura Waller

Abstract: We demonstrate a new computational illumination technique that achieves large space-bandwidth-time product, for quantitative phase imaging of unstained live samples in vitro. Microscope lenses can have either large field of view (FOV) or high resolution, not both. Fourier ptychographic microscopy (FPM) is a new computational imaging technique that circumvents this limit by fusing information from… ▽ More We demonstrate a new computational illumination technique that achieves large space-bandwidth-time product, for quantitative phase imaging of unstained live samples in vitro. Microscope lenses can have either large field of view (FOV) or high resolution, not both. Fourier ptychographic microscopy (FPM) is a new computational imaging technique that circumvents this limit by fusing information from multiple images taken with different illumination angles. The result is a gigapixel-scale image having both wide FOV and high resolution, i.e. large space-bandwidth product (SBP). FPM has enormous potential for revolutionizing microscopy and has already found application in digital pathology. However, it suffers from long acquisition times (on the order of minutes), limiting throughput. Faster capture times would not only improve imaging speed, but also allow studies of live samples, where motion artifacts degrade results. In contrast to fixed (e.g. pathology) slides, live samples are continuously evolving at various spatial and temporal scales. Here, we present a new source coding scheme, along with real-time hardware control, to achieve 0.8 NA resolution across a 4x FOV with sub-second capture times. We propose an improved algorithm and new initialization scheme, which allow robust phase reconstruction over long time-lapse experiments. We present the first FPM results for both growing and confluent in vitro cell cultures, capturing videos of subcellular dynamical phenomena in popular cell lines undergoing division and migration. Our method opens up FPM to applications with live samples, for observing rare events in both space and time. △ Less

Submitted 19 September, 2015; v1 submitted 13 June, 2015; originally announced June 2015.

Journal ref: Optica Vol. 2, Issue 10, pp. 904-911 (2015)