Motivation
Most mathematical tools for processing signals and images rely on the manipulation of signal/pixel intensities at `fixed' positions on a pixel grid. The success of modern algorithms for data analysis such as registration/tracking, dynamic time-warping, neural networks, etc., however, has taught us that both 1) nonlinearity, and 2) modeling the location of pixel intensities are essential concepts to keep in mind when solving modern problems related to estimation and classification. Our goal is thus to establish a mathematical signal representation method based on the idea of matching signals & images by pixel displacement operations that are physically related to the concept of transport phenomena. The figure shown on the left gives examples of matching signals in 1D using the Transport/Wasserstein geometry, versus matching signals with the traditional Euclidean/Lp-based methods. It can be seen the Lagrangian framework enables signal intensities to be carried/transported through the signal domain, thus enabling more meaningful comparisons.
There are multiple benefits offered by representing signals with the Lagrangian framework including the ability to model signal texture and shapes simultaneously, obtaining more parsimonious representations, as well as simplifying the solution to pattern recognition problems (e.g. classification). For example, the Lagrangian signal representation framework can be shown to make signal classes (e.g. classifying handwritten digits, benign vs malignant cancer cells, etc.) more convex and thus more easily (e.g. linearly) separable.
In addition to theory, our project also aims to test these idea in diverse fields including machine learning, optics, computer vision and biomedical imaging. Specific applications include: able to distinguish between benign and malignant tumors in medical images, learning models (e.g., dictionaries) for solving inverse problems, identifying people from images of faces, voice profiles, or fingerprints; and many others.
Publications
Overviews
- S Kolouri, SR Park, M Thorpe, D Slepcev, GK Rohde, Optimal mass transport for signal processing and machine learning, IEEE Signal Processing Magazine, 34(4), pp 43-59, 2017. paper , code
- S Kolouri, SR Park, M Thorpe, D SlepĨev, GK Rohde, Transport-based analysis, modeling, and learning from signal and data distributions, Arxiv preprint , 2016.
Signal & image representation/transforms:
- The signed cumulative distribution transform (SCDT) for signal analysis and classification, SIAM Foundation of data science, 2022. Preprint , Software: PyTransKit .
- Partitioning signal classes using transport transforms, SaSiDa, 2021, preprint
- SR Park, S Kolouri, S Kundu, GK Rohde, The Cumulative Distribution Transform and Linear Pattern Classification, Applied and Computational Harmonic Anaysis, 2018. preprint , journal , Matlab software , iPython notebook example , github tutorial
- M Thorpe, SR Park, S Kolouri, GK Rohde, D Slepcev, A Transportation Lp Distance for Signal Analysis, Journal of Mathematical Imaging and Vision, 2017. paper , code (warning ~2GB)
- S Kolouri, SR Park, GK Rohde, The Radon cumulative distribution transform and its application to image classification, IEEE Transactions on Image Processing, 25(2), pp 920-934, 2016. Preprint available at arXiv:1511.03206. link , Matlab software , iPython notebook example
- S Kolouri, AB Tosun, JA Ozolek, GK Rohde, A Continuous Linear Optimal Transport Approach for Pattern Analysis in Image Datasets, Pattern Recognition, 2016. preprint , journal site .
- W Wang, D Slepcev, JA Ozolek, S Basu, GK Rohde, A
linear optimal transportation framework for quantifying and visualizing
variations in sets of images. International Journal of Computer Vision,
vol. 101(2), pp. 254-269, 2013. Journal
site , pdf .
Machine learning:
- Rubaiyat et al, End-to-end Signal Classification in Signed Cumulative Distribution Space, 2022, Paper, Software: PyTransKit .
- Shifat-E-Rabbi et al, Invariance encoding in sliced-Wasserstein space for image classification with limited training data, preprint , 2022
- Shifat-E-Rabbi et al, R-CDT subspace models for image classification, JMIV, 2021 paper , python code , Matlab code , PyTransKit .
- S Kolouri, N Nadarializadeh, GK Rohde, H Hoffman, Wasserstein embedding for graph learning, 2020, paper
- S Kolouri, K Nadjahi, U Simsekli, R Badeau, GK Rohde Generalized sliced Wasserstein distances: NeurIPS 2019 paper
- S Kolouri, CE Martin, GK Rohde, Sliced-Wasserstein Autoencoder, ICLR 2019. preprint , code
- S Kolouri, GK Rohde, H Hoffman, Sliced Wasserstein Distance for Learning Gaussian Mixture Models, CVPR 2018. paper , code
- S Kolouri, Y Zou, GK Rohde, Sliced Wasserstein Kernels for Probability Distributions, arXiv:1511.03198. CVPR 2016. link , software
Applications
- Zhuang et al, Illumination invariant face recognition using optimal transport Preprint , code , PyTransKit .
- Pre-symptomatic early detection of osteoarthritis from knee MRIs. In press, PNAS, 2020. paper
- Rubaiyat, Hallam, Nichols, Hutchinson, Li, Rohde, Parametric signal estimation using the cumulative distribution transform, IEEE Transactions on Signal Processing 2020, paper
- JM Nichols, MN Hutchinson, N Menkart, GA Cranch, GK Rohde, Time Delay Estimation Via Wasserstein Distance Minimization, IEEE Signal Processing Letters, 2019, paper
- JM Nichols, TH Emerson, L Cattell, S Park, A Kanaev, F Bucholtz, A Watnik, T Doster, and GK Rohde, Transport-based model for turbulence-corrupted imagery, Applied Optics, 57(16) pp. 4524-4536, 2018. paper
- SR Park, L Cattell, JM Nichols, A Watnik, T Doster, GK Rohde, De-multiplexing vortex modes in optical communications using transport-based pattern recognition. Optics express 26(4), pp 4004-4022, 2018. paper
- S Kundu, S Kolouri, KI Erickson, AF Kramer, E McAuley, GK Rohde, Discovery and visualization of structural biomarkers from MRI using transport-based morphometry, Neuroimage 2018, paper
- L Cattell, CH Meyer, FH Epstein, GK Rohde, Reconstructing High-Resolution Cardiac MR Movies from Under-Sampled Frames, Asilomar conference on signals, systems, and computers, 2017. paper
- S Kolouri, GK Rohde, Transport-based single frame super resolution of very low resolution face images. IEEE CVPR 2015, pp 4876-4884. pdf
- AB Tosun, A Yergiev, S Kolouri, J Silverman, GK Rohde, Detection of malignant mesothelioma using nuclear structure of mesothelial cells in effusion cytology specimens, Cytometry A, 87(4), 326-333, 2015. pdf
- S Kolouri, S Basu, GK Rohde, Learning and visualizing statistical relationships betwen protein distributions from microscopy images. IEEE ISBI 2014, pp 381-384. paper
- JA Ozolek, AB Tosun, W Wang, C Chen, S Kolouri, S Basu, H Huang, GK Rohde. Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning. Medical Image Analysis. 18(5), 772-780, 2014. link
- S Basu, S Kolouri, GK Rohde, Detecting and visualizing
cell phenotype differences from microscopy images using transport-based
morphometry. PNAS 111 (9), 3448-3453, 2014. pdf , publisher , software
- W Wang, JA Ozolek, D Slepcev, AB Lee, C Chen, GK Rohde, An optimal transportation approach for nuclear
structure-based pathology, IEEE Transactions on Medical Imaging, 30,
pp. 621-631, 2011. (pdf)
Presentations & Classes & Tutorials
- Youtube lecture series. Transport Transforms for Signal Processing and Machine Learning: videos
- Class Transport methods in signal processing and machine learning: Site.
- Mini course, Fields Institute, Toronto, Cannada, 2022.
- Introduction to optimal transport: slides , video
- Transport transforms (embeddings) for signal and image analysis: slides , video
- Applications: slides, video
- Plenary: Transport transforms for signal processing and machine learning. slides , video
- MICCAI, Granada, Spain, 2018.
- IEEE International Symposium on Biomedical Imaging (ISBI), Washington, DC, 2018.
- IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016.
Codes
- PyTransKit, Transport and other Lagrangian transforms python toolkit code
- R-CDT subspaces for image classification: python code
-
CDT iPython notebook example: iPynb paper
-
Radon-CDT iPython notebook example:
iPynb paper
-
Tutorial code:
Zip Directory paper
-
DiscreteLOT: Matlab software for computing a particle-based linear optimal transport embedding of images of cells, as well as applying the transport-based morphometry pipeline described in Basu et al, PNAS 2014. To run the software, download and run the script "Main_LOT_Particle.m".
PDF Software
-
1D CDT: Python software for computing the 1D cummulative distribution transform (CDT), as described in Park et al arXiv:1507.05936, 2015. To run the software, download and run the iPythonNoteBook script provided. If desired, a Matlab software for 1D CDT is also provided.
paper Python MATLAB
-
2D Continuous LOT: Matlab software for computing the 2D continuous linear optimal transport (LOT) transform as described in Kolouri et al, Pattern Recognition 2015 (in press). To run the software, download and run the script "main.m".
Preprint Software
-
2D Radon-CDT: Matlab software for computing the 2D Radon cumulative distribution transform as described in Kolouri et al, IEEE TIP 2016. To run the software, download and run the script "main.m".
Preprint Software