Back to Projects

Scientific ML

Computer Vision for Nanoparticle Detection in Cellular Environments

An iSCAT image recognition project focused on separating weak nanoparticle signals from noisy live-cell backgrounds through physics-guided preprocessing, representation engineering, and compact CNN modeling.

Overview

This project grew out of a multi-camera iSCAT problem involving weak interferometric signals, background drift, inconsistency across cameras, and large image datasets. The practical question was whether a nanoparticle-on-cell signal was real under complex live-cell conditions, where direct inspection was slow and unreliable.

What made the project compelling was that the modeling problem could not be separated from the physics of the measurement. Before deep learning could be useful, I needed to build an end-to-end pipeline for structured data engineering, background normalization, signal calibration, and spectral representation so that the model would learn particle scattering properties rather than camera artifacts.

Pipeline Overview

A condensed view of the training logic, from raw microscopy data to the final model and project takeaway.

Phase 1

Data Engineering, Signal Calibration, and Representation Design

1

Raw Data Engineering

  • Traverse multi-camera raw video folders
  • Extract frames and compute per-camera averages
  • Write structured outputs for experiment, camera, background, and sample
2

Physics-Guided Signal Calibration

  • Build background from multiple samples
  • Normalize contrast to reduce drift and bias
  • Add signal conditioning and numerical safeguards
3

Representation and ROI Modeling

  • Correct multi-camera alignment, flip, and drift issues
  • Use ratio-style spectral representations for stability
  • Model ROI context for cell, nanoparticle, and full image
4

Deep Learning Dataset Construction

  • Prepare weak-signal live-cell examples from 10,000+ image datasets
  • Use adjacent-frame difference preprocessing
  • Build stacked 3 x 3 x 256 inputs with negative controls
↓

Phase 2

Model Iteration and Diagnostic Comparison

5

Baseline CNN Iteration Loop

  • Start with a simple CNN and compare 3 to 5 convolution layers
  • Add batch norm, dropout, and cross-dataset validation
  • Introduce focal loss for weak signals and class imbalance
  • Keep the 4-layer CNN as the final domain-specific baseline
6

Transfer Learning Check

  • Run ImageNet-pretrained ResNet18 with partial unfreezing
  • Treat transfer learning as a capacity test, not the final choice
  • Ask whether model depth is really the bottleneck
↓
7

Controlled Comparison and Diagnosis

  • In-distribution performance was similar across the small CNN and ResNet18
  • Cross-size and cross-material shifts degraded both models
  • Signal calibration and domain alignment mattered more than model depth
↓
8

Final Training Logic and Project Takeaway

  • Use physics-guided calibration and representation engineering before model training.
  • Train the compact domain-specific CNN as the final baseline for nanoparticle-versus-background discrimination.
  • Treat transfer learning as a diagnostic comparison rather than the core solution.
  • Future directions point toward tracking, temporal state inference, and distribution-level learning.

What I Worked On

  • Built the preprocessing and data engineering pipeline for multi-camera iSCAT datasets, including frame extraction, camera-wise averaging, and structured outputs.
  • Implemented background modeling, contrast normalization, drift correction, and signal conditioning before any deep learning step.
  • Constructed live-cell training data using adjacent-frame difference preprocessing and stacked multi-channel inputs.
  • Iterated from a simple CNN to a regularized 4-layer baseline with batch norm, dropout, focal loss, and cross-dataset validation.
  • Benchmarked that domain-specific CNN against ImageNet-pretrained ResNet18 to test whether model capacity was the primary bottleneck.

Why It Matters

This project changed how I think about scientific machine learning. The central result was not simply that a model could classify images, but that performance depended more on signal calibration and domain alignment than on choosing a larger architecture. That lesson now shapes how I approach AI for science: understand the measurement first, engineer the representation carefully, and only then decide what kind of model is actually justified.