Searching a database of fluorescence microscopy images by typing keywords works well enough when annotations are complete and consistent. In practice, they rarely are. Images get uploaded with minimal metadata, terminology varies across labs, and no text description fully captures what a subcellular pattern actually looks like.
The real question is: given this image, which other images in the database look like it?
That’s the problem we set out to solve.
What We Built
OMERO.searcher is an open-source plugin for OMERO, the widely used platform for managing microscopy image data. It adds content-based image retrieval to any OMERO database — no changes to the database schema required.
Rather than matching text, the system computes numerical features from each image — by default, subcellular location features (SLFs) developed in the Murphy Lab — and ranks images by visual similarity to a query. Users can refine results iteratively: marking examples as positive (retrieve more like this) or negative (exclude these) to steer the search.
Similarity ranking is handled by a modified FALCON algorithm. Features are stored as HDF5 files and compiled into a master index, which keeps search fast even over large collections.
How Well Does It Work?
We evaluated OMERO.searcher on two distinct fluorescence microscopy databases with different cell types and imaging conditions. Retrieval accuracy, measured by AUC, averaged 0.77 across all patterns in one database and reached 0.976 on the second when using balanced positive and negative examples.
For a tool designed to work with any OMERO instance and any numerical feature set — not just the ones we designed — those numbers are encouraging.