Automatic detection of clustered, fluorescent‐stained nuclei by digital image‐based cytometry

Automatic image‐based cytometry (IC) can conveniently quantify the distributions of several specific, fluorescencelabeled molecules within individual, isolated cells of slide‐ or tissue‐based specimens. However, many specimens contain clusters of cells or nuclei that are not detected as individual entities by existing automatic methods. We have developed analysis algorithms which detect individul nuclei occurring in clusters or as isolated nuclei. Specimens were labeled with a fluorescent DNA stain, imaged and the images were segmented into regions of nuclei and background. Clusters of nuclei, identified by their size and shape, were divided into individual nuclei by searching for dividing paths between nuclei. The paths, which need not be straight, possessed the highest average gradient per pixel. In addition, both high‐ and low‐pass filtered images of the original image were analyzed. For each individual nucleus, one of the three segmented regions representing the nucleus (from either the original or one of two filtered images) was chosen as the final result, based on the closeness of the regions to average nuclear morphology. The algorithms correctly detected a high proportion of isolated (328/333) and clustered (254/271) nuclei when applied to images of 2 μm prostate and breast cancer sections. Thus, these algorithms should enable much more accurate detection and analyses of nuclei in intact specimens. © 1994 Wiley‐Liss, Inc.