Ilastik: Interactive Machine Learning For (bio)image Analysis

2 minute read

<span class="timestamp-wrapper"><span class="timestamp">[2019-11-21 Thu] </span></span> Paper/software: ilastik
  • paper: https://www.nature.com/articles/s41592-019-0582-9
  • software: https://www.ilastik.org (written in Python)
  • Updated from version 1.3.2 to 1.3.3
  • Unlike DL and NNs, ilastik does not benefit from large amount of densely annotated data – better to label iteratively
  • DL may come to ilastik later, but requires dense, not sparse, training data
  • The parts of the image analysis pipeline taking place ex-ilastik is facilitated via plugins for Fiji, KNIME, CellProfiler, but can also be done in Python etc
  • Other ML tools mentioned for comparison: Fiji Weka, CellCognition, SuRVoS, FastER, Microscopy Image Browser, Cytomine
  • Workflows built into ilastik
    1. Pixel classification (semantic segmentation)
      • Instance segmentation is performed outside ilastik, e.g. in Fiji
      • In other words, a connected component analaysis has to be done after the pixel classification, e.g. with watershed or Voronoi tesselation
      • Autocontext: performs pixel classification twice – first on the raw data, then on the raw data with the preditions concatenated as additional channels. More computation but smoother and more stable results
    2. Object classification
      • Probability maps from pixel classification are smoothed and thresholded
      • Segmentations from ex-ilastik can be introduced at this stage
      • For each segmented object, features are calculated – intensities in/around objects, convex hull and other shape descriptors
      • A Python template is available for implementing additional features
      • Per-object descriptors can be exported
      • Training of the classifier is a matter of clicking on objects from different classes
    3. Carving
      • Needs object of interest to have stained boundary (if the object itself is stained differently from the surroundings, normal pixel classification works)
      • First, edge detection is performed (Canny or similar, presumably)
      • Then, watersheds are calculated on the edge-map, forming supervoxels around all local minima (Q: what is the tolerance, how is noise suppressed to avoid too many superpixels?)
      • The user input is simple brush strokes, indicating each object and the background
    4. Multicut (Boundary-based segmentation with multicut)
      • Similar to carving, in being boundary based, but not all object have to be indicated
      • The user clicks on edges in the oversegmented superpixel image, and thus indicate if they are true or false
      • Developed for segmentation of neurons in EM images
      • I suppose this can be used on images without edges by first running a Laplacian - but again, simple pixel classification is likely faster
    5. Counting
      • User indicates background with strokes and objects with clicks
      • Objects can overlap
      • Reggression Random Forest then estimates density in rest of image
      • Count estimates are returned in rectangular regions and entire image
      • This only works in 2D
    6. Tracking
      • Works in 2D and 3D over time
      • User trains classifier for dividing objects (click to indicate dividing and non-dividing) and another for false detections and mergers (click to indicate false merges and correct non-merges)
      • Q: is merging allowed? Yes, it seems so, from the description on the website
      • A probabilistic graphical model (PGM) is constructed with default, but adjustable, weights
      • The PGM estimates how likely an object is to be dividing, a false detection, a merge, and how well is matches neigbors in subsequent frames
      • PGM parameters can be learned if tracklet are annotated, i.e. manually tracking over a short period of time ("tracking with learning")
      • Tracks can be exported and postprocessed e.g. in MaMuT