Table of Contents
The field of connectomics aims to reconstruct the wiring diagram of the brain by mapping the neural connections at the level of individual synapses. Recent advances in electronic microscopy (EM) have enabled the collection of a large number of image stacks at nanometer resolution, but the annotation requires expertise and is super time-consuming. Here we provide a deep learning framework powered by PyTorch for automatic and semi-automatic image segmentation in connectomics. This repository is actively under development by Visual Computing Group (VCG) at Harvard University.
Besides installation guidance and package references, we provide several tutorials covering both semantic and instance segmentation for neurons, and other biological structures like synapses and mitochondria.
For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation.
Zudi Lin, Donglai Wei, Won-Dong Jang, Siyan Zhou, Xupeng Chen, Xueying Wang, Richard Schalek, Daniel Berger, Brian Matejek, Lee Kamentsky, Adi Suissa-Peleg, Daniel Haehn, Thouis Jones, Toufiq Parag, Jeff Lichtman and Hanspeter Pfister. “Two-Stream Active Query Suggestion for Active Learning in Connectomics.” European Conference on Computer Vision (ECCV), 2020 [Paper][Supp.]
- Mitocondria Segmentation on the Lucchi Dataset [Link]
- Synaptic Cleft Detection on the CREMI Dataset [Link]
- Synaptic Polarity Detection on the EM-R50 Dataset [Link]
- Two-Stream Query Suggestion Algorithm [Link]
This project has been partially supported by NSF award IIS-1835231 and NIH award 5U54CA225088-03.
MitoEM Challenge: Large-scale 3D Mitochondria Instance Segmentation
The task is the 3D mitochondria instance segmentation on two 30x30x30 μm^3 datasets, 1000x4096x4096 in voxels at (30, 8, 8) nanometer (nm) resolution. The electron microscopy (EM) image volumes are acquired from a rat (MitoEM-R) and a human (MitoEM-H) tissue, respectively. The mitochondria can display a complex morphology, e.g., mitochondria-on-a-string (MOAS) instances that are connected by thin microtubules, and multiple instances can entangle with each other. Our MitoEM challenge is held at IEEE ISBI 2021.