Image Segmentation
Transformers
Safetensors
English
biology
CV
images
animals
beetles
mask2former
entomology
Instructions to use imageomics/BeetleFlow with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use imageomics/BeetleFlow with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-segmentation", model="imageomics/BeetleFlow")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("imageomics/BeetleFlow", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Formatting fixes, NEON acknowledgment, and link additions
Browse files
README.md
CHANGED
|
@@ -3,6 +3,8 @@ license: mit
|
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
library_name: transformers
|
|
|
|
|
|
|
| 6 |
tags:
|
| 7 |
- biology
|
| 8 |
- CV
|
|
@@ -12,7 +14,8 @@ tags:
|
|
| 12 |
- image-segmentation
|
| 13 |
- mask2former
|
| 14 |
- entomology
|
| 15 |
-
datasets:
|
|
|
|
| 16 |
metrics:
|
| 17 |
- mean_iou
|
| 18 |
model_description: >-
|
|
@@ -48,8 +51,8 @@ Mask2Former with a Swin-Large backbone, fine-tuned for pixel-level semantic segm
|
|
| 48 |
|
| 49 |
### Model Sources
|
| 50 |
|
| 51 |
-
- **Repository:** [
|
| 52 |
-
- **Paper:** BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing — *NeurIPS 2025 Workshop for Imageomics: Discovering Biological Knowledge from Images Using AI*
|
| 53 |
- **Demo:** See [batch inference usage](#how-to-get-started-with-the-model) below
|
| 54 |
|
| 55 |
## Uses
|
|
@@ -74,7 +77,7 @@ Mask2Former with a Swin-Large backbone, fine-tuned for pixel-level semantic segm
|
|
| 74 |
|
| 75 |
## Bias, Risks, and Limitations
|
| 76 |
|
| 77 |
-
- Models are trained on pinned ground-beetle specimens from NEON pitfall-trap imagery; performance may degrade on other beetle families, imaging setups, or specimen preparations.
|
| 78 |
- The 5-class and 9-class label schemes differ in granularity; choose the checkpoint that matches your annotation protocol.
|
| 79 |
- Small or occluded body parts (e.g., legs, antennas) may be harder to segment accurately.
|
| 80 |
- Segmentation quality depends on upstream cropping; poorly cropped inputs will reduce mask quality.
|
|
@@ -147,7 +150,7 @@ Expect `input/test_images/` and `input/test_masks/` with paired RGB mask images.
|
|
| 147 |
|
| 148 |
### Training Data
|
| 149 |
|
| 150 |
-
Models are trained on manually annotated beetle part segmentation data from pinned specimen images in the [Beetles as Sentinel Taxa dataset](https://huggingface.co/datasets/imageomics/sentinel-beetles).
|
| 151 |
|
| 152 |
**5-class**
|
| 153 |
- Parts: head, pronotum, elytra, legs, antennas (+ background)
|
|
@@ -159,7 +162,7 @@ Models are trained on manually annotated beetle part segmentation data from pinn
|
|
| 159 |
- 330 labeled beetles with RGB masks
|
| 160 |
- Split: 264 training / 66 test images
|
| 161 |
|
| 162 |
-
Training uses `train_images/` and `valid_images/` (with corresponding `train_masks/` and `valid_masks/`) under the data root
|
| 163 |
|
| 164 |
### Training Procedure
|
| 165 |
|
|
@@ -169,7 +172,7 @@ Training uses `train_images/` and `valid_images/` (with corresponding `train_mas
|
|
| 169 |
- **Training augmentations** (Albumentations): horizontal flip (p=0.5), random brightness/contrast (p=0.25), rotation (±25°).
|
| 170 |
- **Validation augmentations:** resize and normalize only.
|
| 171 |
- **Normalization:** ADE20K mean `[123.675, 116.280, 103.530]` and std `[58.395, 57.120, 57.375]` (scaled to [0, 1]).
|
| 172 |
-
- RGB masks are converted to integer class labels using fixed color palettes defined in `config.py`.
|
| 173 |
|
| 174 |
#### Training Hyperparameters
|
| 175 |
|
|
@@ -188,7 +191,7 @@ Training was performed on a single NVIDIA A100 GPU (40 GB) with 8 DataLoader wor
|
|
| 188 |
|
| 189 |
## Evaluation
|
| 190 |
|
| 191 |
-
Evaluation follows `pipeline/3_segmentation/test.py`: predictions are compared against held-out RGB masks at 512×512 resolution using the Hugging Face `evaluate` library.
|
| 192 |
|
| 193 |
### Testing Data, Factors & Metrics
|
| 194 |
|
|
@@ -209,7 +212,7 @@ Held-out test images and RGB masks (`test_images/`, `test_masks/`) from the Beet
|
|
| 209 |
|
| 210 |
### Results
|
| 211 |
|
| 212 |
-
Evaluated on the held-out test split using `test.py` at 512×512 resolution.
|
| 213 |
|
| 214 |
#### 5-class
|
| 215 |
|
|
@@ -257,7 +260,7 @@ Not applicable for this release. Validation segmentation overlays are saved duri
|
|
| 257 |
|
| 258 |
## Environmental Impact
|
| 259 |
|
| 260 |
-
Carbon emissions
|
| 261 |
|
| 262 |
- **Hardware Type:** a single NVIDIA A100 GPU (40 GB)
|
| 263 |
- **Hours used:** ~0.5 GPU-hours per model
|
|
@@ -333,9 +336,13 @@ If you use these models, please cite BeetleFlow and the underlying dataset.
|
|
| 333 |
|
| 334 |
## Acknowledgements
|
| 335 |
|
| 336 |
-
This
|
| 337 |
|
| 338 |
-
This work was
|
|
|
|
|
|
|
|
|
|
|
|
|
| 339 |
|
| 340 |
## Glossary
|
| 341 |
|
|
|
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
library_name: transformers
|
| 6 |
+
base_model:
|
| 7 |
+
- facebook/mask2former-swin-large-ade-semantic
|
| 8 |
tags:
|
| 9 |
- biology
|
| 10 |
- CV
|
|
|
|
| 14 |
- image-segmentation
|
| 15 |
- mask2former
|
| 16 |
- entomology
|
| 17 |
+
datasets:
|
| 18 |
+
- imageomics/sentinel-beetles
|
| 19 |
metrics:
|
| 20 |
- mean_iou
|
| 21 |
model_description: >-
|
|
|
|
| 51 |
|
| 52 |
### Model Sources
|
| 53 |
|
| 54 |
+
- **Repository:** [Imageomics/BeetleFlow](https://github.com/Imageomics/BeetleFlow)
|
| 55 |
+
- **Paper:** [BeetleFlow: An Integrative Deep Learning Pipeline for Beetle Image Processing](https://doi.org/10.48550/arXiv.2511.00255) — *NeurIPS 2025 Workshop for Imageomics: Discovering Biological Knowledge from Images Using AI*
|
| 56 |
- **Demo:** See [batch inference usage](#how-to-get-started-with-the-model) below
|
| 57 |
|
| 58 |
## Uses
|
|
|
|
| 77 |
|
| 78 |
## Bias, Risks, and Limitations
|
| 79 |
|
| 80 |
+
- Models are trained on pinned ground-beetle specimens from [NEON](https://www.neonscience.org/) pitfall-trap imagery; performance may degrade on other beetle families, imaging setups, or specimen preparations.
|
| 81 |
- The 5-class and 9-class label schemes differ in granularity; choose the checkpoint that matches your annotation protocol.
|
| 82 |
- Small or occluded body parts (e.g., legs, antennas) may be harder to segment accurately.
|
| 83 |
- Segmentation quality depends on upstream cropping; poorly cropped inputs will reduce mask quality.
|
|
|
|
| 150 |
|
| 151 |
### Training Data
|
| 152 |
|
| 153 |
+
Models are trained on manually annotated beetle part segmentation data from pinned specimen images in the [Beetles as Sentinel Taxa dataset](https://huggingface.co/datasets/imageomics/sentinel-beetles). Details of the subset used for training these models:
|
| 154 |
|
| 155 |
**5-class**
|
| 156 |
- Parts: head, pronotum, elytra, legs, antennas (+ background)
|
|
|
|
| 162 |
- 330 labeled beetles with RGB masks
|
| 163 |
- Split: 264 training / 66 test images
|
| 164 |
|
| 165 |
+
Training uses `train_images/` and `valid_images/` (with corresponding `train_masks/` and `valid_masks/`) under the [BeetleFlow GitHub repo](https://github.com/Imageomics/BeetleFlow) data root; this repo includes mask color conventions in the [data README](https://github.com/Imageomics/BeetleFlow/tree/main/data).
|
| 166 |
|
| 167 |
### Training Procedure
|
| 168 |
|
|
|
|
| 172 |
- **Training augmentations** (Albumentations): horizontal flip (p=0.5), random brightness/contrast (p=0.25), rotation (±25°).
|
| 173 |
- **Validation augmentations:** resize and normalize only.
|
| 174 |
- **Normalization:** ADE20K mean `[123.675, 116.280, 103.530]` and std `[58.395, 57.120, 57.375]` (scaled to [0, 1]).
|
| 175 |
+
- RGB masks are converted to integer class labels using fixed color palettes defined in [`config.py`](https://github.com/Imageomics/BeetleFlow/blob/main/pipeline/3_segmentation/config.py).
|
| 176 |
|
| 177 |
#### Training Hyperparameters
|
| 178 |
|
|
|
|
| 191 |
|
| 192 |
## Evaluation
|
| 193 |
|
| 194 |
+
Evaluation follows [`pipeline/3_segmentation/test.py`](https://github.com/Imageomics/BeetleFlow/blob/main/pipeline/3_segmentation/test.py): predictions are compared against held-out RGB masks at 512×512 resolution using the Hugging Face `evaluate` library.
|
| 195 |
|
| 196 |
### Testing Data, Factors & Metrics
|
| 197 |
|
|
|
|
| 212 |
|
| 213 |
### Results
|
| 214 |
|
| 215 |
+
Evaluated on the held-out test split using [`test.py`](https://github.com/Imageomics/BeetleFlow/blob/main/pipeline/3_segmentation/test.py) at 512×512 resolution.
|
| 216 |
|
| 217 |
#### 5-class
|
| 218 |
|
|
|
|
| 260 |
|
| 261 |
## Environmental Impact
|
| 262 |
|
| 263 |
+
Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://doi.org/10.48550/arXiv.1910.09700):
|
| 264 |
|
| 265 |
- **Hardware Type:** a single NVIDIA A100 GPU (40 GB)
|
| 266 |
- **Hours used:** ~0.5 GPU-hours per model
|
|
|
|
| 336 |
|
| 337 |
## Acknowledgements
|
| 338 |
|
| 339 |
+
This project was in part conceived at [Funcapalooza](https://github.com/Imageomics/FuncaPalooza-2025).
|
| 340 |
|
| 341 |
+
This work was supported by the [Imageomics Institute](https://imageomics.org), which is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) program under [Award #2118240](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118240) (Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning).
|
| 342 |
+
|
| 343 |
+
This material is based in part upon work supported by the [National Ecological Observatory Network (NEON)](https://www.neonscience.org/), a program sponsored by the U.S. National Science Foundation (NSF) and operated under cooperative agreement by Battelle.
|
| 344 |
+
|
| 345 |
+
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
|
| 346 |
|
| 347 |
## Glossary
|
| 348 |
|