DeepLabCut: fast, robust, 3D markerless pose estimation of animals

Mathis et al, Nature Neuroscience 2018
Left: a mouse tracking an odor trail with the snout in the future and past tracked with DeepLabCut (credit: Alexander Mathis in collaboration with the Murthy Lab). Middle: a mouse hand tracked with DeepLabCut (credit: Mackenzie Mathis in collaboration with the Mathis Lab). Right: a fly moving in a 3D chamber with DeepLabCut applied labels (credit: Kevin Cury from the Axel Lab).
Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. Here we present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy.

Read about the latest developments, get the open source code, and find more information here:

Key Papers

A. Mathis, P. Mamidanna, K. Cury, T. Abe, V. Murthy, M. Mathis, and M. Bethge
DeepLabCut: markerless pose estimation of user-defined body parts with deep learning.
Nature Neuroscience, 21(9), 1281-1289, 2018
Code, URL, DOI, BibTex

A. Mathis, T. Nath, A. Chen, A. Patel, M. Bethge, and M. Mathis
Using DeepLabCut for 3D markerless pose estimation across species and behaviors
bioRxiv, 2018
Code, URL, BibTex

University of Tuebingen BCCN CIN MPI