DeepLabCut for Animal Behavior Analysis: A Comprehensive Guide for Biomedical Researchers

Abstract

This article provides a complete resource for researchers and drug development professionals seeking to implement DeepLabCut, a powerful deep learning-based toolkit for markerless pose estimation. We cover the foundational principles of the software, from project setup and installation to its application in both single and multi-animal scenarios. The guide details the complete workflow—including data labeling, network training, and video analysis—and offers practical troubleshooting and optimization strategies to enhance performance. Furthermore, we present evidence validating DeepLabCut's accuracy against traditional tracking systems and commercial solutions, empowering scientists to robustly quantify animal behavior in preclinical research with high precision and reliability.

What is DeepLabCut? Core Principles and Setup for Behavioral Scientists

Defining Markerless Pose Estimation and Its Impact on Behavioral Neuroscience

Markerless pose estimation represents a fundamental shift in behavioral neuroscience, replacing traditional manual scoring and physical marker-based systems with deep learning to track animal body parts directly from video footage. This computer vision approach enables the precise quantification of an animal's posture and movement by detecting user-defined anatomical keypoints (e.g., snout, paws, tail) without any physical markers [1]. Tools like DeepLabCut (DLC) have demonstrated human-level accuracy in tracking fast-moving rodents, typically requiring only 50-200 manually labeled frames for training thanks to transfer learning [1] [2]. This transformation allows researchers to capture subtle micro-behaviors—such as tiny head lifts, brief standing events, or slight changes in stride—that contain critical clues about early pathological signs but are often missed by traditional manual methods [1]. The application of this technology is accelerating our understanding of brain function, neurological disorders, and therapeutic efficacy across diverse species and experimental paradigms.

Core Principles and Workflow of Markerless Pose Estimation

The operational workflow of markerless pose estimation can be broken down into a sequential pipeline that transforms raw video into quantifiable behavioral data. DeepLabCut serves as a prime example of this process, leveraging deep neural networks to achieve robust performance with minimal training data.

The following diagram illustrates the complete workflow from video acquisition to behavioral analysis:

Key Technical Innovations

Several technical breakthroughs have enabled the practical application of markerless pose estimation in neuroscience research:

• Transfer Learning: By initializing networks with weights pre-trained on large-scale human pose estimation datasets (like ImageNet), DeepLabCut achieves high accuracy with minimal training data, dramatically reducing the labeling burden from thousands to hundreds of frames [2] [3].
• Multi-Animal Pose Estimation: Advanced architectures now incorporate Part Affinity Fields (PAFs) and multi-task learning to simultaneously estimate poses, group keypoints into distinct individuals, and track identities across frames—even during occlusions and close interactions [4].
• Foundation Models: The development of pretrained models like SuperAnimal-Quadruped (trained on over 40K images of quadruped animals) and SuperAnimal-TopViewMouse provides researchers with out-of-the-box solutions that can be used without any additional training data [5] [2].

Quantitative Performance of Markerless Pose Estimation Tools

The adoption of markerless pose estimation in behavioral neuroscience is supported by compelling quantitative evidence of its performance across various benchmarks and experimental conditions.

DeepLabCut 3.0 Model Performance Benchmarks

Table 1: Performance comparison of different DeepLabCut 3.0 top-down models on standardized datasets. mAP (mean Average Precision) scores measure pose estimation accuracy, with higher values indicating better performance [5].

Model Name	Type	mAP SA-Q on AP-10K	mAP SA-TVM on DLC-OpenField
topdownresnet_50	Top-Down	54.9	93.5
topdownresnet_101	Top-Down	55.9	94.1
topdownhrnet_w32	Top-Down	52.5	92.4
topdownhrnet_w48	Top-Down	55.3	93.8
rtmpose_s	Top-Down	52.9	92.9
rtmpose_m	Top-Down	55.4	94.8
rtmpose_x	Top-Down	57.6	94.5

Multi-Animal Tracking Performance

Table 2: Performance metrics for multi-animal pose estimation across diverse species and experimental conditions, demonstrating the robustness of modern approaches [4].

Dataset	Animals per Frame	Keypoints Tracked	Test Error (pixels)	Assembly Purity (%)
Tri-Mouse	3	12	2.65	>95%
Parenting Mice	3	15	5.25	>93%
Marmosets	2	14	4.59	>94%
Fish School	14	5	2.72	>92%

Current Adoption and Applications in Rodent Research

A systematic review of rodent pose-estimation studies from 2016-2025 reveals accelerating adoption, with publication frequency more than doubling after 2021 [1]. This analysis of 67 relevant papers shows the distribution of applications:

• Tool-Focused Studies (30 papers): Development or validation of new pose-estimation algorithms and software
• Method-Focused Studies (28 papers): Application of pose-estimation to propose new experimental methods or paradigms
• Study-Focused Papers (9 papers): Use of pose-estimation to address specific biological or disease-related research questions

The technology has been successfully applied to study various disease models, including Parkinson's disease, Alzheimer's disease, and pain models, demonstrating its utility across multiple domains of preclinical research [1].

Experimental Protocols for Behavioral Analysis

Protocol 1: Fear Conditioning and Freezing Behavior Analysis

Purpose: To quantitatively assess learned fear memory in rodents using markerless pose estimation of freezing behavior.

Materials & Methods:

• Animals: Adult mice or rats
• Equipment: Fear conditioning chamber with grid floor shock delivery system, high-speed camera (≥30 fps), computer with GPU
• Software: DeepLabCut for pose estimation, BehaviorDEPOT for freezing detection

Procedure:

• Pose Estimation Model Training:
  • Record a 10-minute baseline video of the animal in the chamber
  • Extract and manually label 100-200 frames across diverse postures using DeepLabCut GUI
  • Train a DeepLabCut network using transfer learning (approximately 4-6 hours on GPU)
  • Validate model performance on held-out video frames (target RMSE <5 pixels)

• Fear Conditioning Protocol:

  • Day 1: Expose animal to conditioning context (3 min)
  • Deliver 3 mild footshocks (0.7 mA, 2 sec duration) with 1-min intervals
  • Day 2: Return animal to same context for 5-min memory test (no shocks)

• Automated Freezing Detection:

  • Process test session video with trained DeepLabCut model
  • Import tracking data into BehaviorDEPOT Analysis Module
  • Apply freezing detection heuristic based on movement velocity threshold
  • Calculate percentage time spent freezing during test session

Validation: BehaviorDEPOT's freezing detection heuristic achieves >90% accuracy compared to human scoring, even in animals wearing tethered head-mounts for neural recording [6].

Protocol 2: Social Behavior Analysis in Group-Housed Mice

Purpose: To quantitatively analyze social interactions and individual behaviors in group-housed rodents.

Materials & Methods:

• Animals: 3-5 group-housed mice
• Equipment: Large home cage, overhead camera with wide-angle lens, infrared lighting for dark cycle recording
• Software: DeepLabCut with multi-animal tracking capabilities

Procedure:

• Multi-Animal Pose Estimation Model:
  • Record 30-minute video of group interactions
  • Label keypoints (nose, ears, paws, tail base) for all animals across 200 frames
  • Train multi-animal DeepLabCut model with animal identity prediction
  • Validate tracking accuracy, particularly during occlusion events

• Social Behavior Analysis:

  • Track animal trajectories and body postures across 24-hour period
  • Calculate inter-animal distances using nose coordinates
  • Define social interaction as inter-animal distance <5 cm with nose orientation toward conspecific
  • Quantify interaction bout duration and frequency

• Individual Behavior Classification:

  • Use unsupervised learning algorithms (B-SOiD, VAME, Keypoint-MoSeq) to identify recurring behavioral motifs
  • Cluster pose sequences to classify behaviors (grooming, rearing, feeding)
  • Analyze temporal sequencing of behaviors across light-dark cycles

Technical Notes: The multi-task architecture in DeepLabCut predicts keypoints, limbs, and animal identity to maintain consistent tracking during occlusions, with assembly purity exceeding 93% in complex multi-animal scenarios [4].

Table 3: Key computational tools and resources for implementing markerless pose estimation in behavioral neuroscience research.

Resource	Type	Primary Function	Key Features
DeepLabCut [5] [2]	Software Toolbox	Markerless pose estimation	GUI and Python API, multi-animal tracking, 3D pose estimation, active learning framework
BehaviorDEPOT [6]	Analysis Software	Behavior classification from pose data	Heuristic-based detection, no coding experience required, excellent freezing detection accuracy
SLEAP [1]	Software Toolbox	Multi-animal pose tracking	Instance-based tracking, high performance in dense populations
SpaceAnimal Dataset [7]	Benchmark Dataset	Algorithm training and validation	Multi-species dataset (C. elegans, Drosophila, zebrafish), microgravity behavior analysis
DeepLabCut Model Zoo [2]	Pretrained Models	Out-of-the-box pose estimation	SuperAnimal models for quadrupeds and top-view mice, minimal training required
B-SOiD, VAME, Keypoint-MoSeq [8]	Unsupervised Learning Algorithms	Behavioral motif discovery	Identify recurring behaviors from pose data without human labeling

Advanced Applications and Integration with Neuroscience Methods

The true impact of markerless pose estimation emerges from its integration with established neuroscience techniques, creating new paradigms for investigating brain-behavior relationships.

Integration with Neural Recording and Manipulation

Modern markerless systems enable precise alignment of behavioral quantification with neural activity data, which is crucial for studying the neural basis of behavior:

• Closed-Loop Experiments: DeepLabCut enables real-time, low-latency pose tracking (up to 1200 FPS inference speed) sufficient for closed-loop feedback in behavioral experiments [9] [3]. This allows researchers to trigger optogenetic manipulations or sensory stimuli based on specific postures or movements.
• Neural Correlation Analysis: BehaviorDEPOT stores behavioral data framewise, facilitating precise alignment with simultaneously recorded neural signals from fiber photometry, miniscope calcium imaging, or electrophysiology [6]. This enables direct correlation of neural dynamics with specific behavioral motifs identified through pose estimation.

Behavioral Analysis in Complex Environments

Recent advances have expanded applications beyond standard laboratory settings to more complex and naturalistic environments:

• Space Research: The SpaceAnimal Dataset provides the first benchmark for analyzing animal behavior in microgravity conditions aboard the China Space Station, tracking multiple species including zebrafish, Drosophila, and C. elegans with specialized keypoint annotations [7].
• Wildlife Research: DeepLabCut has been applied to track cheetahs in the wild, demonstrating robust performance in natural environments with variable lighting, complex backgrounds, and unrestricted animal movement [2] [9].

Technical Architecture and Computational Foundations

The effectiveness of markerless pose estimation rests on sophisticated computational architectures that balance accuracy with efficiency.

Deep Learning Architecture for Multi-Animal Pose Estimation

The technical implementation of advanced pose estimation systems involves multi-task convolutional neural networks that simultaneously address several computational challenges:

This architecture enables:

• Keypoint Detection: Localizing body parts using score maps that encode the probability of keypoint occurrence
• Animal Assembly: Using Part Affinity Fields to group keypoints into distinct individuals based on learned limb connections
• Identity Tracking: Maintaining animal identity across frames using visual re-identification embeddings, particularly crucial after occlusions

Computational Foundations and Performance Optimization

The computational efficiency required for practical neuroscience research relies on several key innovations:

• Vectorized Operations: DeepLabCut leverages NumPy's vectorization capabilities for rapid array manipulation during data augmentation, target scoremap calculation, and keypoint assembly [3].
• Model Optimization: Different backbone architectures (ResNet, HRNet, EfficientNet) provide varying trade-offs between speed and accuracy, allowing researchers to select models appropriate for their specific requirements [5] [4].
• Active Learning: Integrated active learning frameworks identify low-confidence predictions and prioritize these frames for human annotation, continuously improving model performance with minimal additional labeling effort [3].

Markerless pose estimation has fundamentally transformed behavioral neuroscience by enabling precise, automated, and high-throughput quantification of animal behavior. The integration of tools like DeepLabCut with behavioral classification systems like BehaviorDEPOT provides researchers with complete pipelines from raw video to quantitative behavioral analysis. Despite significant advances, challenges remain in standardization, computational resource requirements, and integration across diverse experimental paradigms [1].

Future developments will likely focus on increasing accessibility through more powerful pretrained foundation models, improving real-time performance for closed-loop experiments, and enhancing multi-animal tracking in complex social contexts. As these tools continue to evolve, they will further accelerate our understanding of the neural mechanisms underlying behavior and their disruption in disease states.

DeepLabCut is an open-source toolbox for markerless pose estimation of user-defined body parts in animals using deep learning. Its ability to achieve human-level accuracy with minimal training data (typically 50-200 frames) has revolutionized behavioral quantification across neuroscience, veterinary medicine, and drug development [2] [10]. The platform is animal and object agnostic, meaning that as long as a researcher can visually identify a feature to track, DeepLabCut can be trained to quantify it [5]. This capability is particularly valuable in pharmaceutical research where high-throughput, precise behavioral phenotyping is essential for evaluating therapeutic efficacy and safety in animal models.

Recent advancements have introduced SuperAnimal models [11], which are foundation models pre-trained on vast datasets encompassing over 45 species. These models enable "zero-shot" inference on new animals and experimental setups without requiring additional labeled data, dramatically reducing the barrier to entry and accelerating research timelines. For drug development professionals, this means robust behavioral tracking can be implemented rapidly across diverse testing paradigms, from open-field tests to social interaction assays [12].

Core Workflow: From Raw Video to Pose Data

The standard DeepLabCut pipeline transforms raw video footage into quantitative pose data through a structured, iterative process. This workflow applies to both single-animal projects (sDLC) and multi-animal projects (maDLC), with the latter incorporating additional steps for animal identification and tracking [13].

Workflow Visualization

The following diagram illustrates the complete DeepLabCut workflow, integrating both single-animal and multi-animal pathways:

Project Creation and Configuration

The workflow begins with project creation using the create_new_project function, which generates the necessary directory structure and configuration file [14]. The key decision point at this stage is determining whether the project requires single-animal or multi-animal tracking, as this affects subsequent labeling and analysis steps.

Critical Configuration Parameters (config.yaml):

• bodyparts: List of user-defined body parts to track (e.g., nose, ears, tailbase) [14]
• individuals: For multi-animal projects, names of distinct animals [13]
• colormap: matplotlib colormap for visualization consistency [15]
• video_sets: Paths to source videos for analysis [14]

For multi-animal scenarios where animals share similar appearance, researchers should use the multi-animal mode (maDLC) introduced in DeepLabCut 2.2, which employs a combination of pose estimation and tracking algorithms to distinguish individuals [13].

Frame Selection and Labeling

A critical success factor is curating a training dataset that captures the behavioral diversity expected in experimental conditions [14]. The extract_frames function selects representative frames across videos, ensuring coverage of varying postures, lighting conditions, and backgrounds. For most applications, 100-200 carefully selected frames provide sufficient training data [14] [2].

Labeling involves manually annotating each body part in the extracted frames using DeepLabCut's graphical interface [16]. The platform provides keyboard shortcuts (U, I, O, E, Q) to accelerate this process [16]. For multi-animal projects, each individual must be identified and labeled separately in each frame [13].

Model Training and Evaluation

DeepLabCut supports both TensorFlow and PyTorch backends, with PyTorch becoming the recommended option in version 3.0+ [5] [13]. Training leverages transfer learning from pre-trained networks, with the option to use foundation models like SuperAnimal for enhanced performance [11].

Performance Evaluation Metrics:

• Train Error: Loss on training dataset indicating learning progress
• Test Error: Loss on held-out frames measuring generalization
• Mean Average Precision (mAP): Key metric for pose estimation quality [5]

After training, the model should be evaluated on a separate video to assess real-world performance before proceeding to full analysis [14].

Video Analysis and Pose Estimation

Once a satisfactory model is obtained, researchers can analyze new videos using the analyze_videos function. This generates pose estimation data containing coordinates and confidence scores for each body part across all video frames [14].

For multi-animal projects, an additional step involves assembling body parts into distinct individuals and tracking them across frames using algorithms that combine local tracking with global reasoning [13]. The resulting data can be exported to various formats for downstream analysis.

Model Refinement (Active Learning)

DeepLabCut incorporates an active learning framework where the model identifies frames where it has low confidence, allowing researchers to label these "outlier" frames and retrain the network [5]. This iterative refinement process significantly improves model robustness with minimal additional labeling effort.

Performance Benchmarks and Model Selection

DeepLabCut 3.0 Pose Estimation Performance

The table below summarizes the performance of different model architectures available in DeepLabCut 3.0, measured by mean Average Precision (mAP) on benchmark datasets [5]:

Table 1: DLC 3.0 Pose Estimation Performance (Top-Down Models)

Model Name	Type	mAP SA-Q on AP-10K	mAP SA-TVM on DLC-OpenField
topdownresnet_50	Top-Down	54.9	93.5
topdownresnet_101	Top-Down	55.9	94.1
topdownhrnet_w32	Top-Down	52.5	92.4
topdownhrnet_w48	Top-Down	55.3	93.8
rtmpose_s	Top-Down	52.9	92.9
rtmpose_m	Top-Down	55.4	94.8
rtmpose_x	Top-Down	57.6	94.5

These benchmarks demonstrate that top-down approaches generally provide excellent performance, with RTMPose-X achieving the highest scores on both quadruped (SA-Q) and top-view mouse (SA-TVM) datasets [5].

SuperAnimal Foundation Models

The introduction of SuperAnimal models represents a significant advancement, providing pre-trained weights that can be used for zero-shot inference or fine-tuned with minimal data [11]. The table below compares their performance characteristics:

Table 2: SuperAnimal Model Performance Characteristics

Model	Training Data	Keypoints	Applications	Data Efficiency
SuperAnimal-Quadruped	~80K images, 40+ species	39	Diverse quadruped tracking	10-100× more efficient
SuperAnimal-TopViewMouse	~5K images, diverse lab settings	26	Overhead mouse behavior	Excellent zero-shot performance

These foundation models show particular strength in out-of-distribution (OOD) scenarios, maintaining robust performance on animals and environments not seen during training [11]. For drug development applications where standardized behavioral assays are common, SuperAnimal-TopViewMouse often provides excellent results without custom training.

Table 3: DeepLabCut Research Reagent Solutions

Resource	Type	Function	Application Context
SuperAnimal-Quadruped	Pre-trained Model	Zero-shot pose estimation for quadrupeds	Tracking diverse species without training data
SuperAnimal-TopViewMouse	Pre-trained Model	Zero-shot pose estimation for overhead mouse views	Open-field, home cage monitoring
DeepLabCut-Live	Real-time Module	<1ms latency pose estimation [17]	Closed-loop optogenetics, real-time feedback
DeepOF	Analysis Package	Supervised/unsupervised behavioral classification [12]	Detailed behavioral phenotyping (e.g., social stress)
Docker Environments	Deployment	Reproducible, containerized analysis	Cross-platform compatibility, cloud deployment
Google Colaboratory	Cloud Platform	Accessible computation without local GPU	Resource-constrained environments, education

These resources collectively enable researchers to implement complete behavioral analysis pipelines, from data acquisition to quantitative interpretation. The DeepOF package, for instance, has been used to identify distinct stress-induced social behavioral patterns in mice following chronic social defeat stress [12], demonstrating its utility in psychiatric drug development.

Advanced Applications in Research

Behavioral Analysis in Drug Development

DeepLabCut enables precise quantification of behavioral phenotypes relevant to drug efficacy studies. In one application, researchers used DeepOF to analyze social interaction tests following chronic social defeat stress, identifying distinct stress-induced social behavioral patterns that faded with habituation [12]. This level of granular behavioral resolution surpasses traditional manual scoring methods in sensitivity and objectivity.

The platform's ability to track user-defined features makes it particularly valuable for measuring specific drug-induced movement abnormalities or therapeutic improvements. For example, it can quantify gait parameters in neurodegenerative models or measure subtle tremor reductions following pharmacological interventions.

Multi-Animal Social Behavior Analysis

The multi-animal pipeline (maDLC) enables comprehensive analysis of social behaviors by tracking multiple animals simultaneously and identifying their interactions [13]. This capability is crucial for studying social behaviors in contexts such as:

• Social approach and avoidance in anxiety and depression models
• Aggressive behaviors in territoriality studies
• Maternal-offspring interactions in developmental research

The tracking process involves first estimating poses for all detectable body parts, then assembling these into individual animals, and finally linking identities across frames to create continuous trajectories [13].

Real-Time Applications

DeepLabCut-Live provides real-time pose estimation with latency under 1ms, enabling closed-loop experimental paradigms [17]. This capability allows researchers to:

• Deliver stimuli based on specific behavioral states
• Trigger interventions when animals exhibit target behaviors
• Implement neurofeedback protocols based on posture or movement

These real-time applications are particularly valuable for circuit neuroscience and behavioral pharmacology studies where precise timing between neural activity, behavior, and intervention is critical.

DeepLabCut represents a transformative toolset for quantitative behavioral analysis in animal research. Its comprehensive workflow—from project configuration through model training to final analysis—provides researchers with an end-to-end solution for markerless pose estimation. The recent introduction of SuperAnimal foundation models and specialized analysis packages like DeepOF further enhances its utility for drug development professionals seeking robust, efficient behavioral phenotyping.

The platform's flexibility across species, behaviors, and experimental contexts makes it particularly valuable for preclinical studies where standardized, objective behavioral measures are essential for evaluating therapeutic potential. As these tools continue to evolve, they promise to deepen our understanding of behavior and accelerate the development of novel therapeutics for neurological and psychiatric disorders.

DeepLabCut is an efficient, open-source toolbox for markerless pose estimation of user-defined body parts in animals and humans. It uses transfer learning with deep neural networks to achieve human-level labeling accuracy with minimal training data (typically 50-200 frames). This guide provides a comprehensive framework for installing DeepLabCut by addressing the critical decision of computational hardware selection and dependency management, enabling researchers to implement this powerful tool for behavioral analysis in neuroscience and drug development contexts.

The choice between GPU and CPU installation significantly impacts model training times, inference speed, and overall workflow efficiency in behavioral research pipelines. Proper configuration ensures reproducibility and scalability for analyzing complex behavioral datasets.

Performance Comparison: GPU vs. CPU

Quantitative Performance Metrics

DeepLabCut's performance varies substantially between GPU and CPU configurations. The following table summarizes key performance comparisons based on empirical data:

Table 1: Performance comparison between GPU and CPU configurations

Metric	GPU Performance	CPU Performance	Performance Ratio
Training Speed	Significantly faster (hours)	Slower (potentially days)	~100x faster [18]
Inference Speed	Real-time capable	Slower processing	Substantially faster
Multi-Video Analysis	Parallel processing possible	Sequential processing	Major advantage for GPU
Hardware Cost	Higher initial investment	Lower cost	Variable
Best Use Cases	Large datasets, model development	Small projects, data management	Task-dependent

Technical Considerations for Hardware Selection

For optimal DeepLabCut performance in research settings:

• GPU Requirements: NVIDIA CUDA-compatible GPU recommended for substantial performance gains [19]
• Multi-GPU Setup: While training uses only one GPU, multiple GPUs enable simultaneous video analysis [20]
• CPU Fallback: CPU-only installation suitable for project management, labeling, and small-scale analysis [19]
• Cloud Alternatives: Google Colaboratory provides free GPU access for users without local hardware [19]

Installation Protocols

Pre-Installation Requirements

Table 2: Essential pre-installation components

Component	Function	Research Application
Python 3.10+	Core programming language	Required runtime environment
Anaconda/Miniconda	Package and environment management	Creates isolated, reproducible research environments
CUDA Toolkit	Parallel computing platform	Enables GPU acceleration for deep learning
cuDNN	GPU-accelerated library	Optimizes neural network operations
NVIDIA Drivers	GPU communication software	Essential for GPU access

Protocol 1: Conda-Based Installation with GPU Support

This protocol provides a standardized method for installing DeepLabCut with GPU acceleration, suitable for most research environments.

Step 1: Environment Creation

Step 2: Install Critical Dependencies

Step 3: Install PyTorch with GPU Support Select the appropriate CUDA version for your hardware (example for CUDA 11.3):

Step 4: Install DeepLabCut

Step 5: Verify GPU Access

Expected output: True confirms successful GPU configuration [19].

Protocol 2: CPU-Only Installation

For systems without compatible NVIDIA GPUs:

Step 1: Environment Creation

Step 2: Install PyTorch CPU Version

Step 3: Install DeepLabCut

Protocol 3: TensorFlow Backend Installation

Note: TensorFlow support will be deprecated by end of 2024. This protocol is for legacy compatibility only [19].

Step 1: Create Environment with Specific Python Version

Step 2: Install TensorFlow and Dependencies

Step 3: Create Library Links

Step 4: Install DeepLabCut

Hardware Selection Workflow

Hardware Selection Decision Tree: Systematic approach for selecting the appropriate computational configuration based on available hardware and research needs.

Dependency Management and Troubleshooting

Critical Dependencies and Functions

Table 3: Essential dependencies and their research functions

Dependency	Research Function	Installation Method
PyTables	Data management for large behavioral datasets	Conda installation recommended [19]
PyTorch	Deep learning backend for model training	Conda or Pip with CUDA toolkit
OpenCV	Video processing and computer vision	Automatic with DeepLabCut
NumPy/SciPy	Numerical computations for pose estimation	Automatic with DeepLabCut
Matplotlib	Visualization of tracking results	Automatic with DeepLabCut

Common Installation Issues and Solutions

• CUDA Compatibility: Verify CUDA version matches PyTorch requirements [19]
• Path Conflicts: Ensure conda environment isolation to prevent library conflicts [21]
• Windows-Specific Issues: Always run terminal as administrator for proper symlink creation [14]
• Package Conflicts: Use the provided conda environment files for tested dependency combinations [22]

Experimental Protocol: Validation and Benchmarking

Protocol for System Validation

After installation, validate your DeepLabCut setup using this standardized protocol:

Step 1: GPU Verification Test

Step 2: DeepLabCut Functionality Test

Step 3: Performance Benchmarking

• Track processing time for 100-frame video
• Compare training iteration times
• Verify GUI functionality for labeling interfaces

Research Implementation Workflow

Research Implementation Workflow: End-to-end process for implementing DeepLabCut in behavioral research studies, from hardware selection to research insights.

Proper installation of DeepLabCut with appropriate hardware configuration establishes the foundation for robust, efficient markerless pose estimation in animal behavior research. The GPU-enabled installation provides significant performance advantages for large-scale studies, while CPU options remain viable for specific use cases. As DeepLabCut continues to evolve with improved model architectures and performance optimizations [5], establishing a correct installation workflow ensures researchers can leverage the full potential of this tool for advancing behavioral neuroscience and drug development research.

DeepLabCut is an open-source toolbox for markerless pose estimation based on deep neural networks that allows researchers to track user-defined body parts across species with remarkable accuracy [2]. Its application spans diverse fields including neuroscience, ethology, and drug development, enabling non-invasive behavioral tracking during experiments [23]. For researchers in drug development, precise behavioral phenotyping using tools like DeepLabCut provides valuable insights for investigating therapeutic efficacy and modeling psychiatric disorders [12]. The initial step of project creation is fundamental to establishing a robust and reusable analysis pipeline. This protocol details two complementary methods for project initialization: via the graphical user interface (GUI) recommended for beginners, and via the command line interface offering greater flexibility for advanced users and automation [14].

Prerequisites

Software Installation

Before creating a DeepLabCut project, ensure the software is properly installed. DeepLabCut requires Python 3.10 or later [19]. The recommended installation method uses Anaconda to manage dependencies in a dedicated environment [19]:

• Install Anaconda from anaconda.com/download. For MacBooks with M1/M2 chips, use miniconda3 instead [19].
• Create and activate a Conda environment:

• Install DeepLabCut. For the latest version with GUI support and the PyTorch engine, run:
  For installation with TensorFlow support (to be deprecated after 2024), use pip install "deeplabcut[gui,tf]" [19].

Hardware Considerations

• GPU: For significantly faster model training, an NVIDIA GPU with compatible CUDA and cuDNN libraries is recommended [19].
• CPU: Projects can be managed and data labeled using CPU-only systems, with the option to leverage cloud resources like Google Colaboratory for training [19].

Method 1: Project Creation via Graphical User Interface (GUI)

The GUI is the recommended starting point for new users, providing an intuitive visual workflow [14].

Protocol Steps

• Launch the GUI: Open a terminal (Administer on Windows), activate your DeepLabCut environment (conda activate DEEPLABCUT), and launch the interface [14]:

• Initiate Project Creation: The DeepLabCut Project Manager GUI will open. Select the option to "Create a New Project" [14].
• Configure Project Parameters: A dialog window will appear. Fill in the following required fields [14]:
  • Project Name: A descriptive name for your behavior analysis (e.g., "Reaching-Task").
  • Experimenter Name: Your name (e.g., "Researcher_Name").
  • Videos: Select the path(s) to the video files that will form the initial training dataset.
• Set Advanced Options (Optional):
  • Working Directory: The path where the project folder will be created. Defaults to the current directory.
  • Copy Videos: If True, videos are copied to the project folder. If False, symbolic links are created, saving disk space [14].
  • Multi-Animal: Set to False for standard single-animal projects [14].
• Execute: Click the button to create the project. The GUI will generate a project directory with all necessary subfolders and a configuration file (config.yaml).

Output

The function creates a standardized project structure [14]:

• project-directory/
  • config.yaml: The main project configuration file.
  • videos/: Directory containing the videos or symbolic links.
  • labeled-data/: Will store extracted frames for labeling.
  • training-datasets/: Will hold the generated training datasets.
  • dlc-models/: Will contain the trained models and evaluation results.

Method 2: Project Creation via Command Line

The command line interface (CLI) offers programmatic control, beneficial for automation and integration into larger analysis scripts [14].

Protocol Steps

• Launch Python: Open a terminal, activate your DeepLabCut environment, and start an interactive Python session (ipython for Windows/Linux, pythonw for Mac) [14] [24].
• Import the Library:

• Execute the Create Project Function:
  Critical Path Note for Windows Users: Use raw strings (r"...") or double backslashes ("C:\\Users\\...") for paths [14].

Output

The create_new_project function returns the path to the project's configuration file (config.yaml), which is crucial for all subsequent DeepLabCut functions [14]. Store this path as the config_path variable for future use [24].

Table 1: Core Parameters for the deeplabcut.create_new_project Function

Parameter	Data Type	Description	Example
project	String	Name identifying the project.	"Reaching-Task"
experimenter	String	Name of the experimenter.	"Researcher_Name"
videos	List of Strings	Full paths to videos for the initial dataset.	["/path/video1.avi"]
working_directory	String (Optional)	Path where the project is created. Defaults to current directory.	"/analysis/project/"
copy_videos	Boolean (Optional)	Copy videos (True) or create symbolic links (False). Default is False.	False
multianimal	Boolean (Optional)	Set to True for multi-animal projects. Default is False.	False

Post-Creation Configuration

After project creation, the critical next step is configuring the project by editing the config.yaml file. This file contains all parameters governing the project [14].

• Locate the File: The config.yaml file is in your project directory. Its path was returned as config_path in the CLI method.
• Edit Body Parts: Open the file in a text editor. Under the bodyparts section, list all the points of interest you want to track without spaces in the names [14].

• Set Colormap: The colormap parameter can be set to any matplotlib colormap (e.g., rainbow, viridis) to define colors used in labeling and visualization [14].

Comparative Analysis of Initialization Methods

Table 2: Quantitative Comparison of GUI and Command Line Initialization Methods

Feature	GUI Method	Command Line Method
Ease of Use	High (visual guidance) [14]	Medium (requires parameter knowledge)
Automation Potential	Low	High (scriptable, reproducible) [24]
Initial Setup Speed	Fast for single projects	Faster for batch processing
Customization Control	Basic (via GUI fields)	High (direct access to all parameters)
Error Handling	Guided dialog boxes	Relies on terminal error messages
Best For	Beginners, one-off projects	Advanced users, automated pipelines, HPC

Workflow Visualization

The following diagram illustrates the complete project initialization workflow, integrating both the GUI and CLI methods into the broader DeepLabCut pipeline leading to behavioral analysis.

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Materials and Tools for a DeepLabCut Project

Item	Function/Description	Research Context
Video Recording System	High-quality camera to capture animal behavior. Essential for creating input data.	Critical for data acquisition; resolution and frame rate affect tracking accuracy [23].
DeepLabCut Python Package	Core software for markerless pose estimation.	The primary analytical tool. Installation via pip in a Conda environment is recommended [19].
Configuration File (config.yaml)	Central file storing all project parameters (bodyparts, training settings, etc.).	The experimental blueprint. Editing this file tailors the network to the specific research question [14].
Labeling GUI (Napari)	Interface for manually labeling body parts on extracted frames to create the training set.	Used after project creation. A "good training dataset" that captures behavioral diversity is critical for robust performance [14] [25].
GPU with CUDA Support	Hardware accelerator for drastically reducing model training time.	Recommended but not mandatory. Enables faster iteration in model development [19].
Momordicoside P	Momordicoside P, MF:C36H58O9, MW:634.8 g/mol	Chemical Reagent
Specioside B	Specioside B, MF:C23H24O10, MW:460.4 g/mol	Chemical Reagent

Configuring the config.yaml file is a foundational step in any DeepLabCut pose estimation project, setting the stage for all subsequent analysis in animal behavior research. This file dictates which body parts are tracked, how the model learns, and how predictions are interpreted, directly impacting the quality and reliability of the scientific data generated for fields such as neuroscience and drug development [14].

Core Parameters of the config.yaml File

The project configuration file contains parameters that control the project setup, the definition of the animal's pose, and the training and evaluation of the deep neural network. A summary of the key parameters is provided in the table below.

Table 1: Key Parameters in the DeepLabCut config.yaml File

Parameter	Description	Impact on Research
bodyparts	List of all body parts to be tracked [14].	Defines the pose skeleton and the granularity of behavioral quantification.
skeleton	Defines connections between bodyparts for visualization [14].	Aids in visual inference and can guide the assembly of individuals in multi-animal scenarios [26].
multianimal	Boolean (True/False) indicating if multiple animals are present [14].	Determines the use of assembly and tracking algorithms necessary for social behavior studies [26].
individuals	(Multi-animal only) List of individual identifiers [14].	Enables tracking of specific animals across time, crucial for longitudinal drug efficacy studies.
pcutoff	Confidence threshold for filtering predictions [27].	Ensures only reliable position data is used for downstream analysis, reducing noise.
colormap	Color scheme for bodyparts in labeling and video output [14].	Improves visual distinction of body parts for researchers during manual review.

Defining Body Parts: Strategies for Robust Pose Estimation

The bodyparts list is the most critical user-defined parameter. The choice of body parts must be driven by the specific research question and the animal's morphology.

Naming Conventions and Specificity

Body part names should be clear, consistent, and must not contain spaces [14]. For complex organisms or to disambiguate left and right sides, use specific names like LEFTfrontleg_point1 and RIGHTfrontleg_point1 [27]. This precision is essential for accurately parsing the resulting data and attributing movements to the correct limb.

Handling Occlusion and Visibility

A key decision is how to handle body parts that are frequently occluded. Two validated strategies exist, each with implications for the resulting data:

• Label-Only-Visible: Label a body part only when it is clearly visible. The network will learn to predict it with high confidence only when visible, and its likelihood score (pcutoff) can be used to filter out frames where it is occluded [27]. This strategy is best for achieving the highest positional accuracy for visible points.
• Label-with-"Guess": Label the estimated position of an occluded body part. The network will learn to infer its location [27]. This is useful for maintaining a complete skeletal trajectory for behaviors where continuity is more important than absolute precision, but it introduces estimation bias.

Experimental Protocol: Project Setup and Configuration

The following workflow details the steps for creating a new project and configuring the config.yaml file.

Figure 1: The workflow for initializing a DeepLabCut project and configuring the config.yaml file.

Step 1: Create a New Project Launch the DeepLabCut environment in your terminal or Anaconda Prompt and use the create_new_project function. It is good practice to assign the path of the created configuration file to a variable (config_path) for future steps [14].

Step 2: Edit the config.yaml File Open the config.yaml file from your project directory in a standard text editor. Navigate to the bodyparts section and replace the example entries with your own list of body parts.

Example Configuration for a Mouse Study:

After editing, save the file. The project is now configured, and you can proceed to the next step of extracting frames for labeling.

The Scientist's Toolkit

Table 2: Essential Research Reagents and Computational Tools

Item / Software	Function in Research	Application Note
DeepLabCut [14]	Open-source toolbox for markerless pose estimation based on deep learning.	The core platform for training and deploying pose estimation models.
Anaconda	Package and environment manager for Python.	Used to create an isolated environment with the correct dependencies for DeepLabCut.
Labeling Tool (e.g., Napari in DLC) [7]	Software for manual annotation of body parts on extracted video frames.	Used to create the ground-truth training dataset.
SpaceAnimal Dataset [7] [28]	A public benchmark dataset for multi-animal pose estimation and tracking.	Provides expert-validated data for complex scenarios like occlusions, useful for method validation.
Simple Behavioral Analysis (SimBA) [29]	Open-source software for classifying behavior based on pose estimation data.	Used downstream of DeepLabCut to translate tracked coordinates into defined behavioral events.
Jbir-94	Jbir-94, MF:C24H32N2O6, MW:444.5 g/mol	Chemical Reagent
Yunnancoronarin A	Yunnancoronarin A, MF:C20H28O2, MW:300.4 g/mol	Chemical Reagent

Advanced Configuration: Multi-Animal Projects

For experiments involving social interactions, setting multianimal: True in the config.yaml is crucial. This engages a different pipeline that includes keypoint detection, assembly (grouping keypoints into distinct individuals), and tracking over time [26]. The individuals parameter can then be used to define unique identifiers for each animal (e.g., ['mouse1', 'mouse2', 'mouse3']), which assists in tracking identity across frames, especially during occlusions [14] [26]. Advanced multi-animal networks can also predict animal identity from visual features, further aiding in tracking [26].

The DeepLabCut Workflow in Action: From Data Labeling to Behavioral Analysis

The accuracy and reliability of any DeepLabCut (DLC) model for animal pose estimation are fundamentally constrained by the quality and diversity of the training dataset [30] [14]. Frame extraction—the process of selecting representative images from video sources—constitutes a critical first step in the pipeline, establishing the "ground truth" from which the model learns [31]. A dataset that captures the full breadth of an animal's posture, lighting conditions, and behavioral repertoire is essential for building a robust pose estimation network that generalizes well across experimental sessions [32] [14]. This document outlines structured strategies and protocols for researchers to build comprehensive training datasets, thereby enhancing the validity of subsequent behavioral analyses in fields such as neuroscience and drug development.

The Critical Role of Dataset Diversity in Pose Estimation

Tracking drift, where keypoint estimates exhibit unnatural jumps or instability, is a common failure mode in animal pose estimation that can often be traced back to inadequate training data [32]. Such drift is frequently caused by the model encountering postural or environmental scenarios it was not trained on, such as animals in close interaction, occluded body parts, or unusual lighting [30] [32]. The consequences of a non-robust dataset propagate through the entire research pipeline, potentially compromising gait analysis, behavioral classification, and the statistical outcomes of ethological studies [32].

A robust training dataset acts as a primary defense against these issues. The official DeepLabCut user guide emphasizes that a good training dataset "should consist of a sufficient number of frames that capture the breadth of the behavior," including variations in posture, luminance, background, and, where applicable, animal identity [14]. For initial model training, extracting 100-200 frames can yield good results for many behaviors, though more may be required for complex social interactions or challenging video quality [14].

Table 1: Impact of Dataset Composition on Model Performance and Common Failure Modes

Scenario Missing from Training Data	Potential Model Failure Mode	Downstream Impact on Research
Close animal interactions [30]	Loss of tracking for one animal or specific body parts (e.g., nose, tail) [30]	Inaccurate quantification of social behavior
Significant occlusion	Inability to estimate occluded keypoints [33]	Faulty gait analysis and behavior classification [32]
Extreme postures (e.g., rearing, lying)	Low confidence/likelihood for keypoints in novel configurations	Missed detection of rare but biologically significant behavioral events
Variations in lighting/background	High prediction error under new conditions	Reduced model generalizability across experimental cohorts or sessions

Quantitative Framework for Frame Extraction

A strategic approach to frame extraction involves combining different automated and manual methods to ensure comprehensive coverage. The following table summarizes key strategies and their specific objectives.

Table 2: Frame Extraction Strategies for Building a Robust Training Dataset

Extraction Strategy	Core Objective	DeepLabCut Function/Protocol	Key Quantitative Metric(s)
Uniform Frame Sampling	Capture a baseline of postural and behavioral variance from all videos [14].	deeplabcut.extract_frames	Total frames per video; coverage across entire video duration.
K-Means Clustering	Select a diverse set of frames by grouping visually similar images and sampling from each cluster [14].	deeplabcut.extract_frames(config_path, 'kmeans')	Number of clusters (k); frames extracted per cluster.
Outlier Extraction (Uncertainty)	Identify and label frames where the model is least confident, often due to errors or occlusions [30] [34].	deeplabcut.extract_outlier_frames(config_path, outlieralgorithm='uncertain')	Likelihood value (p-bound) for triggering extraction.
Manual Extraction of Specific Behaviors	Add targeted examples of crucial, potentially rare, behaviors (e.g., close social interaction) [30].	Manually curate videos and use DLC's frame extraction GUI.	Number of frames per user-defined behavioral category.

Protocol: K-Means Based Frame Extraction

Purpose: To automate the selection of a posturally diverse set of frames from input videos by leveraging computer vision clustering algorithms.

Materials:

• DeepLabCut project with configured config.yaml file.
• List of videos for frame extraction.

Methodology:

• Open your DeepLabCut environment: Launch your terminal and activate the conda environment where DeepLabCut is installed.
• Execute extraction command: In your Python environment, run the following command, replacing 'your_config_path' with the actual path to your project's config.yaml file:

• Set parameters: The function will prompt you to select the number of clusters (k) and the number of frames to select from each cluster. The optimal value for k depends on the complexity of the behavior but often ranges from 20 to 50 to ensure sufficient diversity.
• Review extracted frames: The extracted frames will be saved in the labeled-data subdirectories of your project. Visually inspect them to ensure they represent a wide array of the animal's poses.

Protocol: Extracting Outlier Frames from Initial Analysis

Purpose: To refine an existing model by identifying and labeling frames where its predictions were poor, a process critical for iterative improvement.

Materials:

• A trained DeepLabCut model that has been used to analyze a video.
• The resulting analysis file (e.g., *.h5).

Methodology:

• Analyze a video: First, use your model to analyze a video with deeplabcut.analy_videos.
• Extract outliers: Use the following command to extract frames where the average likelihood across all body parts falls below a set threshold (p_bound):
  Note: Presently, this method assesses the likelihood across all body parts. To focus on a specific, problematic body part, manual review of the analyzed video is required [34].
• Label and refine: The extracted outlier frames will be saved. Open the DLC GUI to manually correct the labels on these frames. Adding these corrected frames to your training set and re-training the model directly addresses its previous weaknesses.

Diagram 1: A workflow for constructing a robust training dataset through iterative refinement.

Advanced Annotation and Multi-Animal Considerations

For complex research scenarios, such as multi-animal tracking, basic frame extraction requires supplemental strategies.

Strategies for Multi-Animal Tracking

Social interaction experiments, where multiple animals of similar appearance are tracked, present distinct challenges. Key strategies include:

• Targeted Manual Extraction: Actively extract and label frames where animals are in close contact, as these are common failure points for identity swapping and lost tracks [30].
• Iterative Refinement: After initial training, analyze videos of social interaction and use the outlier extraction protocol to find and correct frames where the model failed. Add these corrected frames to the training set for the next training iteration (e.g., shuffle 1 to shuffle 2) [30].
• Video Quality: Consider using higher-resolution videos if downsampling makes it difficult even for a human to distinguish closely interacting body parts, as this likely also hinders the model [30].

Ensuring Annotation Quality

The quality of manual labeling on extracted frames is paramount. Best practices derived from large-scale annotation projects include:

• Clear Guidelines: Establish detailed annotation guidelines that define the precise location of each keypoint, especially for challenging cases like occluded limbs [33].
• Training and Cross-Checking: Annotators should be trained on a small set of images first. A multi-round process of cross-checking and correction by senior annotators significantly improves label quality and consistency [33].
• Leveraging Animal Physiology: Annotators can be instructed to estimate the position of occluded keypoints based on the animal's body plan, pose, and symmetry, which improves the model's ability to handle partial visibility [33].

The Scientist's Toolkit: Essential Research Reagents

Table 3: Key Software and Hardware for DLC Frame Extraction and Annotation

Item Name	Function/Application	Usage Notes
DeepLabCut [14]	Open-source software platform for markerless pose estimation.	Core environment for all frame extraction, model training, and analysis.
Anaconda	Package and environment management.	Used to create and manage the isolated Python environment for DeepLabCut.
Labeling GUI (DLC) [14]	Integrated graphical tool for manual labeling of extracted frames.	Critical for creating ground truth data.
High-Resolution Camera	Video acquisition.	Higher-quality source videos reduce ambiguity during frame extraction and labeling.
CVAT / Label Studio [31]	Advanced, external annotation tools.	Can be used for complex projects, supporting customizable workflows.
Macedonic acid	Macedonic acid, CAS:39022-00-9, MF:C30H46O4, MW:470.7 g/mol	Chemical Reagent
Urolithin M7	Urolithin M7, MF:C13H8O5, MW:244.20 g/mol	Chemical Reagent

A deliberate and multi-faceted strategy for frame extraction is not merely a preliminary step but a foundational component of reproducible and reliable animal pose estimation research. By systematically combining uniform sampling, clustering-based diversity, outlier-driven refinement, and targeted manual extraction, researchers can construct training datasets that empower DeepLabCut models to perform accurately across the full spectrum of natural animal behavior. This rigorous approach ensures that subsequent analyses, from gait quantification to social interaction studies, are built upon a solid and valid foundation.

A critical phase in the development of a robust markerless pose estimation model for animal behavior research is the efficient creation of high-quality training data. In DeepLabCut, this process involves the manual annotation of user-defined body parts on a carefully selected set of video frames. The Labeling GUI, which is built upon the Napari viewer, provides the interface for this task. The quality, accuracy, and diversity of these manual labels directly determine the performance of the resulting deep learning model in tracking behaviors of interest in pre-clinical research, such as gait analysis in disease models or activity monitoring in response to pharmacological compounds [14]. This protocol details the methodology for using the DeepLabCut Graphical User Interface (GUI) to efficiently and accurately annotate body parts, forming the foundational dataset for a pose estimation project.

Conceptual Foundation and Experimental Strategy

The Principle of Frame Selection for Training

Before annotation begins, a strategic set of frames must be extracted from the source videos. The guiding principle is that the training dataset must encapsulate the full breadth of the behavior and the variation in experimental conditions. A robust network requires a training set that reflects the diversity of postures, lighting conditions, background contexts, and, if applicable, different animal identities present across the entire dataset [14]. For many behaviors, a dataset of 100–200 frames can yield good results, though more may be necessary for complex behaviors, low video quality, or when high accuracy is required [14].

Defining the Annotation Target: The Configuration File

The body parts to be tracked are defined in the project's config.yaml file. This file must be edited before starting the labeling process. Researchers must list all bodyparts of interest under the bodyparts parameter. It is critical that no spaces are used in the names of bodyparts (e.g., use "LeftEar" not "Left Ear") [14]. The colormap parameter can also be customized in this file to define the colors used for different body parts in the labeling GUI [14].

Experimental Protocol: The Labeling Workflow

The following step-by-step protocol guides you through the process of labeling frames using the DeepLabCut GUI.

Prerequisites and Initialization

• Project Configuration: Ensure you have created a DeepLabCut project and have edited the config.yaml file to include your list of target body parts [14].
• Frame Extraction: Use the deeplabcut.extract_frames function to select frames from your videos. DeepLabCut offers several methods for this, including uniform interval, k-means based selection to capture posture variation, and manual selection [14].
• Launch the Labeling Tool: From the DeepLabCut GUI, navigate to the "Label Frames" tab. Select a folder within your project's labeled-data directory that contains the extracted frames (these folders are named after your videos). This action will launch the Napari viewer with the first frame loaded [35] [36].

Annotation Procedure in the Napari Viewer

Table 1: Core Steps for Annotation in the Napari GUI

Step	Action	Description and Purpose
1. Add Points Layer	Click the "Add points" layer button.	This creates a new points layer for annotation. The interface may initially seem to limit the number of points layers, but this is typically tied to the body parts listed in your config.yaml file. [35]
2. Select Body Part	In the points layer properties, select the correct body part from the dropdown menu.	This ensures the points you place are associated with the intended anatomical feature. The list is populated from your config.yaml.
3. Place Landmarks	Click on the image to place a point on the corresponding body part.	For high accuracy, zoom in on the image for sub-pixel placement. The human accuracy of labeling directly influences the model's final performance [37].
4. Save Progress	Save your work frequently using the appropriate button or shortcut.	Napari does not auto-save, so regular saving is critical to prevent data loss.
5. Navigate Frames	Use the frame slider to move to subsequent frames.	Repeat steps 1-4 for every body part in every frame that requires labeling.

Table 2: Key Symbolism in the Labeling and Evaluation GUI

Symbol	Represents	Context
+ (Plus)	Ground truth manual label.	The label created by the human annotator.
· (Dot)	Confident model prediction.	A prediction from an evaluated model with a likelihood above the pcutoff threshold.
x (Cross)	Non-confident model prediction.	A prediction from an evaluated model with a likelihood below or equal to the pcutoff threshold. [38]

Workflow Visualization

The following diagram illustrates the complete workflow from project creation to model refinement, highlighting the central role of the labeling process.

Table 3: Key Research Reagent Solutions for DeepLabCut Projects

Item / Resource	Function / Purpose
DeepLabCut Project Environment	A configured Conda environment with DeepLabCut and its dependencies (e.g., PyTorch/TensorFlow). Essential for ensuring software compatibility and reproducibility.
config.yaml File	The central project configuration file. Defines all body parts, training parameters, and project metadata. Serves as the experimental blueprint. [14]
pose_cfg.yaml File	Contains the hyperparameters for the neural network model (e.g., global_scale, batch_size, augmentation settings). Crucial for optimizing model performance. [39]
Labeled-data Directory	Stores the extracted frames and the associated manual annotations in HDF5 or CSV format. This is the primary output of the labeling process and the core training asset. [14] [40]
Napari Viewer	The multi-dimensional image viewer that hosts the DeepLabCut labeling tool. Provides the interface for accurate, sub-pixel placement of body part labels. [35]
Jupyter Notebook	An optional but recommended tool for logging and executing the project workflow. Enhances reproducibility and provides a clear record of the analysis steps. [40]

Troubleshooting and Technical Validation

• Issue: Inability to Add More Points: If the Napari GUI restricts you from adding more than a few points, first verify that all desired body parts are correctly listed in the config.yaml file. The points layers are linked to this configuration [35].
• Issue: KeyError when Clicking on Individuals: In multi-animal projects, a KeyError (e.g., KeyError: 'mouse2') when clicking on the color scheme reference is a known interface bug. This does not affect the core labeling functionality, and you can proceed without interacting with that part of the GUI [36].
• Validation: Labeling Accuracy: To quantify the consistency of your annotations, a best practice is to re-label a small subset of frames and compare the coordinates. The variability between labeling sessions provides an estimate of the human error, which sets a practical upper limit on model accuracy [37].
• Optimization for Low-Resolution Data: For videos with low contrast or resolution, consider cropping the frames further and then upsampling them before labeling. Furthermore, during training, setting global_scale: 1.0 in the pose_cfg.yaml file can prevent downsampling and preserve spatial accuracy [37].

The meticulous annotation of body parts in selected frames is a critical, human-in-the-loop step that directly fuels the DeepLabCut pose estimation pipeline. By adhering to the protocols outlined in this document—strategically selecting diverse frames, accurately using the Napari-based labeling GUI, and understanding the key parameters and common pitfalls—researchers can generate high-fidelity training data. This rigorous approach ensures the development of a robust, reliable, and reusable deep learning model capable of providing quantitative behavioral phenotyping for a wide range of scientific and pre-clinical drug development applications.

DeepLabCut is a widely adopted open-source toolbox for markerless pose estimation of animals and humans. Its power lies in using deep neural networks, which can achieve human-level accuracy in labeling body parts with relatively few training examples (typically 50-200 frames) [41]. The software has undergone significant evolution, with its backend now supporting PyTorch, offering users performance gains, easier installation, and greater flexibility [5]. A core strength of DeepLabCut is its use of transfer learning, where a neural network pre-trained on a large dataset (like ImageNet) is re-trained (fine-tuned) on a user's specific, smaller dataset. This allows for high-performance tracking without the need for massive amounts of labeled data [42].

When creating a project, users must select a network architecture (model) to train. These architectures are the engine of the pose estimation process, and their selection involves trade-offs between speed, memory usage, and accuracy [43]. The available models can be broadly categorized into several families, each with unique characteristics and recommended use cases, which will be detailed in the following sections.

Performance Comparison of Network Architectures

Selecting the appropriate network architecture is crucial for balancing performance requirements with computational resources. The table below summarizes the key characteristics and performance metrics of popular models available in DeepLabCut.

Table 1: Performance and Characteristics of DeepLabCut Model Architectures

Model Name	Type	Key Strengths	Ideal Use Cases	Inference Speed	mAP on SA-Q (AP-10K)	mAP on SA-TVM (DLC-OpenField)
ResNet-50 [43] [42]	Top-Down / Bottom-Up	Excellent all-rounder; strong performance for most lab applications	Default, general-purpose tracking; recommended starting point	Standard	54.9 [5]	93.5 [5]
ResNet-101 [43] [42]	Top-Down / Bottom-Up	Higher capacity than ResNet-50 for complex problems	Challenging postures, multiple humans/animals in complex interactions	Slower	55.9 [5]	94.1 [5]
MobileNetV2-1 [43]	Bottom-Up	Fast training & inference; memory-efficient; good for CPUs	Real-time feedback, low-resource GPUs, or CPU-only analysis	Up to 4x faster on CPUs, 2x on GPUs [43]	Not Specificed	Not Specificed
HRNet-w32 [5]	Top-Down	Maintains high-resolution representations	Scenarios requiring high spatial accuracy	Slower	52.5 [5]	92.4 [5]
HRNet-w48 [5]	Top-Down	Enhanced version of HRNet-w32	When higher accuracy than HRNet-w32 is needed	Slower than HRNet-w32	55.3 [5]	93.8 [5]
DEKR_w32 [44]	Bottom-Up (Multi-animal)	Improved animal assembly in multi-animal scenarios	Bottom-up multi-animal projects with occlusions	Fast	Not Specificed	Not Specificed
EfficientNets [43]	Bottom-Up	More powerful than ResNets; faster than MobileNets	Advanced users willing to tune hyperparameters	Fast	Not Specificed	Not Specificed
DLCRNet_ms5 [4]	Bottom-Up (Multi-animal)	Custom multi-scale architecture for multi-animal	Complex multi-animal datasets with occlusions [4]	Not Specificed	Not Specificed	Not Specificed

Model Selection Guidance

For most single-animal applications in laboratory settings, ResNet-50 provides the best balance of performance and efficiency and is the recommended starting point [43]. Its performance has been validated across countless studies, including for gait analysis in humans and various animal behaviors [42]. If you are working with standard lab animals like mice and do not have extreme computational constraints, ResNet-50 is your best bet.

For multi-animal projects, the choice is more nuanced. The bottom-up approach (using models like ResNet-50, DLCRNet_ms5, or DEKR) detects all keypoints for all animals in an image first and then groups them into individuals. This is efficient for scenes with many animals. In contrast, the top-down approach first detects individual animals (e.g., via bounding boxes) and then estimates pose within each box. Top-down models are a good choice if animals do not frequently interact and are often separated, as they simplify the problem of assigning keypoints to the correct individual [44].

MobileNetV2-1 and EfficientNets are excellent choices when computational resources are limited or when very fast analysis is required, such as for real-time, closed-loop feedback experiments [43]. MobileNetV2-1 is particularly user-friendly for those with low-memory GPUs or who are running analysis on CPUs.

Training Parameters and Configuration

Achieving optimal model performance requires careful configuration of training parameters. The settings control how the model learns from the labeled data and can significantly impact training time and final accuracy.

Core Training Parameters

Table 2: Key Training Parameters and Their Functions in DeepLabCut

Parameter	Description	Default/Common Values	Impact & Tuning Guidance
Batch Size	Number of training images processed per update	1 (TF [45]) to 8 (PyTorch [45])	Larger batches train faster but use more GPU memory. If you increase batch size, you can also try increasing the learning rate [44].
Learning Rate (lr)	Step size for updating network weights during training	e.g., 0.0005 [45]	Crucial for convergence. Too high causes instability; too low leads to slow training. A smaller batch size may require a smaller learning rate [44].
Epochs	Number of complete passes through the training dataset	200+ (e.g., 200 [45], 5000+ [45])	Training should continue until evaluation loss/metrics plateau. More complex tasks require more epochs.
Global Scale (global_scale)	Factor to downsample images during training	e.g., 0.8 [45]	Setting this to 1.0 uses full image resolution, which can improve spatial accuracy for small body parts but is slower [37].
Data Augmentation	Artificial expansion of training data via transformations (rotation, scaling, noise)	Rotation: 25 [45] to 30 [45]; Scaling: 0.5-1.25 [45]	Critical for building a robust model invariant to changes in posture, lighting, and background.

Advanced Parameter Scheduling

For challenging projects, such as tracking low-resolution or thin features, a multi-step learning rate schedule can be beneficial. This involves reducing the learning rate at predefined intervals, allowing the model to fine-tune its weights more precisely as training progresses. An example from the community is: cfg_dlc['multi_step'] = [[1e-4, 7500], [5*1e-5, 12000], [1e-5, 50000]] [37]. This schedule starts with a learning rate of 0.0001 for 7,500 iterations, then reduces it to 0.00005 for the next 4,500 iterations, and finally to 0.00001 for the remaining iterations.

Experimental Protocols for Model Training and Validation

This section provides a detailed, step-by-step protocol for creating a DeepLabCut project, training a model, and validating its performance, as exemplified by a real-world gait analysis study [42].

Protocol: Creating a Custom-Trained Model for Gait Analysis

Objective: To train and validate a DeepLabCut model for accurate 2D pose estimation of human locomotion using a single camera view, achieving performance comparable to or exceeding pre-trained models.

Materials and Reagents:

• Hardware: RGB camera (e.g., 25 fps, 640x480 resolution), a computer with a CUDA-enabled GPU is highly recommended.
• Software: DeepLabCut (Python package).
• Subjects: 40 healthy adult subjects (or appropriate sample size for the model organism).
• Experimental Setup: A 5-meter walkway with force platforms time-synchronized with the camera [42].

Workflow:

Step-by-Step Procedure:

• Project Creation:

  • Use deeplabcut.create_new_project() to initialize a new project, specifying the project name, experimenter, and paths to the initial videos [14].
  • This function creates the project directory, necessary subdirectories (labeled-data, training-datasets, videos, dlc-models), and the main configuration file (config.yaml).

• Configuration:

  • Open the config.yaml file in a text editor.
  • Under the bodyparts section, list all the keypoints you want to track (e.g., heel, toe, knee, hip for gait analysis). Do not use spaces in the names [14].
  • You can also set the visualization colormap at this stage.

• Frame Selection and Labeling:

  • Select a representative set of frames from your videos using the built-in k-means clustering algorithm (deeplabcut.extract_frames()). This method selects frames that capture the diversity of postures and appearances [42].
  • In the cited study, 10 frames were extracted from each of the 40 subject videos, resulting in a total training set of 400 frames [42].
  • Manually label the body parts in each extracted frame using the DeepLabCut GUI (deeplabcut.label_frames()). Zoom in for sub-pixel accuracy where necessary.

• Dataset Creation and Model Training:

  • Generate the training dataset from the labeled frames using deeplabcut.create_training_dataset(). At this stage, you must select your network architecture (e.g., net_type='resnet_101') [42].
  • Begin training the model with deeplabcut.train_network(). The system will automatically save snapshots (checkpoints) during training.
  • Monitor the training and evaluation loss. Training should typically continue until this loss plateaus, which may require hundreds of thousands of iterations (equivalent to several thousand epochs, depending on your dataset size) [45].

• Model Evaluation and Video Analysis:

  • Evaluate the model's performance on the held-out test frames using deeplabcut.evaluate_network(). This generates metrics and plots that allow you to assess the model's accuracy.
  • Use the trained model to analyze new videos and generate pose estimation data (deeplabcut.analyze_videos()).

• Refinement (Active Learning):

  • A critical step for achieving optimal performance is to refine the training dataset. Use the deeplabcut.extract_outlier_frames() function to identify frames where the model is least confident.
  • Manually label these outlier frames and add them to the training dataset. This iterative process, known as active learning, helps the model learn from its mistakes and greatly improves robustness [42].
  • Create a new training dataset and re-train the model incorporating the newly labeled frames.

Validation against Ground Truth: In the gait study, the temporal parameters (heel-contact and toe-off events) derived from the custom-trained DeepLabCut model (DLCCT) were compared against data from force platforms, which served as the reference system. The DLCCT model, especially after refinement, showed no significant difference in measuring grooming duration compared to manual scoring, demonstrating high validity [41] [42].

The Scientist's Toolkit: Essential Research Reagents and Materials

This table outlines the key "research reagents"—the software, hardware, and data components—required to successfully implement a DeepLabCut pose estimation project.

Table 3: Essential Research Reagents and Materials for DeepLabCut Projects

Item Name	Specification / Example	Function / Role in the Experiment
DeepLabCut Python Package	Version 2.3.2+ or 3.0+ [42] [5]	Core software environment providing pose estimation algorithms, GUIs, and training utilities.
Network Architecture (Model)	ResNet-50, ResNet-101, MobileNetV2, etc. [43]	The pre-defined neural network structure that is fine-tuned during training to become the pose prediction engine.
Pre-trained Model Weights	ImageNet-pretrained ResNet weights [42]	Initialization point for transfer learning, allowing the model to leverage general feature detection knowledge.
Video Recording System	RGB camera (e.g., 25 fps, 640x480) [42]	Captures raw behavioral data for subsequent frame extraction and analysis.
Computer with GPU	NVIDIA GPU with CUDA support [5]	Accelerates the model training and video analysis processes, reducing computation time from days to hours.
Labeled Training Dataset	50-200 frames per project, labeled via GUI [41]	The curated set of images with human-annotated keypoints used to teach the network what to track.
Ground Truth Validation System	Force platforms, manual scoring by human raters [41] [42]	Provides objective, reference data against which the accuracy of the pose estimation outputs is measured.
Acetylsventenic acid	Acetylsventenic acid, MF:C22H32O4, MW:360.5 g/mol	Chemical Reagent
Poricoic Acid G	Poricoic Acid G, MF:C30H46O5, MW:486.7 g/mol	Chemical Reagent

Application Notes

The application of trained DeepLabCut (DLC) models for pose tracking in new experimental videos represents a critical phase in the pipeline for high-throughput, quantitative behavioral analysis. This process enables researchers to extract markerless pose estimation data across species and experimental conditions, facilitating the study of everything from fundamental neuroscience to pharmacological interventions [41] [46]. When a model trained on a representative set of labeled frames is applied to novel video data, it estimates the positions of user-defined body parts in each frame, generating a dataset of temporal postural dynamics. The validity of this approach is underscored by studies showing that DLC-derived measurements for behaviors like grooming duration can correlate well with, and show no significant difference from, manual scoring by human experts [41]. The integration of pose tracking with specialized software like Simple Behavioral Analysis (SimBA) further allows for the classification of complex behavioral phenotypes based on the extracted keypoint trajectories [41].

Successful application of a trained model hinges on several factors. The new video data should closely match the training data in terms of animal species, camera perspective, lighting conditions, and background context to ensure optimal model generalizability [47]. Furthermore, the process can be integrated with other systems, such as anTraX, for pose-tracking individually identified animals within large groups, enhancing the scope of analysis in social behavior studies [48].

Experimental Protocols

Protocol: Applying a Trained DeepLabCut Model to Novel Videos

This protocol details the steps for using a previously trained DeepLabCut model to analyze new experimental videos, from data preparation to the visualization of results.

Pre-requisites:

• A trained and evaluated DeepLabCut model that has achieved satisfactory performance on a test set.
• New video files for analysis in a supported format (e.g., .avi, .mp4, .mov).

Procedure:

• Video Preparation and Project Configuration:

  • Ensure the new videos are in a directory accessible by your DeepLabCut environment.
  • Open your DeepLabCut project using the GUI by starting the environment and launching DLC (python -m deeplabcut), then loading your existing project [47].
  • If the new videos are from a similar experimental setup as the training data, they can be added directly to the project for analysis.

• Pose Estimation Analysis:

  • Navigate to the "Analyze videos" tab within the DeepLabCut GUI.
  • Select the new video files you wish to analyze.
  • Choose the correct trained model and shuffle value (typically 1) from the dropdown menus.
  • Adjust the cropping parameters if needed, which can speed up analysis and improve accuracy for certain videos [37] [47].
  • Click "Analyze Videos" to initiate the pose estimation process. This step uses the trained neural network to predict the location of each defined body part in every frame of the new video [47]. The processing time depends on the video length, hardware (GPU is recommended), and model complexity.

• Post-processing and Result Visualization:

  • Once analysis is complete, navigate to the "Create labeled video" tab.
  • Select the analyzed video and configure the plotting options (e.g., displaying trails, skeleton lines, point coloring).
  • Click "Create Video" to generate a new video file with the predicted body parts overlaid on the original frames. This visual inspection is crucial for a qualitative assessment of the tracking accuracy [47].
  • The precise coordinate data for all keypoints, along with the confidence scores for each prediction, are saved in a structured file (e.g., an HDF5 file) within the project directory for further quantitative analysis.

Protocol: Integrating anTraX for Individual Animal Pose Tracking

For experiments involving multiple, identical-looking animals, anTraX can be used in conjunction with DeepLabCut to track individuals and their poses over time [48].

Pre-requisites:

• An anTraX-tracked experiment.
• A DeepLabCut model trained on single-animal images exported from anTraX.

Procedure:

• Run the Trained DLC Model within anTraX:

  • Use the command-line interface to execute the trained DLC model on the anTraX session data.
  • Command: antrax dlc <experiment_directory> --cfg <path_to_dlc_config_file> [48].
  • This command processes the cropped single-animal tracklets generated by anTraX through the DeepLabCut model.

• Load and Analyze Postural Data:

  • The pose tracking results are saved and can be loaded into the Python environment for analysis using the axAntData object from the antrax module.
  • Key commands include:

  • This integration allows for the combined analysis of an animal's identity, position, and fine-scale posture [48].

Data Presentation

Table 1: Key Performance Metrics from a Comparative Study of Behavioral Analysis Pipelines (Adapted from [41])

Analysis Method	Measured Behavior	Comparison to Manual Scoring	Key Findings
DeepLabCut/SimBA	Grooming Duration	No significant difference	High correlation with manual scoring; suitable for high-throughput duration measurement.
DeepLabCut/SimBA	Grooming Bouts	Significantly different	Did not reliably estimate bout numbers obtained via manual scoring.
HomeCageScan (HCS)	Grooming Duration	Significantly elevated	Tended to overestimate duration, particularly at low levels of grooming.
HomeCageScan (HCS)	Grooming Bouts	Significantly different	Reliability of bout measurement depended on treatment condition.

Table 2: Summary of the SpaceAnimal Dataset for Benchmarking Pose Estimation in Complex Environments [7]

Animal Species	Number of Annotated Frames	Number of Instances	Key Points per Individual	Primary Annotation Details
C. elegans	~7,000	>15,000	5	Detection boxes, key points, target IDs
Zebrafish	560	~2,200	10	Detection boxes, key points, target IDs
Drosophila	>410	~4,400	26	Detection boxes, key points, target IDs

The Scientist's Toolkit

Table 3: Essential Research Reagents and Computational Tools for DeepLabCut Pose Tracking

Item Name	Function/Application in the Protocol
DeepLabCut	Open-source toolbox for markerless pose estimation of user-defined body parts using deep learning [49] [41].
anTraX	Software for tracking individual animals in large groups; integrates with DLC for individual pose tracking [48].
Simple Behavioral Analysis (SimBA)	Open-source software used downstream of DLC to classify complex behavioral phenotypes from pose estimation data [41].
Labelme	Image annotation tool used for creating ground truth data by labeling bounding boxes and key points [7].
SpaceAnimal Dataset	A benchmark dataset for developing and evaluating pose estimation and tracking algorithms for animals in space and complex environments [7].
Phyllostadimer A	Phyllostadimer A, MF:C42H50O16, MW:810.8 g/mol
Pseudolaric Acid C2	Pseudolaric Acid C2, MF:C22H26O8, MW:418.4 g/mol

Workflow Visualization

Workflow for Analyzing New Videos with a Trained DLC Model

anTraX and DLC Integration Workflow

Multi-animal pose estimation represents a significant computational challenge in behavioral neuroscience and psychopharmacology. Frequent interactions cause occlusions and complicate the association of detected keypoints to correct individuals, with animals often appearing more similar and interacting more closely than in typical multi-human scenarios [50] [26]. DeepLabCut (DLC) has been extended to provide high-performance solutions for these challenges through multi-animal pose estimation, identification, and tracking (maDLC) [50] [26]. This framework enables researchers to quantitatively study social behaviors, repetitive behavior patterns, and their pharmacological modulation with unprecedented resolution [41] [51]. This article details the technical protocols and application notes for implementing maDLC in a research setting, providing benchmarks and methodological guidelines for scientists in behavioral research and drug development.

Core Computational Challenges and the maDLC Framework

The maDLC pipeline decomposes the complex problem of tracking multiple animals into three fundamental subtasks: pose estimation (keypoint localization), assembly (grouping keypoints into distinct individuals), and tracking (maintaining individual identities across frames) [50] [26]. Each step presents distinct challenges that maDLC addresses through an integrated framework.

Pose Estimation: Accurate keypoint detection amidst occlusions requires training on frames with closely interacting animals. maDLC utilizes multi-task convolutional neural networks (CNNs) that predict score maps for keypoint locations, location refinement fields to mitigate quantization errors, and part affinity fields (PAFs) to learn associations between body parts [50] [26].

Animal Assembly: Grouping detected keypoints into individuals necessitates a method to determine which body parts belong to the same animal. maDLC introduces a data-driven skeleton finding approach that eliminates the need for manually designed skeletal connections. The network learns all possible edges between keypoints during training, and the least discriminative connections are automatically pruned at test time to form an optimal skeleton for assembly [50].

Tracking and Identification: Maintaining identity during occlusions or when animals leave the frame is crucial for behavioral analysis. maDLC incorporates a tracking module that treats the problem as a network flow optimization, aiming to find globally optimal solutions. Furthermore, it includes unsupervised animal re-identification (reID) capability that uses visual features to re-link animals across temporal gaps when tracking based solely on temporal proximity fails [50] [26].

Table 1: Benchmark Performance of maDLC on Diverse Datasets

Dataset	Individuals	Keypoints	Median Test Error (pixels)	Assembly Purity
Tri-mouse	3	12	2.65	Significant improvement with automatic skeleton pruning [50]
Parenting	2 (+1 unique)	5 (+12)	5.25	Data not available in sources
Marmoset	2	15	4.59	Significant improvement with automatic skeleton pruning [50]
Fish School	14	5	2.72	Significant improvement with automatic skeleton pruning [50]

Experimental Protocols and Workflow

Project Configuration and Data Preparation

The initial setup requires creating a properly configured multi-animal DeepLabCut project. This is achieved through the create_new_project function with the multianimal parameter set to True [40]. The project directory will contain several key subdirectories: dlc-models for storing trained model weights, labeled-data for extracted frames and annotations, training-datasets for formatted training data, and videos for source materials [40].

Critical configuration occurs in the config.yaml file, where users must define the bodyparts list specifying all keypoints to be tracked. For multi-animal projects, the multianimalproject setting must be enabled, and the identity of each individual must be labeled during the annotation phase to support identification training [40].

Network Architecture Selection and Training

maDLC employs multi-task CNN architectures that simultaneously predict keypoints, limbs (PAFs), and animal identity. Supported backbones include ImageNet-pretrained ResNets, EfficientNets, and a custom multi-scale architecture (DLCRNet_ms5) that demonstrated top performance on benchmark datasets [50]. The network uses parallel deconvolution layers to generate the different output types from a shared feature extractor [50] [26].

Training requires annotation of frames with closely interacting animals to ensure robustness to occlusions. The ground truth data is used to calculate target score maps, location refinement maps, PAFs, and identity information [50]. For challenging datasets with low-resolution or low-contrast features, specific hyperparameter adjustments are recommended, including setting global_scale: 1.0 to retain original resolution and using multi-step learning rates [39] [37].

Hyperparameter Optimization for Challenging Conditions

The pose_cfg.yaml file provides access to critical training parameters that require adjustment based on dataset characteristics [39]:

• global_scale: Default is 0.8. For low-resolution images or those lacking detail, increase to 1.0 to retain maximum information [39] [37].
• batch_size: Default is 8 for maDLC. This can be increased within GPU memory limits to improve generalization [39].
• pos_dist_thresh: Default is 17. This defines the window size for positive training samples and may require tuning for challenging datasets [39].
• pafwidth: Default is 20. This controls the width of the part affinity fields that learn associations between keypoints [39].
• Data Augmentation: Parameters like scale_jitter_lo (default: 0.5) and scale_jitter_up (default: 1.25) should be adjusted if animals vary significantly in size. rotation (default: 25) helps with viewpoint variation [39].

Diagram 1: maDLC Workflow - Key steps in multi-animal pose estimation.

Validation and Benchmarking

Performance Metrics and Benchmark Datasets

The maDLC framework was validated on four publicly available datasets of varying complexity (tri-mice, parenting mice, marmosets, and fish schools), which serve as benchmarks for future algorithm development [50] [26]. Performance is evaluated through:

• Keypoint Detection Accuracy: Measured as root-mean-square error (r.m.s.e.) between predictions and ground truth. DLCRNet_ms5 achieved median errors of 2.65-5.25 pixels across datasets, with 93.6 ± 6.9% of predictions within acceptable normalized range [50].
• Assembly Purity: The fraction of keypoints grouped correctly per individual. maDLC's data-driven skeleton pruning significantly outperformed naive skeleton definitions across all datasets [50].
• Part Affinity Field Discrimination: Measured by area under the ROC curve (auROC), with PAFs achieving near-perfect discrimination (0.99 ± 0.02) between correct and incorrect keypoint associations [50].

Comparative Validation in Behavioral Pharmacology

In a comparative study measuring repetitive self-grooming in mice, DeepLabCut with Simple Behavioral Analysis (SimBA) provided duration measurements that did not significantly differ from manual scoring, while HomeCageScan (HCS) tended to overestimate duration, particularly at low grooming levels [41]. However, both automated systems showed limitations in accurately quantifying the number of grooming bouts compared to manual scoring, indicating that specific behavioral parameters may require additional validation [41].

Table 2: Validation Metrics for maDLC Components

Component	Metric	Performance	Validation Method
Keypoint Detection	Root-mean-square error (pixels)	2.65 (tri-mouse) to 5.25 (parenting)	Comparison to human-annotated ground truth [50]
Part Affinity Fields	Discrimination (auROC)	0.99 ± 0.02	Ability to distinguish correct vs. incorrect keypoint pairs [50]
Animal Assembly	Purity improvement	Up to 3.0 percentage points	Comparison to baseline skeleton method [50]
Grooming Duration	Correlation with manual scoring	No significant difference	Comparison to human scoring in pharmacological study [41]

The Scientist's Toolkit: Essential Research Reagents

Table 3: Key Research Reagent Solutions for Multi-Animal Pose Estimation

Reagent / Tool	Function / Application	Specifications
DeepLabCut with maDLC	Primary framework for multi-animal pose estimation, identification, and tracking	Open-source Python toolbox; requires GPU for efficient training [50] [40]
Graphical User Interface (GUI)	Annotation of training frames, trajectory verification, and result refinement	Integrated into DeepLabCut for accessible data labeling and analysis [50] [40]
Simple Behavioral Analysis (SimBA)	Behavioral classification from pose estimation data	Downstream analysis tool for identifying behavioral episodes from tracking data [41]
Benchmark Datasets	Validation and benchmarking of model performance	Four public datasets (mice, marmosets, fish) with varying complexity [50]
LabGym	Alternative for user-defined behavior quantification	Learning-based holistic assessment of animal behaviors [51]
Cap1-6D	Cap1-6D, MF:C43H68N10O15, MW:965.1 g/mol	Chemical Reagent
Echinotocin	Echinotocin, MF:C41H66N12O11S2, MW:967.2 g/mol	Chemical Reagent

Advanced Applications in Drug Development

The quantitative capabilities of maDLC offer significant advantages for preclinical drug development. By enabling high-resolution tracking of social interactions and repetitive behaviors in animal models, researchers can obtain objective, high-throughput behavioral metrics for evaluating therapeutic efficacy [41] [51]. Specific applications include:

• Pharmacological Studies: Automated quantification of treatment effects on social behaviors in group-housed animals, with sufficient precision to detect dose-dependent responses.
• Genetic Model Validation: Characterization of social and repetitive behavioral phenotypes in genetic models of neuropsychiatric disorders such as autism spectrum disorder and obsessive-compulsive disorder [41].
• Long-Term Behavioral Monitoring: Continuous tracking of behavioral changes throughout disease progression or therapeutic intervention in home-cage environments [50] [26].

Diagram 2: maDLC Architecture - Core components and information flow.

Expert Tips: Troubleshooting Common Issues and Optimizing Model Performance

Selecting the appropriate DeepLabCut (DLC) project mode is a critical initial decision in markerless pose estimation pipelines for animal behavior research. This guide provides a structured framework for researchers to choose between single-animal and multi-animal DeepLabCut modes based on their experimental requirements, model capabilities, and analytical objectives. The decision directly impacts data annotation strategies, computational resource allocation, model selection, and the biological interpretations possible in preclinical and drug development studies. Proper mode selection ensures optimal tracking performance while maximizing experimental efficiency and data validity in behavioral phenotyping.

Core Decision Framework

Defining Project Requirements

The choice between single-animal and multi-animal modes hinges on specific experimental parameters and research questions. Researchers must evaluate their experimental designs against the core capabilities of each DeepLabCut mode to determine the optimal approach for their behavioral tracking applications.

Table 1: Project Mode Selection Criteria

Decision Factor	Single-Animal Mode	Multi-Animal Mode
Number of Subjects	One animal per video	Two or more animals per video
Visual Distinguishability	Not applicable	Animals may be identical or visually distinct
Tracking Approach	Direct pose estimation	Pose estimation + identity tracking
Annotation Complexity	Label body parts only	Label body parts + assign individual identities
Computational Demand	Lower	Higher
Typical Applications	Single-animal behavioral assays	Social interaction studies, group behavior

When to Use Single-Animal Mode

Single-animal DeepLabCut (multianimal=False) represents the standard approach for projects involving individual subjects. This mode is recommended when:

• Videos contain only one animal whose pose needs to be estimated
• The research focuses on individual behavioral patterns rather than social interactions
• Computational resources are limited
• Researchers are new to DeepLabCut and prefer a simpler workflow
• High-throughput screening of individual animal responses to pharmacological manipulations is required

The single-animal workflow follows the established DeepLabCut pipeline: project creation, frame extraction, labeling, network training, and video analysis [14]. This approach provides robust pose estimation for individual subjects across various behavioral paradigms including reaching tasks, open-field tests, and motor performance assays commonly used in drug development pipelines.

When to Use Multi-Animal Mode

Multi-animal DeepLabCut (multianimal=True) extends capability to scenarios with multiple subjects, employing a more sophisticated four-part workflow: (1) curated annotation data, (2) pose estimation model creation, (3) spatial and temporal tracking, and (4) post-processing [13]. This mode is essential when:

• Multiple animals appear in the same video frame
• Studying social interactions, aggression, or group dynamics
• Tracking identical-looking animals that cannot be distinguished by visual features alone
• Research requires understanding how individuals within a group respond to experimental manipulations

Multi-animal mode introduces critical configuration options, particularly for identity-aware scenarios. When animals can be visually distinguished (e.g., via markings, implants, or size differences), researchers should set identity=true in the configuration file to leverage DeepLabCut's identity recognition capabilities [52] [53]. For completely identical animals, the system uses geometric relationships and temporal continuity to maintain identity tracking across frames.

Quantitative Performance Comparison

Understanding the performance characteristics of each mode enables informed decision-making for specific research applications. Performance metrics vary based on model architecture, number of keypoints, and tracking scenarios.

Table 2: Performance Comparison of DLC 3.0 Pose Estimation Models

Model Name	Type	mAP SA-Q on AP-10K	mAP SA-TVM on DLC-OpenField
top_down_resnet_50	Top-Down	54.9	93.5
top_down_resnet_101	Top-Down	55.9	94.1
top_down_hrnet_w32	Top-Down	52.5	92.4
top_down_hrnet_w48	Top-Down	55.3	93.8
rtmpose_s	Top-Down	52.9	92.9
rtmpose_m	Top-Down	55.4	94.8
rtmpose_x	Top-Down	57.6	94.5

Performance data indicates that RTMPose models generally achieve higher mean Average Precision (mAP) on both quadruped (SA-Q) and top-view mouse (SA-TVM) benchmarks, with rtmpose_x achieving the highest scores [5]. These metrics are particularly relevant for single-animal projects, while multi-animal performance depends additionally on tracking algorithms and identity management.

Experimental Protocols

Project Creation and Configuration

Single-Animal Project Initialization

For Windows users, path formatting requires specific attention: use r'C:\Users\username\Videos\video1.avi' or 'C:\\Users\\username\\Videos\\video1.avi' [14].

Multi-Animal Project Initialization

Post-creation, edit the config.yaml file to define body parts, individuals (for multi-animal), and project-specific parameters. For identity-aware multi-animal tracking, set identity: true in the configuration file [13] [53].

Annotation Strategies by Mode

Single-Animal Annotation Protocol

• Extract frames representing behavioral diversity using deeplabcut.extract_frames(config_path)
• Label body parts across frames using deeplabcut.label_frames(config_path)
• Ensure 100-200 frames with diverse postures, lighting conditions, and backgrounds for robust training [14]

Multi-Animal Annotation Protocol

• Extract frames using the same function as single-animal mode
• Label all visible body parts for all animals in each frame
• Assign consistent individual identities when animals are distinguishable
• For identical animals, assign arbitrary but consistent identities during labeling
• Include more body parts than minimally required - additional points improve occlusion handling and identity tracking [53]

Critical consideration: Multi-animal projects require labeling all instances of animals in each frame, not just a single subject. For complex social interactions with frequent occlusions, increase frame count to ensure sufficient examples of separation events.

Model Training and Evaluation

Training Dataset Creation

Create training datasets using deeplabcut.create_training_dataset(config_path). DeepLabCut supports multiple network architectures (ResNet, HRNet, RTMPose) with PyTorch backend recommended for new projects [13] [5].

Model Training

Train networks using deeplabcut.train_network(config_path). Monitor training progress via TensorBoard or PyTorch logging utilities. For multi-animal projects, focus initially on pose estimation performance before advancing to tracking evaluation.

Evaluation and Analysis

Evaluate model performance using deeplabcut.evaluate_network(config_path). Analyze videos using deeplabcut.analyze_videos(config_path, ["/path/to/video.mp4"]). For multi-animal projects, additional tracking steps assemble body parts into individuals and link identities across frames [13].

Workflow Visualization

DeepLabCut Project Mode Selection Workflow

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential Research Reagents and Computational Solutions

Item	Function/Purpose	Implementation Notes
DeepLabCut Python Package	Core pose estimation platform	Install via pip: pip install "deeplabcut[gui]" (with GUI support) or pip install "deeplabcut" (headless) [5]
NVIDIA GPU	Accelerated model training and inference	Recommended for large datasets; CPU-only operation possible but slower [52]
PyTorch Backend	Deep learning engine	Default in DLC 3.0+; improved performance and easier installation [13] [5]
Project Configuration File (config.yaml)	Stores all project parameters	Defines body parts, training parameters, and project metadata; editable via text editor [14]
Identity Recognition	Distinguishes visually unique individuals	Enable with identity: true in config.yaml for distinguishable animals [52] [53]
Multi-Camera System	3D tracking and occlusion handling	Synchronized cameras provide multiple viewpoints for complex social interactions [54]

Advanced Applications and Specialized Scenarios

Real-Time Behavioral Feedback

DeepLabCut enables real-time pose estimation for closed-loop experimental paradigms. Implementation requires optimized inference pipelines achieving latencies of 10.5ms, suitable for triggering feedback based on movement criteria (e.g., whisker positions, reaching trajectories) [55]. This capability is particularly valuable for neuromodulation studies and behavioral pharmacology in both single-animal and multi-animal contexts.

Special Case: Single Animal with Multi-Animal Mode

Researchers may employ multi-animal mode for single-animal scenarios when skeletal constraints during training would improve performance. This approach is beneficial for complex structures like hands or mouse whiskers where spatial relationships between points remain consistent. However, this method is not recommended for tracking multiple instances of similar structures (e.g., individual whiskers) as independent "individuals" - single-animal mode performs better for such scenarios [52].

Conversion Between Project Types

Existing single-animal projects can be converted to multi-animal format, allowing researchers to leverage enhanced capabilities without restarting annotation work. Dedicated conversion utilities transfer existing labeled data to multi-animal compatible formats [13].

Troubleshooting Common Challenges

Multi-Animal Tracking Failures

The "tracklets are empty" error in multi-animal projects typically indicates failure in the animal assembly process. Solutions include:

• Increasing the number of labeled body parts to provide more spatial context
• Expanding training datasets to include more occlusion examples
• Adjusting tracking parameters in the configuration file
• Verifying consistent identity labeling across frames for distinguishable animals [56]

Adding Body Parts to Existing Projects

Appending new body parts to previously labeled datasets requires specific procedures beyond simply editing the configuration file. After adding body parts to bodyparts: in config.yaml, researchers must relabel frames to include the new points, as the labeling interface won't automatically show newly added body parts without proper dataset refreshing [57].

Alternative Tracking Approaches

For scenarios requiring only center-point tracking without detailed pose estimation (e.g., tracking animal positions without postural details), object detection models like YOLO combined with tracking algorithms such as SORT may outperform DeepLabCut, particularly for very similar-looking objects [56].

In the field of animal behavior research using DEEPLabCut (DLC) pose estimation, the principle of "Garbage In, Garbage Out" is paramount [58]. The performance of any pose estimation model is fundamentally constrained by the quality of its training data. For researchers and drug development professionals, this translates to a critical dependency: the reliability of behavioral insights derived from DLC models is directly proportional to the quality of the annotated data used for training. Errors in labeled data, such as inaccurate landmarks, missing labels, or misidentified individuals, propagate through the analysis pipeline, potentially compromising experimental conclusions and drug efficacy assessments [59]. This application note provides a structured framework for evaluating and enhancing labeled dataset quality within DLC projects, complete with quantitative assessment protocols and practical refinement workflows.

Quantitative Assessment of Data Quality

Before refining a training set, one must systematically evaluate its current state. The following table catalogs common data quality issues alongside metrics for their identification. These errors are a primary cause of model performance plateaus [59].

Table 1: Common Labeled Data Errors and Quantitative Assessment Metrics

Error Type	Description	Potential Impact on Model	Quantitative Detection Metric
Inaccurate Labels [59]	Loosely drawn or misaligned landmarks (e.g., bounding boxes, keypoints).	Reduced precision in pose estimation; inability to track subtle movements.	Measure the deviation (in pixels) from the ideal landmark location.
Mislabeled Images [59]	Application of an incorrect label to an object (e.g., labeling a "paw" as a "tail").	Introduction of semantic confusion, severely degrading classification accuracy.	Count of images where annotated labels do not match the ground truth visual content.
Missing Labels [59]	Failure to annotate all relevant objects or keypoints in an image or video frame.	Model learns an incomplete representation of the animal's posture.	Percentage of frames with absent annotations for required body parts.
Unbalanced Data [59]	Over-representation of certain poses, viewpoints, or individuals, leading to bias.	Poor generalization to under-represented scenarios or animal morphologies.	Statistical analysis (e.g., Chi-square) of label distribution across categories.

Research from MIT suggests that even in best-practice datasets, an average of 3.4% of labels can be incorrect [59]. Establishing a baseline error rate is, therefore, a crucial first step in the refinement process.

When to Refine Your Training Set

Refinement is not a one-time task but an iterative component of the model development lifecycle. Key triggers for refining your DLC training set include:

• Performance Plateau: When model accuracy, precision, or recall metrics stop improving on a validation set despite continued training, the model may have learned all it can from the current data, including its noise and biases [59].
• Poor Generalization: If a model performs well on its training data but fails on new, out-of-domain data (e.g., a different species, lighting condition, or camera angle), the training set likely lacks sufficient diversity or contains domain-specific artifacts [60].
• Introduction of New Edge Cases: Incorporating data from new experimental conditions, animal species, or unexpected behaviors necessitates adding and labeling these edge cases to maintain model robustness [59].

Experimental Protocols for Data Refinement

Protocol 1: Quality Assurance and Error Identification

This protocol outlines a method for proactively identifying poorly labeled data before it impedes model training.

• Objective: To systematically find and flag the types of errors described in Table 1 within a labeled DLC dataset.
• Materials: A curated set of labeled images or videos, DLC project configuration file, and a tool for quality control such as Encord Active [59].
• Methodology:
  • Step 1: After the initial labeling phase (whether manual or automated), export the labeled-data for review.
  • Step 2: Leverage an open-source active learning framework like Encord Active to programmatically scan the dataset. These tools can calculate metrics related to label ambiguity, image similarity, and potential outliers [59].
  • Step 3: Manually review a statistically significant sample of the data, with a focus on the examples flagged by the automated tool as potential errors. For multi-animal projects, pay special attention to identity switches and occluded body parts.
  • Step 4: Quantify the error rates for each error type and prioritize the most prevalent issues.
• Expected Output: A curated list of images/frames requiring re-annotation, accompanied by a quantitative report on label quality.

Protocol 2: Iterative Labeling with Semi-Supervised Learning

This protocol uses Semi-Supervised Learning (SSL) to efficiently expand your training set with minimal manual effort, which is particularly useful for scaling up multi-animal projects [58].

• Objective: To leverage a small, manually labeled dataset to generate high-confidence proxy labels for a larger pool of unlabeled data.
• Materials: A small "bootstrap" set of accurately labeled data, a large corpus of unlabeled video data, and computational resources for training.
• Methodology:
  • Step 1: Train an initial DLC pose estimation model on the small, high-quality bootstrap set.
  • Step 2: Use this model to perform inference on the unlabeled data, generating proxy labels [58].
  • Step 3: Apply a confidence threshold (e.g., p-value > 0.9) to filter the predictions. Only the highest-confidence proxy labels are added to the training set [58].
  • Step 4: Re-train the model on the expanded training set. This process can be repeated until no more data satisfies the confidence criteria or the desired accuracy is achieved.
• Expected Output: A significantly larger, high-quality training dataset with a fraction of the manual labeling effort.

Workflow Visualization for Data Refinement

The following diagram illustrates the integrated cyclical process of assessing and refining a training set within a DLC project, incorporating the protocols outlined above.

The Scientist's Toolkit: Essential Research Reagents and Solutions

The following table details key software and methodological solutions essential for implementing an effective data refinement strategy.

Table 2: Key Research Reagent Solutions for Data Refinement

Item Name	Function/Benefit	Use Case in DLC Context
DeepLabCut (DLC) [13]	An open-source platform for markerless pose estimation of animals.	The core framework for building, training, and deploying pose estimation models on user-defined behaviors.
Semi-Supervised Learning (SSL) [58]	A machine learning technique that uses a small amount of labeled data and a large amount of unlabeled data.	Efficiently scaling up training sets by generating proxy labels for unlabeled frames, reducing manual annotation costs.
Active Learning Frameworks [59]	Tools that help identify the most valuable data points to label or the most likely errors in a dataset.	Pinpointing mislabeled images or under-represented edge cases in a DLC project to optimize labeling effort.
Dynamic Automatic Conflict Resolution (DACR) [61]	A methodology for resolving inconsistencies in human-labeled data without a ground truth dataset.	Improving the consistency and accuracy of human-generated labels by resolving annotation conflicts in multi-annotator settings.
Complex Ontological Structures [59]	A defined set of concepts and the relationships between them, used to structure labels.	Providing clear, hierarchical definitions for labeling complex multi-animal interactions or composite body parts in DLC.

For researchers relying on DEEPLabCut, the journey to a robust and reproducible model is iterative. A disciplined approach to training set refinement—knowing when to employ quality assurance protocols and how to leverage techniques like semi-supervised learning—is not merely a technical step but a scientific necessity. By systematically implementing the assessment and refinement strategies outlined in this document, scientists can ensure their pose estimation models produce high-fidelity behavioral data, thereby strengthening the validity of downstream analyses and accelerating discovery in neuroscience and drug development.

The DeepLabCut Model Zoo represents a paradigm shift in animal pose estimation, providing researchers with access to high-performance, pre-trained models that eliminate the need for extensive manual labeling and training. This application note details the architecture, implementation, and practical application of these foundation models within the context of behavioral research and drug development. We provide structured protocols for leveraging SuperAnimal models for zero-shot inference and transfer learning, enabling researchers to rapidly deploy state-of-the-art pose estimation across diverse experimental conditions.

The DeepLabCut Model Zoo, established in 2020 and significantly expanded with SuperAnimal Foundation Models in 2024, provides a collection of models trained on diverse, large-scale datasets [62]. This resource fundamentally transforms the approach to markerless pose estimation by offering pre-trained models that demonstrate remarkable zero-shot performance on out-of-domain data, effectively reducing the labeling burden from thousands of frames to zero for many applications [62]. For researchers in neuroscience and drug development, this capability enables rapid behavioral analysis across species and experimental conditions without the substantial time investment traditionally required for model training.

The Model Zoo serves four primary functions: (1) providing a curated collection of pre-trained models for immediate research application; (2) facilitating community contribution through crowd-sourced labeling; (3) offering no-installation access via Google Colab and browser-based interfaces; and (4) developing novel methods for combining data across laboratories, species, and keypoint definitions [62]. This infrastructure supports the growing need for reproducible, scalable behavioral analysis in preclinical studies.

Available Models and Performance Specifications

SuperAnimal Model Families

The Model Zoo hosts several specialized model families trained on distinct data domains. These SuperAnimal models form the core of the Zoo's offering, each optimized for specific research contexts [62]:

• SuperAnimal-Quadruped: Designed for diverse quadruped species including horses, dogs, sheep, rodents, and elephants. These models assume a side-view camera perspective and typically include the animal's face. They are provided in multiple architectures balancing speed and accuracy [62].

• SuperAnimal-TopViewMouse: Optimized for laboratory mice in top-view perspectives, crucial for many behavioral assays involving freely moving mice in controlled settings [62].

• SuperAnimal-Human: Adapted for human body pose estimation across various camera perspectives, environments, and activities, supporting applications in motor control studies and clinical movement analysis [62].

Model Architecture Variants

Each SuperAnimal family includes multiple model architectures to address different research needs:

Table: SuperAnimal Model Architecture Variants [62]

Model Family	Architecture	Engine	Type	Keypoints
SuperAnimal-Quadruped	HRNetW32	PyTorch	Top-down	39
SuperAnimal-Quadruped	DLCRNet	TensorFlow	Bottom-up	39
SuperAnimal-TopViewMouse	HRNetW32	PyTorch	Top-down	27
SuperAnimal-TopViewMouse	DLCRNet	TensorFlow	Bottom-up	27
SuperAnimal-Human	RTMPose_X	PyTorch	Top-down	17

Top-down models (e.g., HRNetW32) are paired with object detectors (typically ResNet50-based Faster-RCNN) that first identify animal instances before predicting keypoints, while bottom-up models (e.g., DLCRNet) predict all keypoints in an image before grouping them into individuals [62]. The choice depends on the trade-off between accuracy requirements and processing speed, with bottom-up approaches generally being faster but potentially more error-prone in crowded scenes.

Performance Benchmarks

The SuperAnimal models have demonstrated robust performance on out-of-distribution testing, making them particularly valuable for real-world research applications where laboratory conditions vary.

Table: Model Performance on Out-of-Domain Test Sets [5]

Model Name	Type	mAP SA-Q on AP-10K	mAP SA-TVM on DLC-OpenField
topdownresnet_50	Top-Down	54.9	93.5
topdownresnet_101	Top-Down	55.9	94.1
topdownhrnet_w32	Top-Down	52.5	92.4
topdownhrnet_w48	Top-Down	55.3	93.8
rtmpose_s	Top-Down	52.9	92.9
rtmpose_m	Top-Down	55.4	94.8
rtmpose_x	Top-Down	57.6	94.5

These benchmarks demonstrate that the models maintain strong performance even when applied to data not seen during training, a critical feature for research applications where animals may exhibit novel behaviors or be recorded under different conditions [5].

Installation and Setup

Software Environment Configuration

To utilize the Model Zoo, researchers must first establish a proper Python environment. The current implementation requires Python 3.10+ and supports both CPU and GPU execution, though GPU utilization significantly accelerates inference [19].

Protocol: Environment Setup

• Create and activate a new conda environment:

• Install PyTorch with appropriate CUDA support for your GPU:

• Install DeepLabCut with Model Zoo support:

• Verify GPU accessibility:

  This should return True if GPU access is properly configured [19].

Research Reagent Solutions

Table: Essential Software and Hardware Components [62] [19]

Component	Specification	Function
DeepLabCut	Version 2.3+ with PyTorch backend	Core pose estimation platform with Model Zoo access
Python Environment	Python 3.10-3.12	Execution environment for DeepLabCut pipelines
GPU (Recommended)	NVIDIA CUDA-compatible (8GB+ VRAM)	Accelerates model inference and training
Model Weights	SuperAnimal family	Pre-trained foundation models for various species
Video Data	Standard formats (.mp4, .avi)	Input behavioral recordings for analysis

Experimental Protocols

Protocol 1: Zero-Shot Inference Using SuperAnimal Models

This protocol enables researchers to analyze novel video data without any model training, leveraging the pre-trained SuperAnimal models' generalization capabilities [62].

Procedure:

• Video Preparation: Ensure videos are properly formatted and cropped to focus on the animal of interest. For specific applications like pupil tracking, close cropping around the region of interest improves performance [63].

• Model Selection: Choose the appropriate SuperAnimal model based on species and camera perspective:

• Inference Execution:

• Spatial Pyramid Scaling (Optional): For videos where animal size differs significantly from training data, use multi-scale inference:

  This approach aggregates predictions across multiple scales to handle size variations [62].

• Video Adaptation (Optional): Enable self-supervised adaptation to reduce temporal jitter:

Protocol 2: Transfer Learning for Custom Applications

When zero-shot performance is insufficient for specific experimental conditions, transfer learning adapts the foundation models to new contexts with minimal labeled data [62].

Procedure:

• Project Creation:

• Configuration Modification: Edit the generated config.yaml file to define custom body parts matching the experimental requirements.

• Frame Extraction and Labeling:

• Transfer Learning Initialization:

• Dataset Creation and Training:

  The superanimal_transfer_learning=True parameter enables training regardless of keypoint count mismatch, while setting it to False performs fine-tuning when the body parts match the foundation model exactly [62].

Protocol 3: Model Refinement via Active Learning

For challenging datasets with consistent failure modes, this protocol implements an active learning loop to iteratively improve model performance [63].

Procedure:

• Initial Analysis:

• Outlier Frame Extraction:

• Label Refinement:

• Dataset Expansion and Retraining:

Workflow Visualization

Model Zoo Application Workflow: Decision pathway for implementing SuperAnimal models in research applications.

Troubleshooting and Optimization

Addressing Common Failure Modes

Researchers may encounter specific challenges when applying foundation models to novel data:

• Spatial Domain Shift: Occurs when video spatial resolution differs significantly from training data. Mitigation involves using the scale_list parameter to aggregate predictions across multiple resolutions, particularly important for videos larger than 1500 pixels [62].

• Pixel Statistics Domain Shift: Results from brightness or contrast variations between training and experimental videos. Enable video adaptation (video_adapt=True) to self-supervise model adjustment to new luminance conditions [62].

• Occlusion and Crowding: In multi-animal scenarios, bottom-up models may struggle with keypoint grouping. Consider switching to top-down architectures or implementing post-processing tracking algorithms [7].

Performance Optimization Strategies

• Hardware Utilization: Ensure GPU acceleration is active by verifying torch.cuda.is_available() returns True [19].

• Video Preprocessing: For large video files, consider re-encoding or cropping to reduce processing time while maintaining analysis quality [64].

• Batch Processing: Utilize the deeplabcut.analyze_videos function for efficient processing of multiple videos in sequence [65].

The DeepLabCut Model Zoo represents a significant advancement in accessible, reproducible behavioral analysis. By providing researchers with robust foundation models that require minimal customization, this resource accelerates the pace of quantitative behavioral science in both basic research and drug development contexts. The protocols outlined herein provide a comprehensive framework for implementing these tools across diverse experimental paradigms, from initial exploration to refined application-specific models. As the Model Zoo continues to expand with community contributions, its utility for cross-species behavioral analysis and translational research will further increase, solidifying its role as an essential resource in the neuroscience and drug development toolkit.

The transition from traditional "black box" methods to open, intelligent approaches is revolutionizing animal behavior analysis in neuroscience and ethology. This shift is largely driven by advances in deep learning-based pose estimation and tracking, which enable the extraction of key points and their temporal relationships from sequence images [7]. Within this technological landscape, skeleton assembly—the process of correctly grouping detected keypoints into distinct individual animals—emerges as a critical computational challenge in multi-animal tracking. The data-driven method for animal assembly represents a significant advancement that circumvents the need for arbitrary, hand-crafted skeletons by leveraging network predictions to automatically determine optimal keypoint connections [4].

Traditional approaches required researchers to manually define skeletal connections between keypoints, which introduced subjectivity and often failed to generalize across different experimental conditions or animal species. In contrast, data-driven assembly employs a method where the network is first trained to predict all possible graph edges, after which the least discriminative edges for deciding body part ownership are systematically pruned at test time [4]. This approach has demonstrated substantial performance improvements, yielding skeletons with fewer errors, higher purity (the fraction of keypoints grouped correctly per individual), and reduced numbers of missing keypoints compared to naive skeleton definitions [4].

Quantitative Benchmarks and Performance Metrics

The SpaceAnimal Dataset Benchmark

The development of robust data-driven assembly methods depends on high-quality annotated datasets. The SpaceAnimal Dataset serves as the first public benchmark for multi-animal behavior analysis in complex scenarios, featuring model organisms including Caenorhabditis elegans (C. elegans), Drosophila, and zebrafish [7]. This expert-validated dataset provides ground truth annotations for detection, pose estimation, and tracking tasks across these species, enabling standardized evaluation of assembly algorithms.

Table 1: SpaceAnimal Dataset Composition and Keypoint Annotations

Species	Number of Images	Total Instances	Number of Keypoints	Keypoint Purpose
C. elegans	~7,000	>15,000	5	Analysis of head/tail oscillation frequencies and movement patterns [7]
Zebrafish	560	~2,200	10	Comprehensive characterization of postures and abnormal behaviors under weightlessness [7]
Drosophila	>410	~4,400	26	Description of posture from different angles and skeleton-based behavior recognition [7]

Assembly Performance Across Species

Data-driven skeleton assembly has demonstrated significant performance improvements across multiple species and experimental conditions. Comparative analyses reveal that the automatic skeleton pruning method achieves substantially higher assembly purity compared to naive skeleton definitions, with gains of up to 3.0, 2.0, and 2.4 percentage points in tri-mouse, marmoset, and fish datasets respectively [4]. This enhancement in purity—defined as the fraction of keypoints correctly grouped per individual—is statistically significant (P<0.001 for tri-mouse and fish, P=0.002 for marmosets) and consistent across various graph sizes [4].

Table 2: Performance Comparison of Assembly Methods

Dataset	Assembly Purity (%)	Error Reduction	Statistical Significance	Processing Speed
Tri-mouse	+3.0	Fewer unconnected body parts	P<0.001	Up to 2,000 fps [4]
Marmoset	+2.0	Higher purity	P=0.002	Not specified
Fish (14 individuals)	+2.4	Reduced missing keypoints	P<0.001	≥400 fps [4]

The computational efficiency of these methods enables real-time processing, with animal assembly achieving at least 400 frames per second in dense scenes containing 14 animals, and up to 2,000 frames per second for smaller skeletons with two or three animals [4]. This balance between accuracy and efficiency makes data-driven approaches particularly suitable for long-term behavioral studies where both precision and computational tractability are essential.

Experimental Protocols for Data-Driven Assembly

Multi-Animal Project Configuration in DeepLabCut

The implementation of data-driven skeleton assembly begins with proper project configuration within the DeepLabCut ecosystem. For multi-animal projects, researchers should utilize the Project Manager GUI, which provides customized tabs specifically designed for multi-animal workflows when creating or loading projects [13].

Protocol 1: Initial Project Setup

• Launch DeepLabCut using either the terminal command python -m deeplabcut or an IPython session with import deeplabcut [13].
• Create a new multi-animal project using the create_new_project function with the multianimal=True parameter [13]:

• Specify individuals using the individuals parameter or default to ['individual1', 'individual2', 'individual3'] [13].
• Configure the project by editing the config.yaml file to define bodyparts, individuals, and the colormap for downstream steps [13].

Annotation and Training Workflow

The quality of annotations directly impacts the performance of data-driven assembly methods. The SpaceAnimal dataset construction provides a robust framework for annotation protocols [7].

Protocol 2: Frame Selection and Annotation

• Video Selection: Choose video clips representing diverse scenes, including variations in experimental environment, control group configurations, illumination conditions, developmental stages, and animal vitality [7].
• Frame Extraction: For each video, annotate the first 20 consecutive frames followed by one frame every 5 frames, though the complete dataset should consist of continuous frames to support temporal modeling [7].
• Annotation Tool: Utilize LabelMe or similar tools to annotate bounding boxes, keypoints, and assign target IDs for multiple objects in single images [7].
• Data Splitting: Divide annotated frames into training and validation sets using an 8:2 ratio with stratified random sampling to prevent data leakage and ensure evaluation reliability [7].

Protocol 3: Network Training for Assembly

• Architecture Selection: Implement multi-task convolutional neural networks that simultaneously predict score maps (keypoint localization), location refinement fields (offset quantization errors), and part affinity fields (limb connections) [4].
• Multi-scale Features: Employ architectures like DLCRNet_ms5 that incorporate multi-scale visual features to accommodate varying animal sizes and occlusion patterns [4].
• Limb Prediction: Train networks to predict part affinity fields (PAFs) that encode the location and orientation of limbs between keypoints, enabling discriminative pairing of keypoints belonging to the same animal [4].
• Data-Driven Pruning: After initial training, identify and prune the least discriminative edges based on their performance in distinguishing correct versus incorrect keypoint pairs [4].

Structure-Aware Pose Estimation Framework

Recent advances in structure-aware pose estimation offer enhanced performance for multi-animal tracking in challenging conditions, such as those encountered in space biology experiments [28].

Protocol 4: Implementing Structure-Aware Pose Estimation

• Anatomical Prior Integration: Construct species-specific pose group representations based on anatomical priors, organizing keypoints according to biological regions (e.g., head, back, wings, abdomen) [28].
• Multi-scale Feature Sampling: Implement a module that extracts fine-grained visual cues at keypoint locations across varying body sizes, enhancing spatial feature representation [28].
• Two-Hop Regression: Design a regression architecture that first predicts intermediate part points before regressing final keypoint locations, allowing the model to infer spatial relations through both direct and indirect connections [28].
• Structure-Guided Learning: Incorporate a module that captures inter-keypoint structural relationships to enhance robustness under occlusion and overlap conditions [28].

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Reagents and Computational Tools

Tool/Resource	Type	Primary Function	Application Context
DeepLabCut (maDLC)	Software Package	Multi-animal pose estimation, identification, and tracking [13] [4]	General-purpose animal behavior analysis across species
SpaceAnimal Dataset	Benchmark Data	Provides ground truth annotations for space experiment organisms [7]	Method evaluation and benchmarking for multi-animal tracking
LabelMe	Annotation Tool	Image annotation for bounding boxes, keypoints, and ID assignment [7]	Creating training data for custom pose estimation projects
DLCRNet_ms5	Neural Architecture	Multi-scale network for keypoint detection and limb prediction [4]	Handling scale variations in multi-animal scenarios
Structure-Aware Model	Algorithm Framework	Anatomical prior integration for robust pose estimation [28]	Complex scenarios with occlusion and diverse postures
Part Affinity Fields (PAFs)	Representation	Encode limb location and orientation for keypoint grouping [4]	Data-driven skeleton assembly without manual design

Advanced Implementation and Validation

Evaluation Metrics and Validation Protocols

Robust validation is essential for ensuring the reliability of data-driven assembly methods in research applications. The following protocols outline standardized evaluation approaches.

Protocol 5: Performance Validation

• Assembly Purity Assessment: Calculate the fraction of keypoints correctly grouped per individual across the test dataset [4].
• Root-Mean-Square Error: Compute pixel-level errors between detections and their closest ground truth neighbors for each frame and keypoint [4].
• Normalized Error Analysis: Express errors relative to biological benchmarks (e.g., 33% of tip-gill distance for fish, 33% of left-to-right ear distance for mice) [4].
• Temporal Consistency: Evaluate tracking consistency across frames, particularly during occlusion events and re-identification scenarios.

Integration with Downstream Analysis

The ultimate value of optimized skeleton assembly lies in its utility for downstream behavioral analysis. The structured pose data generated through these methods enables sophisticated behavioral quantification.

Protocol 6: Behavioral Feature Extraction

• Kinematic Parameter Calculation: Extract movement trajectories, speed, direction, angle, acceleration, displacement, activity level, and oscillation frequency from assembled pose sequences [7].
• Abnormal Behavior Detection: Identify behavioral anomalies through deviations from established pose sequence patterns [7].
• Social Interaction Analysis: Quantify inter-animal relationships using proximity, orientation, and movement synchronization metrics derived from assembled skeletons.
• Behavioral Distribution Profiling: Generate continuous behavioral distribution profiles to identify patterns and transitions [7].

The integration of data-driven skeleton assembly methods with advanced pose estimation frameworks creates a powerful pipeline for quantitative behavioral analysis. These protocols and resources provide researchers with a comprehensive toolkit for implementing these methods in diverse experimental contexts, from standard laboratory settings to the unique challenges of space biology research.

In the realm of animal behavior research, multi-animal pose estimation using tools like DeepLabCut (DLC) has become indispensable for neuroscience, ethology, and preclinical drug development [50] [26]. However, accurately tracking multiple interacting individuals presents significant challenges, primarily due to occlusions and the difficulty of re-identifying animals after they have been lost from tracking [50]. When animals closely interact, their body parts often become occluded, causing keypoint detection and assignment algorithms to fail. Furthermore, visually similar animals can become misidentified after periods of occlusion or when leaving the camera's field of view, compromising the integrity of behavioral data [50] [26]. These challenges are particularly prevalent in socially interacting animals, such as mice engaged in parenting behaviors or fish schooling in tanks, where close proximity and frequent contact are common [50]. This application note provides a comprehensive framework of technical solutions and detailed protocols to overcome these tracking challenges within the DeepLabCut ecosystem, enabling more robust behavioral analysis for scientific research and drug development.

Technical Solutions in DeepLabCut

DeepLabCut's multi-animal pipeline addresses occlusion and identity tracking through a multi-faceted approach that combines specialized network architectures and sophisticated algorithms. The system breaks down the tracking problem into three core steps: pose estimation (keypoint localization), assembly (grouping keypoints into distinct individuals), and tracking across frames [50] [26].

Table 1: Core Technical Solutions for Tracking Challenges in DeepLabCut

Solution Component	Primary Function	Mechanism of Action	Benefit for Occlusion/Re-ID
Part Affinity Fields (PAFs)	Animal Assembly	Predicts 2D vector fields representing limbs and orientation between keypoints [50]	Enables correct keypoint grouping during occlusions by preserving structural information [50]
Data-Driven Skeleton	Optimal Connection Discovery	Automatically identifies most discriminative keypoint connections from data; prunes weak edges [50]	Eliminates manual skeleton design; improves assembly purity during interactions [50]
Identity Prediction Network	Animal Re-identification	Predicts animal identity from visual features directly (unsupervised re-ID) [50]	Maintains identity across long occlusions/scene exits where temporal tracking fails [50]
Network Flow Optimization	Global Tracking	Frames tracking as network flow problem to find globally optimal solutions [50]	Creates consistent trajectories by stitching tracklets after occlusions [50]

The multi-task convolutional architecture is fundamental to this solution. The network doesn't merely localize keypoints; it also simultaneously predicts PAFs for limb connections and, crucially, features for animal re-identification [50]. This identity prediction capability is particularly valuable when temporal information is insufficient for tracking, such as when animals leave the camera's view or experience prolonged occlusions [50]. The network uses a data-driven method for animal assembly that finds the optimal skeleton without user input, outperforming hand-crafted skeletons by significantly enhancing assembly purity—the fraction of keypoints grouped correctly per individual [50].

Performance Quantification

The performance of these technical solutions has been rigorously validated on diverse animal datasets, demonstrating robust tracking across various challenging conditions.

Table 2: Performance Metrics of Multi-Animal DeepLabCut on Benchmark Datasets

Dataset	Animals & Keypoints	Primary Challenge	Keypoint Detection Error (pixels)	Assembly Purity / Performance Notes
Tri-Mouse	3 mice, 12 keypoints	Frequent contact and occlusion [50]	2.65 (median RMSE) [50]	Purity significantly improved with automatic skeleton pruning [50]
Parenting Mice	1 adult + 2 pups, 5-17 keypoints	pups vs. background/cotton nest [50]	5.25 (median RMSE) [50]	High discriminability of limbs (auROC: 0.99±0.02) [50]
Marmosets	2 animals, 15 keypoints	occlusion, motion blur, scale changes [50]	4.59 (median RMSE) [50]	Animal identity annotated for tracking validation [50]
Fish School	14 fish, 5 keypoints	cluttered scenes, leaving FOV [50]	2.72 (median RMSE) [50]	Processes ≥400 fps with 14 animals [50]

Beyond these benchmark results, DeepLabCut has demonstrated superior performance compared to commercial behavioral tracking systems. In studies comparing DLC-based tracking to commercial platforms like EthoVision XT14 and TSE Multi-Conditioning System, the DeepLabCut approach achieved similar or greater accuracy in tracking animals across classic behavioral tests including the open field test, elevated plus maze, and forced swim test [66]. When combined with supervised machine learning classifiers, this approach scored ethologically relevant behaviors with accuracy comparable to human annotators, while outperforming commercial solutions and eliminating variation both within and between human annotators [66].

Experimental Protocols

Data Collection and Annotation for Robust Tracking

Purpose: To create a training dataset that enables robust pose estimation and tracking under occlusion conditions.

Materials: Video recordings of animal experiments; computing system with DeepLabCut installed [5].

Procedure:

• Video Acquisition: Record multiple videos of animals interacting under various conditions. Ensure adequate resolution and frame rate to capture rapid movements and interactions [67].
• Frame Selection: Extract frames for annotation using DeepLabCut's extract_frames function. Critically, prioritize frames with closely interacting animals where occlusions frequently occur [50] [67]. For a typical project, several hundred annotated frames are required [50] (Table 2).
• Annotation:
  • Use DeepLabCut's graphical user interface (GUI) to label all visible keypoints on each animal in the selected frames [50] [26].
  • For identity-aware tracking, ensure consistent labeling of each individual animal across frames during annotation [50].
  • Pay special attention to frames where animals are partially occluded—label all visible keypoints even if some are hidden [50].
• Dataset Creation: Split the annotated frames into training (typically 70%) and test sets (typically 30%) using DLC's built-in functions [50].

Network Training for Occlusion-Robust Pose Estimation

Purpose: To train a neural network that reliably detects keypoints and predicts animal identity under challenging conditions.

Materials: Annotated dataset from Protocol 4.1; GPU-enabled computing system for efficient training [5].

Procedure:

• Network Selection: Choose an appropriate network architecture. DeepLabCut provides multiple options, with DLCRNet_ms5 demonstrating strong performance on multi-animal datasets [50].
• Configuration: In the pose_cfg.yaml file, ensure that the multi-animal parameters are properly set:
  • Set identity: True if animals are visually distinct and identity tracking is required [67]. If animals are nearly identical (e.g., same strain, no markings), set identity: False and rely on temporal tracking [67].
  • Configure Part Affinity Fields (PAFs) for limb prediction to assist with animal assembly [50].
• Training:
  • For multi-animal projects, the recommended training iterations range from 20,000 to 100,000 with a batch size of 8 [67]. If you must reduce batch size due to memory constraints, increase the number of iterations proportionally [67].
  • Utilize data augmentation techniques (random rotation, scaling, cropping) to improve model generalization.
  • Monitor training and evaluation loss to identify potential overfitting. Evaluate multiple network snapshots if necessary [67].

Video Analysis and Tracking Workflow

Purpose: To analyze new videos and generate robust trajectory data with correct identity maintenance.

Materials: Trained model from Protocol 4.2; experimental videos for analysis; computing system with DeepLabCut.

Procedure:

• Video Analysis:
  • Use deeplabcut.analyze_videos to process your experimental videos with the trained model.
  • This step generates keypoint detections but does not yet assign them to consistent individual identities across frames [67].
• Tracklet Creation:
  • Run deeplabcut.convert_detections2tracklets to form initial short-track fragments (tracklets) using temporal information [50].
  • This step employs temporal coherence to link detections across consecutive frames but may break during occlusions.
• Global Tracklet Stitching:
  • Execute deeplabcut.stitch_tracklets to merge tracklets across longer sequences [50].
  • This step uses network flow optimization to find globally consistent trajectories, reconnecting identities after occlusions [50].
  • When identity=True is used, the re-identification network assists in linking tracklets of the same animal [50].
• Output:
  • After stitching, the final output is saved as an H5 file containing pose data and identity tracks [67].
  • Convert to CSV using deeplabcut.analyze_videos_converth5_to_csv if needed [67].

Trajectory Verification and Validation

Purpose: To manually verify and correct tracking results, ensuring data quality.

Materials: Analyzed videos with tracking data from Protocol 4.3; DeepLabCut GUI.

Procedure:

• Visualization: Use DeepLabCut's graphical user interfaces for trajectory verification [50]. The deeplabcut.refine_labels GUI allows visualization of tracked keypoints overlaid on video frames.
• Validation:
  • Scrub through video sequences, paying special attention to frames with occlusions or complex interactions.
  • Verify that animal identities remain consistent through these challenging periods.
• Correction:
  • If identity swaps are detected, use the refinement tools to manually correct the labels.
  • These corrected trajectories can be used to retrain the model in an active learning framework, progressively improving performance [50].

Workflow Visualization

Diagram 1: Multi-animal tracking workflow with occlusion handling in DeepLabCut.

The Scientist's Toolkit

Table 3: Essential Research Reagents and Computational Tools

Item/Reagent	Specifications / Version	Function in Experiment
DeepLabCut Software	Version 2.2+ (with multi-animal support) [5]	Core pose estimation, animal assembly, and tracking platform [50] [5]
Video Recording System	High-resolution camera (≥1080p), adequate frame rate (≥30fps)	Captures raw behavioral data for analysis [66]
GPU Computing Resources	NVIDIA GPU with CUDA support [5]	Accelerates model training and video analysis [5]
Annotation Training Set	70% of labeled frames [50]	Trains the deep neural network for specific experimental conditions [50]
Annotation Test Set	30% of labeled frames [50]	Validates model performance and prevents overfitting [50]
Part Affinity Fields (PAFs)	Integrated in DeepLabCut network [50]	Encodes structural relationships between keypoints for robust assembly [50]
Identity Prediction Network	Integrated in DeepLabCut network [50]	Provides re-identification capability for maintaining individual identity [50]

Effective management of occlusions and re-identification is paramount for reliable multi-animal tracking in behavioral research. DeepLabCut addresses these challenges through an integrated approach combining data-driven assembly with PAFs, identity prediction networks, and global optimization for tracklet stitching. The protocols outlined herein provide researchers with a comprehensive framework for implementing these solutions across diverse experimental conditions, from socially interacting rodents to schooling fish. By rigorously applying these methods, scientists can generate high-quality trajectory data essential for robust behavioral analysis in neuroscience research and preclinical drug development.

Validating Your Tool: DeepLabCut Accuracy vs. Commercial Systems and Human Raters

The adoption of deep-learning-powered, marker-less pose-estimation has transformed the quantitative analysis of animal behavior, enabling the detection of subtle micro-behaviors with human-level accuracy [1]. Tools like DeepLabCut (DLC) allow researchers to track key anatomical points from video footage without physical markers, providing high-resolution data on posture and movement [1] [14]. However, the advancement of these technologies necessitates robust and standardized benchmarking protocols to evaluate their performance accurately. For researchers in neuroscience and drug development, employing rigorous metrics is critical for validating tools that will be used to assess disease progression, treatment efficacy, and complex behaviors in rodent models [1] [68]. This document outlines the key metrics, experimental protocols, and reagent solutions essential for benchmarking pose-estimation accuracy within the DeepLabCut ecosystem, providing a framework for reliable and reproducible research.

Key Quantitative Metrics for Pose Estimation Evaluation

Evaluating the performance of pose-estimation models requires a multifaceted approach, assessing not just raw positional accuracy but also the quality of predicted postures. The metrics below form the core of a comprehensive benchmarking strategy. They are officially utilized in the DeepLabCut benchmark suite [69].

Table 1: Core Metrics for Evaluating Pose Estimation Accuracy

Metric Name	Definition	Interpretation and Clinical Relevance
Root Mean Square Error (RMSE)	The square root of the average squared differences between predicted and ground-truth keypoint coordinates. Calculated as: ( \sqrt{\frac{1}{n} \sum{i=1}^{n} (x{i,pred} - x{i,true})^2 + (y{i,pred} - y_{i,true})^2 } ) [69].	A lower RMSE indicates higher precision in keypoint localization. Essential for detecting subtle gait changes in neurodegenerative models like Parkinson's disease [68].
Mean Average Precision (mAP)	The mean of the Average Precision (AP) across all keypoints. AP summarizes the precision-recall curve for a keypoint detection task, often using Object Keypoint Similarity (OKS) as a similarity measure [69].	A higher mAP (closer to 1.0) indicates better overall model performance in correctly identifying and localizing all body parts, even under occlusion. Critical for social behavior analysis [1].
Object Keypoint Similarity (OKS)	A normalized metric that measures the similarity between a predicted set of keypoints and the ground truth. It accounts for the scale of the object and the perceived uncertainty of each keypoint [69].	Serves as the basis for calculating mAP. Allows for a fair comparison across animals and videos of different sizes and resolutions.
Pose RMSE	A variant of RMSE that is computed after aligning the predicted pose to the ground-truth pose via translation and rotation, minimizing the overall error [69].	Focuses on the accuracy of the entire posture configuration rather than individual keypoints. Important for classifying overall body poses and identifying behavioral states.

Experimental Protocol for Benchmarking DeepLabCut Models

This protocol provides a step-by-step methodology for evaluating the performance of a DeepLabCut pose-estimation model on a new dataset, ensuring the assessment is standardized, reproducible, and clinically relevant.

Phase 1: Preparation of Benchmark Dataset

Objective: To create a high-quality, annotated dataset that reflects the biological variability and experimental conditions relevant to your research question.

• Video Selection: Select a representative set of videos that capture the breadth of behaviors, lighting conditions, animal identities, and camera angles your model is expected to encounter. For robust performance, the benchmark set should include data from different behavioral sessions and animals [14].
• Frame Extraction: Use the deeplabcut.extract_frames function to sample frames from the selected videos. A diverse training dataset should consist of a sufficient number of frames (e.g., 100-200 for simpler behaviors, but more may be needed for complex contexts) that capture the full posture repertoire [14].
• Expert Annotation: Manually label the anatomical keypoints on the extracted frames using the DeepLabCut GUI. Consistent and accurate annotation is critical, as this ground truth data is the benchmark for all subsequent evaluations. Alternatively, for data annotated outside DLC, use deeplabcut.convertcsv2h5 to import the coordinates into the correct format [70].
• Dataset Splitting: Divide the annotated dataset into training and test sets. A typical split is 90% for training and 10% for testing, ensuring that frames from the same video are not spread across both sets to prevent data leakage and overfitting.

Phase 2: Model Training & Prediction

Objective: To train a DeepLabCut model and generate pose predictions on the held-out test set.

• Configure Project: Ensure the config.yaml file is correctly set up with the list of bodyparts, and the training parameters (e.g., number of iterations, network architecture) are defined [14].
• Create Training Dataset: Run deeplabcut.create_training_dataset to generate the network-ready training data from the annotated frames.
• Model Training: Train the network using deeplabcut.train_network. Monitor the training loss to ensure convergence.
• Evaluate on Test Set: Use deeplabcut.evaluate_network to generate predictions for all the frames in the test set. This function will output a file containing the predicted keypoint coordinates for the test images.

Phase 3: Metric Calculation and Analysis

Objective: To quantitatively assess model performance by comparing predictions against the ground truth.

• Run Official Benchmark Metrics: Utilize the high-level API from the DeepLabCut benchmark package to compute the standard metrics. The following code can be executed in an IPython environment after installing the benchmark tools [69]:

• Calculate mAP: The calc_map_from_obj function will be called internally during evaluation. It uses the OKS to compute the mean Average Precision, providing a single-figure metric for model quality [69].

• Calculate RMSE: The calc_rmse_from_obj function calculates the Root Mean Square Error for each keypoint, giving insight into the localization accuracy of specific body parts [69].
• Result Interpretation: Analyze the results from the previous step.
  • High RMSE/Low mAP: Indicates potential issues such as insufficient training data, lack of diversity in the training set, or a need for model architecture adjustment. Focus on keypoints with the highest error for targeted refinement.
  • Benchmark Comparison: Compare your model's metrics against the official leaderboards for standard benchmarks like Trimouse or Marmoset to gauge its performance relative to the state-of-the-art [69].

The following workflow diagram summarizes the entire benchmarking protocol.

The Scientist's Toolkit: Research Reagent Solutions

Successful benchmarking and deployment of pose-estimation models rely on a suite of computational and experimental "reagents." The following table details these essential components.

Table 2: Essential Research Reagents for Pose-Estimation Benchmarking

Item Name	Function in Benchmarking	Specification and Notes
DeepLabCut (DLC)	The core software framework for markerless pose estimation of animals. Provides the entire workflow from data management and model training to evaluation [14].	Available via pip or conda. Choose between TensorFlow or PyTorch backends. The project configuration file (config.yaml) is the central control point.
Standard Benchmark Datasets	Pre-defined datasets with ground-truth annotations that serve as a universal reference for comparing model performance and tracking progress in the field [69].	Examples include the TrimouseBenchmark (3 mice, top-view) and MarmosetBenchmark (2 marmosets). Using these allows for direct comparison on the official DLC leaderboard.
DLC Benchmark Package	A specialized Python package containing the code to run standardized evaluations and compute key metrics like RMSE and mAP in a consistent manner [69].	Import as deeplabcut.benchmark. Contains functions like evaluate(), calc_rmse_from_obj(), and calc_map_from_obj().
High-Quality Video Data	The raw input from which frames are extracted and keypoints are predicted. The quality and diversity of this data directly determine the real-world applicability of the model [1].	Should be high-resolution with minimal motion blur. Must encompass the full range of behaviors, animal postures, and lighting conditions relevant to the biological question.
Computational Environment	The hardware and software infrastructure required to run computationally intensive deep learning models for both training and inference.	Requires a modern GPU (e.g., NVIDIA CUDA-compatible) for efficient training. Adequate storage is needed for large video files and extracted data [14].
Expert-Annotated Ground Truth	A set of frames where keypoint locations have been manually and precisely labeled by a human expert. This is the "gold standard" against which all model predictions are measured.	Can be created within the DLC GUI or imported from other sources using the convertcsv2h5 utility [70]. Accuracy is paramount for meaningful benchmark results.

Preclinical research relies heavily on the precise analysis of animal behavior to study brain function and assess treatment efficacy. For decades, the gold standard for quantifying ethologically relevant behaviors has been manual scoring by trained human annotators. However, this method is plagued by high time costs, subjective bias, and significant inter-rater variability, limiting scalability and reproducibility [66]. The emergence of deep-learning-based markerless pose estimation tools, particularly DeepLabCut (DLC), promises to overcome these limitations. This application note synthesizes evidence from rigorous studies demonstrating that DeepLabCut, when combined with supervised machine learning, does not merely approximate but can achieve and exceed the accuracy of human annotation in scoring complex behaviors, thereby establishing a new benchmark for behavioral analysis in neuroscience and drug development [66] [41].

Performance Comparison: DeepLabCut vs. Commercial Systems & Human Raters

Quantitative validation is crucial for adopting any new methodology. Comparative studies have systematically evaluated DeepLabCut against commercial tracking systems and human annotators across classic behavioral tests.

Table 1: Performance Comparison of DeepLabCut vs. Commercial Systems and Human Annotation

Behavioral Test	Metric	Commercial Systems (e.g., EthoVision, TSE)	DeepLabCut + Machine Learning	Human Annotation (Gold Standard)
Open Field Test (OFT)	Supported Rearing Detection	Poor sensitivity [66]	Similar or greater accuracy than commercial systems [66]	High accuracy, but variable
Elevated Plus Maze (EPM)	Head Dipping Detection	Poor sensitivity [66]	Similar or greater accuracy than commercial systems [66]	High accuracy, but variable
Forced Swim Test (FST)	Floating Detection	Poor sensitivity [66]	Similar or greater accuracy than commercial systems [66]	High accuracy, but variable
Self-Grooming Assay	Grooming Duration	Overestimation at low levels (HCS) [41]	No significant difference from manual scoring [41]	Gold Standard
Self-Grooming Assay	Grooming Bout Count	Significant difference from manual scoring (HCS & SimBA) [41]	Significant difference from manual scoring (SimBA) [41]	Gold Standard
General Tracking	Path Tracking Accuracy	Suboptimal, lacks flexibility [66]	High precision, markerless body part tracking [66]	High accuracy, but labor-intensive

A landmark study provided a direct comparison by using a carefully annotated set of videos for the open field test, elevated plus maze, and forced swim test. The research demonstrated that a pipeline using DeepLabCut for pose estimation followed by simple post-analysis tracked animals with similar or greater accuracy than commercial systems [66]. Crucially, when the skeletal representations from DLC were integrated with manual annotations to train supervised machine learning classifiers, the approach scored ethologically relevant behaviors (such as rearing, head dipping, and floating) with accuracy comparable to humans, while eliminating variation both within and between human annotators [66].

Further validation comes from a 2024 study focusing on repetitive self-grooming in mice. The study found that for measuring total grooming duration, the DLC/SimBA pipeline showed no significant difference from manual scoring, whereas a commercial software (HomeCageScan) tended to overestimate duration. However, it is important to note that both automated systems (SimBA and HCS) showed limitations in accurately quantifying the number of discrete grooming bouts, indicating that the analysis of complex behavioral sequences remains a challenge [41].

Experimental Protocols for Validation and Application

To achieve human-level accuracy, a structured workflow from data collection to final behavioral classification is essential. The following protocol outlines the key steps for leveraging DeepLabCut in a behavioral study, based on established methodologies [66] [14] [41].

DeepLabCut Workflow for Robust Behavioral Phenotyping

Project Creation and Configuration

• Create a New Project: Use the deeplabcut.create_new_project() function in Python or the DeepLabCut GUI. Input the project name, experimenter, and paths to initial videos [14].
• Configure the Project: Edit the generated config.yaml file to define the list of bodyparts (e.g., nose, ears, paws, tailbase) to be tracked. Avoid spaces in bodypart names. This file also allows setting the colormap for all downstream steps [14].

Data Collection and Preparation

• Video Acquisition: Record videos of animals (e.g., mice) performing the behavior of interest under consistent lighting conditions. For robust model generalization, ensure the training dataset reflects the breadth of the behavior, including different postures, sessions, and animal identities if applicable [66] [14].
• Critical Consideration: A well-chosen set of 100-200 frames can be sufficient for good results, but more may be needed for complex behaviors or variable conditions [14].

Frame Selection and Labeling

• Extract Frames: Use the deeplabcut.extract_frames() function to select a representative set of frames from your videos. This can be done manually or automatically (e.g., using k-means clustering) [14].
• Label Frames: Manually annotate the bodyparts on the extracted frames using the DeepLabCut GUI. This creates the ground truth data for training. Best Practice: Have multiple annotators label the same frames to create a consolidated, high-quality training set that reduces individual rater bias [66].

Model Training and Pose Estimation

• Train the Network: Execute deeplabcut.train_network() to train the deep neural network. Training times vary based on network size and iterations. Use the provided plots to monitor training loss and determine when to stop [14].
• Evaluate the Model: Use deeplabcut.evaluate_network() to assess the model's performance on a held-out test set of frames. The model is typically suitable for analysis if it achieves a mean test error of less than 5 pixels (relative to the animal's body size) [66] [4].
• Analyze Videos: Run deeplabcut.analyze_videos() to process new videos and obtain the pose estimation data (X, Y coordinates and likelihood for each bodypart in every frame) [14].

Behavioral Classification and Validation

• Create a Time-Resolved Skeleton Representation: From the DLC-tracked coordinates, create a skeletal representation for each frame. Compute features based on distances, angles, and areas between body parts (e.g., 22 features were used in a published study [66]).
• Train a Supervised Machine Learning Classifier: Use a subset of videos manually labeled for specific behaviors (e.g., 'supported rear', 'grooming') to train a classifier (e.g., a neural network) that maps the skeletal features to behavioral labels [66].
• Validate Against Human Scoring: Compare the output of the automated pipeline (pose estimation + classifier) against manual scoring from human annotators not involved in the training process. Use metrics like accuracy, precision, and recall to quantify performance [66] [41].

Advanced Applications: Multi-Animal and Real-Time Analysis

The core DeepLabCut workflow is highly adaptable to more complex experimental paradigms.

Multi-Animal Pose Estimation and Tracking

Social behavior experiments require tracking multiple interacting animals, which introduces challenges like occlusions and identity swaps. DeepLabCut's multi-animal module (maDLC) addresses this with a comprehensive pipeline [4].

• Pose Estimation with Part Affinity Fields (PAFs): The network is trained not only to detect keypoints but also to predict "limbs" (PAFs) that encode the location and orientation of connections between body parts. This helps group keypoints into distinct individuals during close interactions [4].
• Data-Driven Animal Assembly: Instead of a hand-crafted skeleton, maDLC uses a data-driven method to automatically determine the optimal set of connections (skeleton) for assembly, improving performance and reducing user input [4].
• Identity Tracking and Re-identification: The network can also be trained to predict an animal's identity from visual features. This "re-ID" capability is crucial for re-linking identities after prolonged occlusions, a common failure point for tracking algorithms that rely solely on temporal information [4].

Table 2: The Scientist's Toolkit: Essential Research Reagents and Resources

Item / Resource	Function / Description	Example Use Case / Note
DeepLabCut Software	Open-source toolbox for markerless 2D and 3D pose estimation.	Core platform for all steps from project management to analysis. [5]
Pre-trained Models (Model Zoo)	Foundation models (e.g., SuperAnimal-Quadruped) for pose estimation without training.	Accelerates workflow; achieves good performance out-of-domain. [2] [5]
Graphical Processing Unit (GPU)	Hardware to accelerate deep learning model training and video analysis.	Essential for efficient processing of large video datasets. [71]
SimBA (Simple Behavioral Analysis)	Open-source software for building classifiers for complex behaviors from pose data.	Used post-DLC to classify behaviors like grooming. [41]
HomeCageScan (HCS)	Commercial software for automated behavioral analysis.	Used as a comparator in validation studies. [41]
Custom R/Python Scripts	For post-processing DLC coordinates and training behavioral classifiers.	Critical for creating skeletal features and custom analyses. [66]

Real-Time Closed-Loop Experiments

Beyond offline analysis, DeepLabCut has been validated for real-time applications, enabling closed-loop feedback based on animal posture. One study demonstrated tracking of individual whisker tips in mice with a latency of 10.5 ms, fast enough to trigger stimuli within the timescale of rapid sensorimotor processing [71].

• Implementation: A deep neural network is trained offline on high-speed video data. The trained network is then transferred to a real-time system that performs continuous image acquisition, position estimation, evaluation of user-defined Boolean conditions (e.g., "whisker A angle > threshold"), and trigger generation [71].
• Application: This allows for sophisticated experiments where neural stimulation or environmental changes are triggered by specific, naturalistic movements of the animal, providing a powerful tool for probing brain-behavior relationships [71].

The convergence of deep learning and behavioral science, exemplified by DeepLabCut, is transforming preclinical research. Robust experimental protocols validate that this tool is not merely an automated convenience but a means to achieve a new standard of accuracy and objectivity in behavior scoring, matching and in some aspects surpassing the traditional human gold standard. Its flexibility to be applied to diverse species and behaviors, from single animals in classic tests to complex social groups and even real-time closed-loop paradigms, makes it an indispensable asset for researchers and drug development professionals aiming to generate rigorous, reproducible, and high-throughput behavioral data.

In the field of animal behavior research, the shift from traditional observation to automated, quantitative analysis represents a significant paradigm shift. Deep learning-based pose estimation has emerged as a powerful tool, with DeepLabCut (DLC) leading this transformation by enabling markerless tracking of user-defined body parts [72]. However, established commercial systems like EthoVision XT and traditional solutions from companies like TSE Systems continue to play vital roles in research laboratories worldwide. This comparative analysis examines the technical capabilities, implementation requirements, and research applications of these systems within the context of modern behavioral neuroscience and drug development.

Each platform embodies a different approach to behavioral analysis. DeepLabCut represents the cutting edge of deep learning technology, offering unprecedented flexibility at the cost of technical complexity [5]. EthoVision XT offers a polished, integrated solution that has been widely validated across thousands of publications [73] [74]. Meanwhile, TSE Systems provides specialized hardware-software integrations for specific behavioral paradigms, though detailed technical specifications for TSE Systems were limited in the search results. Understanding their comparative strengths and limitations is essential for researchers selecting the appropriate tool for their specific experimental needs.

Technical Comparison of System Capabilities

The following tables provide a detailed comparison of the technical specifications and performance characteristics of DeepLabCut and EthoVision XT, based on current literature and manufacturer specifications. Direct technical data for TSE Systems was not available in the search results, but it is generally recognized in the field as providing integrated systems for specific behavioral tests.

Table 1: Core technical specifications and system requirements

Feature	DeepLabCut	EthoVision XT	TSE Systems
Tracking Method	Deep learning-based markerless pose estimation [5]	Deep learning & contour-based tracking [73] [74]	Information Limited
Pose Estimation	Full body point detection (user-defined) [75]	Contour-based with optional point tracking [72]	Information Limited
Multi-Animal Support	Yes (Social LEAP Estimates Animal Poses) [72]	Yes (up to 16 animals per arena) [74]	Information Limited
Species Support	Animal-agnostic (any visible features) [5]	Rodents, fish, insects [73] [74]	Information Limited
Technical Barrier	High (Python coding, GPU setup required) [72] [5]	Low (graphical user interface) [73] [74]	Information Limited
Hardware Requirements	GPU recommended for training and inference [5]	Standard computer [73]	Integrated systems

Table 2: Performance metrics and experimental flexibility

Characteristic	DeepLabCut	EthoVision XT	TSE Systems
Tracking Speed	Varies (depends on hardware) [5]	Faster than real-time [74]	Information Limited
Accuracy Validation	Comparable to manual scoring [75]	High reliability validated [73] [74]	Information Limited
Customization Level	Very high (code-based) [5]	Moderate (module-based) [72]	Information Limited
Implementation Time	Weeks (training data required) [72]	Immediate use [74]	Information Limited
Data Output	Raw coordinates, probabilities [5]	Processed metrics, statistics [73]	Information Limited
Cost Structure	Free, open-source [5]	Commercial license [72] [74]	Commercial systems

A 2023 comparative study directly analyzing obese rodent behavior found that both DeepLabCut and EthoVision XT produced "almost identical results" for basic parameters like velocity and total distance moved [75]. However, the study noted that DeepLabCut enabled the interpretation of "more complex behavior, such as rearing and leaning, in an automated manner," highlighting its superior capacity for detailed kinematic analysis [75].

Experimental Protocols and Methodologies

DeepLabCut Implementation Protocol

Protocol Title: Markerless Pose Estimation Using DeepLabCut for Rodent Behavioral Analysis

Background: DeepLabCut enables markerless tracking of user-defined body parts through transfer learning with deep neural networks. The protocol below adapts the workflow used in a 2025 gait analysis study [42] for rodent behavior analysis.

Materials and Equipment:

• RGB camera (minimum 25 fps recommended)
• GPU-enabled computer (for efficient training)
• Python environment (3.10+)
• DeepLabCut package (v2.3.2 or newer)

Procedure:

• Video Acquisition

  • Record behavioral sessions with consistent lighting
  • Ensure animals are visible throughout the sequence
  • Use recommended resolution: 640 × 480 pixels or higher [42]

• Project Setup

  • Create new project: deeplabcut.create_new_project()
  • Define body parts to track (e.g., nose, ears, limbs, tail base)
  • Select network architecture (ResNet-50/101 recommended) [42]

• Frame Extraction and Labeling

  • Extract training frames using k-means clustering (400 frames recommended) [42]
  • Manually label body parts on extracted frames
  • Create training dataset

• Model Training

  • Utilize transfer learning from pre-trained models
  • Train network for 103,000 iterations [42]
  • Evaluate network performance on held-out data

• Video Analysis

  • Analyze novel videos using trained model
  • Extract pose estimation data (X,Y coordinates and probabilities)

• Post-processing

  • Filter predictions based on likelihood
  • Correct outliers using refinement function [42]
  • Export data for statistical analysis

Troubleshooting:

• Poor tracking performance: Increase training frames and diversify examples
• Training instability: Adjust learning rate or batch size
• Runtime errors: Verify CUDA and cuDNN installations for GPU support

EthoVision XT Implementation Protocol

Protocol Title: Automated Behavioral Phenotyping Using EthoVision XT

Background: EthoVision XT provides integrated video tracking solutions for behavioral research with minimal programming requirements. The protocol below reflects the standard workflow for rodent open field testing.

Materials and Equipment:

• EthoVision XT software (any recent version)
• Compatible camera (USB or GigE)
• Standard computer system
• Behavioral apparatus (open field, plus maze, etc.)

Procedure:

• Experiment Setup

  • Launch EthoVision XT and create new experiment
  • Select appropriate template or start from scratch
  • Define arena type and size

• Animal Detection Configuration

  • Configure detection method (contrast-based, fur color, or deep learning)
  • Calibrate distance measurements
  • Set up animal identification (single or multiple animals)

• Variable Definition

  • Define zones of interest (center, periphery, etc.)
  • Select behavioral parameters (distance moved, velocity, zone time)
  • Configure data sampling rate (standard: 6-8 fps) [76]

• Data Acquisition

  • Record sessions or analyze pre-recorded videos
  • Use batch processing for multiple videos
  • Monitor tracking accuracy in real-time

• Data Analysis

  • Review automated analysis outputs
  • Generate heat maps, movement trajectories
  • Export data to Excel or other statistical packages

Troubleshooting:

• Poor detection: Adjust contrast settings or detection method
• Inaccurate zone entries: Verify arena calibration
• System performance issues: Reduce video resolution or sampling rate

Workflow Visualization

DeepLabCut Experimental Workflow

DeepLabCut Experimental Workflow: This diagram illustrates the multi-stage process for implementing DeepLabCut, highlighting the data preparation, model training, and analysis phases.

EthoVision XT Experimental Workflow

EthoVision XT Experimental Workflow: This diagram shows the streamlined workflow for EthoVision XT, emphasizing its integrated approach from setup to analysis.

Research Reagent Solutions and Essential Materials

Table 3: Essential research materials for behavioral tracking experiments

Item	Specification	Application	Considerations
Recording Camera	RGB camera, 25+ fps, 640×480+ resolution [42]	Video acquisition	Higher fps enables better movement capture
Computer System	GPU (for DLC) or standard computer (for EthoVision) [5] [74]	Data processing	GPU reduces DLC training time significantly
Behavioral Apparatus	Open field, elevated plus maze, etc.	Experimental testing	Standardized dimensions improve reproducibility
Lighting System	Consistent, uniform illumination	Video quality	Avoid shadows and reflections
Analysis Software	DeepLabCut or EthoVision XT license	Data extraction	Choice depends on technical resources
Data Storage	High-capacity storage solution	Video archiving	Raw videos require substantial space

Discussion and Research Implications

Performance Considerations for Different Research Scenarios

The choice between DeepLabCut and EthoVision XT depends significantly on the specific research requirements and available laboratory resources. For basic locomotor analysis and standardized behavioral tests, both systems demonstrate comparable performance in measuring parameters like velocity and total distance moved [75]. However, for complex behavioral phenotyping requiring detailed kinematic data, DeepLabCut offers superior capabilities in tracking specific body parts and identifying novel behavioral patterns [75].

The technical resources of a research group represent another crucial consideration. DeepLabCut requires significant computational expertise for installation, network training, and data processing [72] [5]. In contrast, EthoVision XT provides an accessible interface suitable for researchers without programming backgrounds [73] [74]. This accessibility comes at the cost of flexibility, as EthoVision XT operates as more of a "black box" with limited options for customizing tracking algorithms [74].

Emerging Applications and Future Directions

Recent advances in pose estimation have enabled applications in increasingly complex research scenarios. The SpaceAnimal Dataset, developed for analyzing animal behavior in microgravity environments aboard the China Space Station, demonstrates how deep learning approaches can extend to challenging research environments with severe occlusion and variable imaging conditions [7]. Such applications highlight the growing importance of robust pose estimation in extreme research settings.

Another emerging application is closed-loop optogenetic stimulation based on real-time pose estimation. DeepLabCut-Live enables researchers to probe state-dependent neural circuits by triggering interventions based on specific behavioral states [17]. This integration of pose estimation with neuromodulation represents a significant advancement for causal neuroscience studies.

DeepLabCut, EthoVision XT, and TSE Systems each occupy distinct niches in the behavioral research ecosystem. DeepLabCut provides unparalleled flexibility and detailed pose estimation capabilities for researchers with technical expertise and computational resources. EthoVision XT offers a validated, user-friendly solution for standardized behavioral assessment with extensive support and documentation. TSE Systems provides integrated hardware-software solutions for specific behavioral paradigms, though detailed technical information was limited in the current search results.

The selection of an appropriate tracking system should be guided by specific research questions, available technical expertise, and experimental requirements. As pose estimation technology continues to evolve, the integration of these different approaches may offer the most powerful path forward, combining the standardization of commercial systems with the flexibility of deep learning-based methods. This comparative analysis provides researchers with the necessary framework to make informed decisions about implementing these technologies in their behavioral research programs.

Within the field of animal behavior research, high-fidelity 3D pose estimation has become a cornerstone for quantifying movement, behavior, and kinematics. The markerless approach offered by DeepLabCut (DLC) provides unprecedented flexibility for analyzing natural animal movements. However, the validation of its 3D tracking accuracy remains a critical scientific challenge. Electromagnetic Tracking Systems (EMTS) offer a compelling solution, providing sub-millimeter accuracy for establishing ground truth data in controlled volumes. This application note details the methodologies and protocols for using EMT systems as a gold-standard reference to quantitatively assess the performance of 3D DeepLabCut models, thereby bolstering the reliability of pose estimation data in neuroscientific and pharmacological research.

Electromagnetic Tracking Systems: A Primer for Validation

Electromagnetic Tracking Systems (EMTS) are a form of positional sensing technology that operate by generating a controlled electromagnetic field and measuring the response from miniature sensors. Their fundamental principle makes them exceptionally suitable for validating optical systems like DeepLabCut.

Core Components and Working Principles

An EMTS typically comprises a field generator (FG) that produces a spatially varying magnetic field, and one or more sensors (often micro-coils or magnetometers) that are attached to the subject or instrument being tracked [77] [78]. The system calculates the position and orientation (6 degrees-of-freedom) of each sensor within the field volume by analyzing the induced signals [78]. Two primary technological approaches exist:

• Dynamic Field Systems (e.g., NDI Aurora): Use alternating magnetic fields at frequencies in the hundreds of kilohertz. While offering high update rates (e.g., 40 Hz), they are susceptible to conductive distortions from eddy currents induced in metallic objects [77].
• Quasi-Static Field Systems (e.g., ManaDBS): Employ sequentially activated coils generating static magnetic fields. This approach demonstrates inherent resistance to conductive distortions, though typically at lower update rates (e.g., 0.3-10 Hz) [77].

Advantages for Pose Estimation Validation

The key attributes that make EMTS valuable for validating DeepLabCut include:

• High intrinsic accuracy: Commercial systems like the NDI Aurora report localization errors of 0.5 mm and 0.3° at the center of the tracking volume [77], providing a reliable metric for comparison.
• Non-line-of-sight operation: Unlike optical motion capture, EMTS can track sensors regardless of visual occlusion, enabling validation in complex experimental setups where body parts may be temporarily hidden [78].
• Direct 3D measurement: EMTS provides inherent 3D positional data without requiring multi-camera calibration or triangulation, serving as an independent source of ground truth.

Performance Benchmarking: EMT System Capabilities

The selection of an appropriate EMT system for validation depends heavily on the specific experimental requirements. The table below summarizes the performance characteristics of representative systems as reported in the literature.

Table 1: Performance Characteristics of Representative EMT Systems

System / Characteristic	NDI Aurora V2	ManaDBS	Miniaturized System [79]
Technology	Dynamic Alternating Fields	Quasi-Static Fields	Not Specified
Reported Position Error	0.66 mm (undistorted) [77]	1.57 mm [77]	2.31 mm within test volume [79]
Reported Orientation Error	0.89° (undistorted) [77]	1.01° [77]	1.48° for rotations up to 20° [79]
Error with Distortion	Increases to 2.34 mm with stereotactic system [77]	Unaffected by stereotactic system [77]	Not Reported
Update Rate	40 Hz [77]	0.3 Hz [77]	Not Specified
Optimal Tracking Volume	50 × 50 × 50 cm³ [77]	15 × 15 × 30 cm³ [77]	320 × 320 × 76 mm³ [79]
Key Advantage	High speed, commercial availability	Robustness to EM distortions [77]	Compact size

Experimental Protocol: Cross-Validation Methodology

This protocol describes a comprehensive framework for validating 3D DeepLabCut pose estimates against an electromagnetic tracking system.

Equipment and Software Requirements

Table 2: Essential Research Reagents and Equipment

Item Category	Specific Examples	Function in Validation
EMT System	NDI Aurora, ManaDBS, or similar [77]	Provides ground truth position/orientation data
EMT Sensors	NDI flextube (1.3 mm), Custom sensors (1.8 mm) [77]	Physical markers attached to subject for tracking
Cameras	High-speed, synchronized cameras (≥2)	Capture video for DeepLabCut pose estimation
Calibration Apparatus	Custom 3D calibration board, checkerboard	Correlate EMT and camera coordinate systems
Animal Model	Mice, rats, zebrafish, Drosophila [80] [7]	Subject for behavioral tracking
Software	DeepLabCut (with 3D functionality) [14], DLC-Live! [81], Custom MATLAB/Python scripts	Data processing, analysis, and visualization

Sensor Integration and Co-localization

The foundation of accurate validation requires precise spatial correspondence between EMT sensors and DLC keypoints.

• Sensor Attachment: Securely affix miniature EMT sensors (e.g., NDI flextubes) to anatomically relevant locations on the animal subject. For larger animals, sensors can be directly attached to the skin or fur. For smaller organisms, consider miniaturized sensors or custom fixtures [77] [78].

• Visual Marker Design: Create highly visible, distinctive visual markers that are physically co-registered with each EMT sensor. These should be easily identifiable in video footage and designed for precise keypoint labeling in DeepLabCut.

• Coordinate System Alignment: Perform a rigid transformation to align the EMT coordinate system with the camera coordinate system using a custom calibration apparatus containing both EMT sensors and visual markers at known relative positions.

Data Collection and Synchronization

Precise temporal alignment is critical for meaningful comparison between systems.

• Hardware Synchronization: Implement a shared trigger signal to simultaneously initiate data collection from the EMT system and all cameras. Alternatively, use a dedicated synchronization box to generate timestamps across all devices.

• Recording Parameters: Collect data across diverse behavioral repertoires to ensure validation covers the full range of natural movements. For the EMT system, record at its maximum stable frame rate. For cameras, ensure frame rates exceed the required temporal resolution for the behavior of interest.

• Validation Dataset Curation: Extract frames representing the breadth of observed postures and movements. Ensure adequate sampling of different orientations, velocities, and potential occlusion scenarios.

Data Processing and Analysis

The following workflow outlines the core computational steps for comparative analysis.

Diagram: Computational workflow for comparing DeepLabCut and EMT data

• Trajectory Interpolation: Resample EMT and DLC trajectories to a common time base using appropriate interpolation methods (e.g., cubic spline for continuous movements).

• Coordinate System Transformation: Apply the calibration-derived transformation matrix to convert all EMT measurements into the camera coordinate system for direct comparison with DLC outputs.

• Error Metric Computation: Calculate the following key performance indicators for each matched keypoint:

  • Positional Error: Euclidean distance between DLC-predicted and EMT-measured 3D positions
  • Angular Error: For orientation comparisons (when applicable)
  • Temporal Consistency: Phase relationships between time-series of matched keypoints

• Statistical Analysis: Compute summary statistics (mean, median, standard deviation, RMS error) across all frames and keypoints. Generate Bland-Altman plots to assess agreement between systems and identify any bias related to movement speed or position within the tracking volume.

Representative Experimental Results

Implementation of this validation methodology typically yields comprehensive performance metrics for 3D DeepLabCut models.

Table 3: Sample Validation Results for Canine Gait Analysis [80]

Body Part	Mean Position Error (mm)	Notes on Performance
Nose	1.2	Well-defined morphology enabled high accuracy
Eye	1.4	Consistent visual features improved tracking
Carpal Joint	2.1	Good performance despite joint articulation
Tarsal Joint	2.3	Moderate error in high-velocity movements
Shoulder	4.7	Less morphologically discrete landmark
Hip	5.2	Challenging due to fur and skin deformation
Overall Mean	2.8	ANOVA showed significant body part effect (p=0.003)

The data demonstrates a common pattern where well-defined anatomical landmarks (nose, eyes) achieve higher tracking accuracy compared to less discrete morphological locations (shoulder, hip) [80]. This highlights the importance of careful keypoint selection during DeepLabCut model design.

Advanced Applications and Integration

Real-Time Closed-Loop Validation

The emergence of real-time pose estimation systems like DeepLabCut-Live! enables validation of dynamic behavioral interventions. This system achieves low-latency pose estimation (within 15 ms, >100 FPS) and can be integrated with a forward-prediction module that achieves effectively zero-latency feedback [81]. Such capabilities allow researchers to not only validate tracking accuracy but also assess the timing precision of closed-loop experimental paradigms.

Multi-Animal Tracking Scenarios

For social behavior studies, multi-animal pose estimation presents additional validation challenges. Approaches like vmTracking (virtual marker tracking) use labels from multi-animal DLC as "virtual markers" to enhance individual identification in crowded environments [82]. When combining this methodology with EMT validation, researchers can quantitatively assess both individual animal tracking accuracy and identity maintenance during complex interactions.

Electromagnetic tracking systems provide a rigorous, quantifiable framework for validating 3D DeepLabCut pose estimation models in animal behavior research. The methodology outlined in this application note enables researchers to establish error bounds and confidence intervals for markerless tracking data, which is particularly crucial for preclinical studies in pharmaceutical development where quantitative accuracy directly impacts experimental outcomes. As both EMT and DeepLabCut technologies continue to advance—with improvements in sensor miniaturization, distortion compensation, and computational efficiency—this cross-validation approach will remain essential for ensuring the reliability of behavioral metrics in neuroscience and drug discovery.

DeepLabCut is an open-source, deep-learning-based software toolbox designed for markerless pose estimation of user-defined body parts across various animal species, including humans [5]. Its animal- and object-agnostic framework allows researchers to track virtually any visible feature, enabling detailed quantitative analysis of behavior [5]. By leveraging state-of-the-art feature detectors and the power of transfer learning, DeepLabCut requires surprisingly little training data to achieve high precision, making it an invaluable tool for neuroscience, ethology, and drug development [5]. This case study explores how DeepLabCut's multi-animal pose estimation capabilities provide superior sensitivity for uncovering ethologically relevant behaviors in complex social and naturalistic settings.

DeepLabCut's Architecture and Performance

Multi-Animal Pose Estimation Capabilities

Expanding beyond single-animal tracking, DeepLabCut's multi-animal pose estimation pipeline addresses the significant challenges posed by occlusions, close interactions, and visual similarity between individuals [4]. The framework decomposes the problem into several computational steps: keypoint estimation (localizing body parts), animal assembly (grouping keypoints into distinct individuals), and temporal tracking (linking identities across frames) [4].

To tackle these challenges, the developers introduced multi-task convolutional neural networks that simultaneously predict:

• Score maps for keypoint localization
• Part Affinity Fields (PAFs) for associating body parts to individuals
• Animal identity embeddings for re-identification after occlusions [4]

A key innovation is the data-driven skeleton determination method, which automatically identifies the most discriminative connections between body parts for robust assembly, eliminating the need for manual skeleton design and improving assembly purity by up to 3 percentage points [4].

Quantitative Performance Benchmarks

DeepLabCut has been rigorously validated on diverse datasets, demonstrating state-of-the-art performance across species and behavioral contexts. The following tables summarize its performance on benchmark datasets:

Table 1: Multi-Animal Pose Estimation Performance on Benchmark Datasets [4]

Dataset	Animals	Keypoints	Test RMSE (pixels)	Assembly Purity (%)
Tri-Mouse	3	12	2.65	>95
Parenting	3	5 (adult), 3 (pups)	5.25	>94
Marmoset	2	15	4.59	>93
Fish School	14	5	2.72	>92

Table 2: Model Performance Comparison in DeepLabCut 3.0 [5]

Model Name	Type	mAP SA-Q on AP-10K	mAP SA-TVM on DLC-OpenField
topdownresnet_50	Top-Down	54.9	93.5
topdownresnet_101	Top-Down	55.9	94.1
topdownhrnet_w32	Top-Down	52.5	92.4
topdownhrnet_w48	Top-Down	55.3	93.8
rtmpose_m	Top-Down	55.4	94.8
rtmpose_x	Top-Down	57.6	94.5

The performance metrics demonstrate DeepLabCut's robustness across challenging conditions, including occlusions, motion blur, and scale variations [4]. The recently introduced SuperAnimal models provide exceptional out-of-distribution performance, enabling researchers to achieve high accuracy even without extensive manual labeling [5].

Experimental Protocols

Project Setup and Configuration

Protocol 1: Creating a New DeepLabCut Project

• Installation: Install DeepLabCut with the PyTorch backend in a Python 3.10+ environment:

  [5]

• Project Creation: Create a new project using either the GUI or Python API:

  [14]

• Project Configuration: Edit the generated config.yaml file to define:

  • bodyparts: List of all body parts to track
  • individuals: List of individual identifiers (for multi-animal projects)
  • uniquebodyparts: Body parts that are unique to each individual
  • identity: Whether to enable identity prediction [14]

Protocol 2: Frame Selection and Labeling

• Frame Extraction: Select representative frames across videos:

  This samples frames to capture behavioral diversity, including different postures, interactions, and lighting conditions [14].

• Manual Labeling: Label body parts in the extracted frames using the DeepLabCut GUI:

  For multi-animal projects, assign each labeled body part to the correct individual [4].

• Create Training Dataset: Generate the training dataset from labeled frames:

  This creates the training dataset with data augmentation and splits it into train/test sets [14].

Network Training and Optimization

Protocol 3: Configuring Training Parameters

The pose_cfg.yaml file controls critical training hyperparameters. Key parameters to optimize include:

• Data Augmentation: Enable and configure augmentation in pose_cfg.yaml:

  • scale_jitter_lo and scale_jitter_up (default: 0.5, 1.25): Controls scaling augmentation
  • rotation (default: 25): Maximum rotation degree for augmentation
  • fliplr (default: False): Horizontal flipping (use with symmetric poses only)
  • cropratio (default: 0.4): Percentage of frames to be cropped [39]

• Training Parameters:

  • batch_size: Increase based on GPU memory availability
  • global_scale (default: 0.8): Basic scaling applied to all images
  • pos_dist_thresh (default: 17): Window size for positive training samples
  • pafwidth (default: 20): Width of Part Affinity Fields for limb association [39]

Protocol 4: Model Training and Evaluation

• Train the Network:

  Monitor training loss until it plateaus, indicating convergence [14].

• Evaluate the Model:

  This calculates test errors and generates evaluation plots [14].

• Video Analysis:

  Run pose estimation on new videos [14].

• Refinement (Active Learning): If performance is insufficient, extract outlier frames and refine labels:

  Then create a new training dataset and retrain [14].

Visualization and Workflow

DeepLabCut Experimental Workflow

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Research Tools for DeepLabCut-Based Behavioral Analysis

Tool/Resource	Function	Application Notes
DeepLabCut Core Software	Markerless pose estimation	Available via pip install; PyTorch backend recommended for new projects [5]
SuperAnimal Models	Pre-trained foundation models	Provide out-of-domain robustness for quadrupeds and top-view mice [5]
DeepLabCut Model Zoo	Repository of pre-trained models	Enables transfer learning, reducing required training data [5]
Imgaug Library	Data augmentation	Integrated into training pipeline; enhances model generalization [83]
Active Learning Framework	Iterative model refinement	Identifies outlier frames for targeted labeling [14]
Multi-Animal Tracking Module	Identity preservation	Handles occlusions and interactions; uses PAFs and re-identification [4]
Behavioral Analysis Pipeline	Quantification of ethological behaviors	Transforms pose data into behavioral metrics [84]

Advanced Applications in Ethological Research

DeepLabCut enables researchers to address classical questions in animal behavior, framed by Tinbergen's four questions: causation, ontogeny, evolution, and function [85]. The sensitivity of multi-animal pose estimation allows for:

Social Behavior Analysis: Tracking complex interactions in parenting mice, marmoset pairs, and fish schools reveals subtle communication cues and social dynamics [4]. The system maintains individual identity even during close contact and occlusions, enabling precise quantification of approach, avoidance, and contact behaviors.

Cognitive and Learning Studies: By tracking body pose during cognitive tasks, researchers can identify behavioral correlates of decision-making and learning. The high temporal resolution captures preparatory movements and subtle postural adjustments that precede overt actions.

Drug Development Applications: In pharmaceutical research, DeepLabCut provides sensitive measures of drug effects on motor coordination, social behavior, and naturalistic patterns. The automated, high-throughput nature enables screening of therapeutic compounds with finer resolution than traditional observational methods.

DeepLabCut's multi-animal pose estimation framework provides researchers with an unprecedentedly sensitive tool for quantifying ethologically relevant behaviors. By combining state-of-the-art computer vision architectures with user-friendly interfaces, it enables precise tracking of natural behaviors in socially interacting animals. The protocols and resources outlined in this case study offer a roadmap for researchers to implement this powerful technology in their behavioral research, ultimately advancing our understanding of animal behavior in fields ranging from basic neuroscience to drug development.

Conclusion

DeepLabCut has firmly established itself as a transformative tool in behavioral neuroscience and preclinical research, enabling precise, markerless, and flexible quantification of animal posture and movement. By mastering its foundational workflow, researchers can reliably track both single and multiple animals, even in complex, socially interacting scenarios. The software's performance has been rigorously validated, matching or exceeding the accuracy of both human annotators and traditional commercial systems while unlocking the analysis of more nuanced, ethologically relevant behaviors. Looking forward, the continued development of features like unsupervised behavioral classification and the expansion of pre-trained models in the Model Zoo promise to further democratize and enhance the scale and reproducibility of behavioral phenotyping. For the biomedical research community, this translates to more powerful, cost-effective, and insightful tools for understanding brain function and evaluating therapeutic efficacy in animal models.

References

• 1. Does advancement in marker-less pose-estimation mean ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12411509/]
• 2. DeepLabCut — The Mathis Lab of Adaptive Intelligence [http://www.mackenziemathislab.org/deeplabcut]
• 3. Case Study: DeepLabCut 3D Pose Estimation [https://numpy.org/case-studies/deeplabcut-dnn/]
• 4. Multi-animal pose estimation, identification and tracking ... [https://www.nature.com/articles/s41592-022-01443-0]
• 5. Official implementation of DeepLabCut [https://github.com/DeepLabCut/DeepLabCut]
• 6. BehaviorDEPOT is a simple, flexible tool for automated ... [https://elifesciences.org/articles/74314]
• 7. Pose estimation and tracking dataset for multi-animal ... [https://www.nature.com/articles/s41597-025-05111-8]
• 8. Evaluation of unsupervised learning algorithms for the ... [https://pubmed.ncbi.nlm.nih.gov/40486967/]
• 9. DeepLabCut - The BRAIN Initiative Alliance [https://www.braininitiative.org/toolmakers/resources/deeplabcut/]
• 10. Using DeepLabCut for 3D markerless pose estimation ... [https://www.nature.com/articles/s41596-019-0176-0]
• 11. SuperAnimal pretrained pose estimation models for ... [https://www.nature.com/articles/s41467-024-48792-2]
• 12. Automatically annotated motion tracking identifies a distinct ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10354203/]
• 13. DeepLabCut for Multi-Animal Projects - GitHub Pages [https://deeplabcut.github.io/DeepLabCut/docs/maDLC_UserGuide.html]
• 14. DeepLabCut User Guide (for single animal projects) [https://deeplabcut.github.io/DeepLabCut/docs/standardDeepLabCut_UserGuide.html]
• 15. How can I select the color on GUI [https://forum.image.sc/t/how-can-i-select-the-color-on-gui/22891]
• 16. Label DeepLabCut - Python Video Annotator's Documentation [https://pythonvideoannotator.readthedocs.io/en/master/modules/label-deeplabcut.html]
• 17. Using DeepLabCut-Live to probe state dependent neural ... [https://www.sciencedirect.com/science/article/pii/S0165027025001360]
• 18. Performance on CPU vs GPU · Issue #76 [https://github.com/AlexEMG/DeepLabCut/issues/76]
• 19. How To Install DeepLabCut - GitHub Pages [https://deeplabcut.github.io/DeepLabCut/docs/installation.html]
• 20. Optimal number of graphic cards for operating deeplabcut [https://forum.image.sc/t/optimal-number-of-graphic-cards-for-operating-deeplabcut/77305]
• 21. DeepLabCut and Conda environments - Usage & Issues [https://forum.image.sc/t/deeplabcut-and-conda-environments/57580]
• 22. Installation Tips — DeepLabCut - GitHub Pages [https://deeplabcut.github.io/DeepLabCut/docs/recipes/installTips.html]
• 23. Using DeepLabCut for 3D markerless pose estimation ... [https://pubmed.ncbi.nlm.nih.gov/31227823/]
• 24. Loading projects in the terminal - deeplabcut [https://forum.image.sc/t/loading-projects-in-the-terminal/69242]
• 25. Documentation for new DeepLabCut GUI 2.3.3 [https://forum.image.sc/t/documentation-for-new-deeplabcut-gui-2-3-3/79307]
• 26. Multi-animal pose estimation, identification and tracking ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9007739/]
• 27. How does Deeplabcut deal with bodyparts that are most of ... [https://forum.image.sc/t/how-does-deeplabcut-deal-with-bodyparts-that-are-most-of-the-time-occluded/30985]
• 28. Structure-Aware Multi-Animal Pose Estimation for Space ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12606747/]
• 29. Rapid, open-source, and automated quantification of the ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12190173/]
• 30. Issue with improving extracted frames #2552 [https://github.com/DeepLabCut/DeepLabCut/issues/2552]
• 31. Data Collection and Annotation Strategies for Computer ... [https://docs.ultralytics.com/guides/data-collection-and-annotation/]
• 32. Anti-drift pose tracker (ADPT): A transformer-based network for robust ... [https://elifesciences.org/reviewed-preprints/95709]
• 33. GitHub - airhors/ Animal - Pose - Estimation -i: NeurIPS 2021 Datasets ... [https://github.com/airhors/Animal-Pose-Estimation-i]
• 34. How to extract outliers in DeepLabCut for a particular ... [https://forum.image.sc/t/how-to-extract-outliers-in-deeplabcut-for-a-particular-bodypart/99239]
• 35. adding bodyparts to label · Issue #2565 · DeepLabCut/ ... [https://github.com/DeepLabCut/DeepLabCut/issues/2565]
• 36. KeyError Question - Usage & Issues [https://forum.image.sc/t/keyerror-question/108075]
• 37. Training settings for low resolution/contrast and thin features [https://forum.image.sc/t/training-settings-for-low-resolution-contrast-and-thin-features/37941]
• 38. DLC network evaluation - what do labels mean? [https://forum.image.sc/t/dlc-network-evaluation-what-do-labels-mean/104835]
• 39. The pose_cfg.yaml Guideline Handbook — DeepLabCut [https://deeplabcut.github.io/DeepLabCut/docs/recipes/pose_cfg_file_breakdown.html]
• 40. DeepLabCut - A Student's Quick Start Guide [https://pigeonsupermodel.com/DLC_StudentGuide.html]
• 41. A comparison of machine learning methods for quantifying ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10859524/]
• 42. DeepLabCut custom-trained model and the refinement ... [https://www.nature.com/articles/s41598-025-85591-1]
• 43. What neural network should I use? (Trade offs, speed ... [https://github.com/DeepLabCut/DeepLabCut/wiki/What-neural-network-should-I-use%3F-(Trade-offs,-speed-performance,-and-considerations)]
• 44. K-means, labeling, network selection, and more [https://forum.image.sc/t/k-means-labeling-network-selection-and-more-q5-selecting-a-model/107890]
• 45. Better results with DeepLabCut==2.3.0 than ... [https://forum.image.sc/t/better-results-with-deeplabcut-2-3-0-than-deeplabcut-3-0-with-pytorch/116017]
• 46. Realtime Monitoring of Animal Behavior Using Deep Learning ... [https://www.jaeid.it/index.php/jaeid/article/view/16397]
• 47. Hands-on: DeepLabCut [https://www.fabriziomusacchio.com/teaching/teaching_assessing_animal_behavior/05_deeplabcut]
• 48. Using DeepLabCut to pose-track individuals - anTraX [https://antrax.readthedocs.io/en/latest/dlc/]
• 49. Introduction to DeepLabCut — Tracking Animal Behavior [https://guillermo-hidalgo-gadea.github.io/Seminar-TrackingAnimalBehavior/Lecture3.html]
• 50. - Multi , animal and pose with... estimation identification tracking [https://www.nature.com/articles/s41592-022-01443-0?error=cookies_not_supported&code=79d452b3-32a8-4e7d-a3a6-64e1b2ef77f6]
• 51. LabGym: Quantification of user-defined animal behaviors ... [https://www.sciencedirect.com/science/article/pii/S2667237523000267]
• 52. 🥳 Get started with DeepLabCut: our key recommendations — DeepLabCut [https://deeplabcut.github.io/DeepLabCut/docs/UseOverviewGuide.html]
• 53. multi-animal DeepLabCut — The Mathis Lab of Adaptive Intelligence [http://www.mackenziemathislab.org/deeplabcutblog/2021/4/2/multi-animal-deeplabcut-the-full-22-release]
• 54. Noninvasive Tracking of Every Individual in Unmarked ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10264345/]
• 55. Real-Time Closed-Loop Feedback in Behavioral ... [https://www.eneuro.org/content/8/2/ENEURO.0415-20.2021]
• 56. DeepLabCut Multi-Animal vs. Single-Animal Project for Point Tracking - Image Analysis - Image.sc Forum [https://forum.image.sc/t/deeplabcut-multi-animal-vs-single-animal-project-for-point-tracking/97341]
• 57. cannot add new labels to a labelled dataset · Issue #2723 [https://github.com/DeepLabCut/DeepLabCut/issues/2723]
• 58. Techniques for Labeling Data in Machine Learning [https://www.phdata.io/blog/techniques-for-labeling-data-in-machine-learning/]
• 59. 5 Ways to Improve The Quality of Labeled Data [https://encord.com/blog/improve-quality-of-labeled-data-guide/]
• 60. Deeplabcut trainbing a model on multiple species of the ... [https://forum.image.sc/t/deeplabcut-trainbing-a-model-on-multiple-species-of-the-same-animal-fish/114743]
• 61. Improving Human-Labeled Data through Dynamic ... [https://machinelearning.apple.com/research/improving-human-labeled-data]
• 62. The DeepLabCut Model Zoo! - GitHub Pages [https://deeplabcut.github.io/DeepLabCut/docs/ModelZoo.html]
• 63. Using ModelZoo models on your own datasets — DeepLabCut [https://deeplabcut.github.io/DeepLabCut/docs/recipes/UsingModelZooPupil.html]
• 64. Input/output manipulations with DeepLabCut [https://deeplabcut.github.io/DeepLabCut/docs/recipes/io.html]
• 65. Helper & Advanced Optional Function Documentation [https://deeplabcut.github.io/DeepLabCut/docs/HelperFunctions.html]
• 66. Deep learning-based behavioral analysis reaches human ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7608249/]
• 67. DeepLabCut Detections and tracking - Usage & Issues [https://forum.image.sc/t/deeplabcut-detections-and-tracking/63454]
• 68. Pose estimation for health data analysis: advancing AI in ... [https://www.frontiersin.org/journals/neurology/articles/10.3389/fneur.2025.1596408/full]
• 69. DeepLabCut benchmark - GitHub Pages [https://deeplabcut.github.io/DeepLabCut/docs/benchmark.html]
• 70. Using labeled data in DeepLabCut that was annotated ... [https://github.com/DeepLabCut/DeepLabCut/wiki/Using-labeled-data-in-DeepLabCut-that-was-annotated-elsewhere-(or-merge-across-labelers)]
• 71. Real-Time Closed-Loop Feedback in Behavioral ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8174057/]
• 72. Best Alternatives to DeepLabCut for Behavioral Tracking ... [https://conductscience.com/deeplabcut-alternatives-behavioral-tracking-2025/?srsltid=AfmBOoo4KLWIY5_kYRxNim0X-A0aoFfBwXM7W21-OgbVTgRf2r0Hl87o]
• 73. EthoVision XT | Benefits [https://noldus.com/ethovision-xt/benefits]
• 74. EthoVision XT: Superior & User-Friendly [https://noldus.com/blog/ethovision-superior-video-tracking-software]
• 75. A comparative analysis using EthoVision and DeepLabCut [https://pmc.ncbi.nlm.nih.gov/articles/PMC10079875/]
• 76. Best ANY-maze Alternatives for Behavioral Tracking (2025) [https://conductscience.com/alternatives-any-maze-behavioral-tracking-2025/?srsltid=AfmBOooWtpJidZ_tuSvk5ckQKgP1NSfAhhH7gB3Uori4ZvIA6iZeShPC]
• 77. In Vitro Assessment and Comparison of a Novel ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12075333/]
• 78. Electromagnetic Tracking System for Medical Micro Devices [https://www.mdpi.com/2072-666X/16/10/1175]
• 79. Optimal configurations of an electromagnetic tracking ... [https://www.sciencedirect.com/science/article/pii/S135045332400122X]
• 80. Gait tracking in dogs using DeepLabCut: A markerless ... [https://www.sciencedirect.com/science/article/pii/S0168159125002679]
• 81. Real-time, low-latency closed-loop feedback using ... [https://elifesciences.org/articles/61909]
• 82. vmTracking enables highly accurate multi-animal pose ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11845028/]
• 83. Explanation of Data Augmentation using Imgaug in ... [https://forum.image.sc/t/explanation-of-data-augmentation-using-imgaug-in-deeplabcut/64120]
• 84. Examples of Animal Behavior Research [https://noldus.com/blog/examples-animal-behavior-research]
• 85. Leveraging social media and other online data to study ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11389916/]