In the
School of Biological Sciences
TUCKER LANCASTER
Will defend his dissertation
SYNTHETIC EYES ON NATURAL LIVES: MACHINE LEARNING APPROACHES TO
BEHAVIORAL ANALYSIS IN ECOLOGICALLY VALID CICHLID ENVIRONMENTS
https://gatech.zoom.us/j/96134265799?pwd=2HVbs8fx9mlcPzuQYt774PPH9hIuwT.1
Patrick McGrath, Ph.D.
School of Biological Sciences
Georgia Institute of Technology
Jeffrey Todd Streelman, Ph.D.
School of Biological Sciences
Georgia Institute of Technology
Gordon Berman, Ph.D.
Department of Biology
Emory University
Eva Dyer, Ph.D.
Department of Bioengineering
University of Pennsylvania
Anqi Wu, Ph.D.
College of Computing
Georgia Institute of Technology
ABSTRACT: A central tension runs through computational ethology: constraining the experimental environment simplifies automated measurement but can degrade or remove the very behaviors that make measurement worthwhile. This dissertation develops six computational tools that navigate this tension for cichlid fishes of Lake Malawi, enabling automated behavioral measurement at the level of ecological validity each biological question demands. The tools are organized across three aims, progressing from constrained single-camera experiments to multi-camera 3D monitoring in large naturalistic aquaria.
The first two aims develop small-tank tools that enabled neurogenomic studies linking behavioral phenotypes from automated measurement to cell-type-specific gene expression. For sand-dwelling bower-builders, whose primary behaviorally relevant outcome is the physical structure the animal builds rather than its body configuration alone, the CichlidBowerTracking system combines depth sensing of sand surface topography with video-based action recognition at human-level classification accuracy, scaling to 33 simultaneous experiments on low-cost single-board computers. For rock-dwelling mbuna, whose courtship leaves no environmental trace, two complementary tools—SARTAB for edge-deployed real-time courtship detection and DemBA for identity-resolved post-hoc behavioral characterization—extended this neurogenomic paradigm to a second behavioral system. These tools are scalable and cost-effective but operate within simplifying assumptions: a single camera, a small arena, and at most two focal animals.
The third aim moves to 650-gallon community tanks housing mixed-sex groups monitored by multi-camera arrays, confronting the simplifications the earlier aims relied on: single-camera coverage, tractable identity among few animals, well-defined spatial regions of interest, and 2D treatment of an inherently 3D volume through a refractive medium. Three integrated tools share an explicit refractive projection model throughout, ensuring geometric consistency from calibration through to behavioral metrics. AquaCal provides refraction-aware multi-camera calibration with sub-millimeter accuracy. AquaMVS embeds the same refractive model into dense multi-view stereo for 3D environmental reconstruction of substrate topography. AquaPose combines detection, pose estimation, tracking, cross-view identity association, and refraction-aware triangulation to produce per-fish 3D midline trajectories, maintaining identity at a rate of 0.012 swaps per fish per 1,000 frames across 12 synchronized cameras. Because the refractive model reduces to standard pinhole geometry when refraction is absent, the algorithmic contributions that under-pin AquaPose—stateless cross-view identity association via community detection, keypoint-based tracking, and 3D-first identity resolution—are in principle applicable to terrestrial multi-camera systems, where they address open challenges independent of refraction.
Together, these tools demonstrate that automated behavioral measurement can achieve the precision and scale that neurogenomic studies require without sacrificing the ecological validity the biology demands. With this measurement infrastructure in place, the constraint on the next generation of cichlid neurogenomic studies shifts from whether behavior can be quantified in naturalistic settings to which biological questions warrant the investment of doing so.
]]>BEHAVIORAL ANALYSIS IN ECOLOGICALLY VALID CICHLID ENVIRONMENTS
]]>