We explore data-centric approaches for exploiting metainformation and mitigating model regression to aid neural networks. We first focus on data-centric approaches for exploiting metainformation in multiple settings. We aid the training of neural networks for optical coherence tomography scans by incorporating metadata such as clinical data and biomarkers in unsupervised and supervised learning frameworks, and use patient ID to choose training samples in active learning frameworks. In a video active learning setting, exploting metainformation from spatial and temporal features in a video provises insight on the time associated with annotating a video sequence. We identify sequences that take the shortest time to annotate, thus minimizing the cost associated with creating video dataset annotations. All neural networks are susceptible to catestrophic forgetting, which is the tendency of neural networks to forget previously learned information when they are trained on new or unrelated tasks. This results in a model’s performance degrading or regressing from its current state as it is sequentially trained on additional data. We design a suite of optimization functions that perform “learn,” “train,” and “restore” tasks to consistently alleviate model regression better than the existing benchmarks.