The goal of this thesis is to study the collaboration in teams of humans and partially autonomous systems. In particular, we address the challenges of these collaborative scenarios using algorithms inspired by barrier functions and control barrier functions techniques. First, we study how to introduce meaningful information about the human operator into the control strategy. We conducted two separate human studies with over 20 externally recruited participants each. In the first study, we simulated an specialized aviation mission in which the participant was paired with an automated pilot. The automated pilot was in charge of the aircraft low-level navigation and piloting duties, and used control barrier functions to avoid areas that could damage the aircraft. This study confirmed that control barrier functions improve the team's safety, especially when the human is highly tasked and is unable to supervise the automated pilot. However, safety came at the cost of longer mission times. As a solution, in the second human study, we developed attention-tunable control barrier functions which modulated their conservatism according to the human's on board attention level. The human's attention was measured using an eye tracker. As hypothesized, these attention based control barrier functions reduced the total length of the mission, with non statistical differences from those with no additional safety mechanisms. This new control strategy constitutes the first human-aware control barrier function controller. Second, we develop the theory to extend the most important results of the smooth control barrier function literature to nonsmooth functions. We defined a generalized derivative named the Mandalay derivative, and derived a set of tools that allows to use this nonlinear operator to prove both forward invariance and asymptotic stability of the superzero levelset of nonsmooth functions. Thanks to the Mandalay derivative and the techniques presented in this thesis, we can now provide theoretically sound results for those collaborative scenarios that may need to be modeled by nonsmooth functions.