The Newton-Raphson Controller is a tracking controller for dynamical systems. Existing results have shown the effectiveness of the controller for a number of nonlinear systems, for example, inverted pendulums, autonomous vehicles and quadrotors. Although this controller has reached a performance comparable to other traditional controllers, important problems still exist with the original controller. First, the controller depends on the mathematical model of the controlled plant to generate the control input. If the model of the plant is inaccurate or unknow, then the tracking control will also be inaccurate or even impossible. Second, although there has been a stability criterion of the controller for linear systems, such a criterion for nonlinear systems is yet to be developed. Third, this controller can only be applied for single-agent systems. To control a multi-agent system, the Newton-Raphson Controller has to manipulate individual agents separately by assigning them reference signals, which reduces the flexibility of the multi-agent system. In this dissertation, we attempt to solve these problems for single-agent and multi-agent systems. For the single-agent system, we propose to apply the Neural Network, such that the Newton-Raphson Controller does not rely on the mathematical model and can operate on a model-free manner. Also, we prove the stability of the controller for a class of nonlinear single-agent systems using differential flatness. We also want to extend the original tracking controller into the case of multi-agent systems. Specifically, we consider two different scenarios, namely the leaderless consensus control and the leader-follower consensus control. We prove that using the extended Newton-Raphson Controller, the leaderless consensus and the leader-follower consensus can be achieved for a class of nonlinear systems. Finally, since the agents may collide with each other during the consensus control, we study the safety-critical control using the Integral-Control Barrier Functions. We show that with the Barrier Function, the agents can maintain a desired distance between each other, making the controller and the system safe.