This dissertation enhances electric power system optimization algorithms using optimization and machine learning-inspired techniques. Power system optimization problems are inherently nonlinear and large-scale, posing challenges for real-time applications and complex models like bilevel formulations, mixed-integer nonlinear programs, and stochastic programs. Simplifications such as relaxations, approximations, machine learning models, and reduced networks help manage complexity but introduce approximation errors that can lead to suboptimal or infeasible operational decisions. Drawing from computational methods in machine learning, the dissertation proposes algorithms for optimally selecting parameters in common power system approximations, constructing reduced network models, and restoring AC power flow feasibility from simplified solutions. Firstly, an enhanced DC power flow model is presented, which adaptively selects coefficients and bias parameters through machine learning techniques. This improves approximation accuracy while maintaining structural simplicity. Secondly, a network reduction algorithm optimizes network equivalencing to better align DC power flow solutions of reduced networks with AC power flow results of the original network, enhancing inter-zonal flow predictions. Thirdly, a parameter optimization algorithm addresses the nonlinearities of the DistFlow model in distribution systems, enhancing the accuracy of the LinDistFlow approximation for single-phase and three-phase models by optimizing coefficients and bias parameters using sensitivity information. Furthermore, an algorithm improves the accuracy of DC Optimal Power Flow (DC-OPF) problems relative to AC-OPF problems by tuning coefficients and bias parameters using a machine learning-inspired methodology. An enhanced DC Optimal Transmission Switching (DC-OTS) model is also proposed, optimizing DC-OPF parameters to better align with AC-OPF results and capturing apparent power flows to improve congestion modeling and transmission switching decisions. Finally, an AC power flow feasibility restoration algorithm employs a state estimation-based post-processing technique to adjust solutions from simplified OPF problems, leveraging machine learning to optimize weight and bias parameters for accurate adjustments.