The objective of the proposed research is to better analyze physical side-channel vulnerabilities, with a specific focus on the power and electromagnetic (EM) side-channels of both software and hardware implementations of cryptographic algorithms. First, we observed that, while performing the side-channel analysis (SCA) of such implementations, the existing body of works primarily focused on proposing better neural network (NN) models to achieve higher accuracy at recovering the secret information (i.e., key or message), which is why portability (profiling and attacking different devices) and interpretability (how the leakages are learned) issues of the NN models were largely overlooked. We demonstrated how this portability issue manifests in the NN-based power/EM SCA on a software implementation of the current national institute of standards and technology (NIST) symmetric-key encryption standard, namely advanced encryption standard (AES). We proposed an efficient cross-device attack technique using multi-device training and pre-processing of traces under practical settings. Second, we investigated the interpretability of NN models used in SCA to gain insight into which features (i.e., points or time samples) contribute the most to the classification decision by validating the relevance scores of features from the NN models using gradient-based post hoc explanation methods to the ones obtained by traditional points-of-interest (PoI) selection methods. Third, we performed a power side-channel vulnerability assessment of a hardware implementation of one of the finalists of the NIST lightweight cryptography (LWC) competition, namely XOODYAK. We developed novel hypothetical attack models specific to the algorithm and demonstrated successful attacks on its INITIALIZE and ABSORB phases using correlation power analysis (CPA) and NN-based profiling techniques. Fourth, we demonstrated a single-trace profiling attack on a constant-time hardware implementation of cumulative distribution table (CDT)-based discrete Gaussian sampler used in lattice-based cryptography (LBC) algorithms which rely on the learning with errors (LWE) problem. Finally, we also developed generic mitigation strategies, such as a sensor to proactively detect an ongoing attack and signature attenuation techniques to reduce the signal-to-noise ratio (SNR) of side channel traces observable by an adversary to ensure implementation security against such attacks.