Online services have become deeply integrated into our everyday interactions with the internet. These services are typically powered by chains of microservices distributed across large-scale datacenters. Many of these microservices are highly latency-sensitive and play a critical role in determining the overall end-to-end latency experienced by users. As user demand grows, compute nodes within datacenters are increasingly burdened with processing high volumes of network I/O. However, conventional commodity NICs are static and unaware of application-level behavior, resulting in inefficient handling of network traffic. This disconnect limits the ability of servers to fully utilize their I/O capabilities, leaving significant potential performance untapped. The emergence of smartNICs provides a new opportunity to offload hardware functions to reprogrammable NICs to assist with accelerating the processing of network traffic. These offloads can are made possible and effective by exposing application-aware semantics to the network interface. This architectural shift enables novel optimizations that can improve throughput while adhering to stringent tail latency requirements. This dissertation explores novel ways to leverage smartNICs to accelerate latency-critical microservices, with the goal of enhancing overall datacenter efficiency and performance. It achieves this by employing smartNICs to balance RPC packets across multi-core servers (i.e., Turbo), optimize NIC-to-CPU data movement for shallow network functions (i.e., NFSlicer), and notify the CPU of connection activity to enable accelerated processing of RDMA connections at scale (i.e., Sassy).