The continued growth of data-intensive applications and increasing processor core counts has made the memory system a fundamental bottleneck in modern server architectures. Conventional DDR memory interfaces are constrained by limited processor pin resources, restricting memory bandwidth, while large multi-socket systems (scale-up) or distributed systems (scale-out) suffer from significant Non-Uniform Memory Access (NUMA) penalties or network overhead for accessing remote memory. The emergence of the Compute Express Link (CXL) provides new opportunities to rethink memory system design through its high-bandwidth, pin-efficient, and memory-sharing capabilities. This thesis presents CXL-based architectural techniques that address memory bottlenecks across individual servers, scale-up systems, and scale-out systems. COAXIAL targets the individual server level by replacing conventional DDR channels with CXL in throughput-oriented manycore processors. This pin-efficient interface substantially increases available memory bandwidth, reducing contention and improving the performance of bandwidth-intensive workloads. StarNUMA targets scale-up systems by augmenting large multi-socket NUMA architectures with a centralized CXL memory pool. By placing heavily shared vagabond pages in shared memory, StarNUMA reduces remote memory accesses and improves the efficiency of shared-memory execution. CLEAR targets scale-out systems by accelerating Tensor Parallel LLM inference across distributed servers. It replaces network-centric AllReduce communication with an active CXL memory pool featuring in-memory reduction hardware, enabling collective operations to be performed through centralized shared-memory reductions. Together, these contributions demonstrate how CXL can be leveraged to address memory bandwidth, latency, and communication bottlenecks at multiple levels of server architecture, providing a scalable foundation for future high-performance computing systems.