The objective of the proposed research is to improve the end-to-end efficiency of agentic AI systems through workload-aware hardware–software co-design. These systems combine large language model (LLM) inference on accelerators with tool execution on CPUs. CPU processing can dominate tool-intensive workloads, while LLM-intensive workloads introduce challenges in model routing, load balancing, and memory management. For LLM-intensive agents, ORACLE uses task-specific verification and asynchronous feedback to improve model selection. Its scheduler accounts for KV cache capacity and balances the benefit of shorter waiting times against potential losses in accuracy. For tool-intensive agents, OneDSE uses workload information to predict CPU configurations and simulation feedback to guide further search, reducing the number of evaluations needed for design exploration. The thesis also presents a cycle-accurate simulator for exploring accelerator designs, along with dataflow optimizations that reduce memory traffic in symbolic reasoning. The proposed research further investigates mixed-precision acceleration based on each model’s role in an agentic program and joint routing and scheduling for model pools that change with the workload. Together, these efforts aim to improve performance and resource utilization across heterogeneous CPU–accelerator platforms by coordinating hardware capabilities with the computational and memory demands of agentic execution.