The optimization of structures, uncertainty quantification, and the solution of inverse problems typically require the solution of many linear systems, some with the same matrix (multiple right hand sides) and some with nearby or slowly changing matrices. We discuss a number of approaches to solve such systems approximately at very low cost using model reduction and stochastic approaches.