We investigate the applicability of multi-agent systems as a control approach for district heating systems. The consumers, i.e., the heat exchange systems, in current district heating systems are purely reactive devices without communication capabilities. In this work, the possibilities of a new type of heat exchanger system that has an open software environment and communication capabilities are explored. Operators of district heating systems have several, often conflicting, goals, e.g., to satisfy the demand of the customers and to minimize production costs. Our approach is to embed a cooperative agent in each consumer. Results from a simulation study indicate that the approach makes it possible to reduce production while maintaining the quality of service. In another experiment in a controlled physical environment, two agent-based approaches are evaluated and compared to existing technologies. The experiment shows that it is possible to automatically load balance a small district heating network using agent technology.