Much effort has been spent on suggesting and implementing new architectures of Multi-Agent Systems. However, we believe the time has come to compare and evaluate these architectures in a more systematic way. Rather than just studying a particular application, we suggest that more general problem domains corresponding to sets of applications should be studied. Similarly, we argue that it is more useful to study the properties of classes of multi-agent system architectures than particular architectures. Also, it is important to evaluate the architectures in several dimensions, both different performance-related attributes, which are domain dependent and more general quality attributes, such as, robustness, modifiability, and scalability. As a case study we investigate the general problem of "dynamic resource allocation" and present four classes of multi-agent system architectures that solve this problem. These classes are discriminated by their degree of distribution of control and degree of synchronization. Finally, we instantiate each of these architecture classes and evaluate, through simulation experiments, how they solve a concrete dynamic resource allocation problem, namely load balancing and overload control of Intelligent Networks.