Selecting Hardware for Clusters

Problem Description

This knowledge base outlines some hardware recommendations if you plan to set up your own compute cluster for running distributed COMSOL jobs.

Solution

Depending upon the type of problems that you want to solve you will want to consider different cluster architectures. From the point of view of selecting cluster hardware, COMSOL applications can be classified as either sweeps of small models over a large set of parameters that are unique variations, or as single unique models that are very large, requiring more computational resources to solve than what is available on any one computer.

Sweeps

When performing sweeps over different parameters, such as geometric dimensions or frequencies, each different combination of parameters can be solved on a different node of the cluster. Each node in the cluster must have enough memory to solve each unique model, and each node should be of similar type. Follow the Hardware Recommendations to determine what is appropriate.

The communication between nodes is often a bottleneck, so Infiniband or an equivalent high-performance network is recommended. The number of nodes can be up to approximately 1/10 of the total number of simultaneous cases you want to run. For example, if you regularly want to sweep over 1000 different parameter values, up to a 100-node cluster is reasonable. Solving across more nodes will usually not result in much more speedup since there is an overhead in sending model data between nodes.

For guidance in solving a distributed sweep, please see the documentation for: Joule Heating of a Microactuator - Distributed Parameter Version

Large Models

For solving a single large model distributed over many nodes, the cluster architecture becomes more important. Communication between nodes becomes more of a bottleneck. Infiniband or an equivalent high-performance network is strongly recommended.

First make sure that you know how much memory you anticipate you will need to solve your largest models. For guidance in predicting memory requirements please see: How Much Memory Is Needed to Solve Large COMSOL Models?.

Once you know how much memory you will need, you can determine the number of nodes in your cluster. You will notice a speedup as you solve a problem on more nodes, but beyond a certain number the speedup becomes marginal and may even go down. When this occurs depends on the model size. Use the guidelines in Knowledge Base 866 for selecting processors and memory.

The solution time will be a function of the number of nodes in the cluster, the cluster architecture, the interconnect speed, the number of processors per node, the memory bandwidth between the processors and the on-node memory, the number of cores per processor, and the solver type being used in COMSOL Multiphysics.

General Notes

• The selection between Linux or Windows HPCS2008R2 operating system will not notably affect performance.

• COMSOL distributes computations on compute clusters using the MPI (Message Passing Interface) model.

• Cloud computing can be achieved through one of our partners. See Running COMSOL Multiphysics® and COMSOL Server™ in the Cloud for more details details.

Browse by Category