General descriptions: the typical applications of CAE client are fully considered in the design of Inspur TS10K system and the system is tailor-made to customer applications. Fat nodes are configured in response to hidden finite element analysis and eight channels of fat nodes are optional.
I. Demands and challenges
CAE (computer aided engineering) is an approximation analytical method in which a computer is used to aid the analysis and calculation of complicated projects and product structural strength, rigidity, yield stability, dynamic response, heat exchange, 3D multi-object contact, elastoplasticity, and other mechanical properties and the optimization of structural performances. Ever since it was initiated in the early 1960s, CAE has become an essential calculation tool in engineering and product structural analysis (e.g. aeronautic, astronautic, mechanical and civil structure sectors) and an important method for the analysis of various problems of continuous media dynamics.
The cluster system with distributive memory is a high-performance server system that has rapidly become popular in recent years. Clusters are an aggregate of independent computers (nodes) which are interconnected via a high-performance network and can work collaboratively as one single and concentrated computing resource (singular system mapping) for parallel computation. This type of server has a low building cost and has a high performance-price ratio and scalability. As the mainstream architecture of high-performance computers today, cluster accounts for over 75% of the top 500. Therefore, the mainstream CAE software supports cluster architecture and MPI.
High-performance computation was adopted in CAE in the 1980s and it has undertaken lots of critical computational applications. CAE-HPC’s goal is to “simulate the real world at the lowest cost”, i.e. “performing complicated mono-field simulation and multi-field coupling analysis with the fewest human and financial resources within the shortest possible time”. CAE undertakes the critical duties in a manufacturing enterprise and therefore selection of its high-performance platform is very important. HPC platform directly affects the operational performance, general cost and system maintenance of CAE.
Generally speaking, CAE analysis mainly includes: pretreatment, calculation and analysis and post-treatment. Pretreatment mainly involves: establishing geometric model of the problem, grid division, establishing number model for calculation and analysis and determining the boundary conditions and initial conditions of model. Calculation and analysis involves: analysis of the number model, ordinarily the large-scale linear equation set, which is the process requiring the largest calculation load and having the highest requirement on hardware performances in CAE analysis. Post-treatment is the inspection and treatment of the calculation results in a graphical manner.
II. Inspur CAE high-performance solution
Analysis of CAE software shows that most CAE applications well support parallel MPI between nodes and therefore are suitable for running on dual-channel computational nodes and generic computations. However, some CAE software has high requirements on memory and is applicable for computation in the OpenMP mode on a fat node. Therefore, we recommend high-capacity four or eight-channel server as the fat nodes of the cluster.
Inspur performs professional analysis and summarization of CAE application performances to provide customized solutions. Inspur high-performance application cluster mainly address three main problems in CAE software application:
It is mainly used for pretreatment and post-treatment and for grid division and visual display. As it has rigorous requirements on server display and processing capacity, we recommend workstation configuration with a professional graphic card.
Obvious algorithm and hidden algorithm have different requirements on computing capacity. The solution offers dual-channel nodes and is therefore applicable for the obvious computation of distributive memory and the computation of fluid software CFD. It is furnished with multiple fat nodes for the hidden computation of shared memory. The design of different architectures reflects the professionalism of Inspur solutions.
Network bandwidth problem
Large-scale CAE software application has very high requirements on network delay and bandwidth. We have configured Ethernet exchange environment or faster 40Gb Infiniband network to ensure the non-blocking computation exchange of all nodes and reduce network delay.
Storage bandwidth problem
Massive temporary files will be created during the computation of some CAE software. An excellent storage system can meet the software’s requirements on network bandwidth. Inspur provides not only professional direct storage, but also optical fiber storage system with an 8Gb interface. Special storage node is adopted to establish Lustre parallel file system and access the Ethernet and even 40GB Infiniband network, so as to avoid CPU waiting for data and substantially increase the calculation efficiency.
III. Advantages and value of Inspur solution
Close combination of computation system and client application
The typical applications of CAE client are fully considered in the design of Inspur TS10K system and the system is tailor-made to customer applications. Fat nodes are configured in response to hidden finite element analysis and eight channels of fat nodes are optional. Dual-channel mainstream computation nodes are furnished for fluid applications; graphical workstations of a tower and rack type are optional for pretreatment and post-treatment; NAS storage, fiber storage and IB SAN storage solutions are available; Inspur Ts Exastor parallel file system may be furnished where there is a particularly high requirement on IO bandwidth.
Well-established interconnection system
Inspur TS10K system includes 4 sets of networks. The management network is a 1G network that has a 10G network as the trunk network. Independent IP may be established for educational network/public network so that WAN users can log in and use the cluster; IPMI network is a non-linear 1G switch for interconnection of all nodes and the administrator can realize the off-band administration of the cluster via an office computer. KVM local monitoring network can realize KVM monitoring of 960 computers at the maximum in a daisy chain mode. The computation network is characterized by interconnection at fully linear speed via IB network with a bandwidth of 56Gb/s and a delay within 1 millisecond to fully guarantee data exchange during computation.
Advanced and efficient dispatching system
Inspur ClusterEngine cluster administration software is customized for Inspur TianSuo HPS cluster. It has a B/S architecture and operates via browser (IE, Firefox, etc.) for all-around cluster monitoring, administration and alarm. The user submits its scientific computation task via the ClusterEngine graphical interface and ClusterEngine will reasonably dispatch the tasks submitted by the user according to the use of cluster resources and the operation’s demand for resources, hence increasing the utilization ratio and efficiency of resources. ClusterEngine can also summarize the resource consumption and resource use conditions and generate a statement for the cluster administrator as the criteria of system optimization.
Green and energy efficiency of cluster system
All nodes are furnished with 80PLUS highly efficient power sources to increase the efficiency of power source switching and reduce user’s operating and maintenance costs. Coupled with ClusterEngine energy-saving components, the free resources are automatically set in a standby or shutdown state, hence reducing the general energy consumption by 20%.