Discussion Forum

Posted: 1 decade ago Aug 13, 2010, 1:22 p.m. EDT

well i did the same last year mostly because i was bored.. I wont do it again... So let me give you my thinking 1-Choice of OS: linux not windows because of the openess and ability to tweak machine performance more easily. the only minus I can think of nt using windows is that if you also need to use a "windows only" soft coupled with comsol ... 2- memory size depends the size of your problems 1GB is NOT ENOUGH.. think for 3d specially you should think along the line of 24 GB AT THE VERY least or 64 GB if you can afford it . you will then get a dual proc mother board. I spoke about a comsol person at the time of the building and she told me that for processor, high clock frequency was the parameter to favor when the budget was constrained. in the end you end up with 8 or 12 cores machines depending upon your budget and YES YOU WANT 64 bits to address all the memory you will use for 3d simulations. Nobody today will seriously consider buying a new 32b its workstation for scientific computing with linux the graphic part can be tricky for you have to met comsol requirement strictly [ Open GL driver and so forth.. ] but it is pretty well documented on their website 3 why do I think it is a bad idea... ? speaking from experience... building your own machine still represent a significant budget. what you dont do well usually, if you dont build pc on a regular basis, is thermal engineering..[ putting compnents and wiring them is rather trivial otherwise nwadays... but in the end you wil end up with either an overheating machine [ very bad and potentialy destructive] or a very noisy one [bad] as in my case... I had to put the UC in another room form my office just because of the fan noise... SO You better go to a local pc dealer and tell him what you want and he will do it for you or just use the "customize feature" that most major manufacturer propose on their website....for you the cost will be in the end the same or possibly less compared to as if you buy the components say on amazon and you will have for sure a better machine and a warranty ...much less headache risk and time wasted... JF

Posted: 1 decade ago Aug 14, 2010, 5:30 a.m. EDT

What about a MacPro? Up to 12 cores, up to 32GB RAM, up to 8TB HD, or 4x512GB solid state HD, 64bit MacOS (even Windows), superb graphics, clever design, both COMSOL and Matlab compatible, etc. Ideal for numerical simulation. cheers Andras

Posted: 1 decade ago Aug 15, 2010, 4:44 a.m. EDT

Hi to add a little, for me the ONLY reason I also have a Windows OS (Win 7 recently) is to have the LiveLink to SolidWorks, as my CAD models come often from my colleagues, and I do not want to redo everything and SW runs onlyunder Windows. Furhtermore I can do shape optimisation easily. For that other points I agree with François. Mac or Linux/Unix for me is the same issue. I have a dual boot, hence I can use both, but I notice not that great differences, UX seems superior once the system start to swap RAM to disk. For the RAM and CPU's, I would say definitively as much RAM as possible, BUT high speed RAM, and check carefully the bottlenecks on RAM to cach transfer, where systems like this sits and wait for access the most of the time. A 3D model uses easily dozen of GbRAM, and in this case 64-bit is required to address it. Then it's better to have as many cores as possible, as this distributes the RAM on their local cache and a maximum of parallelism can be achieved, on the same PC COMSOL runs on what is available of parallel cores, when possible (but not always then the single CPU running is giving the pace) I have tested the multiple cores on my PC 2x6x2 cores-multiplexed, or 2x6 standard, I see little differences, perhaps slightly better in 2x6 than 2x6x2-multiplexed (Idid this with the -np command, not via the BIOS cold still be an issue. In anycase under win-7 one core is mostly used by the OS For the graphics, a recent Nvidia GPU car with sufficint RAM is OK, but upload the latest BIOS, it anyhow not that expensive today Buid or buy: I'll prefer to buy you can get "big" WS if you look carefully on the advertisements, then it's often cheaper to buy Last: a high speed disk for efficient swapping (I have three, one for each OS + swap, one for the soft and the users and the scratch -- Good luck Ivar

Posted: 1 decade ago Aug 15, 2010, 7:06 p.m. EDT

I run Win7 64-bit, 24 GB, dual quad cpus, nVidia GTX260. Very pleased with the results. Re the Linux issue: Depends on whether you intend to use the WS as a dedicated Comsol engine. If so, and if you like Linux, then go for it. Of course, one might wonder if it makes sense to be concerned about Open Source issues when dealing with a proprietary package such as Comsol. I use my machine for other things, too, and given that many s'ware vendors assume Windows as the default standard, this tips the advantage towards Windows for me. So far everything has worked exceedingly well.

Posted: 1 decade ago Aug 16, 2010, 12:30 a.m. EDT

@David re "open source" Open source "per se" is irrelevant of course... but because of the "open source spirit" you are provided with options that windows, to the best of my knowledge, [not sure about mac] dont provide ..and that can, depending upon your need, impact significantly your computing performance when you know what you are doing. One example: swap file on my machine all my user disks partitions are raid 1 for redundancy [regular read/ write on disk have marginal impact on performance for me so I dont bother going with a more sophisticated raid configuration] but my swap files are distributed along all my drives in a "raid 0" mode [a standard option with most linux distro]designed for maximum bandwidth when you hae to access the swap files. I have a total of 4 drives so it make a LOT of difference when comsol has to use VM. Not sure how you will do that with windows.

Posted: 1 decade ago Aug 16, 2010, 3:44 a.m. EDT

Hi, It is not difficult in Windows to set up Raid0 across multiple disks and then assign this to be the swap file. It does indeed make a speed difference in applications that need it, as you say. In my current configuration with 24 GB memory it rarely is needed, however. My standard setup is to have the OS and its supporting files on their own separate HD, the swap file on another on (perhaps, but not always, a small but fast Raid0 array), apps on another one, and data on still another one or more.

Posted: 1 decade ago Aug 17, 2010, 7:53 p.m. EDT

Thank you all for your helpful responses! In order to save costs, another idea I had was to use the Amazon EC2 -- does anyone have any experience using COMSOL with the cloud? Would there be any strong negatives to using the cloud instead of local hardware?

Posted: 1 decade ago Aug 23, 2010, 11:06 a.m. EDT

I would interrested in the Cloud solution also. Has anyone any information on COMSOL plans about it ?

Posted: 1 decade ago Nov 1, 2010, 9:26 a.m. EDT

About the fast hard drive: Is there a reason to buy a SSD for swapping?

Posted: 1 decade ago Nov 17, 2010, 4:26 a.m. EST

[QUOTE] [...] For the graphics, a recent Nvidia GPU car with sufficint RAM is OK, but upload the latest BIOS, it anyhow not that expensive today [...] Good luck Ivar [/QUOTE] Hi Ivar There is quite a big span within the graphic cards on the market. Is it worth investing in an expensive graphic card with a lot of RAM like the Nvidia Quadro 6000 for example? Or is a cheap graphic card sufficient (lets say Nvidia Quadro FX 580) that is good enough to visualize your computational results? Is the graphic card especially the additional RAM helping COMSOL in any case during solving the problem? Many thanks in advance Daniel

Posted: 1 decade ago Nov 18, 2010, 5:10 a.m. EST

Hi for me most advanced graphical cards should do and a few hundred Euro is not the most expensive, be sure yu have a "CAD" card and not really a Gaming one video and CAD graphics are not fully the same thing, but today most graphical card do no longer have this distinction. But I must admit have a FX8500, not 580 as I'm working now on two high res screens, and that is really comportable, the ROI is rapidly there when you spend hours behind your screen -- Good luck Ivar

Posted: 1 decade ago Jan 5, 2012, 7:30 a.m. EST

Anna, here is a link I've found very usefull when I'm willing to buy a new computer or workstation: http://www.comsol.com/support/knowledgebase/866/ cheers, -- Felipe

Posted: 1 decade ago Jan 5, 2012, 9:09 a.m. EST

[QUOTE] Hi to add a little, for me the ONLY reason I also have a Windows OS (Win 7 recently) is to have the LiveLink to SolidWorks, as my CAD models come often from my colleagues, and I do not want to redo everything and SW runs onlyunder Windows. Furhtermore I can do shape optimisation easily. For that other points I agree with François. Mac or Linux/Unix for me is the same issue. I have a dual boot, hence I can use both, but I notice not that great differences, UX seems superior once the system start to swap RAM to disk. For the RAM and CPU's, I would say definitively as much RAM as possible, BUT high speed RAM, and check carefully the bottlenecks on RAM to cach transfer, where systems like this sits and wait for access the most of the time. A 3D model uses easily dozen of GbRAM, and in this case 64-bit is required to address it. Then it's better to have as many cores as possible, as this distributes the RAM on their local cache and a maximum of parallelism can be achieved, on the same PC COMSOL runs on what is available of parallel cores, when possible (but not always then the single CPU running is giving the pace) I have tested the multiple cores on my PC 2x6x2 cores-multiplexed, or 2x6 standard, I see little differences, perhaps slightly better in 2x6 than 2x6x2-multiplexed (Idid this with the -np command, not via the BIOS cold still be an issue. In anycase under win-7 one core is mostly used by the OS For the graphics, a recent Nvidia GPU car with sufficint RAM is OK, but upload the latest BIOS, it anyhow not that expensive today Buid or buy: I'll prefer to buy you can get "big" WS if you look carefully on the advertisements, then it's often cheaper to buy Last: a high speed disk for efficient swapping (I have three, one for each OS + swap, one for the soft and the users and the scratch -- Good luck Ivar [/QUOTE] Hi Evar, Wit regard to the specifications. What according to you will help me compute 3D problems with 1million and above mesh elements. Have 2 options 1) 12 GB with quad core 2) 16 GB with dual core. All the other specs are the same. Have a 64 bit operating system. Speed is not an issue. I want the model to run. Regards Glenston

Posted: 1 decade ago Jan 5, 2012, 9:45 a.m. EST

Hi 1st of all ask "support" they can give a definitive answer. Personally, I would say maximum RAM to store a larger model, while 2 or 4 cores makes a difference in time to solve but not any true difference on the model mesh size (maximum matrix sizes) (note: this applies for V4, I'm not sure exactly how it was with v3.5a) -- Good luck Ivar

Posted: 1 decade ago Mar 9, 2012, 2:24 a.m. EST

On Linux vs. Windows: Just a few remarks from personal experience with a Linux (Centos) WS w/ 2x6C + 96 GB RAM Dell T7500 and W7 WS w/ 2x6c + 54 GB RAM Dell T5500 and an EliteBook 8540w w/ 4C+ 8 GB RAM. 1.) Drivers: Windows is almost painless here; but Linux requires reinstalling/updating especially NVIDIA graphics drivers now and then. 2.) Remote access: Definitely linux! It is so friendly that if you have a good network it does not have a glitch. (In fact, we face the graphics drivers problems only when we want to login to the machine directly.) 3.) Memory use: Linux is better. Windows is taking too much memory itself. It shouldn't matter if you are using plenty of RAM, but it's a hassle for example on laptops. 4.) Ease of use: Windows. New generation of engineers/grad students are not familiar with linux issues to solve them on the spot. Once in a while for trivial things they have to wait IT support to fix them. Windows 7 is almost hassle free from the user's point. 5) Performance: Our 96 GB behemoth linux WS chews anything we send almost with no issues; same with the W7 WS but it has less GHz ( 2.6 vs 3.2)and is preferred less. However, W7 WS is a lot fun to open 3D models for postprocessing. Same is not true for the Linux WS; often it hangs with graphics drivers and makes the life miserable for couple of hours if not days to understand the problem and fix it. Often it turns out to be the graphics drivers; but still the OS error may come in any form and requires serious debugging. Bottom line: Get a fast and big RAM server (I believe you can go up to 4x8C here with 256GB RAM for about 20K USD) with Linux. You don't need to pay extra for the graphics because it will cause more trobule than good. Then get a couple of relatively big RAM (at least 12 GB), reasonable CPU (2x4c would be fine) and good graphics windows WSs. Remotely use the Linux server and post-process on windows stations on which you can occasionally run small jobs as well.

Posted: 1 decade ago Aug 31, 2012, 3:18 p.m. EDT

Hi Anna, Here is my parts list along with links to the best deals I found at the time… AMD 100-505647 FirePro V7900 2GB 256-bit – 1 card seamed to hold up fine, and I built one with a V4800 that worked great and might be a better budget choice http://www.newegg.com/Product/Product.aspx?Item=N82E16814195107 $689 or $154 Dynatron A6 77mm 2 Ball CPU Cooler – X2 for 2 processors, consult AMD’s website for temps on Opterons, these do a great job at extended full load times and held up under prime95 torture tests http://www.newegg.com/Product/Product.aspx?Item=N82E16835114113 $35 X 2 ASUS KGPE-D16 SSI EEB 3.61 Server Motherboard – this is a server board, but the resulting build benchmarks pretty darn close to a $10K Xeon 8 Core workstation, per Everest http://www.newegg.com/Product/Product.aspx?Item=N82E16813131643 $400 COOLER MASTER Silent Pro Gold Series RSA00-80GAD3-US – this might be a little overkill, but I got a good deal on this and I’m familiar with this power supply http://www.newegg.com/Product/Product.aspx?Item=N82E16817171056 $210 ARK IPC-4U600 Black 1.2mm SECC Zinc-Coated Steel 4U Rackmount Server Chassis – this is a big 4U box and sense space was not an issue I took advantage of the thermal control advantages. I removed the front cover and mounted 2 X 120mm fans on the front blowing out, negative pressure kept the machine cooler than pointing the fans in. I had to get a little creative with the mounting of a second by cutting a mounting plate for it out of foam-core board from the office supply store. It is VERY IMPORTANT to have sufficient air replacement in the box http://www.newegg.com/Product/Product.aspx?Item=N82E16811128020 $91 Crucial M4 CT128M4SSD2 2.5" 128GB – these have the Marvell controllers that I like most at the moment, and I know that they work well with the SATA chipset on this motherboard. I used one for OS, one for programs, and one for scratch and user files with mklink http://www.newegg.com/Product/Product.aspx?Item=N82E16820148442 $110 X 2 or 3 for $330 Kingston 8GB 240-Pin DDR3 SDRAM ECC Registered DDR3 1333 – Total of 128GB of RAM, I probably got too much RAM for this build, but at the time I couldn’t find out was necessary to avoid bottle necking for the rest of the build http://www.newegg.com/Product/Product.aspx?Item=N82E16820139140 $65 X 16 for $1040 AMD Opteron 6128 Magny-Cours 2.0GHz 8 x 512KB L2 Cache 12MB L3 Cache Socket G34 – since I went with AMD for economy these landed in a sweet spot for my budget. In retrospect, I might try to go with a faster processor and less RAM, but that could also lead to changing the motherboard, too http://www.newegg.com/Product/Product.aspx?Item=N82E16819105266 $290 X 2 for $580 My cost was less than $4500 each, closer to $3500 with the slower graphics card. I built 2 of them initially. I ran into a couple of snags with some of the hard drives that sold with a buggy firmware, but after an update the machines are stable and are constantly running with 15 to 20 simulations running simultaneously for many days in a row. I like Linux SuSE, but my user needs SolidWorks and I didn’t feel like adding Vitualbox or something to get that running. I think the fact that these are located on the other side of the state and users always have Linux questions helped me chose windoze. This was like most other builds for me, but to a beginner it might be intimidating. I’ve been doing this for about 15 years. There was plenty of trouble shooting and the 5035-hour bug with the hard drives was not documented when I discovered it. The most important thing to consider with a workstation in my humble opinion is production and what that production worth? To me time is a factor of worth, so this machine can reduce the amount of time required to produce. I could make a faster machine, but the two of these can solve more problems in the same amount of time than a XEON that costs twice as much. And I a big fan of redundant redundancy… Hope that helps and good luck!!

Posted: 1 decade ago Feb 25, 2014, 11:16 a.m. EST

When I am doing a mesh for 3D structure, the windows system use only 7% CPU (one thread). Is there anyway to make it use as many CPU resource as possible? [QUOTE] Hi 1st of all ask "support" they can give a definitive answer. Personally, I would say maximum RAM to store a larger model, while 2 or 4 cores makes a difference in time to solve but not any true difference on the model mesh size (maximum matrix sizes) (note: this applies for V4, I'm not sure exactly how it was with v3.5a) -- Good luck Ivar [/QUOTE] -- J.L.

Posted: 8 years ago Aug 17, 2016, 9:46 a.m. EDT

This note describes how various configurations of workstations/server systems performed for COMSOL metamaterial simulations. The benchmarking simulation (WaveOptics module) was a frequency domain study of a metamaterial model with periodic conditions and PML layers at the top and bottom. The simulation had 2813822 degrees of freedom (mesh had 417057 elements). I used a direct solver, PARDISO. Three systems were used, none as a cluster, only as standalone workstations. All three were dell dual core Xeon based workstations. Configurations are as follows: BrownDell poweredge T620 with 16 core (2x8) Xeon E5-2650 @ 2.0GHz, 128 GB RAM (DDR3 assumed), Linux only Bob Dell power edge C6220 II, 20 cores (2x10) Xeon E5-2660 v2 @ 2.2 GHz, 160 GB RAM (DDR3), Linux only Thunder  Del Precision Tower 7910 , 20 cores (2x10) Xeon E5-2687w v3, 256 GB RAM (DDR4 2Rx4 RDIMM 2400Mhz, 32GB sticks), with SSD. RAM was configured to have one module per channel, as suggested by COMSOL (https://www.comsol.com/support/knowledgebase/866/) . Win7 and Linux. Hyperthreading (HT) was most likely enabled on Brown and Bob, I had no control over this hence cannot turn it off. I tested Thunder with and without the HT. Table 1 shows the results of simple controlling the number of cores being used and the best possible times from the three machines that could be achieved. Time is in minutes (decimal value is seconds/60) for solving a single frequency point. ********************************************View in PDF********************** Cores Brown Bob Thunder (Win7, HT off) Thunder (Linux, HT off) 20 11.4 9.8 7.99 16 13.5 12.4 10.3 8.8 14 14.5 13 10.86 9.68 10 18.5 17.2 13.55 11.86 8 22 21.4 6 29 26.6 *****Table properly viewed in the attached PDF******** Table 1: Time results for the benchmarking******** These were also plotted in figure 1 with additional benchmarking tests as well. It is quite evident that the trend of increasing core counts helps up to roughly 16 cores, beyond which the reduction in time is subtler. However increasing the core speed certainly drop as we can see that from 2 GHz (Brown) to 2.2 GHz (Bob), the overall times drop for the same core counts. Thunder (3.1Ghz) being 1.5 times faster than Brown, gives almost double the speed for the simulation time with Linux. The memory is of great importance as well as the older systems have DDR 3 whereas Thunder has 2400 MHz DDR4 which is nearly 1.5 times higher in clock speeds. The newer v3 Xeon processors also have higher overall bus speeds (Thunder with 9.6 GT/s) should be quite important here as well. I believe that quite a bit of the overhead falls from memory communications and hence should be optimized. Hence the single DIMM per channel configuration. *****Figure 1 cannot be plotted in the box here, please look at the attached pdf******* Figure 1: Plot of benchmarking. Brown (Brown line), Bob (orange), Thunder with Win7 HT on (Grey), Thunder with Win7 HT off (Yellow), Thunder with Linux and HT off (Blue). There are two separate points for Thunder Win7 without multicore control enabled in COMSOL for HT on (green) and HT off (red).********** HT does add an overhead and this gets worse for more cores as can be seen with the Win7 results in figure 1. Interestingly, even though people mentioned on forums that swapping between Linux and Win7 shouldn’t matter, results show that it matters a lot. I used Scientific Linux (v7) and the time for 20 cores dropped by nearly 2 mins!!! Also Win7 won’t be able to make use of the 256 GB of RAM, it is limited to 192GB, hence for higher mesh simulations requiring over 200GB (some crazy simulations I plan to do!), Linux would need to be used. From experience, using a batch script to run the simulation also drop a bit of time, probably due to with the GUI using the resources. All the results for Brown and Bob were with script runs for optimal results, I have not done so with Thunder as yet. On Win7 closing any additional CPU monitors of programs shaved a few seconds as well. Lastly, unchecking the multicore control changed simulation times slightly as well, though this could be just a bit of variation from sim to sim, it increased for when HT was on and decreased when HT was off. I hope this helps others! Affar Karimullah.

Attachments:

• Workstation benchmark report.pdf