Note: This discussion is about an older version of the COMSOL Multiphysics® software. The information provided may be out of date.
Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.
Running two GUI's to utilize remaining memory
Posted Jan 13, 2014, 1:39 p.m. EST Studies & Solvers Version 4.3b 4 Replies
Please login with a confirmed email address before reporting spam
COMSOL Community:
I currently have a frequency-swept simulation running that comprises 30GB physical /41GB virtual memory, off of an 8-core, 128GB system. Even though a sweep in off-normal incidence angles is desired, currently I am only solving at one angle, and over a smaller bandwidth, out of concerns of failure over days/weeks-long nested sweeps.
I feel that with this lack of full memory usage I am not working as efficiently as possible. My question is: would there be any benefit in running a second GUI to concurrently solve at the next angle, or even solve at the same angle, but over a different bandwidth? Is there any other solution to be more efficient--i.e. maximize memory usage?
Thanks!
-Bryan
I currently have a frequency-swept simulation running that comprises 30GB physical /41GB virtual memory, off of an 8-core, 128GB system. Even though a sweep in off-normal incidence angles is desired, currently I am only solving at one angle, and over a smaller bandwidth, out of concerns of failure over days/weeks-long nested sweeps.
I feel that with this lack of full memory usage I am not working as efficiently as possible. My question is: would there be any benefit in running a second GUI to concurrently solve at the next angle, or even solve at the same angle, but over a different bandwidth? Is there any other solution to be more efficient--i.e. maximize memory usage?
Thanks!
-Bryan
4 Replies Last Post Jan 15, 2014, 5:06 p.m. EST
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Bryan,
efficiency is not just about memory usage. In practice it will depend on the specific problem, particularly on the solver you use. If your problem can be parallelized well and the solver is capable to comply, you won't take much benefit from a second COMSOL instance, you will even loose efficiency due to swapping.
If the process cannot be distributed on many CPUs a second instance might help.
If efficiency is really critical, I would recommend to try and benchmark several approaches.
Cheers
Edgar
--
Edgar J. Kaiser
emPhys Physical Technology
www.emphys.com
efficiency is not just about memory usage. In practice it will depend on the specific problem, particularly on the solver you use. If your problem can be parallelized well and the solver is capable to comply, you won't take much benefit from a second COMSOL instance, you will even loose efficiency due to swapping.
If the process cannot be distributed on many CPUs a second instance might help.
If efficiency is really critical, I would recommend to try and benchmark several approaches.
Cheers
Edgar
--
Edgar J. Kaiser
emPhys Physical Technology
www.emphys.com
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Thank you greatly, Edgar!
The drop in efficiency from splitting processing time was the biggest concern I had. I have read that there are diminishing returns with increasing cores, beyond some certain number (reference: www.comsol.com/paper/download/181871/frommelt_abstract.pdf). So I may be more efficient if I could assign a single job to 4 cores, and another job to the remaining 4 cores. This paper states it is possible to address jobs to individual cores by running individual servers and running in single-thread mode, but do you have any idea if it can be done as I have stated (splitting into two multicores)?
EDIT: I know I can set the # of processors, but if I set it to 4, would this allow me to run a 2nd instance utilizing the other 4 processors? I would lose VM in this case; however, if I set it to 8, it would run the 1st instance using all 8 cores, and there would be no utility in running a 2nd instance! Seems like a lose-lose situation!
The drop in efficiency from splitting processing time was the biggest concern I had. I have read that there are diminishing returns with increasing cores, beyond some certain number (reference: www.comsol.com/paper/download/181871/frommelt_abstract.pdf). So I may be more efficient if I could assign a single job to 4 cores, and another job to the remaining 4 cores. This paper states it is possible to address jobs to individual cores by running individual servers and running in single-thread mode, but do you have any idea if it can be done as I have stated (splitting into two multicores)?
EDIT: I know I can set the # of processors, but if I set it to 4, would this allow me to run a 2nd instance utilizing the other 4 processors? I would lose VM in this case; however, if I set it to 8, it would run the 1st instance using all 8 cores, and there would be no utility in running a 2nd instance! Seems like a lose-lose situation!
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Bryan,
you have the machine. So you can simply try it. Let us know about your results. And keep in mind that different solvers will give different results.
Cheers
Edgar
--
Edgar J. Kaiser
emPhys Physical Technology
www.emphys.com
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
SOLUTION: use batch sweeps!
This was a feature I'd never used before. After looking into it, a Batch Sweep will allow me to sweep over a parameter (setting each parameter to a separate .mph and separate job), set any number of simultaneous jobs, and evenly distribute the # of cores based on the number of simultaneous jobs (under "Batch 1" under "Job Configurations"). So for my system, I can run two jobs with 8 dedicated cores each (I misspoke earlier, I have 16 cores, not 8), or 4 jobs with 4 cores each.
It seems this is a good way for people to better utilize single-node servers that have a large # of cores, since several results on parallel computing show diminishing returns after several cores/threads.
I am in a time crunch, so I cannot perform any elaborate benchmarking. BUT, I will try to post the average simulation time/frequency of my single job/all-core vs multi-jobs/distributed cores.
-Bryan
This was a feature I'd never used before. After looking into it, a Batch Sweep will allow me to sweep over a parameter (setting each parameter to a separate .mph and separate job), set any number of simultaneous jobs, and evenly distribute the # of cores based on the number of simultaneous jobs (under "Batch 1" under "Job Configurations"). So for my system, I can run two jobs with 8 dedicated cores each (I misspoke earlier, I have 16 cores, not 8), or 4 jobs with 4 cores each.
It seems this is a good way for people to better utilize single-node servers that have a large # of cores, since several results on parallel computing show diminishing returns after several cores/threads.
I am in a time crunch, so I cannot perform any elaborate benchmarking. BUT, I will try to post the average simulation time/frequency of my single job/all-core vs multi-jobs/distributed cores.
-Bryan