Note: This discussion is about an older version of the COMSOL Multiphysics® software. The information provided may be out of date.

Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.

3D RF Module Calculation on Multiple Cores

Please login with a confirmed email address before reporting spam

I am planning to build a computer for running Comsol, and was hoping to find out how well Comsol can utilize multiple cores for RF Module calculations. The most demanding calculation that we would like to run is calculating the evolution of the electromagnetic field inside a waveguide in 3D.

Does anyone have experience running similar calculations on multiple cores? We are worried that the matrix inversions required cannot easily be split over multiple cores, or that the memory requirements would increase significantly if the job were split over multiple cores. If that is the case, we may be better off purchasing a computer with fewer, faster cores (for example, maybe 8 high-end Xeon cores, 128 GB memory). If parallel computing works well for this problem, we can get more total computing power for cheaper by building a computer with more, slower cores (say 24 mid-range Opteron cores).

Thanks very much for any suggestions!

8 Replies Last Post 20 dic 2013, 06:27 GMT-5
Edgar J. Kaiser Certified Consultant

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 15 ago 2013, 12:33 GMT-4
Hi

I am sometimes using a dual Xeon machine with 16 physical cores (2 x 8) with 256 GBytes and most times a 4 core i7 machine with 64 GByte under Windows. I don't see that memory demand depends on the number of cores.
The task manager shows 32 or 8 cores (physical + hyperthreaded). It shows CPU load at 50 % in each machine, so hyperthreading is not used.

Interestingly, if I start a second COMSOL instance on the same machine and run two models in parallel, the CPU load reaches 100%. But I haven't done benchmarking so far, so I don't know if each of the two instances runs as fast as a single instance on the machine.
--
Edgar J. Kaiser
emPhys Physical Technology
Hi I am sometimes using a dual Xeon machine with 16 physical cores (2 x 8) with 256 GBytes and most times a 4 core i7 machine with 64 GByte under Windows. I don't see that memory demand depends on the number of cores. The task manager shows 32 or 8 cores (physical + hyperthreaded). It shows CPU load at 50 % in each machine, so hyperthreading is not used. Interestingly, if I start a second COMSOL instance on the same machine and run two models in parallel, the CPU load reaches 100%. But I haven't done benchmarking so far, so I don't know if each of the two instances runs as fast as a single instance on the machine. -- Edgar J. Kaiser emPhys Physical Technology

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 15 ago 2013, 15:58 GMT-4
Thanks Edgar! I am aware that Comsol is capable of running on multiple cores in general. What I specifically want to know is if RF module calculations of electromagnetic fields are a problem which scales well to multiple cores using the current version of Comsol. I am worried that it might have one of the following issues:
1) The problem cannot be parallelized efficiently. If the step which takes the majority of the calculation time can only utilize one core, then running on multiple cores would not improve calculation time significantly.
2) Asking this calculation to run on multiple cores will increase the memory required significantly. I believe Edgar is saying that for his tasks the amount of memory required does not change when running on more cores. However, the hardware recommendations here (www.comsol.com/support/knowledgebase/866/) suggest that you have 4-8 GB memory per core, which suggests to me that at least some tasks must have memory requirements which scale with the number of cores used. RF module calculations are very demanding, and often run out of memory on the machine we use in a collaborator's lab (which has 64 GB RAM) even when running only on a single core.

With regard to hyperthreading - you never want to use hyperthreading for running any type of calculation. Hyperthreading causes each CPU core to be time-shared between two tasks, and switch back and forth between working on each one. Forcing the computer to run two instances of Comsol and use hyperthreading will cause each instance to run less than half as fast as if it were running on its own, resulting in the overall calculation time being slower, because time is wasted when swapping the two tasks between the cores. Not to mention it will use twice as much memory. Hyperthreading is useful for multitasking (for example, a server communicating with many clients) but not for running a single demanding task.
Thanks Edgar! I am aware that Comsol is capable of running on multiple cores in general. What I specifically want to know is if RF module calculations of electromagnetic fields are a problem which scales well to multiple cores using the current version of Comsol. I am worried that it might have one of the following issues: 1) The problem cannot be parallelized efficiently. If the step which takes the majority of the calculation time can only utilize one core, then running on multiple cores would not improve calculation time significantly. 2) Asking this calculation to run on multiple cores will increase the memory required significantly. I believe Edgar is saying that for his tasks the amount of memory required does not change when running on more cores. However, the hardware recommendations here (http://www.comsol.com/support/knowledgebase/866/) suggest that you have 4-8 GB memory per core, which suggests to me that at least some tasks must have memory requirements which scale with the number of cores used. RF module calculations are very demanding, and often run out of memory on the machine we use in a collaborator's lab (which has 64 GB RAM) even when running only on a single core. With regard to hyperthreading - you never want to use hyperthreading for running any type of calculation. Hyperthreading causes each CPU core to be time-shared between two tasks, and switch back and forth between working on each one. Forcing the computer to run two instances of Comsol and use hyperthreading will cause each instance to run less than half as fast as if it were running on its own, resulting in the overall calculation time being slower, because time is wasted when swapping the two tasks between the cores. Not to mention it will use twice as much memory. Hyperthreading is useful for multitasking (for example, a server communicating with many clients) but not for running a single demanding task.

Edgar J. Kaiser Certified Consultant

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 15 ago 2013, 16:21 GMT-4
Robert,

I also suspect that running parallel instances is not a benefit regarding time. Sometimes I have different projects and I can run parallel tasks over the weekend, when there may be spare time. So coming back on Monday morning I have two models solved and not just one.

Regarding my experience with multiple cores. Those models were RF models with full wave equations, such as microwave antennas.

And one additional tip: you can tell COMSOL how many cores to use in the preference settings. This might be useful for testing.

Cheers
Edgar

--
Edgar J. Kaiser
emPhys Physical Technology
Robert, I also suspect that running parallel instances is not a benefit regarding time. Sometimes I have different projects and I can run parallel tasks over the weekend, when there may be spare time. So coming back on Monday morning I have two models solved and not just one. Regarding my experience with multiple cores. Those models were RF models with full wave equations, such as microwave antennas. And one additional tip: you can tell COMSOL how many cores to use in the preference settings. This might be useful for testing. Cheers Edgar -- Edgar J. Kaiser emPhys Physical Technology

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 15 ago 2013, 16:32 GMT-4
Oh - great! Then it sounds like running on multiple cores should work at least decently well. Thanks very much for clarifying that you were solving a similar problem.

I think, as you suggest, we may have to try running our problem (perhaps in a coarse grained version on a desktop PC) using different numbers of cores to see how the required time and memory usage scales with the number of cores used.

With regard to running multiple tasks at once - you might want to consider telling each Comsol instance to only use a fraction of the real cores on your computer, such that all running instances use all the real cores, but no hyperthreaded cores. This would be more efficient. Then again, if you problem gets done while you are not there, maybe it doesn't matter if it could have been finished a little faster.

Thanks again for your help!
Oh - great! Then it sounds like running on multiple cores should work at least decently well. Thanks very much for clarifying that you were solving a similar problem. I think, as you suggest, we may have to try running our problem (perhaps in a coarse grained version on a desktop PC) using different numbers of cores to see how the required time and memory usage scales with the number of cores used. With regard to running multiple tasks at once - you might want to consider telling each Comsol instance to only use a fraction of the real cores on your computer, such that all running instances use all the real cores, but no hyperthreaded cores. This would be more efficient. Then again, if you problem gets done while you are not there, maybe it doesn't matter if it could have been finished a little faster. Thanks again for your help!

Robert Koslover Certified Consultant

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 15 ago 2013, 21:21 GMT-4
I run 3D RF/microwave computations often, usually across 12 cores. The parallel direct solvers (PARDISO, MUMPS) make excellent use of them. For problems too large for those solvers, even the memory-efficient iterative solvers (e.g., GMRES) can make pretty good use of the multiple cores. I've found the performance advantage of using 12 cores vs. 4 cores to be well worth it.
I run 3D RF/microwave computations often, usually across 12 cores. The parallel direct solvers (PARDISO, MUMPS) make excellent use of them. For problems too large for those solvers, even the memory-efficient iterative solvers (e.g., GMRES) can make pretty good use of the multiple cores. I've found the performance advantage of using 12 cores vs. 4 cores to be well worth it.

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 19 dic 2013, 06:49 GMT-5
I hope that it is okay to bump and old thread.

I currently work an HP Z420 with 6 physical cores and 6 hyper-threading cores. Just as you experienced COMSOL never used more than 50% of my CPU power on one model. If a ran two or more it would use up all of my CPU power.

However by manually setting the number of cores to 12 (the actually number the OS sees) COMSOL know uses all the CPU power even when solving just one model.

I am currently testing if the is a speed increase or not.
I hope that it is okay to bump and old thread. I currently work an HP Z420 with 6 physical cores and 6 hyper-threading cores. Just as you experienced COMSOL never used more than 50% of my CPU power on one model. If a ran two or more it would use up all of my CPU power. However by manually setting the number of cores to 12 (the actually number the OS sees) COMSOL know uses all the CPU power even when solving just one model. I am currently testing if the is a speed increase or not.

Edgar J. Kaiser Certified Consultant

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 19 dic 2013, 07:03 GMT-5

Hi Simon,

interesting! Let us know how it works out for you.

Cheers
Edgar

--
Edgar J. Kaiser
emPhys Physical Technology
www.emphys.com
Hi Simon, interesting! Let us know how it works out for you. Cheers Edgar -- Edgar J. Kaiser emPhys Physical Technology http://www.emphys.com

Please login with a confirmed email address before reporting spam

Posted: 1 decade ago 20 dic 2013, 06:27 GMT-5
Hmm it seems that there is no gain - perhaps even a loss. The initial/first test showed the following:

6 cores: 4 hours, 8 minutes 42 sec
12 cores: 4 hours, 32 minutes, 28 sec

This is for a problem regarding the flow in a pipe system and for 35 different conditions.

I deleted both the study and the data set between the two calculations.
Hmm it seems that there is no gain - perhaps even a loss. The initial/first test showed the following: 6 cores: 4 hours, 8 minutes 42 sec 12 cores: 4 hours, 32 minutes, 28 sec This is for a problem regarding the flow in a pipe system and for 35 different conditions. I deleted both the study and the data set between the two calculations.

Note that while COMSOL employees may participate in the discussion forum, COMSOL® software users who are on-subscription should submit their questions via the Support Center for a more comprehensive response from the Technical Support team.