Right, this is TR 2000/ Zen+ versus Ryzen 3000/Zen2, which makes a difference here.

Yep, very obvious here after looking at the Top Hosts list. Nothing but Threadripper 3000's and Ryzen 3000's. The codepath through the FPU must be really optimized for the BHspin2 application.

Does anybody know if it uses advanced SIMD instructions like AVX2 or FMA? Those are the major architectural differences between Zen+ and Zen 2 plus the 256bit wide AVX registers.

Can you not see this in the Boinc startup? I have this line:
Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx smx tm2 dca pbe

I'm not speaking of the cpu features, I am asking what features the application uses. Nothing descriptive in the application name for any hints. Also as I stated previously, no parameter string is visible on the application when examining a task. Both Einstein and Milkyway show for example a huge string of parameters attached to very task. That gives some clues about the application codepath.

[Edit] Not much can be gleaned about the parameter set in the slot either. The param.in file has this structure which I don't understand yet.

BHSPIN large grid v:190723
SET num_tested 20000
SET idum -705100
SET OUTPUT 3
SET PSN 1
SET PPSN 0
SET Sal -1.9
SET Alfa 0.1
SET WIND1 1
SET WIND2 1
SET kick 6
SET Sigma3 132.5
SET ECSlower 0.18867924528302

SET Fa 0
SET Beta 1
SET ZZ 0.0065

---

A proud member of the OFA (Old Farts Association)

Keith Myers wrote:

Does anybody know if it uses advanced SIMD instructions like AVX2 or FMA?

It most certainly does not. On Broadwell-EP for instance, BHspin v2 does not trigger the AVX2/FMA clock frequency offset.


Keith Myers wrote:

Those are the major architectural differences between Zen+ and Zen 2 plus the 256bit wide AVX registers.

Classic FPU code throughput per core was circa doubled as well, wasn't it?

Keith Myers wrote:

Does anybody know if it uses advanced SIMD instructions like AVX2 or FMA?

It most certainly does not. On Broadwell-EP for instance, BHspin v2 does not trigger the AVX2/FMA clock frequency offset.


Keith Myers wrote:

Those are the major architectural differences between Zen+ and Zen 2 plus the 256bit wide AVX registers.

Classic FPU code throughput per core was circa doubled as well, wasn't it?

Thanks for the information that AVX isn't being used. Thought that might be the key. But if the AVX offset for Broadwell isn't being triggered, I guess it isn't.

From a techreport article:

AMD also addressed a major competitive shortcoming of the Zen architecture for high-performance computing applications. The first Zen cores used 128-bit-wide registers to execute SIMD instructions, and in the case of executing 256-bit-wide AVX2 instructions, each Zen floating-point unit had to shoulder half of the workload. Compared to Intel's Skylake CPUs (for just one example), which have two 256-bit-wide SIMD execution units capable of independent operation, Ryzen CPUs offered half the throughput for floating-point and integer SIMD instructions.

Zen 2 addresses this shortcoming by doubling each core's SIMD register width to 256 bits. The floating-point side of the Zen 2 core has two 256-bit floating-point add units and two floating-point multiply units that can presumably be yoked together to perform two fused multiply-add operations simultaneously.

That capability would bring the Zen 2 core on par with the Skylake microarchitecture for SIMD throughput (albeit not the Skylake Server core, which boasts even wider data paths and 512-bit-wide SIMD units to support AVX-512 instructions.) To feed those 256-bit-wide execution engines, AMD also widened the load-store unit, load data path, and floating-point register file to support 256-bit chunks of data.

---

A proud member of the OFA (Old Farts Association)

Yes. We got a developer who try to make GPU app for the project, unfortunately he resign because he got struggles to port application algorithm to GPU. This is not a surprise as our application base was written since 2002 when nobody expect GPU's with compute possibilities...

This is very common problem with science app that not all algorithms can be ported to GPU.

But...

We still think about it, slowly change and clear code and maybe... But this not happens in this year definitely :(

That is a shame. But I will be adding 48 cores shortly on two xeon servers I'm building from scrap parts. If the pentathlon hasn't eaten all your tasks or melted your ethernet I'll do some more.

You'd get much faster runtimes if you switched to Linux ;)

You'd get much faster runtimes if you switched to Linux ;)

I find it hard to believe the OS has anything to do with processing time. Sure, maybe Windows uses 2% of the CPU and Linux 1%, but that only changes Boinc from 98% to 99%, not worth doing. Surely the Universe task is written as well as the Universe coders write it, and makes use of whatever instruction set my CPU has. The OS it's under cannot change how fast it is.

Some explanation.

BHSPIN large grid v:190723
SET num_tested 20000
SET idum -705100
SET OUTPUT 3
SET PSN 1
SET PPSN 0
SET Sal -1.9
SET Alfa 0.1
SET WIND1 1
SET WIND2 1
SET kick 6
SET Sigma3 132.5
SET ECSlower 0.18867924528302

SET Fa 0
SET Beta 1
SET ZZ 0.0065

"BHSPIN large grid v:190723"
This is our internal information about source code used for this tasks batch.

"SET num_tested 20000"
Quantity of simulations to be done with this particular task.

"SET idum -705100"
Parameter from 0 to 1'000'000, every single batch is an array of all elements from this array with step of 100.

"SET OUTPUT 3"
Informs application which method of simulation have to be done. "3" is BHSpin.

Other parameters are simulation starting values and are taken from large list of values. Most of them are part of arrays with 3 to 32 values except ECSlower - it is calculated by work generator from different variables.

---

Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT

You'd get much faster runtimes if you switched to Linux ;)

I find it hard to believe the OS has anything to do with processing time. Sure, maybe Windows uses 2% of the CPU and Linux 1%, but that only changes Boinc from 98% to 99%, not worth doing. Surely the Universe task is written as well as the Universe coders write it, and makes use of whatever instruction set my CPU has. The OS it's under cannot change how fast it is.

Universe@home is much faster under Linux than under Windows.
Confirmed by the project's admins. And by my observations and that of others.

See this post and thread: https://universeathome.pl/universe/forum_thread.php?id=526&postid=4183

OS does make a difference. Universe is faster on Linux than on Windows, TN-Grid is faster on Linux than on Windows, all the AutoDock VINA sciences at WCG were much much faster under Linux than Windows. Some applications just run better on Linux. Others run the same.

For example, most of my Universe@home tasks on Ubuntu 20.04 on my AMD Ryzen 5 1400 3.2 GHz take just over 1 hour to finish. On WIndows,and from memory, closer to 3 hours.

If those 2 servers are dedicated crunchers, try a Live USB of Ubuntu on one of them and see if it's worth it.

Universe@home is much faster under Linux than under Windows.
Confirmed by the project's admins. And by my observations and that of others.

See this post and thread: https://universeathome.pl/universe/forum_thread.php?id=526&postid=4183

OS does make a difference. Universe is faster on Linux than on Windows, TN-Grid is faster on Linux than on Windows, all the AutoDock VINA sciences at WCG were much much faster under Linux than Windows. Some applications just run better on Linux. Others run the same.

For example, most of my Universe@home tasks on Ubuntu 20.04 on my AMD Ryzen 5 1400 3.2 GHz take just over 1 hour to finish. On WIndows,and from memory, closer to 3 hours.

If those 2 servers are dedicated crunchers, try a Live USB of Ubuntu on one of them and see if it's worth it.

I won't begin to try to understand why it's faster. And I've never used Linux before. But you're correct, those two dual xeons were built out of scrap parts to do nothing but Boinc. I guess it can't be that hard to install Linux. And since they're almost identical (all 4 CPUs are the same, one has a slightly newer motherboard), I can then do direct comparisons on speed of the 5 projects I run (Rosetta, Universe, LHC on CPU) (Einstein, Milkyway on GPU). Now which flavour? I'm going to go for Ubuntu, since that seems to be the one for beginners. Am I correct in assuming one won't make the projects run much faster than the other? And the only difference is the interface? I have a couple of 8GB USB sticks, so I'll try....

Ubuntu is fine. The only difference between Ubuntu flavours is RAM usage and maybe a little bit of CPU because of whatever desktop environments they use (the interface)

Xubuntu should use less RAM and maybe a little bit less of CPU too. Other than that, no difference.

You can just run it off the USB if you'd like to try it first (once you reboot, everything you install or save is reset though).

Ubuntu is fine. The only difference between Ubuntu flavours is RAM usage and maybe a little bit of CPU because of whatever desktop environments they use (the interface)

Xubuntu should use less RAM and maybe a little bit less of CPU too. Other than that, no difference.

You can just run it off the USB if you'd like to try it first (once you reboot, everything you install or save is reset though).

So where are programs installed? Into a RAM drive? I'd assumed the USB stick was behaving like the hard disk.

Anyway that will do. I can quickly install Boinc and attach to a single project and directly compare it to the other server still running Windows 10.

Ubuntu is fine. The only difference between Ubuntu flavours is RAM usage and maybe a little bit of CPU because of whatever desktop environments they use (the interface)

Xubuntu should use less RAM and maybe a little bit less of CPU too. Other than that, no difference.

You can just run it off the USB if you'd like to try it first (once you reboot, everything you install or save is reset though).

So where are programs installed? Into a RAM drive? I'd assumed the USB stick was behaving like the hard disk.

Anyway that will do. I can quickly install Boinc and attach to a single project and directly compare it to the other server still running Windows 10.

I believe it boots into the RAM so everything is lost afterwards.
Once you boot up, just find the software manager, look for BOINC and install.

I believe it boots into the RAM so everything is lost afterwards.
Once you boot up, just find the software manager, look for BOINC and install.

I've downloaded xubuntu-20.04-desktop-amd64.iso and put it onto a bootable 8GB USB stick using Rufus 3.10. I will have a go at it tomorrow (as in Monday afternoon, GMT). If Universe is significantly faster, I'll try the other projects aswell, if the majority are faster, I'll stick with it.

For example, most of my Universe@home tasks on Ubuntu 20.04 on my AMD Ryzen 5 1400 3.2 GHz take just over 1 hour to finish. On WIndows,and from memory, closer to 3 hours.

If they go three times faster, does that mean the CPUs are working 3 times harder? And generating 3 times as much heat by using 3 times the electricity? I'll use a watt meter on the wall socket when I do this to find out. If they aren't hotter, it's even stranger that they can do more work without using more power (or is Windows inserting extra pointless instructions in there?). If they are hotter, I'm going to have to buy bigger fans.

For example, most of my Universe@home tasks on Ubuntu 20.04 on my AMD Ryzen 5 1400 3.2 GHz take just over 1 hour to finish. On WIndows,and from memory, closer to 3 hours.

If they go three times faster, does that mean the CPUs are working 3 times harder? And generating 3 times as much heat by using 3 times the electricity? I'll use a watt meter on the wall socket when I do this to find out. If they aren't hotter, it's even stranger that they can do more work without using more power (or is Windows inserting extra pointless instructions in there?). If they are hotter, I'm going to have to buy bigger fans.

No difference in temperatures. Around 50º Celsius on both Operating systems.
I did not check with Universe@home, but with the SCC project over at WCG, which ran much faster on Linux, power usage was the about same on both Operating systems (around 83 watts).

Over at WCG, it was speculated by some members that the speedup verified with the AutoDock Vina sciences (SCC, etc) was due to better Linux memory management.

I can't comment on GPU performance under Linux, nor on LHC CPU, but Rosetta@home, considering that the runtime is fixed, we would have to run the same workunit on Windows and Linux and check the number of decoys/structures computed over the same 8/12/16/24 hours (whatever is set on your Roseta@home preferences) to see under which OS more structures/decoys were computed. So it will be hard to judge which OS is better on the Rosetta project.

For example, most of my Universe@home tasks on Ubuntu 20.04 on my AMD Ryzen 5 1400 3.2 GHz take just over 1 hour to finish. On WIndows,and from memory, closer to 3 hours.

If they go three times faster, does that mean the CPUs are working 3 times harder? And generating 3 times as much heat by using 3 times the electricity? I'll use a watt meter on the wall socket when I do this to find out. If they aren't hotter, it's even stranger that they can do more work without using more power (or is Windows inserting extra pointless instructions in there?). If they are hotter, I'm going to have to buy bigger fans.

No difference in temperatures. Around 50º Celsius on both Operating systems.
I did not check with Universe@home, but with the SCC project over at WCG, which ran much faster on Linux, power usage was the about same on both Operating systems (around 83 watts).

Over at WCG, it was speculated by some members that the speedup verified with the AutoDock Vina sciences (SCC, etc) was due to better Linux memory management.

I can't comment on GPU performance under Linux, nor on LHC CPU, but Rosetta@home, considering that the runtime is fixed, we would have to run the same workunit on Windows and Linux and check the number of decoys/structures computed over the same 8/12/16/24 hours (whatever is set on your Roseta@home preferences) to see under which OS more structures/decoys were computed. So it will be hard to judge which OS is better on the Rosetta project.

The runtime on Rosetta isn't fixed, look at my recent validated tasks:
https://boinc.bakerlab.org/rosetta/results.php?userid=104432&offset=0&show_names=0&state=4&appid=
They vary from 25,000 to 43,000 seconds (7 to 12 hours) over 6 computers.

Ok, so perhaps the runtime is not precisely fixed, but it tries to be. Observe how I got much more credit for some tasks than others, even though the CPU time was the same. The amount of credit given would be a good measure I think. Presumably it's related to the number of FLOPS done.

It's quite odd, because if I look at a running Rosetta task on my fastest computer and a running Rosetta task on my slowest computer, they both have the same task size and estimated speed in the task's properties. Perhaps those are just markers to give Boinc an idea of run time.

Over at WCG, it was speculated by some members that the speedup verified with the AutoDock Vina sciences (SCC, etc) was due to better Linux memory management.

We could find the answer very quickly by looking at a few hosts of different users in the results and looking at times for two computers with the same CPU but different OS. But I'm not sure how (if it's even possible without admin rights) to search through and find two similar computers.

https://boinc.bakerlab.org/rosetta/prefs.php?subset=project

"Target CPU Time"

The runtime is fixed. Default is 8 hours (usually ends a little before in my experience). However, if the task is working on a model that hasn't finished by the time it reaches 8 hours of CPU time (or whatever is set on the preferences), it has 4 more CPU hours to finish that model. If it doesn't, the "watchdog" software kicks in and kills the task. Which I guess explains your 7 hours (8 in prefs, I assume) up to 12 hours (the 4-hour cut-off) range across your tasks..


See my post https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13644&postid=92449 and the following post from a Rosetta moderator.

The link you sent didn't work but I've seen you on the Rosetta forums so I easily found your tasks. Most of your tasks seem to respect the 8 hour target.
As to why some end earlier, I don't really know but I do remember 1 or 2 that ended below 7 hours. Maybe the Rosetta software thinks it wouldn't finish the next model in the time left till the target runtime so it just stops there?

The way I understand Rosetta is that if not for a CPU runtime target or the watchdog, the Rosetta software would go on generating structures and decoys for whatever protein input "forever".

https://boinc.bakerlab.org/rosetta/prefs.php?subset=project

"Target CPU Time"

The runtime is fixed. Default is 8 hours (usually ends a little before in my experience). However, if the task is working on a model that hasn't finished by the time it reaches 8 hours of CPU time (or whatever is set on the preferences), it has 4 more CPU hours to finish that model. If it doesn't, the "watchdog" software kicks in and kills the task. Which I guess explains your 7 hours (8 in prefs, I assume) up to 12 hours (the 4-hour cut-off) range across your tasks..


See my post https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13644&postid=92449 and the following post from a Rosetta moderator.

The link you sent didn't work but I've seen you on the Rosetta forums so I easily found your tasks. Most of your tasks seem to respect the 8 hour target.
As to why some end earlier, I don't really know but I do remember 1 or 2 that ended below 7 hours. Maybe the Rosetta software thinks it wouldn't finish the next model in the time left till the target runtime so it just stops there?

The way I understand Rosetta is that if not for a CPU runtime target or the watchdog, the Rosetta software would go on generating structures and decoys for whatever protein input "forever".

Interesting. And cool that every project can run completely differently within the Boinc structure.

I think the best way to test Rosetta, and I'll try it tomorrow, is to set it doing some tasks in Ubuntu, wait until they get validated, then see how many points I get compared to the same amount of run time in Windows. I'll test Universe first though as that should be easiest, then LHC (if I can work out how to install Virtualbox in Ubuntu!), then Milkyway and Einstein on GPU (if I can get a Linux driver for the GPU), then I'll leave it overnight on Rosetta and check the validated results the next day.

https://boinc.bakerlab.org/rosetta/prefs.php?subset=project

"Target CPU Time"

The runtime is fixed. Default is 8 hours (usually ends a little before in my experience). However, if the task is working on a model that hasn't finished by the time it reaches 8 hours of CPU time (or whatever is set on the preferences), it has 4 more CPU hours to finish that model. If it doesn't, the "watchdog" software kicks in and kills the task. Which I guess explains your 7 hours (8 in prefs, I assume) up to 12 hours (the 4-hour cut-off) range across your tasks..


See my post https://boinc.bakerlab.org/rosetta/forum_thread.php?id=13644&postid=92449 and the following post from a Rosetta moderator.

The link you sent didn't work but I've seen you on the Rosetta forums so I easily found your tasks. Most of your tasks seem to respect the 8 hour target.
As to why some end earlier, I don't really know but I do remember 1 or 2 that ended below 7 hours. Maybe the Rosetta software thinks it wouldn't finish the next model in the time left till the target runtime so it just stops there?

The way I understand Rosetta is that if not for a CPU runtime target or the watchdog, the Rosetta software would go on generating structures and decoys for whatever protein input "forever".

Interesting. And cool that every project can run completely differently within the Boinc structure.

I think the best way to test Rosetta, and I'll try it tomorrow, is to set it doing some tasks in Ubuntu, wait until they get validated, then see how many points I get compared to the same amount of run time in Windows. I'll test Universe first though as that should be easiest, then LHC (if I can work out how to install Virtualbox in Ubuntu!), then Milkyway and Einstein on GPU (if I can get a Linux driver for the GPU), then I'll leave it overnight on Rosetta and check the validated results the next day.

Good luck.

Installing VirtualBox should be as easy as installing BOINC. Both are in the Ubuntu repositories.
Open the software Center, search for BOINC and VirtualBox, install. Then run.

I do not know if you need to reboot because of Virtualbox so that's something to consider when running of the USB. Anyway, try installing Virtualbox first and then BOINC and see if it works.

If you encounter issues, feel free to post here!

Good luck.

Installing VirtualBox should be as easy as installing BOINC. Both are in the Ubuntu repositories.
Open the software Center, search for BOINC and VirtualBox, install. Then run.

I do not know if you need to reboot because of Virtualbox so that's something to consider when running of the USB. Anyway, try installing Virtualbox first and then BOINC and see if it works.

If you encounter issues, feel free to post here!

Thanks for your help. I'll get round to this eventually, but have been too busy with other things, I'll post the speeds I get on the 5 projects I use when I get Linux running.