Message boards : Number crunching : Double your task throughput on Linux
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
shift
Avatar

Send message
Joined: 22 May 18
Posts: 4
Credit: 4,414,233
RAC: 0
Message 4634 - Posted: 16 Feb 2021, 21:08:16 UTC - in response to Message 4629.  

From eyeballing my results from windows vs ubuntu I reckon that universe@home is 3 times faster on ubuntu and einstein@home is 4.5 times faster on ubuntu. It looks likes rosetta may be faster on windows and the other projects I run seem to be similar on windows and ubuntu. A pity I can't get Roblox (the kids game) to play on ubuntu then.
ID: 4634 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mikey
Avatar

Send message
Joined: 4 Apr 15
Posts: 46
Credit: 43,128,567
RAC: 0
Message 4637 - Posted: 23 Feb 2021, 4:00:26 UTC - in response to Message 4634.  

From eyeballing my results from windows vs ubuntu I reckon that universe@home is 3 times faster on ubuntu and einstein@home is 4.5 times faster on ubuntu. It looks likes rosetta may be faster on windows and the other projects I run seem to be similar on windows and ubuntu. A pity I can't get Roblox (the kids game) to play on ubuntu then.


Could be the overhead Windows imposes on everything as opposed to the much smaller Linux overhead.
ID: 4637 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mikey
Avatar

Send message
Joined: 4 Apr 15
Posts: 46
Credit: 43,128,567
RAC: 0
Message 4638 - Posted: 23 Feb 2021, 4:00:44 UTC - in response to Message 4634.  

From eyeballing my results from windows vs ubuntu I reckon that universe@home is 3 times faster on ubuntu and einstein@home is 4.5 times faster on ubuntu. It looks likes rosetta may be faster on windows and the other projects I run seem to be similar on windows and ubuntu. A pity I can't get Roblox (the kids game) to play on ubuntu then.


Could be the overhead Windows imposes on everything as opposed to the much smaller Linux overhead.
ID: 4638 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rsNeutrino

Send message
Joined: 1 Nov 17
Posts: 29
Credit: 291,940,933
RAC: 415
Message 4641 - Posted: 24 Feb 2021, 20:03:20 UTC - in response to Message 4638.  
Last modified: 24 Feb 2021, 20:04:07 UTC

Could be the overhead Windows imposes on everything as opposed to the much smaller Linux overhead.


I'm not sure this is it, because I also did some tests with Linux Mint before using Ubuntu in Hyper-V and there was a split in task accelleration... some tasks took 3 hours while some just took 1 hour like in Ubuntu:
Another possibility is that Virtualbox did something funky with the emulation, but then also only in Mint and not Ubuntu...

ID: 4641 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 308
Credit: 4,733,484,700
RAC: 14,480
Message 4642 - Posted: 25 Feb 2021, 1:21:46 UTC

When comparing Mint to Ubuntu, make sure both are running a glibc library of at least 2.31.

We noticed the speedup was specific to the glibc library when running Ubuntu 18.04 with kernel 5.4 and glibc 2.29 against Ubuntu 20.04 with kernel 5.4 and glibc library 2.31. So even with the kernel versions being exactly the same, it was the glibc version that mattered that caused the significant speedup in Universe tasks.

A proud member of the OFA (Old Farts Association)
ID: 4642 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum
Avatar

Send message
Joined: 9 Sep 17
Posts: 14
Credit: 580,314,930
RAC: 0
Message 4644 - Posted: 25 Feb 2021, 18:59:46 UTC

Just checked Linux Mint 20.1 (Ubuntu 20.04) with the latest kernel 5.4.0-66-generic x86_64 and it installs glibc 2.31. But when I checked https://www.gnu.org/software/libc/ it says they've released 2.33. The Synaptic Manager does not list glibc so I don't know what package includes it.
When I searched for how to get LM to upgrade to glibc 2.33 I didn't find anything but did see a warning not to try it since stuff will break.
Is glibc 2.31 sufficient or might glibc 2.33 give better throughput???
ID: 4644 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 308
Credit: 4,733,484,700
RAC: 14,480
Message 4645 - Posted: 25 Feb 2021, 20:55:32 UTC - in response to Message 4644.  

No, you can't update the glibc library via any package manager. It is tied too closely to the kernel and other libraries. The main problem is all the symbol libraries have to be updated for EVERY piece of software on the host. Just not realistically doable.

The 2.31 glibc library is totally sufficient. I have not noticed any noticeable performance gain for hosts running newer distros with higher glibc versions.

Whatever was in the 2.31 glibc library that the Universe apps liked, it still is present in later versions but they produce no significant improvements over 2.31.

A proud member of the OFA (Old Farts Association)
ID: 4645 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum
Avatar

Send message
Joined: 9 Sep 17
Posts: 14
Credit: 580,314,930
RAC: 0
Message 4646 - Posted: 25 Feb 2021, 21:24:31 UTC

Thanks Keith, ULX & BHspin are running at speeds similar to reported here.
ID: 4646 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
wolfman1360

Send message
Joined: 18 Feb 17
Posts: 7
Credit: 6,070,100
RAC: 0
Message 4650 - Posted: 7 Mar 2021, 19:33:00 UTC

Luckily I have a few Linux machines - well, more than a few, but Ubuntu 20.04 has been all kinds of trouble for me.
Still, this project is the only one that Windows truly falls flat on. A shame. There used to be some WCG subprojects where Linux faired much better, but now Windows is doing far better on one or two there. Sometimes Linux isn't always the end all, we won't talk about how many issues I've had with Nvidia GPU drivers and why I love my AMD. LHC at home has now conqored me, for now. I don't have the time or inclination to try and troubleshoot each evening after work, much less on weekends.
No idea why windows would be a contributing factor - it isn't as though this project demands a ton of resources, unless it is lacking a library of some sort?
ID: 4650 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ianab

Send message
Joined: 21 Jul 18
Posts: 1
Credit: 33,960,071
RAC: 99
Message 4699 - Posted: 30 Apr 2021, 7:52:51 UTC

I stumbled on this tread when trying to work why my Xeon E5 (non HT) machine was taking longer to do work units than my slower Xeon E3 (with HT) but running Linux. I had suspected that Linux vs Windows was the issue, just looking at results from different wingmen. . Linux almost always won, even against faster Windows machines.

So just wanted to say thanks for the confirmation / explanation, and the suggestion of using a VM if I get keen enough.

For the Windows guys that want to experiment with Linux, my suggestion is to dig out an old PC and set it up with Linux Mint. First Gen i3 etc can't be given away locally, it costs actual money to take them to the recycle centre. But throw a cheap SSD and another stick of RAM in there, load up Linux Mint, and you have a perfectly usable machine for Internet use. The machines that are too old or not worth repair I use as "organ donors". Then I can strip them out and sell for scrap, better to get $2 at the scrap yard than pay $20 at the transfer station.

Anyway, you can get a machine that will run Linux just fine for practically nothing. This is little "project". Started out as a free i3 based slimline business PC that I got for free. Used lower spec Xeons are cheap on AliExpress, so it's worth looking up the system board and seeing what it will support.
https://universeathome.pl/universe/show_host_detail.php?hostid=593120
If I switched it to Linux it would get a big boost on this project.
ID: 4699 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 8,758
Message 4779 - Posted: 24 Aug 2021, 15:45:58 UTC

https://universeathome.pl/universe/show_host_detail.php?hostid=599526

Hurray!
Made it into the top 40 :)

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 4779 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 8,758
Message 5028 - Posted: 2 Jan 2022, 3:33:13 UTC

Re-aimed 3950x CPU to mostly do U@h and the rac is rising nicely.
A proud member of the OFA (Old Farts Assoc.)
ID: 5028 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JohnDK

Send message
Joined: 5 Mar 22
Posts: 3
Credit: 36,465,333
RAC: 59
Message 5165 - Posted: 12 Mar 2022, 17:43:18 UTC

Installed Linux Mint in VirtualBox under Windows 11, works very good, much better runtimes.
ID: 5165 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
franz

Send message
Joined: 15 Jan 17
Posts: 5
Credit: 20,712,233
RAC: 0
Message 5199 - Posted: 26 Mar 2022, 7:23:33 UTC - in response to Message 4645.  
Last modified: 26 Mar 2022, 7:24:54 UTC

Keith: FWIW, on Linux part of glibc is called "libm", a math library, and the Universe app uses math functions from it (quite heavily). Every now & then the math functions in libm get updated with new more optimized versions that are faster. That's why you see improved times with newer glibc versions. Windows probably doesn't bother with providing any optimized versions.
ID: 5199 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
xii5ku

Send message
Joined: 9 Nov 17
Posts: 21
Credit: 563,207,000
RAC: 13
Message 5202 - Posted: 26 Mar 2022, 9:43:47 UTC - in response to Message 4642.  

On 25 February 2021 Keith Myers wrote:
We noticed the speedup was specific to the glibc library when running Ubuntu 18.04 with kernel 5.4 and glibc 2.29 against Ubuntu 20.04 with kernel 5.4 and glibc library 2.31.
Are you sure about the glibc version of your Ubuntu 18.04 test case? According to distrowatch, Ubuntu 18.04 comes with glibc 2.27.

Empirically, I didn't narrow the relevant glibc update down closer than >2.27 && ≤2.31 myself. But from reading glibc release notes, it appears to me that glibc 2.29 introduced the change which sped up BHspin v2 and ULX. (2.29 release notes, relevant patchset)
ID: 5202 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 308
Credit: 4,733,484,700
RAC: 14,480
Message 5203 - Posted: 26 Mar 2022, 22:15:48 UTC - in response to Message 5202.  

Correct. Ubuntu 18.04 shipped with glibc version 2.27. I misremembered the correct version number.

A proud member of the OFA (Old Farts Association)
ID: 5203 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 8,758
Message 5230 - Posted: 28 Mar 2022, 12:44:45 UTC
Last modified: 28 Mar 2022, 12:55:46 UTC

I spoke too soon on another thread. My U@H centered box had locked up for some reason.
Then the application wouldn't restart. So un-archived the zip over the top of it and copied in the Pandora BOINC file. Also set the tasks for E@H down to 60 in case the issue was/is an infection from the other system.

Crossing fingers. X X

==edit=== Crossing Fingers didn't work. Perhaps bumping the cpu volts to 1.30 will ===edit===

Tom M

https://universeathome.pl/universe/show_host_detail.php?hostid=617626
A proud member of the OFA (Old Farts Assoc.)
ID: 5230 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 8,758
Message 5234 - Posted: 28 Mar 2022, 22:01:51 UTC - in response to Message 5230.  

I spoke too soon on another thread. My U@H centered box had locked up for some reason.


It is still running. I bumped the cpu threads/cores up to 100% so I could run 30 threads on U@H and let it munch on E@H on a couple of gpus (gtx 1080's) that are living there for the moment.

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5234 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 8,758
Message 5428 - Posted: 15 May 2022, 12:44:47 UTC

Has your RAC been going down due to the in-ability to upload due to Pentathlon contest?

Mine apparently has.
A proud member of the OFA (Old Farts Assoc.)
ID: 5428 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Xenu666

Send message
Joined: 4 May 21
Posts: 4
Credit: 5,927,311,700
RAC: 2,350
Message 5438 - Posted: 15 May 2022, 21:11:12 UTC - in response to Message 5428.  
Last modified: 15 May 2022, 21:12:33 UTC

Yes, very much so :(
My machines are idle most of the time due to not being able to upload finished WU's and thus not being able to download new ones. I've been trying to hit update on the project but it doesn't seem to do any difference.

I hope this contest will end soon...
ID: 5438 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : Double your task throughput on Linux




Copyright © 2024 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek