Message boards : Number crunching : Double your task throughput on Linux
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Jim1348

Send message
Joined: 28 Feb 15
Posts: 219
Credit: 99,164,814
RAC: 247,850
Message 4456 - Posted: 1 Sep 2020, 12:50:59 UTC

See the discussion on the benefits of upgrading to Ubuntu 20.04:
https://universeathome.pl/universe/forum_thread.php?id=526&postid=4446

Note especially how to prevent login problems with Nvidia cards:
https://universeathome.pl/universe/forum_thread.php?id=550&postid=4454

Here is more info:
https://askubuntu.com/questions/1229821/login-loop-after-installing-20-04
ID: 4456 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Feb 15
Posts: 219
Credit: 99,164,814
RAC: 247,850
Message 4457 - Posted: 2 Sep 2020, 16:35:31 UTC - in response to Message 4456.  
Last modified: 2 Sep 2020, 16:37:06 UTC

I am now averaging about 51 minutes for the BHspin v2. They range from 42 minutes to a little over an hour.
Just for comparison, I ran one on my Win7 machine, an i7-4771 with about the same speed as the Ryzen 2700.
It took 3 hours 20 minutes.

I think I will stick to Linux.

PS - The ULX are also speeding up by about the same amount, but I don't have an average yet.
ID: 4457 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rsNeutrino

Send message
Joined: 1 Nov 17
Posts: 9
Credit: 68,852,933
RAC: 101,046
Message 4458 - Posted: 3 Sep 2020, 3:11:05 UTC - in response to Message 4457.  

Glad that the upgrade worked on your side, too!

BHspin v2 task series 325 just commenced on my side, with the first 6 taking between 32 and 33 minutes.
It seems the start of each series takes the shortest amount of time to calculate in general.
The last ones of series 324 took between 54 and 56 minutes.

Ubuntu 20.04 inside Hyper-V VM on Windows 10 1909 host
Ryzen 1700 @ 3.6 GHz
32 GB DDR4 @ 3.2 GHz Cl14
ID: 4458 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Feb 15
Posts: 219
Credit: 99,164,814
RAC: 247,850
Message 4459 - Posted: 3 Sep 2020, 13:18:39 UTC - in response to Message 4458.  

It seems the start of each series takes the shortest amount of time to calculate in general.

Yes, I got a couple at 36 minutes, but they don't last long.

I have no idea about what libraries are improved. This upgrade does not speed up WCG (OPN or MIP), so they must not use them.
I have a lot of other projects to check out though.
ID: 4459 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile JohnMD
Avatar

Send message
Joined: 21 Feb 15
Posts: 13
Credit: 10,108,919
RAC: 2,629
Message 4469 - Posted: 11 Sep 2020, 16:22:11 UTC - in response to Message 4459.  

I've noticed the same trend on Windows. Right now they take double the time compared with a week ago
ID: 4469 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 76
Credit: 285,586,700
RAC: 1,741,010
Message 4471 - Posted: 12 Sep 2020, 20:00:25 UTC - in response to Message 4469.  

I've noticed the same trend on Windows. Right now they take double the time compared with a week ago

Yes, I see the same thing. Noticeable in host production graphs. Rise to a peak, then slowly drop to a minima, then slowly rise back to another peak. Then repeat the cycle again. Each cycle defined by the next task series number.

A proud member of the OFA (Old Farts Association)
ID: 4471 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 729
Credit: 140,227,798
RAC: 12,573
Message 4472 - Posted: 13 Sep 2020, 10:54:07 UTC - in response to Message 4471.  

Each cycle defined by the next task series number.

This is correct.
Each task series are very similar, but every one have a bit changed starting task parameters.
All possible combinations is 265'420'800 for BHSpin.in current version (about 50-60millions are done for now).
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home project team
My Patreon profile
ID: 4472 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tom M

Send message
Joined: 18 Jul 17
Posts: 4
Credit: 73,025,083
RAC: 665,425
Message 4500 - Posted: 13 Nov 2020, 13:40:33 UTC

Thank you all. I was looking at 1 hour tasks and wondering what I broke. The answer is "nothing" just normal processing variation.
A proud member of the OFA (Old Farts Assoc.)
ID: 4500 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4513 - Posted: 23 Nov 2020, 7:17:36 UTC

Interesting thread.
I was wondering why my 32 threaded processors appear woefully lacking in regards to credit. It looks like Linux is twice as fast.
Same I don't run Linux or Raspberry Pi's :(

ID: 4513 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tom M

Send message
Joined: 18 Jul 17
Posts: 4
Credit: 73,025,083
RAC: 665,425
Message 4514 - Posted: 24 Nov 2020, 12:02:39 UTC - in response to Message 4513.  

Interesting thread.
I was wondering why my 32 threaded processors appear woefully lacking in regards to credit. It looks like Linux is twice as fast.
Same I don't run Linux or Raspberry Pi's :(


Yup. That is the motivation to run a Linux 20 version (version 18 isn't any faster, 20 is).

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 4514 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4515 - Posted: 24 Nov 2020, 18:57:41 UTC

Well sadly for me I'll need to decide on crunching elsewhere then or put up with the woeful credits. I won't be installing Linux :(
Choice is mine I guess but it's a real shame for Windows users.

ID: 4515 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 76
Credit: 285,586,700
RAC: 1,741,010
Message 4516 - Posted: 24 Nov 2020, 22:59:10 UTC

Well, you can get almost the same performance by running Linux in a Windows 10 VM instance.

A proud member of the OFA (Old Farts Association)
ID: 4516 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4517 - Posted: 25 Nov 2020, 7:09:34 UTC

Thanks for the suggestion Keith but it sounds too technical for me.
Its actually your 3950X that I'm comparing mine to. Mine's close to 400k/day, your is nearly 1.3Mil/day. MASSIVE difference. :(
I'll just crunch elsewhere.

ID: 4517 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 76
Credit: 285,586,700
RAC: 1,741,010
Message 4519 - Posted: 25 Nov 2020, 20:31:01 UTC

Just need to point out the fault does not lie with the project's application. The fault lies with the Windows operating system.

A proud member of the OFA (Old Farts Association)
ID: 4519 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4520 - Posted: 26 Nov 2020, 7:23:46 UTC

Yeah I understand that, thanks Keith.
I MIGHT.... try installing it side by side with one of my crunchers but my every day rig is my 3950X so I definitely won't be installing it on this machine.
I imagine Linux and Universe@Home is one of the projects with the largest difference between credits. Are there any other projects? (I'm not talking about Linux specific projects like MLC, just projects that produce more credits than Win10)

TIA

ID: 4520 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rsNeutrino

Send message
Joined: 1 Nov 17
Posts: 9
Credit: 68,852,933
RAC: 101,046
Message 4565 - Posted: 1 Dec 2020, 6:19:17 UTC - in response to Message 4520.  

You seem to have many computers running Windows 10 Pro, which has Hyper-V built in.

Do the following if you want to try using Linux and Windows together on the same machine at the same time (like I do):
- enable the virtualization extentions in the BIOS (AMD-V / Intel VT-x)
- enable Hyper-V in Windows Features
- best use the latest Ubuntu version, so download the latest iso
- use the Wizard in Hyper-V Manager to create a new VM
- configure the amount of CPU cores (host and VM share them, and they are not fixed to physical cores, host usage takes priority) and RAM your Linux programs may need (fixed works best)
- attach iso to the VM before starting the VM, install Ubuntu, install BOINC using the Ubuntu Software, done
ID: 4565 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Keith Myers
Avatar

Send message
Joined: 10 May 20
Posts: 76
Credit: 285,586,700
RAC: 1,741,010
Message 4566 - Posted: 1 Dec 2020, 16:43:49 UTC - in response to Message 4520.  

I don't run that many projects so have limited knowledge of any other projects that show a large discrepancy in runtimes between cpu tasks run on Windows and Linux.

Universe jumps out easily as a case.

A proud member of the OFA (Old Farts Association)
ID: 4566 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4568 - Posted: 1 Dec 2020, 19:11:49 UTC
Last modified: 1 Dec 2020, 19:14:54 UTC

Thanks for the reply Keith/rsNeutrino.
Yes all my machines are Windows
.
I found some time to install a VM and with much pain I worked out how to install Ubuntu..... the issue rsNeutrino, I have is that I've never even looked at Linux before. I've no idea how to do the most basic things with it.
For instance, when I downloaded the BOINC program, it was just a shell extension. I got stuck on that.
Then a team mate told me what to type into a terminal to install it. (I had to find out what a terminal was) (sudo apt install boinc-client boinc-manager) I couldn't find that simple information ANYWHERE! Not even on the BOINC website!

Ok...got it installed...or did I? I now have no idea where it was installed. Again, with help I was told I'd find the files in a usr/bin folder. It took me quite a while just to FIND that folder! Grrrrr. I've found it and I see some files called BOINC and BOINCMGR etc ..... but I've no idea how to run the program.

This is like teaching a 5year old.... I feel completely out of my depth trying to figure out a program that most other people seem to know quite easily.
**sigh...... so I have made progress but have hit another wall. I find I have to chip away at it because I find it all a bit depressing.... especially when you are used to the simplicity of Windows when it comes to installing software.

Appreciate your help fellow crunchers. This baby will at least learn some things about Linux I guess. :(

ID: 4568 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rsNeutrino

Send message
Joined: 1 Nov 17
Posts: 9
Credit: 68,852,933
RAC: 101,046
Message 4569 - Posted: 1 Dec 2020, 19:43:41 UTC - in response to Message 4568.  
Last modified: 1 Dec 2020, 19:55:41 UTC

https://www.chip.de/ii/8/4/2/2/1/7/2/a4e7ac17686754fc.jpg
After installation of Ubuntu, its desktop should look like this.
To install BOINC or any other popular software, there is no need to use the terminal at all.
Look for the orage bag icon on the left with the "A" on it, that is the lanch icon vor the "Ubuntu Software Center".
(You can find other apps by clicking on the button with the 3x3 white dots, which is like the Start button in Windows.)
In Ubuntu Software, click on the magnifiing glass in the upper left corner and search for BOINC.
It should now list BOINC Manager as a result, click on it, install, done!
You should now have BOINC with the familiar user interface like in Windows.
Its launch icon can be found with the 3x3 dot start button.

You may have to remove the stuff you installed by terminal first, if above fails.
"sudo apt remove boinc-client boinc-manager"

Also maybe don't use Oracle Virtualbox for VMs, because it lags like hell if you want to use many cores with it, I tried that route...
Windows 10 PROFESSIONAL specifically has Hyper V with works absolutely the best, and you have it as an option.
Hope that helps. :)
ID: 4569 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Chooka
Avatar

Send message
Joined: 30 Jun 16
Posts: 31
Credit: 123,041,695
RAC: 93,499
Message 4570 - Posted: 2 Dec 2020, 6:52:56 UTC

Hi rsNeutrino,

Thank you very much for taking the time to reply! I didn't know about any of what you mentioned, including the 3x3 white dots! lol.
I had to do an update and also at the bottom of the page for the 3x3 dots select "All" applications instead of "frequent" The BOINC Icon was hiding in there.
I can now see that I can add projects!!! YAY!!

I've run out of time to do more today but so far I'd like to say a HUGE thank you for taking the time to reply and help :D !!! Really appreciated.
I'll start with baby steps before looking to change to Hyper-V. I have no idea how to use that if it's build into my Windows 10.One thing at a time lol.

I'll keep you updated on the progress.
Thanks once again!

Chooka

ID: 4570 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Double your task throughput on Linux




Copyright © 2020 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek