Message boards : Number crunching : Very high disk activity
Message board moderation

To post messages, you must log in.

AuthorMessage
Rasputin42

Send message
Joined: 11 Mar 15
Posts: 9
Credit: 6,641,411
RAC: 2
Message 331 - Posted: 31 May 2015, 12:38:52 UTC

Hi,
It seems universe uses very little memory but extreme number of disk write activity.
This strikes me as very very inefficient.
Why don't you do this in memory and write the results to disk, instead of constantly having a large number of small data chunks being written to disk?
ID: 331 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 841
Credit: 144,180,465
RAC: 0
Message 332 - Posted: 31 May 2015, 12:42:05 UTC - in response to Message 331.  

Hi,
It seems universe uses very little memory but extreme number of disk write activity.
This strikes me as very very inefficient.
Why don't you do this in memory and write the results to disk, instead of constantly having a large number of small data chunks being written to disk?

What version of BOINC Manager do you use?
This activity exists on v. 6.x
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 332 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42

Send message
Joined: 11 Mar 15
Posts: 9
Credit: 6,641,411
RAC: 2
Message 333 - Posted: 31 May 2015, 12:46:15 UTC

I am using 7.4.42, the latest version, win 8.1 64 bit
ID: 333 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 841
Credit: 144,180,465
RAC: 0
Message 334 - Posted: 31 May 2015, 13:45:17 UTC - in response to Message 333.  

Application saves 20 times checkpoint during execution and periodically saves some debug info (less than 20kb for each WU).
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 334 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42

Send message
Joined: 11 Mar 15
Posts: 9
Credit: 6,641,411
RAC: 2
Message 335 - Posted: 31 May 2015, 13:49:16 UTC

It is saving files constantly, all the time, no break
ID: 335 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42

Send message
Joined: 11 Mar 15
Posts: 9
Credit: 6,641,411
RAC: 2
Message 336 - Posted: 31 May 2015, 14:09:16 UTC

These are the files, the 4 executables are constantly writing to.
The rate of saved data NEVER falls below 25000 Bytes/sec.

4x (universe-xray_sources_v2_3_windows_intelx86.exe)




Datei Write (B/s)
-0 23.491
-0 18.736
-0 18.512
-0 16.665
G:\Boinc\ProgramData\BOINC\slots\9 13.377
G:\Boinc\ProgramData\BOINC\slots\11 11.277
G:\Boinc\ProgramData\BOINC\slots\10 11.133
G:\Boinc\ProgramData\BOINC\slots\12 10.097
G:\Boinc\ProgramData\BOINC\slots\9\tmp5467.dat 696
G:\Boinc\ProgramData\BOINC\slots\10\tmp5467.dat 588
G:\Boinc\ProgramData\BOINC\slots\12\tmp5467.dat 521
G:\Boinc\ProgramData\BOINC\slots\11\tmp5467.dat 405
ID: 336 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grzegorz Wiktorowicz
Project developer
Project scientist

Send message
Joined: 25 Feb 15
Posts: 50
Credit: 5,684,705
RAC: 0
Message 341 - Posted: 1 Jun 2015, 12:55:01 UTC - in response to Message 336.  

Hi,

There is indeed a file for temporary parameters storage. Although the usage of it is outdated, we had some more-urgent issues recently. We will do our best to fix this problem before the next release of code.

Thanks for reporting this! Such feedback is a great help for us.
Grzegorz Wiktorowicz
National Astronomical Observatories,
Chinese Academy of Sciences
ID: 341 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42

Send message
Joined: 11 Mar 15
Posts: 9
Credit: 6,641,411
RAC: 2
Message 343 - Posted: 1 Jun 2015, 17:38:21 UTC

Thanks for recognizing the issue.I only noticed it, because i am using a usb flash drive for boinc.However, this continuous writing is especially bad for flash or ssd drives, in the long run.
ID: 343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
cyrusNGC_224@P3D

Send message
Joined: 21 Feb 15
Posts: 46
Credit: 926,538,317
RAC: 62
Message 352 - Posted: 4 Jun 2015, 20:04:09 UTC

On Linux you can specualte this with iotop; every second disk activity:
Total DISK READ : 0.00 B/s | Total DISK WRITE : 77.21 K/s
Actual DISK READ: 0.00 B/s | Actual DISK WRITE: 77.21 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO COMMAND
14019 be/7 boinc 0.00 B/s 7.72 K/s 0.00 % 1.85 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
19826 be/7 boinc 0.00 B/s 7.72 K/s 0.00 % 0.23 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
7665 be/7 boinc 0.00 B/s 7.72 K/s 0.00 % 0.14 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
11676 be/7 boinc 0.00 B/s 19.30 K/s 0.00 % 0.08 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
10008 be/7 boinc 0.00 B/s 15.44 K/s 0.00 % 0.06 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
18498 be/7 boinc 0.00 B/s 7.72 K/s 0.00 % 0.05 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
15378 be/7 boinc 0.00 B/s 7.72 K/s 0.00 % 0.03 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
14108 be/7 boinc 0.00 B/s 3.86 K/s 0.00 % 0.01 % ../../projects/universeathome.pl_universe/universe-xray_sources_v2_3_x86_64-pc-linux-gnu
ID: 352 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Very high disk activity




Copyright © 2024 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek