Message boards : Number crunching : Upload failure: file size too big
Message board moderation

To post messages, you must log in.

AuthorMessage
Alien Seeker
Avatar

Send message
Joined: 5 Mar 20
Posts: 6
Credit: 2,622,833
RAC: 1,682
Message 4163 - Posted: 16 Apr 2020, 18:48:50 UTC

One of my tasks was refused because the output file was too big. Either the limits should be increased or these tasks should be detected before the end to limit the lost computing power.
ID: 4163 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Life v lies...Dont be a DNA-de...
Avatar

Send message
Joined: 13 Aug 15
Posts: 2
Credit: 59,592,097
RAC: 11
Message 4164 - Posted: 16 Apr 2020, 19:40:01 UTC

PLEASE address this problem. While sporadic and not affecting the majority of WUs, nonetheless, I do NOT wish to be wasting my PCs power nor time.

PROSZĘ rozwiązać ten problem. Chociaż sporadycznie i nie wpływa na większość WU, nie chcę marnować mocy komputera ani czasu.

There have been at least three other post regarding this problem, at least two posted a month ago: forum_thread.php?id=507&postid=4052
and postid=4078

4/15/2020 3:54:19 AM Computation for task universe_ulx_511_7675_20000_1-999999_840000_1 finished
4/15/2020 3:54:24 AM Output file universe_ulx_511_7675_20000_1-999999_840000_1_r1989911120_1 for task universe_ulx_511_7675_20000_1-999999_840000_1 exceeds size limit.
4/15/2020 3:54:24 AM File size: 2174869177.000000 bytes. Limit: 1700000000.000000 bytes

4/16/2020 6:00:41 AM Computation for task universe_ulx_511_10025_20000_1-999999_330000_0 finished
4/16/2020 6:00:46 AM Output file universe_ulx_511_10025_20000_1-999999_330000_0_r596292448_1 for task universe_ulx_511_10025_20000_1-999999_330000_0 exceeds size limit.
4/16/2020 6:00:46 AM File size: 2211380125.000000 bytes. Limit: 1700000000.000000 bytes
God is Love, Jesus Proves it.
Bóg jest miłością, Jezus to potwierdza.
Isten szerelem, Jézus ezt bizonyítja.
Gott ist Liebe, Jesus beweist es.
ID: 4164 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
James W

Send message
Joined: 14 Aug 17
Posts: 16
Credit: 3,845,928
RAC: 0
Message 4166 - Posted: 19 Apr 2020, 11:47:59 UTC
Last modified: 19 Apr 2020, 12:11:06 UTC

<edit>See thread "WU Error -> WU error -> WU error" and Message 4161 from Project Administrator, where he explains problem. Though he boosted WU size limit to 2G, there are still larger files generated. The task reported below has Peak disk usage of 2.31 GB.

WU=44121708; Task=96754296
Application: Universe ULX v0.12 windows_x86_64
Client state: Compute error
Exit status: 0 (0x00000000)
Stderr output:
upload failure: <file_xfer_error>
<file_name>universe_ulx_511_9023_20000_1-999999_690000_6_r1579669706_1</file_name>
<error_code>-131 (file size too big)</error_code>
</file_xfer_error>
Most of these "ulx" WUs crunching on my Windows machine have failed due to the file size being too big, or other language to same affect. In this WU alone, mine was only one of six failures, with one being an aborted task. Other WUs have similar errors, such as "Exit status: 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED." Appears this type of Task/WU needs to be reviewed. No wonder someone has resorted to aborting this type of task.
ID: 4166 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alien Seeker
Avatar

Send message
Joined: 5 Mar 20
Posts: 6
Credit: 2,622,833
RAC: 1,682
Message 4167 - Posted: 19 Apr 2020, 15:37:29 UTC - in response to Message 4166.  

There are two different problems related to excessive output file size. The one mentioned in the other thread is that the task itself fails during computing because it expects a smaller output: task 96062288 is an example, with a "Disk usage limit exceeded" message. The problem I referred to up in this thread is when computing happens fine, the task itself is configured with big enough limits to fit, but then after computing ends, the upload server rejects the result: see task 96009619 for an example; this time the message is "upload failure: universe_ulx_511_1819_20000_1-999999_150000_0_r262071627_1, -131 (file size too big)".
ID: 4167 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Thyme Lawn

Send message
Joined: 15 Oct 17
Posts: 11
Credit: 3,715,011
RAC: 1,676
Message 4170 - Posted: 20 Apr 2020, 11:19:28 UTC - in response to Message 4167.  
Last modified: 20 Apr 2020, 11:19:50 UTC

The universe_ulx_511 and universe_ulx_512 tasks on my system have the following in client_state.xml.

The <workunit> sections have the maximum disk space set to 2,384MB (2.33GB):

<rsc_disk_bound>2500000000.000000</rsc_disk_bound>

The <result> sections have the _1 upload file limit set to 1,621MB (1.58GB):

<max_nbytes>1700000000.000000</max_nbytes>

My maximum upload speed is 640Kbps so (assuming there's no other activity, which will never be the case) it would take almost 45 minutes to upload a 1.58GB file. ULX tasks average out at around 3¼ hours each on my system, so if that was the happening for all tasks running 4 of them would generate more upload data than my connection could handle. Fortunately the size range for my ULX _1 upload files has ranged from 4.4MB to 132.7MB in the past few days ...
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 4170 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg_BE

Send message
Joined: 24 Nov 17
Posts: 23
Credit: 14,553,170
RAC: 0
Message 4171 - Posted: 23 Apr 2020, 19:56:40 UTC

https://universeathome.pl/universe/result.php?resultid=96764160
Peak disk usage 2.28 GB
Stderr output

<core_client_version>7.16.5</core_client_version>
<![CDATA[
<stderr_txt>
21:31:11 (41908): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
<file_name>universe_ulx_511_10857_20000_1-999999_850000_3_r2085177067_1</file_name>
<error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>


https://universeathome.pl/universe/result.php?resultid=96764010
Peak disk usage 2.10 GB
Stderr output

<core_client_version>7.16.5</core_client_version>
<![CDATA[
<stderr_txt>
02:43:04 (12080): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
<file_name>universe_ulx_511_17115_20000_1-999999_800000_4_r1290607146_1</file_name>
<error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>
ID: 4171 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jan Henrik
Avatar

Send message
Joined: 22 Mar 16
Posts: 2
Credit: 259,080,333
RAC: 820,409
Message 4173 - Posted: 24 Apr 2020, 14:11:41 UTC

app-arently ULX 0.12 keeps craving for attention of your qualified personnel;

Application
Universe ULX 0.12
Name
universe_ulx_512_3963_20000_1-999999_500000
State
Computation error
Received
4/22/2020 6:18:27 AM
Report deadline
5/6/2020 6:18:26 AM
Estimated computation size
807 GFLOPs
CPU time
07:07:37
Elapsed time
07:08:11
Executable
universe-ULX_12_windows_x86_64.exe

how peculiar;

Universe@Home | Output file universe_ulx_512_3963_20000_1-999999_500000_3_r1296757251_1 for task universe_ulx_512_3963_20000_1-999999_500000_3 exceeds size limit.
Universe@Home | File size: 2080570381.000000 bytes. Limit: 1700000000.000000 bytes

whoever can fix this

greetings

"less than a pixel"
ID: 4173 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 748
Credit: 140,962,465
RAC: 8,986
Message 4174 - Posted: 25 Apr 2020, 18:33:19 UTC - in response to Message 4173.  

All tasks with CPU work time bigger then 3600 seconds get credits.
Also, I generate some work units with 10 times shorter. The plan is to create more, shorter tasks and try to compile it results with other on our work machine. In next few days we will see how it works.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 4174 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Upload failure: file size too big




Copyright © 2021 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek