Message boards : Number crunching : Large upload Files

Large upload Files

Post to thread Subscribe


Previous · 1 · 2
AuthorMessage
cykodennis
Avatar

Send message
Joined: 4 Feb 15
Posts: 24
Credit: 6,003,527
RAC: 0
Message 108 - Posted: 27 Feb 2015, 15:37:04 UTC - in response to Message 107.  

Funny, this morning i had the idea to ask zombie :D

Zombie, do you have such problems with uploads & deadlines?
IIRC your location was california?
ID: 108 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 4 Feb 15
Posts: 11
Credit: 44,425,235
RAC: 1
Message 109 - Posted: 27 Feb 2015, 15:49:33 UTC - in response to Message 108.  
Last modified: 27 Feb 2015, 16:12:24 UTC

Funny, this morning i had the idea to ask zombie :D

Zombie, do you have such problems with uploads & deadlines?
IIRC your location was california?


At first, the machines would download way too much work, and everything runs in panic mode. I think the time estimates for the tasks may be too small. However, the BOINC client will apply a Duration Correction Factor to the project as tasks complete. So now my queue is normal size and not running in panic mode.

With regard to the uploads, they usually take ~3 minutes to upload. With task run times of several hours, the uploads never get backed up for me.
Dublin, California
Team: SETI.USA
ID: 109 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Alez

Send message
Joined: 21 Feb 15
Posts: 9
Credit: 10,943,535
RAC: 8,277
Message 129 - Posted: 2 Mar 2015, 1:25:14 UTC

I just had to abort the transfer of over 50 units that have been trying to upload for 3 days on multiple systems. I am getting upload speeds of 3 kb/s and constant http transient errors. I have no problems uploading to any other project from over 30 systems and have never had a problem uploading to the test server. This is from the UK so I should have no issues transferring data to Poland.
ID: 129 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Krzysztof Piszczek - wspieram Polski projekt BOINC
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 555
Credit: 59,426,467
RAC: 1,278
Message 135 - Posted: 6 Mar 2015, 5:34:02 UTC - in response to Message 129.  
Last modified: 6 Mar 2015, 17:15:40 UTC

Currently on test server are some work units for new application version where result files are really smaller. If all work units finish well I will add new application here.

Edit:

Job done.
New application sends back only about one megabyte of data...
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home project team
Radioactive Android Map
Tsu
ID: 135 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
cykodennis
Avatar

Send message
Joined: 4 Feb 15
Posts: 24
Credit: 6,003,527
RAC: 0
Message 139 - Posted: 7 Mar 2015, 2:46:12 UTC
Last modified: 7 Mar 2015, 2:48:35 UTC

We are happy about that.

I hope the project has not to face any difficulties because of this change?
What did the trick? I originally thought that this whole startrack-whatever-thing would simply have to produce files like ~25 MB compressed?

Edit:
Aaaand, i didn't notice:
Is the new application already on production or still on Testserver?
I've seen that the testserver has still 20k WU ready to send?
ID: 139 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Krzysztof Piszczek - wspieram Polski projekt BOINC
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 555
Credit: 59,426,467
RAC: 1,278
Message 140 - Posted: 7 Mar 2015, 2:50:06 UTC - in response to Message 139.  

The trick is really stop to record some data. In one of the next app the data will be processed straight forward in client app as a part of it logic.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home project team
Radioactive Android Map
Tsu
ID: 140 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
DadX

Send message
Joined: 22 Feb 15
Posts: 10
Credit: 3,538,015
RAC: 4,701
Message 142 - Posted: 9 Mar 2015, 1:11:11 UTC

Because I'm crunching this project on a Win 8 laptop that's running 2 of 4 cores at 60% and mostly only 8 hours a day my problem is getting the calculations done in 4 days. Two days was impossible, 4 days is doable but very tight, any chance you can bump this to 6 days?
ID: 142 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Krzysztof Piszczek - wspieram Polski projekt BOINC
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 555
Credit: 59,426,467
RAC: 1,278
Message 143 - Posted: 9 Mar 2015, 10:29:41 UTC - in response to Message 142.  

I cant change to 6 days at this stage of project but I will strongly consider change later.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home project team
Radioactive Android Map
Tsu
ID: 143 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 5 Mar 15
Posts: 4
Credit: 12,251,628
RAC: 11,765
Message 144 - Posted: 9 Mar 2015, 11:15:27 UTC

Not a single result of my so far completed 61 WUs does exceed 1 MB in total size. On the mainpage I read 70 MB would be normal. Is there a problem with the WU batch which I am processing?
In our forum over at Rechenkraft.net other people report the same situation.

Michael.
ID: 144 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
cykodennis
Avatar

Send message
Joined: 4 Feb 15
Posts: 24
Credit: 6,003,527
RAC: 0
Message 145 - Posted: 9 Mar 2015, 11:20:45 UTC - in response to Message 144.  

Ihr bearbeitet bereits den New&Improved Batch, Michael.
ID: 145 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Michael H.W. Weber

Send message
Joined: 5 Mar 15
Posts: 4
Credit: 12,251,628
RAC: 11,765
Message 266 - Posted: 11 Apr 2015, 11:49:43 UTC - in response to Message 145.  
Last modified: 11 Apr 2015, 11:50:37 UTC

Danke für die Info.

Michael.

P.S.: Schnell mal Ace of Spades nachladen... ;)
ID: 266 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2 Post to thread

Message boards : Number crunching : Large upload Files