Message boards : Number crunching : Server Thread
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 14 · Next

AuthorMessage
mikey
Avatar

Send message
Joined: 4 Apr 15
Posts: 45
Credit: 43,117,233
RAC: 1,710
Message 5311 - Posted: 6 May 2022, 13:33:23 UTC - in response to Message 5305.  

It looks like my uploads are stalled.


And when I cycled the Network connectivity everything started moving again. The joys of a 20 second rest.

----edited---
Spoke too soon. Stalled again.


It is in 1-3 hour back off.

Have run completely out of tasks.

Tom M


Focus on the downloads and then at least you can crunch, just keep clicking retry until they come thru
ID: 5311 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 23 Apr 22
Posts: 136
Credit: 64,030,000
RAC: 105,018
Message 5324 - Posted: 6 May 2022, 20:26:45 UTC

Server is still overwhelmed.

Some uploads & downloads are going through without manual intervention, but it's still necessary to keep things moving.
Grant
Darwin NT
ID: 5324 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 124
Credit: 1,231,364,283
RAC: 3,734,128
Message 5334 - Posted: 7 May 2022, 0:31:54 UTC - in response to Message 5324.  

Server is still overwhelmed.

Some uploads & downloads are going through without manual intervention, but it's still necessary to keep things moving.


I just remembered that a lot of the high volume systems at Seti@Home had to use the cc_config.xml file to limited the total number of "reports" so the Seti servers would not stagger to a halt after the Tuesday maintenance cycle. I wonder if that could make a difference here?
Nah, probably not. Would have to get everyone in the contest to do it too.
I wonder though if standard DDOS (distributed denial of service) defense tactics would lower the load so that some of the uploads would go through at full speed?

I also wonder if a discrete upper limit to the connection "sockets" would keep things moving for the ones who can connect.

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5334 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr. Michael Hicks

Send message
Joined: 25 Dec 20
Posts: 11
Credit: 119,980,667
RAC: 1
Message 5336 - Posted: 7 May 2022, 0:52:53 UTC - in response to Message 5334.  

Hi, Folks. I am also suffering from the server issues. On my raspberry pi
nodes I can see up to a dozen 'upload pending' finished task listings per node
waiting in the queue.

Hitting the 'retry pending transfers" button in boinc does not seem to help and
it is sad to see my average wu points go down day by day. I hope the folks
at Universe@Home can get things running as before. Anything specific
that I could do on my end I would be happy to hear about.

-Mike
ID: 5336 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
amontero

Send message
Joined: 30 Mar 22
Posts: 3
Credit: 528,667
RAC: 1,004
Message 5337 - Posted: 7 May 2022, 2:13:39 UTC

Same here. Having intermittent issues, mainly "project servers may be temporarily down" when updating project and attempting uploads, which are failing most of the time. I've got lucky with a couple WUs by retrying (one of them while posting here).
Also, pinging universeathome.pl stalls. However, "curl -IL universeathome.pl" does work, but with quite lag in replying.
ID: 5337 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 124
Credit: 1,231,364,283
RAC: 3,734,128
Message 5340 - Posted: 7 May 2022, 14:21:53 UTC - in response to Message 5336.  
Last modified: 7 May 2022, 14:22:51 UTC

Hi, Folks. I am also suffering from the server issues. On my raspberry pi
nodes I can see up to a dozen 'upload pending' finished task listings per node
waiting in the queue.

-Mike


I added <max_file_xfers_per_project>8</max_file_xfers_per_project> to the options in the cc_config.xml file, re-started the Boinc Manager or re-read the config files. The default max transfers for all projects are 8 which I left alone.

It started actually succeeding on uploading on two of the 8 uploads that are "active". I still have backed up uploads but I am now getting downloads again since it got closer to being "caught up." I just switched it to 4 per project. Will see if it continues to catch up.

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5340 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
pawg

Send message
Joined: 10 Mar 15
Posts: 25
Credit: 15,934,256
RAC: 38,753
Message 5343 - Posted: 7 May 2022, 21:44:22 UTC

Server works fine now
ID: 5343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 23 Apr 22
Posts: 136
Credit: 64,030,000
RAC: 105,018
Message 5344 - Posted: 7 May 2022, 22:35:11 UTC - in response to Message 5343.  

Server works fine now
Yep.
Looks like things came good around 5.5 hours ago.

And another batch of work has been loaded as well.
Grant
Darwin NT
ID: 5344 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
xii5ku

Send message
Joined: 9 Nov 17
Posts: 21
Credit: 563,079,667
RAC: 20,012
Message 5349 - Posted: 8 May 2022, 12:07:59 UTC - in response to Message 5344.  
Last modified: 8 May 2022, 12:08:30 UTC

And another batch of work has been loaded as well.
Most work requests are responded to with "Project has no tasks available" though, because the feeder's(?) buffer of tasks to assign (or something like that) is not refilled frequently enough. IOW this buffer is too small for the current rate of client requests for more work.
ID: 5349 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dr. Michael Hicks

Send message
Joined: 25 Dec 20
Posts: 11
Credit: 119,980,667
RAC: 1
Message 5350 - Posted: 8 May 2022, 17:25:31 UTC - in response to Message 5340.  

thank you, tom. i will try that myself.

-mike
ID: 5350 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
vaughan

Send message
Joined: 4 Feb 15
Posts: 7
Credit: 158,154,501
RAC: 4,086
Message 5351 - Posted: 8 May 2022, 23:28:32 UTC

Unable to get tasks to crunch - kick the server please.
ID: 5351 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 23 Apr 22
Posts: 136
Credit: 64,030,000
RAC: 105,018
Message 5353 - Posted: 9 May 2022, 4:57:40 UTC

Certainly something's going on with the server- over 3.3 million Tasks are ready to send, but "Project has no tasks available" is the only response i'm getting from the Scheduler when requesting work.
The last time i got any work was almost 8 hours ago. And for 4 hours before that the server was only giving "Project has no tasks available" responses to work requests, and prior to getting some work back then, it was unable to supply any work for almost 6 hours.
Grant
Darwin NT
ID: 5353 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Freewill

Send message
Joined: 2 May 20
Posts: 18
Credit: 4,325,695,066
RAC: 4,557,027
Message 5354 - Posted: 9 May 2022, 10:46:18 UTC

Same here on most of my PCs - unable to get new tasks, onboard caches are being drained.
ID: 5354 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
frankhagen

Send message
Joined: 12 Aug 17
Posts: 21
Credit: 58,957,280
RAC: 0
Message 5356 - Posted: 9 May 2022, 12:48:27 UTC

situatin is persistent.

about time to kickstart the server!
ID: 5356 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
supdood

Send message
Joined: 4 May 18
Posts: 11
Credit: 12,761,467
RAC: 4,162
Message 5362 - Posted: 9 May 2022, 14:47:51 UTC - in response to Message 5356.  
Last modified: 9 May 2022, 15:17:28 UTC

situatin is persistent.

about time to kickstart the server!

I'm not sure it needs to be restarted or if it simply can't hand out the tasks any faster. The number of tasks ready to send continues to decrease, so someone is getting the work, but it clearly isn't fast enough for the demand. I would guess that the issue is a need for more resources allocated to the job-processing daemons, but I don't have any insight into our current rate of task delivery compared to regular/max.


EDIT: There may in fact be something that it bottle-necking the process as I can't imagine the regular task delivery is this slow:
From 15:10:03 UTC to 15:13:24 UTC the server released...427 tasks.
I wonder if the increased return rate and data processing that kicks off is hogging the server's resources.
ID: 5362 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dataman
Avatar

Send message
Joined: 24 Feb 15
Posts: 32
Credit: 609,507,165
RAC: 1,417
Message 5364 - Posted: 9 May 2022, 14:59:16 UTC

Me too. Still cannot get any wu's across multiple platforms (Win10, Ubuntu & Raspberry OS).

:(
ID: 5364 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 834
Credit: 144,179,798
RAC: 3,293
Message 5366 - Posted: 9 May 2022, 15:36:07 UTC - in response to Message 5364.  

At the moment, every second to server comes back 46 files (4 millions files per day). Also, at the same time some workunits are downloaded.
I'm afraid this the limit where I can't do much more on server.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 5366 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dataman
Avatar

Send message
Joined: 24 Feb 15
Posts: 32
Credit: 609,507,165
RAC: 1,417
Message 5367 - Posted: 9 May 2022, 16:18:52 UTC - in response to Message 5366.  
Last modified: 9 May 2022, 16:19:35 UTC

At the moment, every second to server comes back 46 files (4 millions files per day). Also, at the same time some workunits are downloaded.
I'm afraid this the limit where I can't do much more on server.

Thank you for the update! At least I know what is happening now. I will hang in there until things get sorted.

Cheers
ID: 5367 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
gemini8

Send message
Joined: 4 Dec 15
Posts: 1
Credit: 18,725,783
RAC: 21,687
Message 5373 - Posted: 9 May 2022, 20:36:28 UTC

I'm running a two day cache and ask for Universe work only on CPU.
No scripting to tickle the server.
Getting about a day's worth of work on each of my four machines running in the Pentathlon.
Works fine for me. :-)
ID: 5373 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 23 Apr 22
Posts: 136
Credit: 64,030,000
RAC: 105,018
Message 5376 - Posted: 10 May 2022, 5:18:27 UTC

14 hours since one of my systems was last able to get any work.
Grant
Darwin NT
ID: 5376 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 . . . 14 · Next

Message boards : Number crunching : Server Thread




Copyright © 2023 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek