Message boards : Number crunching : No WUs right now?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3664 - Posted: 5 Aug 2019, 8:01:37 UTC - in response to Message 3663.  

I tried resetting two of my machines and then just detaching on one of them to no avail.
I still receive the "No Tasks Available" signature.
ID: 3664 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Feb 15
Posts: 253
Credit: 200,562,581
RAC: 0
Message 3670 - Posted: 5 Aug 2019, 15:49:45 UTC - in response to Message 3664.  

When first attaching to Universe, suspend any other projects you are running. That has worked for me twice for getting Universe work units, and you can then resume the others.

Of course, it is not a long-term fix. But once you get work, it might keep flowing if you set the buffer to 0.5 + 0.5 days also. That setting could depend on how fast your machine is also, so you might have to experiment. But once it starts working, it will go for a while. I don't know if my machines are permanently fixed or not.
ID: 3670 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3671 - Posted: 5 Aug 2019, 16:56:38 UTC - in response to Message 3670.  

I appreciate the "tip" and I tried separately on each of the machines to suspend other project (TN-Grid) then reset Universe.... then reboot, then update universe, and it was having none of it...
I reset both Universe and TN and then removed them from boinc.
I rebooted. I reattached to Universe. Not to TN-Grid.
The first message is "No Tasks Sent", the next messages are "No Tasks Available"....
I wish it would just get fixed.
ID: 3671 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Pete Broad

Send message
Joined: 7 Jan 16
Posts: 20
Credit: 208,458,667
RAC: 0
Message 3672 - Posted: 5 Aug 2019, 17:15:00 UTC - in response to Message 3671.  

Further to my post of the other day I am now seeing the number of work units I'm picking up slowly increase on both my Androids and Laptops/Pc's. However, there are still numerous machines that do not get work, in fact some haven't had any work for 2 weeks! I can't find any obvious pattern, all of my laptops are Win 10, some get work, some don't, it just doesn't make sense really.

Pete

,
ID: 3672 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gordon Lack

Send message
Joined: 24 May 19
Posts: 16
Credit: 53,469,167
RAC: 0
Message 3673 - Posted: 5 Aug 2019, 17:18:23 UTC - in response to Message 3671.  
Last modified: 5 Aug 2019, 17:19:16 UTC

I rest the project on all 4 of my systems running it.

One of then then produced:
Sun 04 Aug 2019 02:05:31 BST | Universe@Home | Resetting project
Sun 04 Aug 2019 02:05:34 BST | Universe@Home | Master file download succeeded
Sun 04 Aug 2019 02:05:39 BST | Universe@Home | Sending scheduler request: To fetch work.
Sun 04 Aug 2019 02:05:39 BST | Universe@Home | Requesting new tasks for CPU
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Scheduler request completed: got 18 new tasks
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Resent lost task universe_bh2_180328_263_1307823693_20000_1-999999_825200_1
...16 more resent lost tasks....
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Resent lost task universe_bh2_180328_263_1307928693_20000_1-999999_930200_1
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Project has no tasks available
which was odd, as it had had at most 1 job before the reset. So where did 18 lost ones come from?

Since then it has mostly gotten 0 tasks/Project has no tasks available each time it reporst a complete task (which always seem to get done individually).
Except once, when it managed:
Sun 04 Aug 2019 15:50:42 BST | Universe@Home | Sending scheduler request: To report completed tasks.
Sun 04 Aug 2019 15:50:42 BST | Universe@Home | Reporting 1 completed tasks
Sun 04 Aug 2019 15:50:42 BST | Universe@Home | Requesting new tasks for CPU
Sun 04 Aug 2019 15:50:46 BST | Universe@Home | Scheduler request completed: got 12 new tasks
on that same system. The other 3 aren't getting anything.

So something is weird about the job server.
ID: 3673 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3674 - Posted: 5 Aug 2019, 18:20:38 UTC - in response to Message 3673.  
Last modified: 5 Aug 2019, 18:30:46 UTC

I wonder if a glance over the server logs would be beneficial.....
It doesn't appear that the problem is going to be self correcting....
I would also venture a guess that it is entirely possible that the
Tasks ready to send 3051778
value is inaccurate..... but it seems all task types across the board are affected.
ID: 3674 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Feb 15
Posts: 253
Credit: 200,562,581
RAC: 0
Message 3675 - Posted: 5 Aug 2019, 18:36:18 UTC - in response to Message 3673.  

Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Scheduler request completed: got 18 new tasks
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Resent lost task universe_bh2_180328_263_1307823693_20000_1-999999_825200_1
...16 more resent lost tasks....
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Resent lost task universe_bh2_180328_263_1307928693_20000_1-999999_930200_1
Sun 04 Aug 2019 02:05:41 BST | Universe@Home | Project has no tasks available
which was odd, as it had had at most 1 job before the reset. So where did 18 lost ones come from?

I saw that too once. It loses them just so it can find them.
ID: 3675 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gordon Lack

Send message
Joined: 24 May 19
Posts: 16
Credit: 53,469,167
RAC: 0
Message 3676 - Posted: 5 Aug 2019, 21:22:51 UTC - in response to Message 3674.  

I would also venture a guess that it is entirely possible that the
Tasks ready to send 3051778
value is inaccurate.....
I'm beginning to wonder that,
If this figure were bogus and in fact there actually aren't many BHSpin to send it would explain things.
A small bunch get added - a few systems request jobs and get them all - the rest of the system see there are no jobs.

The average run-time seems to be ~2 hours. There are, apparently, ~124000 jobs in progress for ~1400 users. That's an average of 177 hours outstanding work per user - which is >1 week. That doesn't seem to tie in with what is being reported here,.
ID: 3676 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jim1348

Send message
Joined: 28 Feb 15
Posts: 253
Credit: 200,562,581
RAC: 0
Message 3677 - Posted: 5 Aug 2019, 21:30:59 UTC - in response to Message 3676.  

If this figure were bogus and in fact there actually aren't many BHSpin to send it would explain things.
A small bunch get added - a few systems request jobs and get them all - the rest of the system see there are no jobs.

If your machine is getting work at all, then you will get it about every other request, keeping your machine busy. And there are many people who are not getting work at all.

And the two groups are interchangeable, one week to another. So I don't know of anything that explains that.
ID: 3677 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 847
Credit: 144,180,465
RAC: 0
Message 3678 - Posted: 5 Aug 2019, 22:28:07 UTC

I wrote before that while in server is more than 2k tasks ready to send feeder can't serve all hosts.
This will go better as less tasks will be ready to go.

At the moment I just switch on host with 28 cores/56 threads and got 11 tasks.
I know that is stressful, but I don't want to remove generated tasks because I don't want to make mess in list of work to do.

Link to mentioned host:
https://universeathome.pl/universe/show_host_detail.php?hostid=151325
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 3678 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mmonnin

Send message
Joined: 2 Jun 16
Posts: 169
Credit: 317,253,046
RAC: 0
Message 3679 - Posted: 6 Aug 2019, 1:35:46 UTC

Then don't make so many. Make less more often.
ID: 3679 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3682 - Posted: 6 Aug 2019, 17:23:20 UTC - in response to Message 3678.  

Hello Krzysztof,
I want to believe you but Universe has run out of tasks previously and there never was a recovery issue, for me anyways. Maybe the previous WU's were much quicker to create and these new ones take a much longer time to create?
I have 3 machines that just will not download WU's since July 24th. I have 4 other machines that received WU's on Aug 1st was last time. I have machines that have received Universe WU's everyday since the 24th of July including today.... they have been crunching right along....
I have manually updated all of these hosts that are not receiving WU's with Universe update requests at least 3 times per day. Out of all of my hosts the three that haven't had any Universe WU's since July 24th doesn't match what you said. That's honestly right at a minimum of 40 requests for WU's that have been un-answered. Yea, maybe it's just a roll of the dice.... but 40 requests in 13 days.... on same days other hosts of mine receive WU's...
I think those are Las Vegas odds.....
Seems like an anomaly....
ID: 3682 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gordon Lack

Send message
Joined: 24 May 19
Posts: 16
Credit: 53,469,167
RAC: 0
Message 3683 - Posted: 6 Aug 2019, 18:48:13 UTC - in response to Message 3678.  
Last modified: 6 Aug 2019, 18:48:46 UTC

I wrote before that while in server is more than 2k tasks ready to send feeder can't serve all hosts.
This will go better as less tasks will be ready to go.
This is a chicken and egg problem.
The server routinely has ~3 million tasks ready.
It can't serve them out properly, so the number stays high,
If the number were lower it could serve out more and the number would drop even lower.
ID: 3683 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 847
Credit: 144,180,465
RAC: 0
Message 3684 - Posted: 6 Aug 2019, 19:05:42 UTC - in response to Message 3682.  

Hello Krzysztof,
I want to believe you

Believe me, this third time. It must wait to server hardware upgrade to sort this out.

Usually I'm not generating more then 2-2.5kk tasks to prevent this but this time I have rewrited tasks generation script for new application version and unfortunately I didn't proper check number of loops in it.

The plan was for 300k tasks for first batch, but 3kk was gone :(

Now, in last two days I see in server status that qty of tasks in progress are going slowly higher and I believe that next two-three days will be enough to solve problem (everyday is at least 100k tasks finished).
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 3684 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
JagDoc
Help desk expert

Send message
Joined: 21 Feb 15
Posts: 83
Credit: 873,439,097
RAC: 0
Message 3686 - Posted: 6 Aug 2019, 19:22:47 UTC

I had on some of my host the problem with no wu.
After updating project manually every 1 min for 10 -20 min they got work.
Then work comes without doing something.
ID: 3686 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3688 - Posted: 7 Aug 2019, 7:47:34 UTC - in response to Message 3684.  


Believe me, this third time. It must wait to server hardware upgrade to sort this out.

OK Krzysztof,
I trust that you understand more of this situation than myself so I will just be patient and wait and see.
Thank you.
ID: 3688 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
additude

Send message
Joined: 6 Feb 19
Posts: 41
Credit: 131,325,319
RAC: 0
Message 3693 - Posted: 8 Aug 2019, 14:29:30 UTC - in response to Message 3688.  

Hi Krzysztof,

You were right. I discovered that the 3 machines that I had problems with needed some software libraries updated and a settings change update to run with BHspin v2 v0.19.....

My machines are now all loaded with WU's.
ID: 3693 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gordon Lack

Send message
Joined: 24 May 19
Posts: 16
Credit: 53,469,167
RAC: 0
Message 3694 - Posted: 11 Aug 2019, 0:28:10 UTC - in response to Message 3693.  
Last modified: 11 Aug 2019, 0:52:16 UTC

Has something changed in the last day or so?

For some reason my logs now contains this (previously they did not):
Fri 09 Aug 2019 17:38:12 BST | Universe@Home | Sending scheduler request: Requested by project.
which is happening every 10 minutes.
Given that this can be (often is) followed immediately by:
Fri 09 Aug 2019 17:38:12 BST | Universe@Home | Not requesting tasks: don't need (CPU: not highest priority project; Intel GPU: not highest priority project)
Fri 09 Aug 2019 17:38:15 BST | Universe@Home | Scheduler request completed
it doesn't seem to be the BOINC scheduler doing this.
ID: 3694 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gordon Lack

Send message
Joined: 24 May 19
Posts: 16
Credit: 53,469,167
RAC: 0
Message 3695 - Posted: 11 Aug 2019, 1:55:22 UTC - in response to Message 3694.  

For some reason my logs now contains this (previously they did not):
Fri 09 Aug 2019 17:38:12 BST | Universe@Home | Sending scheduler request: Requested by project.
which is happening every 10 minutes.


And if all 12000+ active hosts are doing this that means your server is getting ~20 requests per second for tasks.
ID: 3695 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 847
Credit: 144,180,465
RAC: 0
Message 3696 - Posted: 11 Aug 2019, 10:02:52 UTC - in response to Message 3695.  

I switch this on temporary to push hosts to ask for work.
As I see on my hosts it helps. Also "work in progress" shows that more hosts get work.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 3696 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Number crunching : No WUs right now?




Copyright © 2024 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek