Message boards : Number crunching : Server Thread
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 . . . 16 · Next

AuthorMessage
frankhagen

Send message
Joined: 12 Aug 17
Posts: 21
Credit: 58,957,280
RAC: 0
Message 5639 - Posted: 25 May 2022, 13:35:56 UTC

about time to hire a nanny for this kindergarten!
ID: 5639 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 30 Oct 16
Posts: 182
Credit: 18,395,933
RAC: 106
Message 5640 - Posted: 25 May 2022, 15:14:21 UTC - in response to Message 5639.  
Last modified: 25 May 2022, 15:14:51 UTC

about time to hire a nanny for this kindergarten!
You missed the capital A. I guess you failed kindergarten. I would have been spanked for writing such poor English.
ID: 5640 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5641 - Posted: 25 May 2022, 19:40:49 UTC

Anyone have an idea of how often the "top computers" update runs?

Thank you.
A proud member of the OFA (Old Farts Assoc.)
ID: 5641 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
frankhagen

Send message
Joined: 12 Aug 17
Posts: 21
Credit: 58,957,280
RAC: 0
Message 5645 - Posted: 30 May 2022, 6:53:38 UTC - in response to Message 5407.  
Last modified: 30 May 2022, 6:57:14 UTC

Zipping the files together would reduce the network overhead, and wouldn't cost a lot of spare cash :). Could that be done with reasonable ease?



yes - implemented since boinc 5.8


https://boinc.berkeley.edu/trac/wiki/JobTemplates

<gzip_when_done/>
use this to compress the output file before uploading, see FileCompression
ID: 5645 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 841
Credit: 144,180,465
RAC: 2
Message 5646 - Posted: 30 May 2022, 18:09:42 UTC - in response to Message 5645.  



use this to compress the output file before uploading, see FileCompression

And our files are zipped already (using ).
But this works for single files, not whole result in 6 files. As I already describe in few posts...
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 5646 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
frankhagen

Send message
Joined: 12 Aug 17
Posts: 21
Credit: 58,957,280
RAC: 0
Message 5647 - Posted: 31 May 2022, 2:44:36 UTC

To zip multiple files in a "mix & match" fashion, you can use the boinc_filelist function provided. Basically, it's a crude pattern matching of files in a directory, but it has been useful for us on the CPDN project. Just create a ZipFileList instance, and then pass this into boinc_filelist as follows:

bool boinc_filelist(
const std::string directory,
const std::string pattern,
ZipFileList* pList,
const unsigned char ucSort = SORT_NAME | SORT_DESCENDING,
const bool bClear = true
);


int do_gzip(const char* strGZ, const char* strInput) {
// take an input file (strInput) and turn it into a compressed file (strGZ)
// get rid of the input file after
FILE* fIn = boinc_fopen(strInput, "rb");
if (!fIn) return 1; //error
gzFile fOut = gzopen(strGZ, "wb");
if (!fOut) return 1; //error
fseek(fIn, 0, SEEK_SET); // go to the top of the files
gzseek(fOut, 0, SEEK_SET);
unsigned char buf[1024];
long lRead = 0, lWrite = 0;
while (!feof(fIn)) { // read 1KB at a time until end of file
memset(buf, 0x00, 1024);
lRead = 0;
lRead = (long) fread(buf, 1, 1024, fIn);
lWrite = (long) gzwrite(fOut, buf, lRead);
if (lRead != lWrite) break;
}
gzclose(fOut);
fclose(fIn);
if (lRead != lWrite) return 1; //error -- read bytes != written bytes
// if we made it here, it compressed OK, can erase strInput and leave
boinc_delete_file(strInput);
return 0;
ID: 5647 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[AF>Le_Pommier] Jerome_C2005

Send message
Joined: 25 Sep 15
Posts: 23
Credit: 6,587,067
RAC: 0
Message 5648 - Posted: 31 May 2022, 14:16:16 UTC

Weather forecast helping the universe !!



(j'attends la team 1er degré de pied ferme ;) )
ID: 5648 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5649 - Posted: 31 May 2022, 14:24:04 UTC

Hi,
I am getting random task files running 6-8+ hours. How long before these will "time out"?

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5649 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 15 Aug 20
Posts: 37
Credit: 8,554,094,169
RAC: 13,451
Message 5650 - Posted: 31 May 2022, 14:41:32 UTC - in response to Message 5647.  

but how much processing overhead will this induce on the server having to unzip ~200,000 results/day?

seems like it'll just trade one bottleneck for another.
ID: 5650 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
frankhagen

Send message
Joined: 12 Aug 17
Posts: 21
Credit: 58,957,280
RAC: 0
Message 5651 - Posted: 31 May 2022, 15:08:15 UTC - in response to Message 5650.  

about nothing compared to the havoc we had.
ID: 5651 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ian&Steve C.

Send message
Joined: 15 Aug 20
Posts: 37
Credit: 8,554,094,169
RAC: 13,451
Message 5652 - Posted: 31 May 2022, 15:38:41 UTC - in response to Message 5651.  
Last modified: 31 May 2022, 15:41:11 UTC

about nothing compared to the havoc we had.


yeah, but you're only having to do it for 100-200 results per day because CPDN has much longer tasks and less work available.

Universe is handling 1000x the amount of results daily they're getting back 2-3 results per SECOND. the overhead will be massive.
ID: 5652 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
arcturus

Send message
Joined: 24 Mar 22
Posts: 17
Credit: 137,338,000
RAC: 1,043
Message 5653 - Posted: 31 May 2022, 23:41:46 UTC

Scale back and redeploy to other projects starting with those most contributing to the backlog?
ID: 5653 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 30 Oct 16
Posts: 182
Credit: 18,395,933
RAC: 106
Message 5654 - Posted: 31 May 2022, 23:53:34 UTC - in response to Message 5653.  

Scale back and redeploy to other projects starting with those most contributing to the backlog?
What?!
ID: 5654 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5791 - Posted: 26 Aug 2022, 22:18:34 UTC

This is a server question for a non-U@H project.

I know "several" people here also participate in the Einstein@Home project for their gpus and over here for the CPU tasks.

Is anyone else getting a Proxy 502 from the e@h messages website?

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5791 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5792 - Posted: 26 Aug 2022, 22:19:35 UTC

This is a server question for a non-U@H project.

I know "several" people here also participate in the Einstein@Home project for their gpus and over here for the CPU tasks.

Is anyone else getting a Proxy 502 from the e@h messages website?

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5792 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mr P Hucker
Avatar

Send message
Joined: 30 Oct 16
Posts: 182
Credit: 18,395,933
RAC: 106
Message 5794 - Posted: 26 Aug 2022, 22:45:55 UTC - in response to Message 5791.  
Last modified: 26 Aug 2022, 22:46:19 UTC

This is a server question for a non-U@H project.

I know "several" people here also participate in the Einstein@Home project for their gpus and over here for the CPU tasks.

Is anyone else getting a Proxy 502 from the e@h messages website?

Tom M
Yes. Subscribe to this thread, lots of stuff is posted there when there are problems with any project.

https://boinc.berkeley.edu/forum_thread.php?id=10279
ID: 5794 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Grant (SSSF)

Send message
Joined: 23 Apr 22
Posts: 156
Credit: 69,772,000
RAC: 582
Message 5803 - Posted: 6 Sep 2022, 9:47:51 UTC
Last modified: 6 Sep 2022, 9:51:15 UTC

We seem to be back.
Uploads are going through, slowly, after a few retries.

Scheduler requests are taking a while to get a response, but work is being allocated- but it can take a few requests to get a response that isn't an error or just a timeout.
However actually being able to download allocated Tasks is near impossible without lots of "Retry pending transfers" action.


So what happened???
Grant
Darwin NT
ID: 5803 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5804 - Posted: 6 Sep 2022, 10:49:43 UTC - in response to Message 5803.  


So what happened???

I propose we had a "Labor (Day)" strike and now we are being hit by a "work slowdown" :)

While I don't know what happened it seems like currently we are experiencing a classic server overloaded by upload symptoms.

Tom M
A proud member of the OFA (Old Farts Assoc.)
ID: 5804 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Tom M

Send message
Joined: 18 Jul 17
Posts: 134
Credit: 1,379,173,617
RAC: 7,900
Message 5805 - Posted: 6 Sep 2022, 11:06:49 UTC - in response to Message 5804.  

Uploads finished for me. Downloads started.
A proud member of the OFA (Old Farts Assoc.)
ID: 5805 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Krzysztof Piszczek - wspieram ...
Project administrator
Project developer
Project tester
Avatar

Send message
Joined: 4 Feb 15
Posts: 841
Credit: 144,180,465
RAC: 2
Message 5807 - Posted: 6 Sep 2022, 15:35:02 UTC

It was a power cut over last night in server room.
Everything should be ok now.
Krzysztof 'krzyszp' Piszczek

Member of Radioactive@Home team
My Patreon profile
Universe@Home on YT
ID: 5807 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · 12 . . . 16 · Next

Message boards : Number crunching : Server Thread




Copyright © 2024 Copernicus Astronomical Centre of the Polish Academy of Sciences
Project server and website managed by Krzysztof 'krzyszp' Piszczek