Remove this ad

Lead

Nov 7 13 5:12 PM

Tags : :

Doesn't allow for enough part files. Simple patch

# diff -u -F3 src/utils/scripts/mldonkey_importer.pl.BAK src/utils/scripts/mldonkey_importer.pl
--- src/utils/scripts/mldonkey_importer.pl.BAK 2013-11-07 16:09:58.642824063 +0000
+++ src/utils/scripts/mldonkey_importer.pl 2013-11-07 00:34:50.945980709 +0000
@@ -394,7 +394,7 @@ $final_string = $final_string . &int64_s
my $n = 1;
my $result = 0;

- while (!$result && !($n>999)) {
+ while (!$result && !($n>9999)) {
open(TEST, " <" . $output_folder . sprintf("/%03d.part.met",$n)) or $result=$n;
close(TEST);
$n++;
Quote    Reply   
Remove this ad
Remove this ad

#3 [url]

Nov 26 13 2:23 PM

Re: 10814 patch: /utils/scripts/mldonkey_importer.pl

There's no "hard" limit on the number of files you can download, or put in the download queue. However, there's a soft limit, raised by common sense... it takes some time to get sources for them, even more for publishing them. Depending on the types of the files, you may very well get the same sources for many of them (like when downloading a TV series), which means you don't gain anything by downloading them all at once.
Just having them sitting in the download queue stopped doesn't hurt, nowadays memory isn't something to worry about (at least on today's desktop machines).

I think the patch is valid, but I'd make it iterate over all *.pat.met files rather than counting up to a preset number...

concordia cum veritate

Quote    Reply   
Remove this ad

#4 [url]

Nov 27 13 9:50 PM

Re: 10814 patch: /utils/scripts/mldonkey_importer.pl

Ok, here is the patched patch.
stoatwblr, please test it. I don't have mldonkey files to import, and I rewrote the loop a bit (should me much faster this way, since it only checks if files exist and doesn't open them).

Stupid board rejects any type of attachment.

[code]sub get_first_free_number {
my $n = 1;

while (-f sprintf("$output_folder/%03d.part.met",$n)) {
$n++;
}

$n;
}
[/code]

Quote    Reply   

#5 [url]

Nov 28 13 12:10 AM

Re: 10814 patch: /utils/scripts/mldonkey_importer.pl

GonoszTopi wrote:
There's no "hard" limit on the number of files you can download, or put in the download queue. However, there's a soft limit, raised by common sense..


And available memory, as you pointed out. There are definitely issues with having lots set active (more than about 600 causes issues on this system) which seems to come down to a mixture of available memory and the number of search requests being sent out.

OTOH There are extremely major issues with having a lot of files made available, including extremely long hang periods when issuing "reload shared", but that's a topic for another thread. (the size of known.met files is a particular pain point. Why are they 5 times larger than XML-style mldonkey files?)

FWIW, if you're targetting 1960s-era TV series with upwards of 35 episodes per season it's relatively easy to exceed 1000 items in the queue. You're right that many times theyr'e all available from the same source but there are a number of edge cases where they aren't or sources only show up once every 6-9 months (or worse).

(This ancient box has 16Gb of ram, of which 12Gb is eaten by ZFS ARC management)

Quote    Reply   

#6 [url]

Nov 28 13 12:14 AM

Re: 10814 patch: /utils/scripts/mldonkey_importer.pl

StuRedman wrote:
stoatwblr, please test it


I deleted the mldonkey part files about a week ago. Checking this will take quite a while (I'll have to reexport to mldonkey, then pull them back to a test area to verify things are working)

Quote    Reply   
Remove this ad
Add Reply

Quick Reply

bbcode help