Very Weird Perl Running Time Difference

[Jeremy]

Hmmmmmm - now here's an odd one:

I'm running perl 5.8.3 from ActiveState (binary build 809) under WinXP SP2.

One little scriptette I've written chunters through about 11k lines in a text file, searches and removes certain lines and then appends the acceptable lines to a data file with some html coding, so the output file can be used as an html doc in another app.

Now here's the rub....

If I append the acceptable data to file.txt the script takes 1min 40 secs to run.

But if instead I append the data to file.html the script takes 50 mins 28 secs to run.

The file extension is the only difference, but I don't understand why it would have such a drastic impact on the script performance.

Any ideas?

[cd34]

is it possible that you have a virus scanner on that is checking .html files for exploits?

Are you doing an open/write/close for each line? or open/append, dump a ton of text, close?

If open/close, I would suspect that something else is checking that html file each time -- that's the only thing I can think of that would do that.

[Jeremy]

Good thinking! That could well be the answer. I'm using AVG, MS Anti-Spyware and also have the MS SP2 Security Center running. Any one of those three could be checking the html file, which opens, appends and closes each time.

I've had a quick look at how WinXP can log things, but not in much details yet. Hopefully, that'd give me the definite answer.

This is the section of the code I'm using with the html output file bolded. It's probably not terribly elegant or efficient but as I'm a relative newbie to perl and it works okay outputting to txt files, it does the job currently. I can always rename the txt to html if necc :-)

Basically it reads each line of a file (http://domain.tld) into the variable and prints it out with an href around it and the domain.tld completing the link.

open (CLEANLINKSFILE, "$WorkingDir/InputLinks.txt");
	foreach $CleanedLinksGrab (<CLEANLINKSFILE>) 
		{
			chomp $CleanedLinksGrab;
			$CleanedLinks = "$CleanedLinksGrab";
			
			open (XENULINKSFILE, ">>$WorkingDir/OutputLinks.html");
			print XENULINKSFILE "<a href=\"$CleanedLinks\">$CleanedLinks</a><br>\n";
			close (XENULINKSFILE);
		
		} 	
close (CLEANLINKSFILE);

[cd34]

try that

$HTML = "";
open (CLEANLINKSFILE, "$WorkingDir/InputLinks.txt");
	foreach $CleanedLinksGrab (<CLEANLINKSFILE>) 
		{
			chomp $CleanedLinksGrab;
			$CleanedLinks = "$CleanedLinksGrab";
			
                        $HTML .= "<a href=\"$CleanedLinks\">$CleanedLinks</a><br>\n";
		
		} 	
close (CLEANLINKSFILE);

open (XENULINKSFILE, ">>$WorkingDir/OutputLinks.html");
print XENULINKSFILE $HTML;
close (XENULINKSFILE);

[Jeremy]

Hell's bells! That changed the total running time to about a second (probably under)! Thanks cd34 :-))

$HTML = ""; is that the same as: undef $HTML; ???

The . concatenates / adds each line to the variable???

Do you know of a decent layman's guide to using . ? I know it's really very powerful but have yet to find a good tute that doesn't speak Zorgian ;-)

[cd34]

undef would do the same thing -- although, generally, you want to declare a variable if you ever decide to use the script in strict mode.

.= is the same as $variable = $variable . "stringtoadd";

I've gotten to the point where I really rarely use inline substitution because there are times that it doesn't work the way I would expect, so, a lot of times you'll see something like:

$html = '<a href="' . $sql->{url} .'">' . $sql->{title} . '</a>';

the same thing could be accomplished with sprintf

$html = sprintf('<a href="%s">%s</a>',$sql->{url},$sql->{title});

With perl, there are dozens of ways to accomplish the same thing. Oddly, that doesn't simplify things.

As for books, I work best from reference rather than tutorial books. I would suspect that one of the O'Reilly zoo books would be worth a look through at a bookstore. You really want to thumb through about 10 different books on perl to find one that has a writing style you like. Advising programming books is quite difficult.