Posts Tagged With 'Perl' RSS Feed

Today's Topic: Perl's $_ Variable

pheaven — Tue, 08 Jan 2008 05:30:29 -0700

When we are speaking we often use words such as "it" to refer to the thing we are currently talking about. For example, "My computer has just had a PSU failure. It is on fire." You could say that "it" refers to the current topic, which we assigned in the previous sentence.

Anyone who's read many of my ramblings will know that one thing that interests me is the use of features of natural language in programming languages. What if we could express the idea of the current topic in a programming language, though?

Turns out that is exactly what the $_ variable in Perl is for. It is sometimes also known as the "default variable". If you have read many Perl scripts you will probably have come across things like this:

chomp;
s/\[b\](.*?)\[\/b\]/<b>$1<\/b>/;
print;

The question that people often ask on seeing this is - chomp what? Bind a substitution to what? Print what? A fairly substantial number of Perl built-ins, when invoked a parameter missing, will use the default variable $_ instead. You could re-write the above as:

chomp $_;
$_ =~ s/\[b\](.*?)\[\/b\]/<b>$1<\/b>/;
print $_;

The next question is, how do you set the current topic? Since $_ is a variable, you can assign to it:

$_ = "[b]badger[/b]\n";

However, there are some constructs that will assign a value to it automatically. For example:

for (A..Z) {
    print;
}

Will print the alphabet in uppercase. What is going on? Basically, (A..Z) creates a list containing all the letters from A to Z. You could have put a number of values separated by commas, an array or a mixture of these between the brackets. Once this list has been created, the loop now iterates through the elements. The first element of the list is "A", the second is "B" and so on. Perl automatically assigns the current element to $_ for each iteration. Note you could have used foreach here too.

There is, however, a neater way to write the above. As what is inside the for construct is very simple, we can get away with writing it like this:

print for (A..Z);
## OR ##
print foreach (A..Z);

This also works when using the diamond operator (which returns one line from a file at a time) in a while loop. Imagine $fh contains a file handle. We could chomp and print every line in the file like this:

while (<$fh>) {
    chomp;
    print;
}

If you want to perform the same operation on many variables, you can use a similar trick:

for ($a, $b, $c, $d) {
    s/'/&#39;/g;
}

This escapes all ' characters, which is a good idea you're going to be feeding any of those variables into an SQL query. You wouldn't want to open a hole for an SQL injection attack now, would you? (An SQL injection attack exploits programs that place user-supplied data into an SQL query without validating or sanitizing it by supplying data that changes the meaning of the query.)

$_ is lexically scoped and works a like a variable declared with my. It exists within the block it's created in only, and outside that block it's gone (and $_ refers to the enclosing block's topic). For example:

for (2..20) {
    for (1..10) {
        print;
    }
    print;
}

Will print the numbers between 1 and 10 followed by a 2, then 1 to 10 again, followed by 3 and so on.

Using $_ and its related tricks is a good way to shorten and neaten up your code, and saves you a little typing. However, remember to take readability into account. If it is a script that you will be doing a lot of work with in the future, consider whether using $_ now will save you time in the long run. You may be better assigning to a "named" variable that you know won't be over-ridden in a deeper scope accidentally during alterations. As always, there's more than one way to do it. Choose the appropriate one.

Perl turns 20, Perl 5.10 released!

pheaven — Tue, 18 Dec 2007 15:17:14 -0700

Today represents exactly twenty years since the very first release of the Perl programming language! On 18th December 1987, Larry Wall released Perl 1.0. Twenty years on there has been another release of Perl: Perl 5.10. Five years in development, Perl 5.10 makes some significant advances on the Perl 5.8 series both in terms of performance and features, while maintaining backwards compatibility with Perl 5.8.

The regex engine, always a strength of Perl, has seen a whole raft of improvements. Some of these include performance enhancements, and others are new features; if is now, for example, much easier to write recursive regexes for matching nested structures. You can also name captures rather than just referring to them by number, which will bring readability enhancements.

There are also some new features borrowed from the forthcoming Perl 6. This includes the // operator (like || but testing definedness rather than truth), the "say" keyword (like print, but automatically putting a newline character on the end), the state keyword for variables that you initialize once, then remember their value even when they go out of scope and the "given" statement, which is a bit like a switch block on steroids.

Today also saw the release of Parrot 0.5.1. Parrot is a virtual machine being developed to run Perl 6, and the Perl 6 compiler included in this release is still at a very early stage, though moving very fast. Most significant is that the Perl 6 compiler itself is now partly written in NQP (or Not Quite Perl 6), a subset of the Perl 6 language that was implemented for bootstrapping purposes. The developers hope that within the next couple of months, the Perl 6 implementation will take some huge steps forward; we'll keep you posted.

So, Happy Birthday to Perl, and here's to the next 20 years!

Splitting CSV with regex

Jonathan — Wed, 21 Nov 2007 06:01:18 -0700

I answered a question on the Perl forum today about splitting CSV. CSV is a comma separated format; for example:

blah,blah,blah

You can put values in quotes:

blah,"blah blah",blah

And those quotes make commas within them meaningless too:

blah,"blah,blah,blah",blah

If we do the naive thing and implement it using split on a comma:

my @fields = split(/,/, $string);

Then we will obviously get the Wrong Answer. The question was, is there a regex we can use with split that will do the Right Thing? And the answer was yes, though it took me a few minutes to come up with it. The thing is that we don't want to match anything more than the commas we are splitting on, but we do need to do some analysis on the string that is up ahead (or behind us) to detect if the comma we are seeing is in quotes.

We can use the (?!...) construct to achieve what we want. This is called negative lookahead; it says "if you can match the pattern here then the regex fails to match". It is a zero-width assertion, just like the start and end of string anchors. My first thought was that we could do something like:

my @fields = split /,(?![^",]+")/, $string;

So we split on a comma, unless the pattern in the negative lookahead matches. And that pattern says, if we see a load of characters after this comma that don't include a comma or a quote, followed by a quote, then we fail to match. That would imply we have seen a comma inside some quotes. This nearly works, but it doesn't handle the case where we see another comma in the quotes (so it would work for "blah,blah" but not "blah,blah,blah"). This isn't hard to solve - we just need say that it is OK to match a comma if there isn't a quote either side of it (so it's inside the quoted string, but not between two quoted strings). That gives us:

my @fields = split /,(?!(?:[^",]|[^"],[^"])+")/, $string;

You can probably transplant the regex (the bit between the slashes) into the regex engine of lots of other languages, though I only tested it in Perl. Hope this is useful, anyway.

Perl 5.10 Coming Soon!

Jonathan — Mon, 19 Nov 2007 10:40:34 -0700

I've spent the last few days at the French Perl Workshop. As well as my brand new talk, I gave two re-runs of two that I had given before. One was entirely new to the French audience, and the other one was from quite a while back and given a face lift so it seemed new (in fact, it was the first talk I ever gave to the Perl community; it was on the topic of web security, and from the web code I encounter it is no less relevant today than it was then). That was 100 minutes worth of talks, which was a pleasure thanks to a great audience.

The keynote talk covered some of the new features and improvements coming in Perl 5.10. While the Perl 6 project progresses to bring us the future version of Perl, Perl 5 is still what people are using to get work done today and thus a new release is most welcome. Many improvements have been made to the regex engine, to fix bugs, increase performance and introduce new features. I was lucky enough to spend considerable time talking about these changes with the guy who was doing most of them back at the German Perl Workshop in February. It is great to see that they will be available in an official release soon. There are also some features taken from Perl 6, including the "say" keyword (like "print" with a newline on the end automatically - something you often want anyway), the "given" block (a bit like a switch statement in other languages, but more powerful) and an short-circuit operator for checking if a value is defined rather than just false due to its value.

There at the workshop, during at lightning talk, Release Candidate 1 of Perl 5.10 was made. It is expected this year, sometime in December, provided no problems show up in the release candidate that blocks this.

Thanks to everyone who was there and made it such an enjoyable time. I'm already really looking forward to my next conference outing - in Israel - and have also submitted talk proposals for another couple of Perl conferences in 2008 too. I don't have any plans for appearances at non-Perl conferences, but I'm open to offers - contact me if you're interested.

Catch Me At Conferences!

Jonathan — Mon, 22 Oct 2007 07:05:28 -0700

I've got a brand new talk in the works, which I will be delivering at both the French Perl Workshiop (Lyon, 16th-17th November) and the Israeli Perl Workshiop (Tel Aviv, 31st December). I will be talking on the topic of parallelism - why we need to do it, how (not) to screw it up, what Perl 6 will bring to the table and some of the latest research for future ideas, including lock-free data structures. I hope to announce more European dates for the talk over the coming months, subject to it being accepted at more conferences.