PERL
Remember the spectrum we discussed? From scripting languages to interpreted languages to compiled lanaguges? Today, we are going to begin discussing PERL. And, in doing so, we are moving into the domain of interpreted languages.You'll find that the building blocks in PERL are less complex than in shell -- you'll be using libraries and language features, rather than whole programs. But, as compared to shell scripting, PERL has less overhead as the libraries are part of the same program -- and don't encounter the overhead of separate processes and interprocess communication via pipes.
PERL also has more powerful operators. In shell, there were string operators -- and then some others added as after thoughts. In PERL, they are very powerful and include such things as exponents and regular expressions. These are, in fact, more powerful than they are even in compiled languages such as C and Java.
The reason for this is that in compiled languages they like the operators to closely map to features frequently available in hardware. They don't like to "hide" tremendous complexity behind a simple-seeming operator. Instead, it is left to libraries so that programmers understand that the operation is complex and also so that there can be different options in the implementation. PERL is, for example, famous for the power of its regular expression and string processing.
Of course compiled languages execute more rapidly as they are ready for the processor and require no parsing, interpretation, or translation. But, again, the trade-off is that the building blocks are smaller and, so, development time is higher -- and maintainence is more costly, making the applications less agile and dynamic, a big concern in changing enviornments or with changing needs.
PERL - The Practical Extraction and Reporting Language
Perl isn't exactly a scripting language. But, we've got a spare day in the schedule -- and Perl is certainly one of the tools of the trade.Perl is an important language for many "quick programs". It was originally designed, as its name implies, as a tool for system administrators and others to "extract and report" -- basically to process log files, &c. As a result, it has a tremendously flexible and powerful regular expression capability -- something lacking in C, C++, and, to some extent, in Java.
And, this capability, combined with a language designed to make the common case convenient, has made Perl the language of choice for not only system administrators, but also as the "glue" used by IT developers, Web developers, &c. Basically, Perl is an interpreted lanaguage -- somewhere between a shell script and a full-fledged compiled HLL (but much closer to a compiled HLL, in many respects).
Shell scripting provides an excellent way to solve complex tasks with very little effort. But, it does this by pulling together powerful programs, usually using files and pipes as IPC tools. And, these techniques can be slow and cumbersome.
By comparison, the building blocks in Perl tend to be a bit smaller, but much more integrated. As a result, shell is often excellent for solving small but complex problems quickly. Perl is often used for medium-sized problems. And, truly large problems might be better done in a compiled language. But, especially with the current availability of tremendous processing power -- economies there are often insignificant.
Hello World!
A Perl program looks much like a shell script, except the program exec'd by the shell to process the script isn't, well, a shell -- it is the Perl interpreter. And the program that it is interpretign isn't, well, written in the language of the shell -- it is written in Perl.The program below shows the invocation of the Perl interpreter at the top of the program -- just like the shell -- and also a quick "Hello World!"
There are a few other features to note. Just like shell, comments begin with a #. Much like C, C++, or Java, lines end with a ";". And, lastly, quote are interpreted just as they are in shell. "Double quoted strings allow for the interpretation of escapes, such as the newline\n", whereas 'single quoted strings are exactly literal -- no interpretation at all.'
#!/bin/perl # The usual hello world program -- an an example of a comment print `Hello world.`; # Much like C, all lines end in a ;
Scalar Variables
Variables in Perl are typeless. They can hold strings, as well as characters, integers, and decimal numbers. Much like in shell, "typeless" really means "stirngs available for interpretation". But, in some ways, this interprettion is more natural in Perl. For example, mathematical operations can be performed without need for an external program.In Perl, scalar variables always have the prefix $. We'll soon see that scalars, lists and arrays have different prefixes.
I guess I should also note that literal values can be used just as in other languages, except that, for example, '3' and 3, are equivalent. Why? Everything is typeless and interpreted on the fly.
The Arithmetic operators
Perl basically uses the same set of arithmetic operators as C -- plus some:
$sum = 1 + 2; $difference = $value1 - $value2; $product = 5 * $value; $quotient = $value1 / $value2; $remainder = $value1 % $value2; $incrementafter++; $++incrementbefore; # and, here's a new and very cool one: The power operator $value = $base ** $exponent; # The following alsow work, as usual...more soon # <, >, <=, >=, ==, !=
String operators
Strings in Perl seem to have been inspired by shell scripts. As we already discussed, the "" vs '' works the same way. And, variable substitution to form strings works the same way:
$firstname = "Greg"; $lastname = "Kesden"; $fullname = "${firstname} ${lastname}"; # and, ge, gt, le, and lt, work as with shell scripts for comparison
In addition, the "." and "x" operators are also lots of fun. "." is concatenate and "x" literally causes a string to be repeated. Incidentally, the ".=" operator works just fine, too.
Please note: Although I don't think we discussed it in the context of shell scripts, the ${var} notation is also part of shell. It is used to offset the name of a variable name within a string. The reason is that in some cases, the string and the variable name would otherwise become impossible to distinguish, $varfollowedbysomethingelse, for example.
$fullname = $firstname." ".$lastname; $treepeat = "Bottle of beer" x 99;
One important note for Java programmers is that Perl treats strings as values, not objects. So, when strings are assigned, values are copied, not aliased via references.
Arrays
Perl provides traditional indexed arrays -- with some really cool operators. In Perl, array variables begin with an @ instead of a $. But, when referencing Arrays the $ is used, because the value is that of a single element, not the entire array. Array indexing begins with 0. As is the case with the Java ArrayList or C++ STL Vector, Perl arrays grow dynamically.The following code segment declares an empty array and also demonstrates the creation of an array with several initialized elements and the access, by index, of a single item within the array:
@winners = (); # An empty array @contestants = ("Greg", "Mark", "Rich", "Tim", "Angie"); # initialized array print "${contestants[2]}\n"; # Prints Rich
Array operations
The push and pop operations should be pretty intuitive once you're thinking in the right context: think LIFO stack. push add an item to the end (high index) of the array and then returns the length of the array. pop removes the last item and returns it. $#arrayname returns the index of the last item in the array -- not its length.
@winners = (); # An empty array @contestants = ("Greg", "Mark", "Rich", "Tim", "Angie"); # initialized array print "${contestants[2]}\n"; # Prints Rich $winner = pop (@contestants); push (@winners, $winner); print "@winners\n"; print "@contestants\n"; print $#contestants
Arrays can also be used to check this one out:
$wildcard = "Jeff"; @winners = (); # An empty array @contestants = ("Greg", "Mark", "Rich", "Tim", "Angie"); # initialized array print "${contestants[2]}\n"; # Prints Rich $winner = pop (@contestants); push (@winners, $winner); print "@winners\n"; print "@contestants\n"; @nextcontestants = ($wildcard, @winners); print "@{nextcontestants}\n";
Here's another nice array trick: Initializing a string from an array:
@namesarray = ("Greg", "Mark", "Jeff", "Rich", "Tim"); $names = "@namesarray"; print "${names}\n";
But, we need to be careful. Check out the example below. Notice the absence of the quotes. This will assign the length of the array to the variable on the left:
@namesarray = ("Greg", "Mark", "Jeff", "Rich", "Tim"); $count = @namesarray; print "${count}\n";
Arrays can also be used in a bizzar way to make parallel assignments:
# $item1 = $item1prime # $item2 = $item2prime ($item1, $item2) = ($item1prme, $item2prime);
Conditionals
Conditionals in perl work exactly like conditionals in C, except that they also offer the optional elsif construct that we saw in shell scripting:
if ($x == $y) { # blah blah } else { # ha ha } if ($x == $y) { # blah blah } elsif ($x == $a) { # blah blah } elsif ($x = $b) { # blah blah } else { # blah blah }
The Traditional for and while loops
Perl has for and while loops that exactly mimic the syntax of C, C++, or Java:
for ($count=0; $count < 10; $count++) { print "$count\n"; } while ( $option ne "Quit") { dosomethinguseful(); }
The foreach Loop
Perl also has a special for of the for loop designed to make array access more convenient. It basically allows you to walk through the array in an iterator-like fashion. Below is an example written with each of a traditional and foreach loop.The new-fangled, but super-convenient foreach version:
@contestants = ("Greg", "Mark", "Rich", "Tim", "Angie"); # initialized array foreach $contestant (@contestants) { print "$contestant\n"; }
The venerable, familiar, good ole' fashion version:
@contestants = ("Greg", "Mark", "Rich", "Tim", "Angie"); # initialized array for ( $count=0; $count <= $#contestants; $count++) { print "${contestants[$count]}\n"; }
Files
Perl file manipulation will be very familiar to those who have worked in Java or C. Files are manipulated through file handles, which are basically special identifiers used for open files. They are not prefixed with a $ and, by convention, they are written in all CAPITALs.Below is a pretty typical example. It opens $filename, as DATA_FILE and then uses the <> operator to construct an array representation. The file is then clsoed and subsequently printed from the array.
# open the file named $filename and associate it with the handle DATA_FILE open (DATA_FILE, $filename); # Read each line of the file into the array @lines @lines = <DATA_FILE>; # Iterate through the lines, printing each one foreach $line (@lines) { print "$line"; # Notice: No \n. This is already at the end of the $line } # Close the file close (DATA_FILE);
Output to a file is often handled simply by specifying the handle before the formatting string of a print, as follows:
print OUTPUT_FILE "This will land in the file!";
In addition to the unrestricted open show above, Perl allows files to be opened for only limited types of access. This is done by placing a <, >, <<, +<, +>, or +>> before the $file, as shown below:
- open (DATA_FILE, $filename); # unrestricted
- open (DATA_FILE, <$filename); # Read-only
- open (DATA_FILE, +<$filename); # Read/Write, must exist prior
- open (DATA_FILE, >$filename); # Write/create
- open (DATA_FILE, +>$filename); # Read/Write/Create
- open (DATA_FILE, >>$filename); # Append
- open (DATA_FILE, +>>$filename); # Read/Write/Create/Append
Additionally, it shouldn't come as a surprise that STDIN, STDOUT, and STDERR are predefined file handles. They work as expect.
The last thing I want to mention is that redirection is really just another form of file input. $filename can be replaced with "| command" or "command |", for output and input piping, respectively:
- # Send output to DATA_FILE to sort, which sends its output to a file
open (DATA_FILE, '| sort > sortedfile.txt');- # Invoke "ls -l" and tie it to DATA_FILE for input
open (DATA_FILE, 'ls -l |');
Regular Expressions and String Manipulation
these days, there are many great reasons to program in perl. One of those happens to be the first among those: its natural ability to play with strings and, in particular, regular expressions.The following two operators, =~ (match) and !~ (no match), are among the most basic. =~ returns the number of times a substring matching the regular expression is found in the supplied string. Sometimes it is interpreted as a true/false expression, where 0 matches is false (not found). The "not in" opertor !~ retunrs true if no matches are found.
The general forms are as follows:
$nummatches = ($somestring =~ /regular expression/); $notin = ($somestring !~ /regular expression/);
If you group parts of a regular expression within ()-parenthesis, and the regular expression is matched, each match within ()-parenthesis will be saved into a special variable -- much as was the case with, for example, sed. These special variables are $1, $2, etc. Careful! Careful! Everyone wants to believe that these variables represent command-line arguments as they do in shell. Notice the difference! It is also worth noting that, although not preferred, Perl will accept the \1, \2, /3, etc, notation common in many other programs. Regardless, here's a quick example:
if ( $somestring ~= /([0-9]+)[a-zA-Z]*([0-9]+)/) { # $1 is the number at the begining of the line # $2 is the number at the ending of the line } else { # $1 and $2 are unchanged }perl also has a special variable, $_, which represents the default string. Several important operators act on this string by default. For example, perl can do sed-style searching and replacing. When this type of expression is defined, it is acting upon $_:
$_ = "This is an example string: Hello World"; $changes = s/World/WORLD/g; print "$_\n"; # "World" is now WORLD print "$changes\n"; # The number of substitutions made; in this case, 1
The tr function is also very powerful. It acts much like the tr command. It allows the user to define a mapping of character-for-character substitutions and applies them to $_. Each character in the first field will be replaced by the corresponding character in the second filed. As with th s function above, it returns the number of substitutions:
$changes = tr/abc/123/; # a becomes 1, b becomes 2, c becomes 3
Please note: In the examples above, there are no quotes around the tr and s expressions. This is important. If the expressions are quoted, they'll be interpreted as strings and assigned, instead of interpreted as regex operations and performed.
Associative Arrays
Perl has a really cool language feature called Associative Arrays. Associative arrays are like regular arrays, except that they allow the use of strings as keys. So, when storing a value into an array, we don't need to use a numerical index -- we can use something descriptive. They are called "associative" arrays, because they they associate the key and the value, as contrasted with traditional arrays which associate a value with a somewhat arbitrary, and certainly less descriptive, numerical index. For example, consider an array of bank balances:
$bankbalance{"Greg"} = 100.00; $bankbalance{"Amy"} = 150.00; $mybalance = $bankbalance{"Greg"};We see that the keys are strings: "Greg" and "Amy". We see that the associative array, in effect, pairs them with their values, 100.00 and 150.00, respectively.
For those of you who happen by chance to be familiar with hash tables, it is easy to see that they are the natural implementation for associative arrays. The string key is hashed and, the resulting hash, is used as the numerical index into an array. Fot those of you who are not presently familiar with hash tables, no worries -- we'll talk about them soon enough.
A "Middle Spectrum" Language Feature
Associative arrays are tremendously useful to programmers. They are useful anytime we want effectively constant time, immediate, access to a value via some key. But, we tend not to see them as a language feature in scripting languages or in compiled high-level languages. Instead, they are generally available as a language feature in the languages in the "middle" of the spectrum from scripting languages to compiled languages, languages such as Perl, Python, AWK, &c.Associative arrays are a type of complex data structure. And, we tend not to see real data structures in scripting languages -- they are hard to share with the programs that do the real work. Instead, we tend to use the file system to store information that can't be held in scalar variabels or arrays in memory. The idea is that we generally pipe data into and out of programs that are capable of building their own complex data structures.
Unlike traditional arrays, which are easily projected into memory, associative arrays really are complex data structures. So, instead of hiding this complexity behind normally simple operators, compiled languages generally leave it to libraries, such as C++'s STL or Java's Collections classes.
Unlike scripting languages, the languages in the "middle" of our language spectrum have a different model. Rather than stringing together external programs to solve problems, they make very heavy use of their own data strcutres and complex libraries. But, unliked compiled languages which have some concern about efficiency, "middle spectrum" languages are more concerned about convenience than efficiency.
As such, they are more than willing to hide the complexity of associative arrays from the programmer within a first-class language feature since it makes programming more efficient. The fact is that the average Perl programmer would rather have associative arrays and not care about the implementation than to have to make use of a library for a very commonly used feature.
Syntax Detail
Recall that, in Perl, scalar variables begin with a $-dollar sign and traditional arrays begin with an @-sign. Associative arrays begin with a %-percent sign. As with traditional arrays, when referencing an individual element of an associative array, a $-dollar sign is used. As with traditional arrays, the reason is simple -- a single element of an array is a scalar value, not an array.Instead of using []-brackets to identify individual elements, associative arrays use {}-squiggles. Take a look a second look at our prior exmple to see the basic use cases: setting and getting the value of an element:
$bankbalances{"Greg"} = 100.00; $bankbalances{"Amy"} = 150.00; $mybalance = $bankbalances{"Greg"};
Initializing an Associative Array
An associative array can be initialized as a whole. To do this, we just feed it a list of key-value pairs. The individual pairs are just run together:
%bankbalance = { "Greg", 100.00, "Amy", 150.00};It is very important to notice that the key-value pairs are grouped together by their relative position within the initializer list. As a consequence, if we want to have an empty value, we can't just leave it out completely -- we need to save the place using an empty position within the list. Notice that "Greg" has no value -- but we still preserved its space within the list. Having discussed this syntax, I'd also like to observe that it is a much better to set some initial value, any initial value, than to leave it out. At least that way, you'll know where you are starting.
%bankbalance = { "Greg", , "Amy", 150.00};
Iterating Through Associative Arrays