Posts Tagged With 'C# 3.0' RSS Feed

When Linq and ref Parameters Meet

pheaven — Tue, 01 Apr 2008 07:17:31 -0700

It's fair to say that I'm a pretty heavy user of Linq. You'll find uses of it scattered across my code, from the obvious (using DLinq to query a database) to the slightly more exotic (writing queries over collections obtained from classes in System.Reflection). Linq often allows you to express a problem very neatly, resulting in compact, readable code. It also factors out the application of operations and leaves you to worry about the operations themselves, likely decreasing bugs.

Fixing A Bug

Today I ran across some code that took a parameter, then used it in a Linq query. Omitting the clutter, it looked something like this:

public void Lookup(string URL, ref int ID, ref int Status)
{
    // ...stuff...
    var Result = from D in DB.Datas
                 where D.URL == URL
                 select D;
    // ...more stuff...
}

It then set the couple of ref parameters based upon the results of the query (they really should have been out parameters, and I'll likely do some further refactoring later). However, the code calling this also appeared to be assuming that that the URL parameter would be updated to exactly match the entry in the database. This mattered because the "where" clause would actually match if the case was different, and the calling code wanted to know when this was the case. Clearly, this code couldn't be updating URL; it wasn't a ref parameter. Making it a ref parameter here and in the calling code was the obvious fix, so I did that. The code now looked more like this:

public void Lookup(string ref URL, ref int ID, ref int Status)
{
    // ...stuff...
    var Result = from D in DB.Datas
                 where D.URL == URL
                 select D;
    // ...more stuff, including updating URL...
}

And then I hit compile.

The Compile Error

Making the above change gave me the following compile error.

Cannot use ref or out parameter 'URL' inside an anonymous
method, lambda expression, or query expression

Or, re-phrased to fit this specific problem, "you can't use ref parameters in a Linq query". My first thought was, "huh, what the...". My second was, "OK, we can easily work around that." But of course, the main question that lingered after the initial "just do something that works so I can fix the bug" was, "but why can't I?"

The Workaround

The workaround is straightforward enough: you just make a local variable and assign the ref parameter to it.

public void Lookup(string ref URL, ref int ID, ref int Status)
{
    // ...stuff...
    var Lookup_URL = URL;
    var Result = from D in DB.Datas
                 where D.URL == Lookup_URL
                 select D;
    // ...more stuff, including updating URL...
}

This compiles and works just fine. So if that's all you were looking for, you have an answer. But now let's dig into the "why".

Introducing ILDASM

ILDASM is short for Intermediate Language Disassembler. A .Net executable contains a sequence of low-level instructions encoded in a binary format that we call bytecode, as well as tables of metadata about the classes and methods it contains. ILDASM turns that bytecode into IL, or Intermediate Language: a human readable representation of the binary data.

To get us started, let's just look at disassembling the Hello World program. First, we compile the following C# into an EXE file:

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Hello, world!");
    }
}

And then we use ILDASM to get the Intermediate Language. Stripping away some sections we need not care about, we get:

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .method private hidebysig static void  Main(string[] args) cil managed
  {
    .entrypoint
    // Code size       13 (0xd)
    .maxstack  8
    IL_0000:  nop
    IL_0001:  ldstr      "Hello, world!"
    IL_0006:  call       void [mscorlib]System.Console::WriteLine(string)
    IL_000b:  nop
    IL_000c:  ret
  } // end of method Program::Main

  .method public hidebysig specialname rtspecialname
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

You'll see that there is a class declaration and two method declarations. The first is the Main method that we wrote. We can see that it loads the string "Hello, world!" onto the stack, then calls the the System.Console class' WriteLine method. Note that it finds its argument on the stack. The second method is actually a constructor and was generated for us; all it does is call our superclass' constructor.

As an aside for those of you wandering, "why the two nop (no operation) instructions", the reason for these is that I build this in debug mode. It inserts nop instructions anywhere that you could want to insert a breakpoint, and then at runtime actually patches the bytecode when you attach a debugger to contain "break" rather than "nop" opcodes.

The .Net CLR Doesn't Do Anonymous Methods

Now we'll write an example with an anonymous method. This code simply creates a list of integer values, then removes all of those that are less than or equal to five. I've highlighted the anonymous method - written with the compact lambda syntax - in red. It takes a single parameter, x, written to the left of the "=>", then checks that it is less than 5, producing a boolean result.

class Program
{
    static void Main(string[] args)
    {
        var TestList = new List<int>() { 1, 3, 8, 9, 10 };
        TestList.RemoveAll(x => x <= 5);
        foreach (var Val in TestList)
            Console.WriteLine(Val.ToString());
    }
}

If you run ILDASM on this program, you get quite a lot of output. I'll spare you the full listing (but feel free to try it) and just show you the parts that matter. The biggest thing you'll notice when going through the listing is that we have a new method.

.method private hidebysig static bool '<Main>b__1'(int32 x)
  cil managed
{
  .custom instance void [mscorlib]System.Runtime.CompilerServices.
    CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 )
  // Code size       12 (0xc)
  .maxstack  2
  .locals init ([0] bool CS$1$0000)
  IL_0000:  ldarg.0
  IL_0001:  ldc.i4.5
  IL_0002:  cgt
  IL_0004:  ldc.i4.0
  IL_0005:  ceq
  IL_0007:  stloc.0
  IL_0008:  br.s       IL_000a

  IL_000a:  ldloc.0
  IL_000b:  ret
} // end of method Program::'<Main>b__1'

This is actually our anonymous method, but it's not so anonymous once compiled. It's name is "<Main>b__1" - a name that we could never write in C# since it contains angle brackets, and thus has no chance of conflicting with anything else in our program. What's notable is that this is a method of its own, with only the name to hint to us what method it was originally contained within.

There are other interesting things in IL that we could spend a while looking at, including the fact that it makes it a delegate for this method and caches it in a private field to aid performance, however we'll put that aside for now and look at the next issue.

The .Net CLR Doesn't Do Lexical Scope

In the .Net CLR, methods can't see each other's local variables (unless we call one and pass it as a "ref"). This is normally the case in C# too, until we start writing anonymous methods. Then we can write anonymous methods that refer to variables in the enclosing method. Let's change our program to exhibit this.

class Program
{
    static void Main(string[] args)
    {
        var TestList = new List<int>() { 1, 3, 8, 9, 10 };
        int Maximum = 5;
        TestList.RemoveAll(x => x <= Maximum);
        foreach (var Val in TestList)
            Console.WriteLine(Val.ToString());
    }
}

This produces the same results as the previous version of the program. Now let's look at the disassembly. The first surprise we get is to discover that our Program class now actually contains a new, nested class!

  .class auto ansi sealed nested private beforefieldinit
    '<>c__DisplayClass2'
         extends [mscorlib]System.Object
  {
    .custom instance void [mscorlib]System.Runtime.CompilerServices.
      CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 )
    .field public int32 Maximum
    .method public hidebysig specialname rtspecialname
            instance void  .ctor() cil managed
    {
      // Code size       7 (0x7)
      .maxstack  8
      IL_0000:  ldarg.0
      IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
      IL_0006:  ret
    } // end of method '<>c__DisplayClass2'::.ctor

    .method public hidebysig instance bool
            '<Main>b__1'(int32 x) cil managed
    {
      // Code size       17 (0x11)
      .maxstack  2
      .locals init ([0] bool CS$1$0000)
      IL_0000:  ldarg.1
      IL_0001:  ldarg.0
      IL_0002:  ldfld      int32 Program/'<>c__DisplayClass2'::Maximum
      IL_0007:  cgt
      IL_0009:  ldc.i4.0
      IL_000a:  ceq
      IL_000c:  stloc.0
      IL_000d:  br.s       IL_000f

      IL_000f:  ldloc.0
      IL_0010:  ret
    } // end of method '<>c__DisplayClass2'::'<Main>b__1'

  } // end of class '<>c__DisplayClass2'

Because methods can not see each other's locals, the compiler has to resort to using fields to exchange information between them. So how is this class used inside our Main method?

First, it stores an instance of this class as a local variable of Main, instantiating it right at the start of the method. Yes, that means that you have an instance of the nested class created per call to the method! This is needed to get the correct semantics in multi-threaded code.

Whenever we reference the Maximum variable in our method, even outside of the lambda expression, it actually gets compiled down to a store field or load field instruction. This means that it's not, at the bytecode level, a local variable any more, which means it will be more expensive to access at runtime.

Code That Uses ref

So what does code that uses ref look like at an Intermediate Language level? Once again, we'll write a small test program:

class Program
{
    static void Main(string[] args)
    {
        var Test = 2;
        DoubleMe(ref Test);
        Console.WriteLine(Test);
    }

    static void DoubleMe(ref int Val)
    {
        Val *= 2;
    }
}

This program produces 4. Next up, we'll disassemble it:

.class private auto ansi beforefieldinit Program
       extends [mscorlib]System.Object
{
  .method private hidebysig static void  Main(string[] args) cil managed
  {
    .entrypoint
    // Code size       19 (0x13)
    .maxstack  1
    .locals init ([0] int32 Test)
    IL_0000:  nop
    IL_0001:  ldc.i4.2
    IL_0002:  stloc.0
    IL_0003:  ldloca.s   Test
    IL_0005:  call       void Program::DoubleMe(int32&)
    IL_000a:  nop
    IL_000b:  ldloc.0
    IL_000c:  call       void [mscorlib]System.Console::WriteLine(int32)
    IL_0011:  nop
    IL_0012:  ret
  } // end of method Program::Main

  .method private hidebysig static void  DoubleMe(int32& Val) cil managed
  {
    // Code size       8 (0x8)
    .maxstack  8
    IL_0000:  nop
    IL_0001:  ldarg.0
    IL_0002:  dup
    IL_0003:  ldind.i4
    IL_0004:  ldc.i4.2
    IL_0005:  mul
    IL_0006:  stind.i4
    IL_0007:  ret
  } // end of method Program::DoubleMe

  .method public hidebysig specialname rtspecialname
          instance void  .ctor() cil managed
  {
    // Code size       7 (0x7)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  call       instance void [mscorlib]System.Object::.ctor()
    IL_0006:  ret
  } // end of method Program::.ctor

} // end of class Program

I have colored the interesting lines in red. The first line shows us using the lodloca instruction rather than ldloc, which loads the address of the local variable onto the stack rather than the value of the local variable. Then, inside DoubleMe, it uses ldind and stind to load and store the value indirectly, that is, using the address.

So could they have made it work?

If all the IL has made you feel woozy, it's about to get worse. Now we've seen all the previous examples, let's try hand-editing some IL to see if we can actually get ref parameters to work in lambdas, and if not working out exactly why we can't. The first step is to write a program that is as close to what we want as we can get C# to compile, since editing IL is a pain.

class Program
{
    static void Main(string[] args)
    {
        var TestList = new List<int>() { 1, 3, 8, 9, 10 };
        int MaxVal = 5;
        RemoveVal(TestList, MaxVal);
        foreach (var Val in TestList)
            Console.WriteLine(Val.ToString());
    }

    static void RemoveVal(List<int> TheList, int Maximum)
    {
        TheList.RemoveAll(x => x <= Maximum);
    }
}

Next we can used ILDASM to get the IL code. Now let's start doing some small changes. First, we'll change the signature of RemoveVal to take an int32&, and the call to it to load the address of the MaxVal local and pass that instead of passing the value itself. Then inside RemoveVal, where we previously just loaded the parameter and assigned it to the Maximum field in the generated nested class, we'll add an extra instruction to load the indirect field.

    IL_0007:  ldarg.1
    IL_00071: ldind.i4
    IL_0008:  stfld      int32 Program/'<>c__DisplayClass2'::Maximum

Using ILASM, we can re-assemble this to an executable file, and discover that the program still works. However, that's not really got us any further than a temporary local would have in C#. What we actually want is to store the indirect reference in a field. Can we have a field of type int32&, though? To find out, in the anonymous class let's add one:

    .field public int32& MaximumTest

If we try and assemble this, it produces an EXE file. It all looks good until we try and run it, at which point the CLR crashes, leaving the following message behind:

Unhandled Exception: System.Runtime.InteropServices.COMException (0x801312E4):
Field of ByRef type. (Exception from HRESULT: 0x801312E4)
   at Program.RemoveVal(List`1 TheList, Int32& Maximum)
   at Program.Main(String[] args)

Why, though? Well, the reason is quite simply that, in general, it is unsafe to allow fields of ByRef type. This is because a ByRef is essentially a pointer into a stack frame, which is where local variables are stored. If you store a pointer into it, and that pointer exists beyond the stack frame (that is, until after we return from the method that declared the local we passed a reference to), you are pointing into unknown and very likely invalid memory. That would compromise the type-safety of the VM. I'd run across this very issue before in a different situation, so it was interesting to see it coming up as the answer to the question we set out to answer.

Summary

So, to summarize all of this drawn-out analysis, the reason we can't use ref and out parameters in an anonymous method (and thus a Lambda expression or a Linq query) is as follows. Anonymous methods compile down to separate methods. If they are accessing parameters or local variables, to share those with the method implementing the lambda expression we are required to put them in a field. To ensure the type safety of the CLR, we can not store a ByRef value (what we get when we use ref and out) in a field.

I suspect that fixing this well would require adding some extra support to the .Net Common Language Runtime itself, which is relatively unlikely to happen; this happened between .Net 1.x and .Net 2.x and I suspect they won't be in a hurry to do that again (the underlying VM between .Net 2.x and .Net 3.x is actually the same, it's just some extra libraries). You could do it by making stack frames garbage-collectible so it's OK for them to live beyond an invocation (but that probably hurts performance a lot), or you could do it by adding support for lexical scopes and closures, so the compiler doesn't have to fake them using fields. The Parrot VM, which I work on, does the second of these.

Anyway, I hope this analysis has given you more of a sense of the Real Answer, rather than the compilers simple, "you can't".

Divvy up my list? Aye!

pheaven — Thu, 31 Jan 2008 04:28:52 -0700

I wasn't born in Yorkshire, but I spent most of my childhood growing up there. Yorkshire, a large county in the north east of England, has its own collection of slang, not least including the word "aye", which it appears is appropriate in response to pretty much any situation. Another slang phrase is "divvy up". Basically, this just means to divide a bunch of things up somehow.

These days, I spend a lot of my time writing C#. If I'm dealing with a bunch of things in C#, I'll usually store them in a generic List collection, or some other collection. This got me thinking: how can I divvy up a List in C# into multiple Lists? And that's when I decided to implement the Divvy extension method.

What we're aiming for

Suppose we have a list of scores:

var Scores = new List<int>()
    { 87, 32, 45, 60, 91, 10, 58, 77, 66, 71 };

We want to categorize them by grade. For example, anything over and including 85 is a grade A, anything from 70 to 84 is a grade B and so on. This is where our Divvy extension method is going to come in useful. We'll look at what the following code means and how to implement the Divvy method in a moment, but using it will look like this:

// Split them up by grade.
var GradeScores = Scores.Divvy(new Dictionary<string, Predicate<int>>()
{
    { "Grade A", x => x >= 85 },
    { "Grade B", x => x >= 70 && x < 85 },
    { "Grade C", x => x >= 50 && x < 70 },
    { "Fail",    x => x < 50 }
});

It will give us back a Dictionary where the keys are the names of the categories we supplied when calling Divvy and the values are lists of items in that category. Therefore, we can loop over like this:

// Display results.
foreach (var Grade in GradeScores.Keys)
{
    Console.Write(Grade + ": ");
    foreach (var Score in GradeScores[Grade])
        Console.Write(Score.ToString() + " ");
    Console.WriteLine();
}

Which will give us the following output:

Grade A: 87 91
Grade B: 77 71
Grade C: 60 58 66
Fail: 32 45 10

Neat, huh? Let's take a look at how on earth we make this work.

Extension Methods

The first problem is that we want to have a method on the List class. However, that class is in the .Net class library, so we can't go changing it. We also don't want to have to start using a subclass of List all over the place just to get the extra method.

C# 3.0 allows you to write extension methods (detailed tutorial here). These are static methods that can be called as if they were instance methods. This way, we can write a method that can be called on instances of the List class. So our starting point is:

namespace Jnthn.ListExtensions
{
    public static class DivvyUp
    {
        public static void Divvy<T>(this List<T> TheList)
        {
            
        }
    }
}

Note the use of the "this" keyword to mark this as an extension method. Also note that List is a generic type - it takes a type parameter T. However, we don't know what this type is, and we want to make Divvy generic too. Thus we declare it as Divvy<T>.

The Return Type

We are going to split the List into many other Lists, based upon the category the item gets placed into. The categories are named by strings, e.g. "Grade A" and "Grade B". Therefore, for our return type we can use:

Dictionary<string, List<T>>

That is, a Dictionary where the keys are strings and the values are List<T>s, where T is the type of the original List that we were called on. Therefore, our method is now:

public static Dictionary<string, List<T>> Divvy<T>(
    this List<T> TheList)
{
    // Implementation goes here.
}

The Categorizers Parameter

Now we need a way to specify how to divvy up the list into the different categories. This is a mapping (from category name to something that determines whether an item is in that category), so we can use a Dictionary. The keys will be category names, which are of type string:

Dictionary<string,?>

However, what should the type of the values be? Well, they will be functions that take an item of type T and return a bool (true or false) depending on whether the item is in that category or not. There is a delegate in the .Net class library named Predicate<T> that specifies this type of function. Therefore, our parameter's type will be:

Dictionary<string, Predicate<T>>

Meaning that our full method declaration is:

public static Dictionary<string, List<T>> Divvy<T>(
    this List<T> TheList,
    Dictionary<string, Predicate<T>> Categorizers)
{
    // Implementation goes here.
}

You'll be glad to know that the implementation is simpler than the signature.

Initializing The Result Dictionary

Before we start categorizing the items, we need to set up the result Dictionary. We'll instantiate a Dictionary with the same type as the return type we worked out earlier. Then we will loop over the keys of the Categorizers parameter, which are the names of the categories, and create mappings in the Result dictionary from the names to empty Lists.

var Result = new Dictionary<string, List<T>>();
foreach (var Category in Categorizers.Keys)
    Result.Add(Category, new List<T>());

Divvying Up

Finally, we get to the code that's going to do the real work, and it's actually really quite straightforward.

foreach (var Item in TheList)
    foreach (var Category in Categorizers.Keys)
        if (Categorizers[Category](Item))
        {
            Result[Category].Add(Item);
            break; // Since we've divvy'd it into a list.
        }

The outer loop iterates over each item in the List that we were invoked on. The inner loop iterates over the names of the categories. The only slightly tricky part is the condition. Remember that the keys of the Dictionary are actually Predicate<T>s - functions that will return a bool when passed a value of type T. So we just call that function, passing the current Item as a parameter. If it returns true, then we know the item is in the current category. We add it to the correct result list, and then break. This takes us out of the inner loop, and we continue onto the next item in the List.

Putting it all together

Here's the Divvy method as a whole.

public static Dictionary<string, List<T>> Divvy<T>(
    this List<T> TheList,
    Dictionary<string, Predicate<T>> Categorizers)
{
    // Initialize result dictionary of lists.
    var Result = new Dictionary<string, List<T>>();
    foreach (var Category in Categorizers.Keys)
        Result.Add(Category, new List<T>());

    // Go through list items and divvy 'em up.
    foreach (var Item in TheList)
        foreach (var Category in Categorizers.Keys)
            if (Categorizers[Category](Item))
            {
                Result[Category].Add(Item);
                break; // Since we've divvy'd it into a list.
            }

    // And that's it.
    return Result;
}

Looking Back At The Call To Divvy

Let's take another quick look at the call to Divvy to better understand what is going on.

var GradeScores = Scores.Divvy(new Dictionary<string, Predicate<int>>()
{
    { "Grade A", x => x >= 85 },
    { "Grade B", x => x >= 70 && x < 85 },
    { "Grade C", x => x >= 50 && x < 70 },
    { "Fail",    x => x < 50 }
});

Here we are using two features of C# 3.0: collection initializers (tutorial here) and lambda expressions (tutorial here). First we instantiate a new Dictionary, with keys of type string and parameters of type Predicate<T> - exactly what we declared the Categorizers parameter of Divvy to be. Then we use a collection initializers to specify a list of key/value pairs. Each goes in curly brackets, separating the key from the value by a comma. Therefore, "Grade A", "Grade B" and so on are keys.

The values are lambda expressions. If you read where "=>" as "where", you can read the first one as, "x where x is greater than or equal to 85". Note that we are declaring x here - it is just a parameter name. Remember that a lambda expression declares an anonymous method (or function), and what comes to the left of the => is a parameter list. "x =>" declares the x, and "x >= 85" uses it.

Aye!

And that's your lot. Enjoy divvying up your lists, and I'm off to tuck into a nice Yorkshire pudding.

Higher Order Programming Is Easy!

pheaven — Tue, 22 Jan 2008 07:09:07 -0700

Higher Order Programming is one of those things that to many people sounds weird, magical, mysterious or just too hard for them to be able to do. It's not. If you have ever passed a function or method as a parameter to another function or method, then you have done higher order programming. If you have ever used a function pointer in C or a delegate in C# or some kind of callback mechanism, then you have done higher order programming.

Define it!

We are all used to writing code that takes parameters. We do it whenever we write subroutines, functions or methods. Normally these parameters are data. In higher order programming, one or more of these parameters is code rather than data. That is, you can pass code, functions and/or methods around, just as you can pass data around. That's all there is to it.

Show me an example

Let's look at C#, because the syntax will not be too unusual to programmers in many languages. Suppose we have a List of scores for a test.

var Scores = new List<int>() { 2, 7, 9, 5, 6, 10, 8 };

We want to get all scores that were considered passes. Suppose the test is out of ten, and to pass you need to get more than half marks. We can write a method to determine whether a score is a pass and that returns a boolean (true or false value).

public bool Pass(int Score)
{
    return Score > 5;
}

Then we can use the FindAll method to find all scores that were considered passes.

var Passed = Scores.FindAll(Pass);

Notice what we have done here. We did not call the Pass method ourself. Instead, we passed the Pass method to the FindAll method as a parameter. Internally, it looped over the values in the list and called the Pass method on each of them, and put those that it returned true for into a new list and returned it.

You are probably now thinking, why go to all the trouble when I could have just written a loop? And it's true - you could have written a loop that did the test in an if statement and build a new list.

var Passed = new List<int>();
foreach (var Score in Scores)
    if (Score > 5)
        Passed.Add(Score);

However, step back and think about what we just achieved from a software engineering perspective. It is a well known principle that separation of concerns helps to reduce code complexity. Here we have separated the concerns of:

Testing each element of a list and building a new list of elements that passed the test

The test itself

Therefore, higher order programming is a tool that enables us to reduce our code complexity by separating concerns, and also increase code re-use along the way.

Code without a name

You might be thinking, "huh, I just wrote more code than I would have; I had to define a whole extra method!" You can also argue, quite reasonably, that we have actually made the program harder to understand now too, because you have to go and find the Pass method to fully understand the code.

While sometimes we may have complex conditions that we wish to re-use in many places in our program - in which case writing a separate function or method is the right thing to do - in this case it would be nice if we could take advantage of higher order programming without having to resort to writing a separate method.

Enter anonymous methods, or lambda expressions. These allow you to define a method (or function, depending on your language) without a name and inside another method. Let's re-write the previous example using a lambda expression.

var Passed = Scores.FindAll(Score => Score > 5);

The "=>" is the syntax for defining an anonymous method in C# 3.0. Anything to the left of it is a parameter that the method takes. In this case, it takes a single parameter named "Score". To the right of it is the body of the method, with an implicit "return" (that is, it will return the boolean result of testing whether its parameter Score is greater than 5).

You could read the "=>" quite nicely in this case as "where". That is, "find all scores where the score is greater than five".

Yes, you can return code too

Suppose that I want to work out how many scores got different grades. For example, Grade A was given for scores between 8 and 10, Grade B for given for between 5 and 7, and so forth. We could write:

var GradeA = Scores.FindAll(Score => Score >= 8 && Score <= 10);
var GradeB = Scores.FindAll(Score => Score >= 5 && Score <= 7);

However, it would be nice if we could make something that read more clearly, such as:

var GradeA = Scores.FindAll(Range(8,10));
var GradeB = Scores.FindAll(Range(5,7));

How could this work? Well, FindAll expects a method that takes an int and returns a bool. This type of method - that is, a method with that signature - has a name: Predicate. Therefore, the Range method needs to have a return type of Predicate (actually, Predicate<int>, which means that the parameter it takes is of type int).

public Predicate<int> Range(int LowerLimit, int UpperLimit)
{
}

And what do we write inside the Range method? Well, we return an anonymous function that takes a single parameter and tests if it is squeezed between the two values we supplied as parameters to the Range method.

public Predicate<int> Range(int LowerLimit, int UpperLimit)
{
    return x => x >= LowerLimit && x <= UpperLimit;
}

And that's it.

What languages support this?

Here's the stinger: not all languages support these kinds of programming techniques. C# started out with the ability to do higher order programming, but without anonymous methods. In C# 2.0 they introduced anonymous methods, but with a fairly cumbersome syntax. In C# 3.0, they introduced the nice "lambda" syntax that I have shown here. Essentially, the language designers have realized more and more the value of higher order programming and worked to make it as easy as they make object oriented programming. The C# of today is a truly multi-paradigm language.

Many of the dynamic languages provide good support for this, including Perl, Python and Ruby. C enables it through function pointers, though there's no anonymous method or lambda style syntax. Java, however, has so far chosen to stick to its object oriented roots rather than embracing the multi-paradigm approach. We'll leave arguing about whether that's a good or a bad thing for another time. And primarily functional languages, such as ML and Haskell, make extremely heavy use of higher order programming techniques.

Taking it further

The examples I've shown here are trivial, but they are enough to give you a taste of what higher order programming is about. Try writing some code like this yourself, or reading up on higher order programming in a language of your choice. And when you're trying to work out a good way to get better code reuse and reduce code complexity, keep it in mind as one of the tools in your toolbox, alongside the other paradigms (such as object oriented programming, procedural programming and generic programming) that you are probably already more familiar with.

C# 3.0 Part 4: Linq

pheaven — Wed, 12 Dec 2007 05:12:31 -0700

The final part of our C# 3.0 series is about Linq - Language Integrated Query.

Learn:

What declarative programming is

How to write Linq queries

How to use ordering, joins, groups and query continuations

What DLinq and XLinq are

Read it now!

Inside PH: C# 3.0 and Linq

Jonathan — Mon, 10 Dec 2007 06:01:09 -0700

This morning I started working on a new library for a new feature we'll be rolling out in the not too distant future. It is the first production code that we're developing at PH using Visual Studio 2008, which brings us C# 3.0 and Linq.

Just one morning into it, it already feels like a vast improvement. It took all of ten minutes to get the DLinq classes generated in Visual Studio; it would have taken under five if it hadn't been my first time doing it and working it out as I went. Then they were ready to use, which was also trivial. No more writing SQL or stored procedures or calling methods on a data reader to get the data out: just instantiate the DataContext (which represents the database), write the query, and it's done.

I have also been enjoying the new Lambda expression syntax and type inferencing, which has saved me a few keystrokes already and makes the code somewhat more readable (provided you recognize the new syntax, of course). Once I know that this all fits into our build process smoothly, I'll probably begin moving some other projects over the Visual Studio 2008 so we can also use the new C# 3.0 features in those. Note that we won't be re-writing things just for the sake of using C# 3.0 features in them; that's probably just a reliable way to introduce new bugs, and I'd advise against that generally unless you've got really good reasons to do it.

Don't forget that PH is currently running a series on C# 3.0; the final part, on Linq, comes out this week! I plan to follow it up with an article on DLinq; I've been holding back on that until I'd got some more practical experience using it, though.

C# 3.0 Part 3: Initializers & Anonymous Types

pheaven — Thu, 06 Dec 2007 01:27:52 -0700

The third article in our C# 3.0 series covers the new features that will help you to build data structures with less and more readable code.

The article covers:

Object initializers

Collection initializers

Anonymous type is

Type equivalence of anonymous types

Check it out.

C# 3.0 Part 2: Extension Methods & Lambdas

pheaven — Tue, 27 Nov 2007 09:02:06 -0700

The second part of our C# 3.0 series is here, covering extension methods and lambda expressions. Learn:

What an extension method is

How to write extension methods

The new lambda expression syntax

To use the higher order programming paradigm

Go to the article!

Learn C# 3.0 With Our New Series!

pheaven — Wed, 21 Nov 2007 09:31:37 -0700

This week we are starting a brand new four part series, covering the new features of C# 3.0! The first article of the series covers type inference. You'll learn:

What is type inference anyway?

How can you take advantage of it?

What does the new "var" keyword do?

Is this at all related to Variant types in Visual Basic?

Will using this hurt the performance of your programs at runtime?

Check out the article!