'$jonathan.ramble(topic => 'work' & 'play');' Blog RSS Feed

Inside PH: Lots of post-launch tweaks

Jonathan — Wed, 23 Jul 2008 13:56:36 -0700

First up, I'd like to thank everyone who has been providing feedback on the PH re-design. As lead developer, I'm trying to factor it all in to our development and prioritize fixes and improvements based on what people are asking for. We went through quite a bit of internal testing before unleashing the new PH for community testing a week prior to launch. Even so, various bugs have been spotted since we launched, both by the PH community and the team. Many are fixed, some are still in our bug tracker.

Some of the more visible things that have been improved in the week or so since the launch include:

Restoring printer friendly mode in various articles where it had gone missing

The submissions area now lists the status of your resources

You can now edit the way that your submissions are tagged, to help make them more findable in the resource directory

New resources now show up in "tag feeds", so get more promotion on tag start pages

Tag pages now show synonyms for a tag, where we have them available

Made corners of homepage boxes a bit thinner, to make it a bit less "heavy", plus added links to user page of latest blog posts. Ah, and some pretty colorful icons.

Fixes to the color and font size controls in the dropdowns in the editor for blog and forum posting

This is in addition to many, many other behind-the-scenes fixes and improvements, meaning you should get faster page load times and allowing us to run things more smoothly. A lot of the changes we have made are a direct result of comments from Programmer's Heaven users, so thanks to everyone who gave this feedback, and please do keep it coming!

Finally, I'd like to apologize for it being so long since the last time I wrote an "Inside PH" blog entry. The months when we were working away on the re-design were incredibly busy for me, both working to bring you the new PH along with the rest of the PH team and also with work on the Perl 6 compiler, another big project I hack on. I hope to write much more soon, including lessons from the development of the new PH, some of the techniques we're using and how we tackled some of the challenges we faced.

As always, happy hacking, and I hope the re-design and subsequent fixes and improvements are making Programmer's Heaven feel more heavenly.

A love letter to ASP.NET

Jonathan — Tue, 05 Feb 2008 04:23:55 -0700

Dear oh so wonderful ASP.NET,

I know that we programmers are far, far more stupid than your code generator. Clearly, if I write:

<form action="BoardEdit.aspx" method="post" runat="server">

Then actually you might as well assume I'd written:

<form method="post" runat="server">

And left you to figure out what the form action should be, because you can do that perfectly every time, fully accounting for the URL re-writing. If I suggest something different, clearly I would never have a good reason for doing this (for example, in the case that you've got no damm clue how to respect URL re-writing rules, but oh no, that would NEVER happen, would it? Oh, wait...).

Love (the thought of punching) you loads,

Jonathan

A wish for C# 4.0

Jonathan — Mon, 04 Feb 2008 08:25:31 -0700

OK, OK, so C# 3.0 is only just out of the door. As anyone who has been reading my stuff for a while will have realized, I like what they've done with the language. Here at PH, we have now fully migrated all our solutions to Visual Studio 2008, and can now use C# 3.0 language and .Net 3.5 framework features everywhere they're useful. This makes me happy (though some annoying issues migrating to Web Deployment Projects 2008 decidedly aren't making me happy).

I haven't thought long and hard about what I'd really like in C# 4.0. I think I need C# 3.0 to settle in a bit more. I've used it quite a bit, but it takes a while to fully realize the power of all that is there, and I certainly don't want to start suggesting additions to the language when there is already a perfectly good solution to that problem.

There are many principles in programming language design. I'm quite heavily influenced by Larry Wall, designer of Perl, probably because I've listened to him talking about language design more than anyone else. One of the principles he has is known as huffmanization. Huffman coding is an approach to compression where you give the most frequently occurring things the shortest codes, so the things that occur most in the file you are compressing have the shortest representations. The analogy in programming language design is things that you do regularly should have a compact syntax, whereas things you do infrequently can be longer - because you do them less.

C# doesn't score badly overall alongside this principle generally, but there is one area that I really feel could use some improvement. How often do you write:

private string _Foo;

And then an accessor and mutator method:

public string Foo
{
    get { return _Foo; }
    set { _Foo = value; }
}

I find myself doing this a lot, and it's 6 lines of boilerplate (OK, only four lines if you ignore the braces, which don't have any understanding cost, but do take up screen space). By contrast, anonymous types let you declare fields and and accessors all in one go (and yes, I know it's a tad different).

What I'd really like is a way, in a single declaration, to declare an underlying private field with either protected, internal or public simple accessors. I'm not sure how it could look, or of some nice syntax for it. I guess an attribute could do it:

[Accessor("Foo", Scope.Public)]
private int _Foo;

Though that's still quite a bit to type. I've still not found any syntax to suggest that I like, though. Ideas, anyone? But anyway, I'd really like to see some improvement here. It's not a Big New Feature, but it would save a lot of trivial, boring code.

Inside PH: Final Builder

Jonathan — Thu, 10 Jan 2008 13:41:06 -0700

Before now I've talked about our use of the Subversion version control system and Visual Studio.Net 2008. These are two of the most important tools we all use in our day-to-day operations. Today I am going to talk about another one that saves us a lot of time and potential mistakes: Final Builder.

Programmer's Heaven is a large site. To keep the code base modular, we break it up into many libraries. Some of these libraries provide primitives that we use all over the place. On top of this bottom "layer" are another set of modules that relate to specific features. For example, we have the wiki engine, the forum, the blog engine and so on. Finally, there is the web project for the site itself.

In all, we have perhaps a dozen solution files to build each time we want to do a full build of the site to test or deploy. Doing that by hand every time would be extremely laborious and error prone, so we automate it. There are numerous ways to do this. Makefiles are perhaps the best known solution. Based on inference rules, they are certainly powerful, and pretty portable too. They are used for other projects I work on, where portability is a large concern. However, at PH we only ever build internally, always on Win32 and for deployment to an environment we have full control over.

Before I took over as lead developer here, Tore (the founder and webmaster here) had already automated the build process using Final Builder. It's GUI-driven, has plug-ins for numerous source control systems, compilers, unit test suites and so on as well as a huge range of file operations - and far more that we've just not needed yet. The build fails if any project fails to build, but we also fail if any of the unit tests fail too (something I'll talk about another time), to help prevent us from deploying broken stuff. Note that we build the unit tests and run them, but don't actually deploy them.

On a system "this big" (yes, I know, there are far, far bigger ones out there), life without an automated build tool would be rather painful. If you have a system of more than a couple of modules, I suggest you invest in something, whether it's a makefile, a home-grown script (I'd hack it up in Perl, of course ) or something like Final Builder, depending on your needs. There is a free trial of Final Builder that you can get if you want to play. (And by way of disclaimer, I'm not involved in any way with the company behind Final Builder; I'm just sharing something that has worked well for us here at PH.)

Upgrading from Vista to XP

Jonathan — Mon, 17 Dec 2007 01:57:17 -0700

For a light start to the week, I was amused by this parody review of Windows XP, written as if it was the successor to Vista. It certainly captures a lot of the reasons why I'm still on XP and have no plans to downgrade. Uh. Upgrade.

I hate "== true"

Jonathan — Thu, 13 Dec 2007 09:19:07 -0700

If you've asked someone to look for something you left at their house, would you say, "give me a call if it's true that you found it"? No, you'd say "give me a call if you found it". So why do so many people write things like:

if (found == true)

Instead of the shorter, clearer, equivalent:

if (found)

Yes, I know, I shouldn't let refactoring get me so cranky.

Inside PH: C# 3.0 and Linq

Jonathan — Mon, 10 Dec 2007 06:01:09 -0700

This morning I started working on a new library for a new feature we'll be rolling out in the not too distant future. It is the first production code that we're developing at PH using Visual Studio 2008, which brings us C# 3.0 and Linq.

Just one morning into it, it already feels like a vast improvement. It took all of ten minutes to get the DLinq classes generated in Visual Studio; it would have taken under five if it hadn't been my first time doing it and working it out as I went. Then they were ready to use, which was also trivial. No more writing SQL or stored procedures or calling methods on a data reader to get the data out: just instantiate the DataContext (which represents the database), write the query, and it's done.

I have also been enjoying the new Lambda expression syntax and type inferencing, which has saved me a few keystrokes already and makes the code somewhat more readable (provided you recognize the new syntax, of course). Once I know that this all fits into our build process smoothly, I'll probably begin moving some other projects over the Visual Studio 2008 so we can also use the new C# 3.0 features in those. Note that we won't be re-writing things just for the sake of using C# 3.0 features in them; that's probably just a reliable way to introduce new bugs, and I'd advise against that generally unless you've got really good reasons to do it.

Don't forget that PH is currently running a series on C# 3.0; the final part, on Linq, comes out this week! I plan to follow it up with an article on DLinq; I've been holding back on that until I'd got some more practical experience using it, though.

When C# bites

Jonathan — Wed, 28 Nov 2007 06:25:35 -0700

So I wanted to do:

private TTo Identity<TFrom,TTo>(TFrom I)
{
    return (TTo)I;
}

Well, actually I didn't. I wanted C# to be smart enough to realize that if S is a subtype of T then List<S> can safely be assumed to be a subtype of List<T>. But anyway, that's not the case, and you have to call ConvertAll on the List to get it to be a List<T> rather than a List<S>. Joy. Anyway, turns out the above fails to compile. Thankfully, this works:

private TTo Identity<TFrom,TTo>(TFrom I)
{
    return (TTo)(object)I;
}

But, argh.

Splitting CSV with regex

Jonathan — Wed, 21 Nov 2007 06:01:18 -0700

I answered a question on the Perl forum today about splitting CSV. CSV is a comma separated format; for example:

blah,blah,blah

You can put values in quotes:

blah,"blah blah",blah

And those quotes make commas within them meaningless too:

blah,"blah,blah,blah",blah

If we do the naive thing and implement it using split on a comma:

my @fields = split(/,/, $string);

Then we will obviously get the Wrong Answer. The question was, is there a regex we can use with split that will do the Right Thing? And the answer was yes, though it took me a few minutes to come up with it. The thing is that we don't want to match anything more than the commas we are splitting on, but we do need to do some analysis on the string that is up ahead (or behind us) to detect if the comma we are seeing is in quotes.

We can use the (?!...) construct to achieve what we want. This is called negative lookahead; it says "if you can match the pattern here then the regex fails to match". It is a zero-width assertion, just like the start and end of string anchors. My first thought was that we could do something like:

my @fields = split /,(?![^",]+")/, $string;

So we split on a comma, unless the pattern in the negative lookahead matches. And that pattern says, if we see a load of characters after this comma that don't include a comma or a quote, followed by a quote, then we fail to match. That would imply we have seen a comma inside some quotes. This nearly works, but it doesn't handle the case where we see another comma in the quotes (so it would work for "blah,blah" but not "blah,blah,blah"). This isn't hard to solve - we just need say that it is OK to match a comma if there isn't a quote either side of it (so it's inside the quoted string, but not between two quoted strings). That gives us:

my @fields = split /,(?!(?:[^",]|[^"],[^"])+")/, $string;

You can probably transplant the regex (the bit between the slashes) into the regex engine of lots of other languages, though I only tested it in Perl. Hope this is useful, anyway.

Perl 5.10 Coming Soon!

Jonathan — Mon, 19 Nov 2007 10:40:34 -0700

I've spent the last few days at the French Perl Workshop. As well as my brand new talk, I gave two re-runs of two that I had given before. One was entirely new to the French audience, and the other one was from quite a while back and given a face lift so it seemed new (in fact, it was the first talk I ever gave to the Perl community; it was on the topic of web security, and from the web code I encounter it is no less relevant today than it was then). That was 100 minutes worth of talks, which was a pleasure thanks to a great audience.

The keynote talk covered some of the new features and improvements coming in Perl 5.10. While the Perl 6 project progresses to bring us the future version of Perl, Perl 5 is still what people are using to get work done today and thus a new release is most welcome. Many improvements have been made to the regex engine, to fix bugs, increase performance and introduce new features. I was lucky enough to spend considerable time talking about these changes with the guy who was doing most of them back at the German Perl Workshop in February. It is great to see that they will be available in an official release soon. There are also some features taken from Perl 6, including the "say" keyword (like "print" with a newline on the end automatically - something you often want anyway), the "given" block (a bit like a switch statement in other languages, but more powerful) and an short-circuit operator for checking if a value is defined rather than just false due to its value.

There at the workshop, during at lightning talk, Release Candidate 1 of Perl 5.10 was made. It is expected this year, sometime in December, provided no problems show up in the release candidate that blocks this.

Thanks to everyone who was there and made it such an enjoyable time. I'm already really looking forward to my next conference outing - in Israel - and have also submitted talk proposals for another couple of Perl conferences in 2008 too. I don't have any plans for appearances at non-Perl conferences, but I'm open to offers - contact me if you're interested.

Inside PH: Now we're using Subversion

Jonathan — Mon, 12 Nov 2007 05:04:05 -0700

I'm now working as lead developer at Programmer's Heaven. Every so often, I'll be bringing you a little look inside, sharing some of the tools and techniques that we use to build and run the site.

Last week I completed migrating the Programmer's Heaven source tree to the Subversion version control system. Before I arrived, we were using a commercial offering, which I won't name here. I tried to give it plenty of chances to work well, and was patient when I heard a new version was coming out. However, the new version introduced new problems more than resolving the old ones, and I could never track down a pattern to the most serious issue, which led to broken builds a couple of times. It was frustrating knowing there were better tools out there; I had worked with Subversion on numerous projects before and knew that it was simple and Just Worked most of the time. It has its quirks, but nothing as close to annoying as what we were using.

Anyway, after months of suggesting we change to another version control system, they decided to make me lead developer, which basically meant that the migration was my problem if it went horribly wrong. Thankfully, after plenty of groundwork in the weeks leading up to it, I had the actual migration itself done and got our automated build scripts fixed up in the space of a day. I also took the opportunity for some re-organization of the repository so there's less top-level folders now - we'd spawned quite a few.

A good version control system is essential if you have multiple developers working on the same code base (and multiple means two and up - Tore and I could never have coped with us both hacking on the site without some form of version control over the last year). It's also useful when there's just one person involved too, especially if you are working on it over multiple computers. Being able to review any change that you did in the past or roll back to an earlier version of the code can get you out of all kinds of tricky situations. I speak from experience.

Anyway, that's your first peak at one of the things we're using here. I'll be brining you some more over the coming weeks and months. Questions welcome.

Catch Me At Conferences!

Jonathan — Mon, 22 Oct 2007 07:05:28 -0700

I've got a brand new talk in the works, which I will be delivering at both the French Perl Workshiop (Lyon, 16th-17th November) and the Israeli Perl Workshiop (Tel Aviv, 31st December). I will be talking on the topic of parallelism - why we need to do it, how (not) to screw it up, what Perl 6 will bring to the table and some of the latest research for future ideas, including lock-free data structures. I hope to announce more European dates for the talk over the coming months, subject to it being accepted at more conferences.

Are Internal DSLs Really Just Well Designed APIs?

Jonathan — Wed, 03 Oct 2007 09:19:40 -0700

Well, I figured I may as well make the first post in my PH blog vaguely controversial. If the title's not enough for you, then let me go on to suggest that the Bible has some useful advice in it for those of us trying to decode the continual stream of acronyms that the computing industry likes to throw at us.

"What has been will be again,
 what has been done will be done again;
 there is nothing new under the sun."
    -- Ecclesiastes 1:9

One of the things that amuses me most about the computing industry is how we seem to keep getting shiny new names for the same old technologies, perhaps just applied in a different problem domain or using a different language.

I have a computer science background, and one of my classes was on the subject of Distributed Systems. Shortly after taking this class, I decided it was high time I found out what on earth this AJAX thing everybody was going on about really was. And I came away saying, "OK, so we use JavaScript to make method invocations on an object that in turn makes a request to a server, which then responds to that request and then we use what is sent back to make changes to the page through some other set of objects...what's the new technology here?" And the answer is: there's not really much in the way of new technology. It's the same old client/server request stuff that got branded Web Services a while back, coupled with what we were doing without the client/server stuff and calling DHTML (Dynamic HTML) years ago.

Don't get me wrong here. I'm not saying that web services or AJAX are bad things. I have seen both used powerfully, and use web services to good effect myself. But in terms of the technology, they are nothing revolutionary. It's just a combination of formats and languages applied to a specific problem domain.

With all that in mind, I've spent some time trying to work out what the deal is with DSLs. DSL is short for Domain Specific Language. This is a (formal) language of some kind that is suited to a very specific task. For example, MathML is a domain specific language suitable for describing mathematical formulas. DSLs break down into Internal DSLs and External DSLs. Internal DSLs are just subsets of an existing more general purpose language, whereas External DSLs are languages of their own.

The advantage of an internal DSL is that you do not need to write your own parser for it, and you can use any of the features of the general purpose programming language amongst the domain specific bits. The advantage of an external one is that you are creating a language all of your own and have all the freedom associated with doing so. The advantages of one approach are the disadvantages of the other.

I see a lot of people talking about Internal DSLs in the Perl and Ruby camps, often with one side saying language X is better at it and the other pointing out that you can do the same kind of thing in language Y. The thing is, the more I look at the examples from both sides, the more I am left thinking: if it's an internal DSL, you are just using the syntax provided by the programming language, and therefore how is creating an internal DSL any different to just defining a nice interface to the module/package/whatever that you are working on?

Nice interfaces to modules and good API design are both of great importance. They make libraries and frameworks easier to learn and use by new programmers, and make the code more clear to those who are maintaining it. So if the whole DSL thing has just got people asking, "how can we use (or abuse) the syntax of our programming language to create a good interface to our module", I welcome it. But I'm failing to see anything new.