“Looks like I picked the wrong week to stop sniffing glue”
Steve McCroskey, Airplane!

• So it’s been a while since my last post. Just over a month, not including The F5 High, which wasn’t “original IP”. Frankly, I just stopped reading pretty much cold turkey. I wanted and needed to go heads down on day job stuff for a while. Since I haven’t been reading, Morning Coffee is going to be a little cold while I ramp back up.
• The new NHL season is upon us, and the Caps are looking good so far. Obviously, they have the new uniforms, but they’re also out to a 2-0 start for the first time in five years. And in those two games, they’ve only allowed one goal and are 100% on the PK. It’s nice to see them start strong, but obviously there’s a long way to go. Here’s hoping the can stay strong all season.
• Speaking of staying strong, the wheels that were rattling last week came off the Trojan bandwagon completely this week. I’m not sure it’s as big an upset as Appalachian State beating Michigan but it’s close. What happened to the team that scored 5 TD’s in a row on Nebraska?
• Big news last week is that MSFT is going to release the source code to much of the .NET Framework. Scott Guthrie has the details. Frankly, between Rotor & Reflector, it wasn’t like you couldn’t see the source code anyway, so this seems like a no-brainer. But integrating it directly into the VS Debugging experience, that’s frakking brilliant.
• I haven’t had a chance to install the new XML Schema Designer (Aug 07 CTP)  but I was really impressed with this video. The XML Team blog has more details. However, I’m not sure what the ship vehicle is. The CTP install on top of VS08 beta 2, but in the video they keep saying “a future version” of VS, implying that it’s not going to be in VS08.
• Dare is spending some time investigating SSB. I think it’s interesting that some of the REST crowd are starting to see the need for durable messaging. Dare argues that the features and usage models are more important than wire protocol. As long as it’s standardized, I don’t care that much about the protocol. Several of the REST folks mentioned AMQP. While I’ve got nothing against AMQP technically (frankly, I haven’t read the spec), but what does it say about durable messaging vendors (including MSFT) that a financial institution felt the need to drive an interoperable durable messaging specification?

A while back, my good friend and MAAB cohort Loren Goodman sent me the following. I was hoping he’d start his own blog and post it (and other stuff) there. But he’s busy working on other things these days, so I doubt he’ll get to the whole blogging thing anytime soon. So I offered to post it here rather than have it sit gathering virtual dust. Enjoy.

Update: I got the timeline wrong. Loren sent me this and I offered to post it. He didn’t ask me to post it originally as I originally wrote. Sorry about that, Loren.

---

The F5 High

Hunting and farming in the next generation of software development

A little while ago I was having an intense discussion with a colleague of mine Scott Colestock while we were both out at Microsoft.  We were discussing the ever present need some developers have to build things themselves instead of looking for something to leverage or using an off the shelf product.  After we were postulating a couple of off the wall, overly judgmental and holier than thou theories behind this phenomenon, Scott said something that really hit home and opened my eyes to an interesting explanation of what might really be going on.  He offhandedly said maybe it is not about ego or fear of less than adequate solutions at all; maybe it is actually bout the F5 high instead.  The F5 high… hmmm have not heard of that one.  What is the F5 high I asked him?  The F5 high, you know, that feeling you get when you hit F5 in the development environment and something happens.  Come on, admit it, It feels good.  As I took in the idea, I realized that he was totally right, it does feel good, it is a sense of accomplishment, it is an ultra quick hit, and more importantly, it makes us want to do it again.  This simple button and the rote process wrapped around it is a gateway to experiencing indisputable proof of our own creativity.  So to some extent, we are able to become our own dealer, dolling out small packets of that near harmless drug called satisfaction whenever we want.    So after this grand near religious realization of our true essence and this new found understanding behind that very special little key, we moved back on topic and continued our discussion about architecture.

Later on I was replaying the discussion between Scott and I with another colleague of mine, Jon Tobey.  Jon is one of the most avid fisherman I know.  Right in the middle of my F5 story, just when I thought mans greatest revelation had been already been achieved, if you can even imaging anything greater, he proceeded to add yet another improvement to the growing theory.  He took the F5 high concept and explained it in the context of fishing.  As a sportsman, fishing is about the hunt, it is about the building feeling you get when there is a bite and the exhilaration that courses through your veins as you attempt to catch and reel in the unknown trophy on the other end of you line.  In the case of fishing, we don’t even keep the fish, we throw it back for reuse.  Considering that we return the fish to the pond or stream from which we caught it, fishing is clearly much more about the rewards of the process and not about the value proposition of the actual fish.

If we approached fishing trips like software projects there would be some very strange consequences.  Firstly, we would have to keep everything we caught – even if it is not a fish and even if we did not catch it.   Then we would bring them back to office and try and probably have to cram them though the little slots on the sharepoint server or something.  Reusing fish would be dramatically less fulfilling, we would just take the dead fish, somehow work a hook into its mouth, throw it across the floor, and then yank the rod with excitement on a par with a kid on the traveling carnival merry go round.  Then drag the lifeless fish corpse across the floor by reeling it in until it reached the end of our rod.  Take it off and throw it back across the floor, repeat.  Man, that sounds exciting doesn’t it?  So where does that leave us?  Right now we have a very smelly server closet and a bunch of drug crazed developers yanking dead fish around the office hoping to somehow score an F5 fix while wondering when fish serialization technology will reach the point where the rehydrated fish are slightly more exciting.  They are all very excited about the new social network called FishTank 2.0 where anyone can reel in anyone else’s dead fish.  This does not look good – I must need a quick hit of F5 – {F5} – crap, well not so exciting, it turns out that F5 in word fires up the find and replace box and I remain unfulfilled.

So it appears that good developers, aka hunters, need to do some fishing every once in a while to keep the monotony of eating fish every day in check.  For when this get too far out of balance, the devs usually start using the project managers as the fish and the requirements of the project as the river.  They bait the project managers and try to yank them off the page and into something new, something that will give them that F5 fix they so desperately need.  When the PMs tell them NO, they might go to the users, a different company, who knows?

After spending almost all of fiscal year 07 (July ’06 thru June ’07) not traveling and not presenting, I’m going to be doing a few public talks to finish out the year. If you, dear reader, are going to one of these please drop me a line. Invariably, it’s the side meetings and discussions that are the most valuable at these conferences.

IT Architect Regional Conference 2007
October 15th – 16th, San Diego, CA

I’m a huge fan of IASA, so I’m thrilled to be doing their west regional conference. I’ve presented to a packed house for the local chapter before, so I think these folks will put on a good conference. They sure have a good selection of topics and speakers.

My session is called “Moving Beyond Industrial Software“. Here’s the abstract:

Computers have been instrumental in ushering in the post-industrial age. Yet, most enterprises today are run with an industrial mindset and the IT department is organized like a factory. This creates a tension between the forces of industrialization that define the organization and the forces of post-industrialization that define today’s marketplace. For example, our post-industrial world is becoming more decentralized by the day. Yet many organizations believe the key to a successful service oriented architecture – a very decentralized system design – is to have a central service repository.

In this session, Harry Pierson will examine this tension, get you thinking outside the industrial mindset and help you think about software development in a post-industrial way.

I’m very excited about this talk.

MS SOA & Business Process Conference
October 29th – November 2nd, Redmond, WA

I’m not presenting at MSSOABPC (that’s a mouthful) but looks like most of my team is going. So if you’re going and want to hang out with the guys who are doing this stuff in the trenches @ MSIT, let me know. Also, I put out the call for anyone interested in a geek dinner. From the agenda, looks like they’re keeping us busy until 8pm every night Mon-Wed, so we can either a) have geek dinner Thursday or Friday or b) have geek beers after one of the receptions in the early part of the week.

patterns & practices Summit USA West
November 5th – 9th, Redmond, WA

I did the p&p Summit back in 2005, a very successful debut of my Developer 2.0 talk. (I’m doing that talk at a different conference this year, details below.) This year, I’m not 100% sure what I’m going to talk about yet. I’m currently slated to talk about the Rome project that I’m doing in MSIT, but given our current slow progress on that project, I’m probably going to talk about something else. I’m thinking either the “Moving Beyond Industrial Software” talk described above or the “Facing the Fallacies of Distributed Computing” talk described below. Any other suggestions?

DevTeach Vancouver 2007
November 26th – 30th, Vancouver, BC

This is a brand new experience for me. Frankly, I’d never heard of DevTeach before my friend Mario Cardnial suggested I submit a couple of sessions. Since it’s only a few hours drive away, I’m bringing the family along. We’ll see how that goes. And when I’m not doing my sessions or hanging out with the family, I might take in a session or two in the XNA track.

Here are the sessions I’m doing:

Developer 2.0
Finding Your Way in the Future of Software Development
The one constant in software development is change. Software development in 2007 is dramatically different than it was in 2000, which was in turn dramatically different than in 1993. You can be guaranteed that the platforms, languages, and tools will continue to evolve. Learn how Harry Pierson, Architect in Microsoft IT, believes software development is going to evolve in the next five years and what you must do today to remain competitive.

Facing the Fallacies of Distributed Computing
Sun Fellow Peter Deutsch is credited with authoring “The Eight Fallacies of Distributed Computing”. These are near-universal assumptions about distributed systems that “All prove to be false in the long run and all cause big trouble and painful learning experiences.” In this session, we will examine these fallacies in depth and learn how to avoid them on the Windows platform by leveraging Web Services, WCF and SQL Service Broker.

Nick responds to my visceral thoughts on the integration business case. There’s no point in excerpting it – go read the whole thing. I’ll wait.

It looks like for case #2, he added the ability to “change readily and inexpensively”, which is to say he made it overlap even further with #4 than it used to. He also changed #3 to make it clear that he was collecting metrics to give us “awareness of process efficiency”. That makes #3 overlap with #4 on efficiency instead of #1 on BI, but either way it’s still redundant.

So we’re still left with the business cases of Business Intelligence, Efficiency and Agility. Nick conflates Efficiency and Agility both in his original post and his follow-up, but I think it makes sense to separate them. I still stand by my original point that the business is only interested in directly funding Business Intelligence.

Nick is willing to bet a nice lunch that MSFT has invested more in improving operational efficiency that we have on BI in the past four years. He’s probably right, but he missed the point I was making. The business will readily invest in improving a specific process they can measure the ROI on improving. MSFT has lots of processes, I’m sure most of them have significant room for improvement.

But Nick’s list isn’t about specific improvements. He’s explicitly wrote that he’s describing a scenario where “our systems are all optimally integrated”. Selling the business on generally improving efficiency is very different that selling the business on improving the efficiency of a specific process. I’d bet the same nice lunch that the vast majority – if not all – of integration infrastructure running at MSFT was originally deployed as part of a specific business scenario that needed to be solved.

My point here is most businesses are better at funding projects to meet specific business needs than it is at funding pure infrastructure projects.

As for agility. Martin Fowler pointed out once that adding flexibility means adding complexity. But chances are, you’ll be wrong about the flexibility you think you’ll need. So you actually end up with the additional complexity but none of the flexibility benefit. Martin recommends “since most of the time we get it wrong, just don’t put the flexibility in there”. Instead, you should strive for simplicity, since simpler systems are easier to understand and thus easier to change.

Does the same philosophy apply to process? I think so, though there is one thing I’d be willing to risk being wrong on. We all expect the steps in a process to change over time, so moving to a declarative model for process definition sounds like a good idea. Luckily, there’s existing platform infrastructure that helps you out here. But beyond that, I can’t think of a flexibility requirement that I’m so sure of that I’m willing to take on the additional complexity.

Again, I’m not saying efficiency or agility (or integration for that matter) are bad things. I’m saying they’re a tough sell to the business in the absence of specific scenarios. Selling the business on automating the ordering processing is feasible. Selling the business on building out integration infrastructure because some future project will leverage it is much tougher. If you can sell them on it, either because the company is particularly forward thinking or because you can sell ice to Eskimos, then more power to you. But for the rest of us, better to focus on specific scenarios that the business will value and keep the integration details under wraps.

I’ve been investigating the new WCF/WF integration in .NET Framework 3.5. I want to understand how the new context features work. Unfortunately, there’s not much info out there (that I could find at any rate). You’re pretty much stuck with the samples and Jesus Rodriguez’s overview of durable services. So I sat down to dig a little deeper.

Note, since there are no docs on this stuff as I write this, many of the links below are Reflector code links.

I started by lifting the DurableCalculator sample contract and service implementation and dumping it into a new WCF Service Library project. I did this for two reasons. First, VS08 has added a WCF Service Host much like VS05 added the ASP.NET Development Server. Very cool. But the existing sample is still written to be hosted in IIS, so I wanted to change that. Second, and much more important, I wanted to start with a vanilla config file. I knew it wouldn’t work out of the box, but the point of this exercise was to learn how this works under the covers.

When you fire up the durable service with the vanilla config file, you get an error (as expected). Services marked with the DurableServiceAttribute require a binding that supports the context protocol. WsHttpBinding, the default binding when you create a new service, doesn’t. However, it’s easy to fix by switching to wsHttpContextBinding instead. Via Reflector, we see that wsHttpContextBinding inherits from wsHttpBinding and adds a ContextBindingElement to the binding element collection created by the base class. BasicHttpContextBinding and netTcpContextBinding work the same way.

Even after changing to wsHttpContextBinding, we’re still getting an error on service start. But it’s a new error, so we’re making progress. Now, we’re told that services marked with DurableServiceAttribute need a persistence provider to be specified. If we look in the original sample’s web.config file, we find a persistenceProvider element in the service behavior. This element references the SqlPersistenceProviderFactory type. Obviously, the point here is to persist durable service instances to the database between calls, much as WF can do.

However, merely configuring the existing SQL persistence provider doesn’t really tell you what’s going on under the hood. Besides, often when you’re experimenting, you don’t really want to go thru the headache of setting up a SQL store for persisting instances to. Somewhere along the line, I implemented a fake persistence service for WF that stored the serialized instances in memory. So I decided to do the same for WCF durable services.

Building a WCF Persistence Provider requires building two classes: a factory and the provider itself. Factories inherit from PersistenceProviderFactory, which exposes only one non-CommunicationObject method: CreateProvider. It appears that the service host creates a single persistence provider factory and calls CreateProvider whenever it needs a persistence provider. Providers themselves inherit from PersistenceProvider, which exposes methods to Load, Save and Delete durable service instances.

My FakePersistenceProvider (and factory) are brain dead simple, though the ratio of “real” code to factory and CommunicationObject scaffolding is quite low (about 40 lines out of 258). The factory keeps a dictionary of serialized service instances, keyed by guid. When providers are created, this key guid is passed as a parameter to CreateProvider. The provider instances delegate Load, Save and Delete back to internal methods on the factory class. The methods themselves use the NetDataContractSerializer to serialize the service instance out to a byte array and deserializing it back again. I chose NetDataContractSerializer because that’s what the SQL persistence provider uses under the hood.

PersistenceProvider supports async versions of Load, Save and Delete but I didn’t implement them. Also, there’s a LockingPersistenceProvider abstract class which adds (you guessed it) instance locking semantics. However, my fake provider doesn’t span machines, much less require locking semantics so I skipped it.

So it looks like DurableServiceAttribute, context-supporting bindings and persistence providers are all inter-related. Certainly, you can’t use the attribute with out the binding and persistence provider. As I continue to dig, I’m want to see how context inter-relates with WF as well as it’s possible usage outside of DurableServiceAttribute based scenarios.

If you’re interested in the code, I’ve stuck it up on my SkyDrive. In addition to the FakePersistenceProvider implementation and the simple Durable Calculator service, it includes a simple client to test the service and persistence provider. The WCF Service Host includes a test client, but it doesn’t appear to support the context protocol, so I had to build a simple test app instead. Enjoy.