Microsoft announced the availability of a contract checking library that will be available in .NET 4.0.  It is a spin-off of Spec#, but is a .NET Framework library, which opens the functionality to all .NET languages.  You can use constraints to define a contract of the inputs, outputs, invariants, and more. There is also a Visual Studio component that allows you to configure static and/or runtime checking.

A related tool, PEX, automatically generates unit tests.  It analyzes your code and creates tests for all the scenarios it can find.  The tests that PEX produces result in 100% code coverage.  PEX cooperates with the constraints from the contract checking library by generating tests for those constraints also.

Instrumenting applications with logging is an incredibly good habit to have.  Today, you can debug on your desktop and a lot of developers think this is sufficient.  However, you need to think about how you will solve problems once your application is deployed.  Most of the time, a customer can’t remember what they did when a certain problem happens; all they know is that your program doesn’t do what they expected it to.  Errors manifest themselves in error messages, crashes, and/or (even worse) incorrect logic.  When the customer sees it,  but you don’t, it can be next to impossible to find or reproduce the bug.  Often, your only saving grace is a log file.

The need for logging becomes more critical with WinAZ.  You will have a deployed service, but you don’t know which or how many machines are processing that service.  This means that you must design for this scenario,  but that’s another topic; what I want to discuss here is the environment that your service is operating in and how it affects your ability to solve problems. Essentially, there isn’t a way to know about a problem in your service without logging.

One of the talks I attended today was about logging and debugging in WinAZ by Steve Garrity and Mark Gilbert; an excellent presentation, based upon their real-world experience.  A couple interesting take-aways (of many) was the need to make logs replayable. This allows you to see what the problem was so that you can reproduce and fix it – otherwise, how do you know (as I said earlier, the user often doesn’t remember what they did).  Besides, if you can see problems occurring through log analysis, it is much easier to detect and fix the problem to minimize its impact. Another interesting principle was to include correlation IDs in your log. Remember, you don’t know what machine is running your service, and you need a way to match an error message with a log entry or just keep track of log entries if you merge logs from multiple machines. To help with this, WinAZ includes a high-performance logging system with an API you can tap into.

Parallel Extensions for the .NET framework includes Parallel LINQ (PLINQ), the Task Parallel Library (TPL), and Concurrent Data Structures.  The parallel extensions have been available for around a year as CTP’s and will be included with VS2010.  Essentially, parallel extensions make it easier to write code that targets multi-core architecture.  PLINQ allows us to parallelize a LINQ query.  It takes advantage of the declarative nature of LINQ by allowing you to code the query as you always have, but append AsParallel to the data source.  The TPL abstracts the concept of a unit of work and encapsulates plumbing functionality that we must implement for threading today; instead of Thread – think Task.  Some of the more remarkable services of a Task includes a built-in facility to safely cancel thread execution – assuming your thread logic cooperates. Another important feature of these API’s is that they are smart enough to know how many processors you have on your system and perform the load balancing automatically, essentially keeping processors busy without overloading memory or incurring unnecessary overhead.  The concurrent data structures provide a reliable set of collections and associated utilities to synchronize access to data.  These collection classes are optimized by avoiding course-grained approaches that would serialize access to the entire collection, but instead take a fine-grained approach that minimizes the amount of overhead required to safely access data.

When should you begin using it?  Some say “Now!”, but think about it for a minute; this is still beta software, so you have to think twice before deploying it.  Perhaps a pilot project with tolerance for beta software would be more appropriate.  Of course, it depends on your organization, but you should consider implementing parallelism into your applications as early as possible because processor’s aren’t getting any faster and the only way you will achieve greater performance going forward is to scale out to multi-core/multi-processing systems.

Enterprise application integration (EAI) is expensive.  You need to build interfaces between heterogenous computing systems.  Some of the considerations include when each system is avalable, communications channel, and API.  Sometimes systems aren’t available 24x7, so you need a way to hold communications until that time.  We have message queuing, but that adds another component to the interface.  There are various communications technologies available, which depends on the specific systems that need to exchange information.  Most of the time, you can find TCP/IP as a least common denominator, but there are many other methods of communication to work with, including Java RMI, Corba, and some form of web service.  API’s and protocols vary, depending on communication channel.  If you’re using TCP/IP, you often need to define your own protocol.  Here, I mentioned only a few variables and a few of the considerations that go into builing enterprise integration solutions, but there are many features and options to select, resulting in an exponential explosion in large corporations who have many years of various technologies supporting their infrastructure.  It isn’t an easy problem to solve, but the industry is evolving to standards-based solutions that could help relieve these problems.

The WinAZ solution for EAI is the Service Bus.  The WinAZ service bus is much the same as service bus solutions being used in the enterprise today.  One of the primary differences is that the WinAZ service bus is in the cloud.  It is also based on the standard WS* web services stack; embraced accross the industry by titans such as IBM, Sun, HP and more.  The WinAZ service bus is open, meaning that you can access it via any technology that can post to an HTTP URL.  Imagine using Perl or Ruby to communicate with another cloud application written in .NET.  Of course, there is a WinAZ API that any .NET developer can code to.  The demos I’ve seen use C#, but the WinAZ service bus API is accessible via any managed .NET language that targets the .NET CLR.  In addition to an open API, the WinAZ service bus can integrate with existing enterprise service bus solutions.  Therefore, you can build upon an existing service bus solution and tie into the WinAZ service bus as needs arise.  Microsoft has made it easy for anyone, regardless of platform, to opt into and integrate with WinAZ.

Microsoft announced several new technologies this morning.  In my mind, none as far-reaching as WinAZ, but important.  For example, there are new controls for both WPF and silverlight.  I was particularly impressed by the new Silverlight chart control, which will ship with source code. 

Another part of the WinAZ stack is Live Services.  Essentially, Live Services will assist in the integration of Mobile, PC, and Web for a single experience accross devices. The enabling technology of this experience is Live Mesh, further enabled by a more basic technology of synchronization services.  What this brings to the development and user experiences is the ability to make updates on any device (PC, Web, or Mobile) at any time and be able to share those changes on any other device.  In a perfect world, this will happen real-time, but the reality is that we must acknowledge the nature of the Web as an inherently imperfect medium of availability; something no tool or software can overcome.  This reality leads to the concept of occasionally connected applications where your application can work off-line as well as on-line.  Getting back to the point, synchronization services enables occasionally connected applications, ensuring we don't loose the benefits of Live Mesh, allowing realization of Live Services.

One of the common problems in the enterprise space is identity managment.  Many organizations already have Active Directory, but they often have other systems that require their own username and password.  Either the company has purchased 3rd party applications with their own identity systems or they are building custom solutions, each with their own login facilities.  For ASP.NET apps, there isn’t a problem if the application is inward facing (because they can configure Windows authentication), but if the same system must be used by company staff and customers together, then you end up with another login system because, for one of many reasons, customers can’t use the companies internal identity management system.  It’s frustrating for customers who need to remember multiple logins for every system they must work with, which is where many companies find themselves because they either can’t afford to budget for building a custom solution on their own or a combination of other reasons.  Building single sign-on systems to bring together the identity management of multiple systems is an expensive undertaking.  Alternately, there are 3rd party applications that facilitate single sign-on; some solutions are proprietary. Fortunately, communications technologies are not difficult to build these days – especially Web Services, but it’s still a hard problem to solve and you have the continuous need to fund new work for each system brought on-line that needs identity managment services. 

WinAZ has built-in identity management facilities for helping to relieve this problem.  Essentially, anyone can plug in their own identity managment system, log in, and the application will recognize this.  The development experience is streamlined, with API’s that help perform the identity system integration. In one of the presentations I saw yesterday, MS showed just that – an application that allowed customers from multiple companies, using their own identity management systems to access a single application. Of course, Active Directory is supported, but it looks like MS has introduced a solution to a hard problem that will save organizations time, money, and yield less frustration for customers and enterprise users.

In Ander’s Heilsberg introduced new features of C#, which includes dynamic types, optional parameters, enhanced interop, and co/contra variant support:

Dynamic Types

A dynamic type allows you to interact with dynamic languages, such as javascript, ruby, and python.  Essentially, the fixup to type members is performed at run-time, rather than compile-time.  One (of many) use case for this feature is in Silverlight, where you’ll want the ability to call into javascript with C#.  Actually, instead of javascript, you’ll be able to use C# code.

Optional Parameterrs

You’ll now be able to specify default values for method parameters, rather than declare multiple overloads that call into a single overload that includes all of the parameters. Besides making it easier to build reusable types, this feature helps a lot with Office application development.  Count how many times you’ve used Type.Missing for optional Excel or Word API examples and perhaps you’ll see this as an incredibly useful feature too.

Better COM-Interop Support

In addition to optional parameters, there is also greater COM Interop support.  I.e. dynamic typing means that we don’t have to perform multiple conversions to index Excel cells and optional parameters allow us to use the Office (And other COM) APIs in a more natural way.  Additionally, there will be a new assembly option that grabs PIA types and includes them in your assembly so that you won’t need to include the PIA in your distribution.

Co-Variance/Contra-Variance Support

The developer experience for passing arrays and generic types to and from methods is now natural and sensible.  You can also specify in and out on Generic type parameters, indicating that they can be passed as input parameters or return values, respectively.

The C# Compiler Service 

Finally, to top off the presentation, Anders introduced some future thinking on what they will be doing with the C# Compiler. This follows the theme about C# supporting more dynamic language principles, especially in the area of meta-programming, similiar to features of Ruby.  Essentially, you can pass code to the C# compiler, which is a service you can call, for evaluation and execution.  This one wasn’t necessarily a C# 4.0 feature, but an early look at some very cool stuff we might have some time in the future.

#PDC2008

During the PDC Keynote, this morning, Ray Ozzie announced a new platform, Windows Azure (WinAZ). What’s interesting about WinAZ is that it will be a cloud-based operating system.  Microsoft has entlisted the experience of Kernel experts and other experience they’ve nurtured within the company for years to create a platform with all of the essential infrastructure that developers have come to expect from an operating system.  Being in the cloud, WinAZ considers additional features necessary for its environment: including federated security, virualization, scalability, availability and more.  What we should see is a new platform that acknowledges the world we compute in and where  we need to be in the future.  Existing infrastructure is fraught with security hazards, availability concerns, and complexity.  With WinAZ we will have a modern platform to help with these modern and real problems that we deal with every day.

Besides being a solution to a growing technical problem, WinAZ has a business case.  Think about all of the time you spend managing infrastructure, the costs of that infrastructure, and planning for future growth and scalability.  It is expensive to pay IT pros who vary in skill set.  WinAZ minimizes this problem by being hosted on a massive set of data centers throughout the world.  You’ll also be able to reduce infrastructure because your platform is managed by WinAZ. The cost benefits for small and growing companies are realized by the virtualization capabilities and inherent scalability built into WinAZ.  The potential ease of management, reliability, and growth potential makes WinAZ a compelling platform.

It was a relief to hear that our existing developer skill sets translate directly into WinAZ technology.  Amazingly, it will support both Managed and Unmanaged code development.  The managed code development support will come from a version of the .NET platform with an extensive set of WinAZ services support.  In addition to the .NET support, Microsoft is porting it’s own enterprise technology to prove the WinAZ platform. You’ll see Live Services (no surprise there), Sharepoint, SQL Services, Exchange, and CRM.  This is just the beginning of the supporting services for WinAZ, but they (along with .NET) appear to be the primary base services of the platform.

Following the Software + Services model, WinAZ will migrate new technology back to the enterprise so that customers that choose not to operate in the cloud will have many of the same capabilities customers on WinAZ.  I have customers that operate off and on the cloud, by thier choice, and I think this is a real-world approach.

Note: Don't click anything other than the *.edmx between then and the time you open the property window. This can be confusing because you'll often have the entity designer open, which has its own property window, which is different from the property window that you see when the *.edmx in the Solution Explorer is selected. Try it, look at the property windows for when the designer is selected and when the *.edmx is selected.

I like the way Visual Studio has evolved over time in the area of being able to install pre-requisites with setup projects.  In VS2008, there is support for pre-requisites where you can right-click on the setup project, select Properties, and then click the Prerequisites button. Many of the items you’ll need, such as .NET Framework versions are available.  There’s even a check box for v3.5 SP1. The thing is that new versions of software are continuously released, but don’ t necessarily find their way into the VS2008 Prerequisites list.  Fortunately, this list is configurable, leaving you with options to customize existing Prerequisites and add your own.

Recently, I built a setup project that needed to include SQL Server 2008 Express. Looking at the Prerequisite list, there was an option for SQL Server 2005 Express SP2, but not the 2008 version.  Rather than construct my own prerequisite package file, I decided to launch a process in a custom installer that I already had working for other items I needed in the setup package.  I found an example of command-line options on Microsoft’s site and gave that a shot.  The only problem is that the example was for SQL 2005, which won’t work because the options changed in SQL 2008. I eventually found the right article at http://msdn.microsoft.com/en-us/library/ms144259.aspx.  This article was okay, except that I used the SQL 2008, rather than the express version of the command-line options. After realizing something wasn’t quite right, I read the next section of the article that explained the Express options, which were different.  BTW, there’s a doc error for the features option: change /FEATURES=SQL_Tools to /FEATURES=SQL,Tools – with the comma instead of underline.

After I got that working, the package still wouldn’t install because PowerShell is a prerequisite when you use the Tools option. After getting PowerShell working, I encountered another error because Windows Installer 4.5 was a pre-requisite.  Of course, I needed .NET 3.5 SP1 as a pre-requisite to my application, so I was building this custom installer with a long sequence of pre-requisites, growing more complex at each step.  Add-in re-starts and bugs that I produce myself and the fun increases.

I had reached a point where I realized that there has got to be a better way to do this. So, I looked at VS2008 setup project prerequisites again.  Along the way, I found this cool open-source tool called the Boostrapper Manifest Generator.  It’s moved a couple times to different sites, but today you can find it here, in the MSDN Code Gallery: http://code.msdn.microsoft.com/bmg. It isn’t polished, but it does come with a help file that’s coherent enough to move you in the right direction.  I got it working for Windows Installer 4.5 in a short amount of time.  Additionally, there often nothing better than a good example, so I looked at how existing packages are set up here: C:\Program Files\Microsoft SDKs\Windows\v6.0A\Bootstrapper\Packages, which is where you put your  package, once complete, so VS2008 can find and list it in the Prerequisites window.

Because my package will install behind a firewall to machines that don’t have Internet access, I needed an off-line solution. Problem is that the .NET 3.5 SP1 prerequisite package only deploys on-line.  There’s an explanation of how to get an off-line install at http://download.microsoft.com/download/A/2/8/A2807F78-C861-4B66-9B31-9205C3F22252/VS2008SP1Readme.htm#General%20Issues, but people have had problems with the instructions here in the past.  I think the article has been updated and should work.  One of the more useful bits of information is from this forum thread: http://social.msdn.microsoft.com/forums/en-US/vssetup/thread/d3c1991a-9353-43f0-be82-2dacedc1ceba/.

After that, I turned my attention to installing SQL Server 2008 Express again, this time via Pre-Requisites.  I was considering using the Bootstrapper Manifest Generator, but this was a little more complex than running it on a Hotfix, like Windows Installer 4.5. I found a work-around on this blog entry: http://blogs.msdn.com/sqlexpress/archive/2008/09/12/faq-installing-the-sql-server-2008-express-clickonce-bootstrapper-for-visual-studio-2008-sp1.aspx.  In short, the pre-requisite packages install with the VS 2008 Express SP1 downloads, but not the VS2008 Pro SKU.  So, you can install a VS2008 Express SP1 and get the prerequisite packages.  Only problem for me is that I already had SQL Express 2008 and SQL Server 2008 Developer Edition installed on my machine, meaning that the VS2008 Express products didn’t install the pre-requisite packages.  I worked around this by spinning up a Virtual Machine, (I keep a base OS around that I can copy any time for testing) and installing on that clean system, and then copying the new pre-requisite packages to my main system.  These pre-requisites also installed on-line, so I looked into Product.xml for Windows Installer, found the name of the file I needed and included the executable in the same folder as Product.xml.  I had already created a pre-requisite for Windows Installer 4.5, but figured the one from MS would be more reliable.  Problem was that the MS Product.xml contained options for multiple OS and CPU architectures, that I wasn’t interested in.  Therefore, I removed those entries from Product.xml and the package worked okay.  The experience for SQL 2008 Express was a little different because Product.xml only pointed to a checking application that determined if a prior version exists.  Instead, I had to go into the culture folder (en in my case), and alter the Package.xml file.  I also needed to add the executable for the SQL 2008 Express setup to the same folder as Package.xml, en, (not the Product.xml folder). Along the way, I encountered warnings on the prerequisites when opening them from the setup project in VS2008.  They turned out to be errors in the Product.xml and Package.xml files.  The warnings went away when I fixed the Product.xml and Package.xml typos I had introduced.

Anyway, I now have a very nice setup package with a simplified custom installer and reliable handling of prerequisites.  Hopefully, if you encounter the same problem, this might save you from all of the research and time that I encountered.