Martijn's blog - E-Commerce, EAI, BizTalk and .NET

Moving my blog to MSDN

2007-01-08T11:24:00.000+01:00

January 1st, 2007, I started working for Microsoft and will be moving to http://blogs.msdn.com/martijnh/. Hope to see you there!

Outlook: Recovering e-mail messages from attachments

2006-12-26T15:41:00.000+01:00

Recently, I was what I thought to be dragging e-mail from my Inbox into my Personal Folders. Outlook warned me that the process could take some time due to the amount of messages selected, but I went ahead with the move. After a while, Outlook popped up a freshly created e-mail message containing my 1000+ e-mail messages!

The new e-mail was about 85MB and the original message were gone from my inbox... I really wanted those e-mails in my Personal Folders. However, Outlook programming is something I've always shied away from. After some experimenting, I discovered what I wanted couldn't be easily done using Outlook APIs...

I wanted to:

Open the e-mail message containing the attached 1000+ 'backups'
Iterate the attachments
Save the attachments as e-mails into my Personal Folders box

After a lot of headache, I discovered Redemption (see http://www.dimastr.com/redemption/). Redemption provides interop between .NET and the Extended MAPI. Using redemption, I finally succeeded in achieving my goal. Just in case someone else outthere is as silly as I am, here's how I did it:

// To use this namespace you must set a
// reference to the Microsoft Outlook 11.0
// COM server.
using Microsoft.Office.Interop.Outlook;
using OutLookApp = Microsoft.Office.Interop.Outlook.Application;
using System.IO;
using rd = Redemption;

ApplicationClass outLookApp = new ApplicationClass();
NameSpace outlookNS = outlookApp.GetNamespace("MAPI");

MailItem attachmentsItem = (MailItem)outlookNS.GetItemFromID(messageId, null);

// left out for clarity: find the Personal Folders->Inbox folder

rd.MAPIUtils utils = new rd.MAPIUtils();
utils.MAPIOBJECT = outlookNS.MAPIOBJECT;

foreach (Attachment a in attachmentsItem.Attachments)
{
string attachmentFile = Path.GetTempFileName();
a.SaveAsFile(attachmentFile);

MailItem underlyingItem =
  (MailItem)personalFolder.Items.Add(OlItemType.olMailItem);

rd.MessageItem neww =
  utils.GetItemFromMsgFile(attachmentFile, false);

underlyingItem.Save();

MailItem placedItem =
  (MailItem)underlyingItem.Move(personalFolder);

neww.CopyTo(placedItem);
}

This code misses all kinds of nice sanity checks, like whether the attachment actually is a MSG file, etc. It sufficed for my scenario and I now have the e-mails back. FYI :-D

WPF - themes in SDK

2006-12-24T12:57:00.000+01:00

I'm probably the last one to notice, but just in case I'm not... The themes included in PresentationFramework are included in the SDK (zipped in WPFSamples.zip) under

WPFSamples\Core\AeroTheme
WPFSamples\Core\ClassicTheme
WPFSamples\Core\LunaTheme
WPFSamples\Core\RoyaleTheme

:-D Definitely inspirational stuff!

WPF - Writing an IMultiValueConverter (update)

2006-11-03T19:15:00.000+01:00

What I didn't realize in the previous blog entry regarding the subject was the fact that binding to a DependencyProperty and performing a string.Format blindly could lead to weird results, e.g.:

<TextBlock>
<TextBlock.Text>
  <MultiBinding
        Converter="{StaticResource StringFormatterConverter}"
        ConverterParameter="{}{0}, {1}">
   <Binding Path="Customer.Lastname" />
   <Binding Path="Customer.Firstname" />
  </MultiBinding>
</TextBlock.Text>
</TextBlock>

Could convert to "DependencyProperty.UnsetValue, DependencyProperty.UnsetValue" The solution is pretty simple:

bool argumentsAreNullOrEmpty = true;

// Check whether all values are null or unset.
foreach (object value in values)
{
if (value != null && value != DependencyProperty.UnsetValue)
{
argumentsAreNullOrEmpty = false;
break;
}
}

// If we have all empty or null arguments, we do nothing.
if (argumentsAreNullOrEmpty)
{
return Binding.DoNothing;
}

Happy WPFing!

CAB: load your modules from your App.Config

2006-11-02T17:18:00.000+01:00

In CAB, there's a 'special voodoo magic' file which defines which modules are to be loaded in the IoC process. The file is called ProfileCatalog.xml by default.

We're currently using CAB in combination with an XBAP application. This leads to deployment issues, as ProfileCatalog.xml will be deployed in to a folder called Data by default and the modules configured in the file cannot be located. You can work around this issue, but I took the issue as an opportunity to replace the default implementation of IModuleEnumerator in CAB with my own.

My implementation looks at the App.Config file for it's modules. All that's needed to use the new enumerator is to configure it:

<configSections>
<section
  name="SolutionProfile"
  type="BankTellerModule.Services.ModulesSection, BankTellerModule" />
<section
  name="CompositeUI"
  type="Microsoft.Practices.CompositeUI.Configuration.SettingsSection, Microsoft.Practices.CompositeUI" />
</configSections>

<CompositeUI>
<services>
  <add
   serviceType="Microsoft.Practices.CompositeUI.Services.IModuleEnumerator, Microsoft.Practices.CompositeUI"    instanceType="BankTellerModule.Services.ConfigurationModuleEnumerator, BankTellerModule" />
</services>
</CompositeUI>

<SolutionProfile>
<Modules>
  <add assemblyFile="BankTellerModule.dll" />
</Modules>
</SolutionProfile>

The code for this behaviour will be available here in a few hours...

ClickOnce deployment of XBAP application to remote IIS

2006-11-02T15:48:00.000+01:00

To ensure you don't encounter the issues we encountered, make sure you set the option "Rename all application files using the .deploy file extension".
You can find the option under Project properties -> Publish -> Options.

If you don't use this option and your IIS is hardened, as it should be, you'll see your assemblies being rejected (404.2 return code in you W3 log files). The .2 subcode means the server is not allowed to serve .dll files.

The deployment renames the files from e.g. BankTellerModule.dll -> BankTellerModule.dll.deploy. This ensures the files get served by IIS. Locally, the files are stored in C:\Documents and Settings\<user account>\local Settings\Apps\2.0 and renamed back to .dll.

ClickOnce publishing XBAP applications and F5 debugging not working

2006-11-02T15:28:00.000+01:00

Some times, when you publish an XBAP application using ClickOnce, you cannot debug the same application afterwards using F5. This is due to a change to the project settings (debug tab). The extension is removed from the command-line argument to PresentationHost.exe (-debug .xbap). Insert the .xbap extension and things work alright again.

Supporting WPF XML Browser Applications (XBAP) in a CompositeUI Application Block application

2006-10-29T17:18:00.000+01:00

Currently, we’re building a real application in WPF for a local government. The application uses the MVC pattern quite heavily, so I did a test to see whether we could use the Composite UI Application Block (CAB, read more about it’s architecture and use, download it from MSDN, play with the labs to understand it) to enable easier composability in order to distribute our use cases more easily throughout the development team, decouple logic from UI even more, etc.

It turns out the CAB currently doesn’t support WPF. After some searching, I discovered Kent Boogaart from Australia (thanks Dion!) is busy writing a support layer for WPF in CAB. So far, so good. Testing the download he provided, however, revealed the implementation doesn’t support so-called XBAP applications.

XBAP application are browser hosted WPF applications and are different in some aspects to ‘traditional’ WPF applications. For instance, the Application object which is the root for all WPF applications, doesn’t allow you to call the Run method if the WPF application is running within a browser. This is because of the asynchronous nature of the webbrowser.

After a few simple modifications to Kent’s code and a little more trickery, I had the BankTeller application working in XBAP. Well, sort of… The thing with CAB is that it assumes synchronous applications. An example of this (taken straight from Microsoft.Practices.CompositeUI.CabApplication):

public void Run()
{
RegisterUnhandledExceptionHandler();
Builder builder = CreateBuilder();
AddBuilderStrategies(builder);
CreateRootWorkItem(builder);

...

rootWorkItem.FinishInitialization();
rootWorkItem.Run();
Start();

// Whoops for non-blocking Start()s
rootWorkItem.Dispose();
if (visualizer != null)
visualizer.Dispose();
}

As I explained earlier, XBAP applications take offense to calling the Run method on their Application, they’re asynchronous. In order to make XBAP applications work, a few modifications have to be made to CAB itself.

Don’t call Run on the Application class.
Ensure the correct UI element get’s displayed.
Remove the dispose calling from the main Run() method of Microsoft.Practices.CompositeUI.CabApplication.
Implement a correct IDisposable pattern for cleaning up resources once the Application is actually done with them.
Ensure we call Dispose to clean up used resources.

For those just interested in the endresult, a modified WPF port capable of supporting CAB based applications in XBAP, you can download the example.

If you’re interested in more detail about the changes I made and how I did this, please read the whitepaper I wrote, which can be downloaded here.

A special thanks to Kent Boogaart for creating the WPF port for CAB, without which I wouldn’t have been able to put CAB to use in my WPF scenario!

WPF - Using CompositeCollection to combine multiple sources for databinding

2006-10-17T20:59:00.000+02:00

As I said before, I'll blog about things I learn and find interesting... ;-) Today, I realized there's a very easy way to combine different sources for a binding. It's done using what's called a CompositeCollection. Here's how it works:

Create a datasource (XmlDataProvider, ObjectDataProvider, ...)
Create a CollectionViewSource to look at the data in a specific way
Bind a control's Source/ItemsSource/... property to the CollectionViewSource
Add a view 'static' items
Bind to another view at the same or different data

An example:

<XmlDataProvider x:Key="MyData" XPath="/Info">
  <x:XData>
    <Info xmlns="">
      <Item ID="12345" Name="Book 1" Price="$10.00" />
      <Item ID="24678" Name="Book 3" Price="$9.00" />
    </Info>
  </x:XData>
</XmlDataProvider>

<CollectionViewSource x:Key='a' Source="{Binding Source={StaticResource MyData}, XPath=Item/@Price}" />
<CollectionViewSource x:Key='b' Source="{Binding Source={StaticResource MyData}, XPath=Item/@Name}" />

We now have the two CollectionViewSource instances up and running. For example purposes, they're looking at the same datasource, but this could be anything else. Now, let's create a simple ListBox control which binds to both these CollectionViewSource instances and adds constant items..

<ListBox>
  <ListBox.ItemsSource>
    <Binding>
      <Binding.Source>
        <CompositeCollection>
          <ListBoxItem>My Constant Item</ListBoxItem>
          <CollectionContainer Collection="{Binding Source={StaticResource a}}" />
          <CollectionContainer Collection="{Binding Source={StaticResource b}}" />
        </CompositeCollection>
      </Binding.Source>
    </Binding>
  </ListBox.ItemsSource>
</ListBox>

It's as easy as that. The output would be:

My Constant Item
$10.00
$9.00
Book 1
Book 3

Just like the MergedDictionaries within the ResourceDictionary, you can easily combine different sources of data and bind to those sources, even TwoWay.

WPF - Writing an IMultiValueConverter

2006-10-15T11:45:00.000+02:00

Currently, I'm working on a project involving WPF. I will post my experiences from time to time. This week, I wrote a simple IMultiValueConverter implementation which can be used to format a set of bindings in a String.Format fashion. This is how it works:

Define the class and assign a key to refer to it:

<Window.Resources>
<c:StringFormatterConverter
x:Key="StringFormatterConverter" />
</Window.Resources>

Bind some element (GridViewColumn in this case) to a datasource, in this case something which has properties called LASTNAME and FIRSTNAME on it:

<GridViewColumn.DisplayMemberBinding>
<MultiBinding>
<Binding Path="LASTNAME" />
<Binding Path="FIRSTNAME" />
</MultiBinding>
</GridViewColumn.DisplayMemberBinding>

Now set a converter on the MultiBinding:

<MultiBinding Converter="{StaticResource StringFormatterConverter}" ConverterParameter="{}{0}, {1}">

Both IValueConverter and IMultiValueConverter implement two methods:

Convert
ConvertBack

One for binding one direction and the other for the opposite direction (from->to).

The ConverterParameter in the Converter is used to pass the format we'll be using to display the bound elements.
Let's look at a code sample (sanity checks are removed for readability purposes):

public class StringFormatterConverter : IMultiValueConverter
{
public object Convert(
  object[] values,
  Type targetType,
  object parameter,
  System.Globalization.CultureInfo culture)
{
  return string.Format(
   culture,
   formatArgument,
   values);
}

public object[] ConvertBack(
  object value,
  Type[] targetTypes,
  object parameter,
  System.Globalization.CultureInfo culture)
{
  throw new NotImplementedException(
   "Unable to convert a formatted argument string back.");
}
}

It's that simple, WPF will pass the values to the converter and expects a converted value back. We format the values using the other supplied arguments, parameter ('{0}, {1}') and culture.

You can use any string.Format argument you'd like; just remember to escape the first '{' sign, so the Xaml interpreter knows you don't mean to use a Markup Extension.

Escaping the first brace is done by placing an empty mark extension in front of
the character ('{}{').

The namespace prefix 'c' refers to the assembly and namespace implementing the converter (e.g. <xmlns:c="clr-namespace:Martijn.Wpf.Converters" />

Clustering under Virtual Server - Don't forget to update the SID!

2006-09-30T19:02:00.000+02:00

When using Virtual Machines, I often copy an existing image to duplicate it. However, currently I'm clustering my BizTalk Server 2006 environment for high availability using Virtual Server 2005.

After setting up the cluster, adding the secondary node didn't work, it kept saying 'Access Denied'. Normally, things like changing the LanManServer parameters or LMCompatibilityLevel fix this, but in this case, no dice..

Until I remembered something trivial when using cloned images, the computer SID is the same in both images! Using the NewSID tool, I updated the SID, rejoined the domain and added the secondary node to the cluster without trouble.

Using my blog as a post-it note ;-)

Renaming your BizTalk Server and Sql Server Agent jobs

2006-09-30T12:50:00.000+02:00

After renaming a BizTalk Server host, you need to configure the server using the BizTalk Server Configuration Wizard (in 2006). However, during the reconfiguration, you might encounter a state I did. In my case, a dialog came up stating

'Failed to create Management database "BizTalkMgmtDb" on server "...". Cannot add, update, or delete a job (or its steps or schedules) that originated from an MSX server.

SQL script file: "C:\Program Files\Microsoft BizTalk Server 2006\Schema\\btf_message_logic.sql"'

This failure is caused by the Sql Server Agent still having the jobs configured by the old server. You cannot simply delete/update those jobs as Sql Server doesn't allow you to.

A remedy to this situation is stated in this KB article.

However, this involves renaming the server back to the original name, rebooting, etc. A simpler solution is to update the msdb..sysjobs table and set the originating_server field to reflect the new name.

After this update, the BizTalk Server Configuration Wizard will complete it's work successfully.

Confusion regarding upgrading an Orchestration in BizTalk Server

2006-09-20T21:06:00.000+02:00

There seems to be some confusion regarding upgrading Orchestrations. Doug Girard has documented this in a blog post here, but recent webcasts indicate to the contrary. I thought I'd save others some time and confirm that Doug's absolutely right, upgrading an Orchestration version is a fact of

deploying and binding the new version
unenlisting the current Orchestration (which doesn't suspend running Orchestration instances, but in fact removes instance subscriptions for the Orchestration)
starting the new version

If you drop in a message between steps 2 and 3, BizTalk will fail with a "No Subscriptions Found" exception in the EventViewer. This means that to ensure your system will be able to pick up messages correctly, step 2 and 3 need to be executed within the scope of a transaction, as Doug suggests. All Orchestration actions for the v1 version will be dealt with appropriately.

Doug Girard mentions a script for performing steps 2 and 3 in a single transaction to avoid dropped messages.

The location of the script within the documentation is this (copy+paste the location, clicking will not work).

You can also find the C# code (it's not really a script) within the contents under Operations -> Managing BizTalk Server -> Deploying and Managing BizTalk Applications -> Updating BizTalk Applications -> Deploying and Starting a New Version of an Orchestration Programmatically.

UPDATE: A colleague of mine, Martin Rienstra, mentioned that in spite of documentation to the contrary, the last two version number elements do matter in side by side and upgrading scenarios. I've tested it and he is right.

BizTalk Server Best Practices Analyzer

2006-08-26T10:52:00.000+02:00

A bit late on noticing the download, but I thought I'd direct your attention to it if you missed it like I did. The Best Practices Analyzer is a tool which can do precisely what it says on the cover. It analyzes a lot of aspects regarding a BizTalk environment to see whether best practices have been implemented. A must have for any release manager, IT pro, and even developers!

The tool can be downloaded here.

Microsoft Business Rules Engine (MS-BRE) - XmlTrackingInterceptor

2006-04-25T07:43:00.000+02:00

A lot of businesses have a need to prove what their software has done in evaluating certain requests from customers, such as loan applications, mortgage application, etc. Using the BRE, there's a few ways to accomplish this goal.

One of them is implementing an interface called the IRuleSetTrackingInterceptor. Implementing this interface consists of writing code for the following methods, then called by the BRE as it's executing a RuleSet:

SetTrackingConfig
Sets the rule set tracking GUID and tracking configuration.
TrackAgendaUpdate
Tracks updates to the rule engine agenda.
TrackConditionEvaluation
Tracks the evaluation of rule conditions.
TrackFactActivity
Tracks changes to working memory.
TrackRuleFiring
Tracks the firing of a rule.
TrackRuleSetEngineAssociation
Tracks the association of a rule set with a rule engine instance.

Once your custom implementation is done, you can associate the class with a RuleSet to be tracked by passing it from code when calling Policy.Execute, e.g.:

Policy executingPolicy = new Policy("My Policy", 1, 4);
executingPolicy.Execute((object[]) facts.ToArray(typeof(object)), interceptor);

After the BRE is done executing the specified Policy, the interceptor class has been called for each task and might contain valueable information as to the execution of the Policy.

The BRE supplies a standard implementation of IRuleSetTrackingInterceptor, called the DebugTrackingInterceptor. This interceptor is also used if you have any "Call Rules" shapes within your Orchestrations. It's the way for the HAT to enable viewing what went on within the business process, BRE wise.

The DebugTrackingInterceptor, however, uses a plain text file format, which can be cumbersome to parse using tools. I've implemented a Xml version, which can be downloaded here. Using it you can track the same information as the DebugTrackingInterceptor does, but in a Xml format.

Tool update: Policy Exporter

2006-04-16T15:13:00.000+02:00

When working with the BRE, the stringent way the Business Rule Composer manages versioning can be a real pain. Undeploying / unpublishing policies and/or unpublishing vocabularies can be very handy. This tool allows you to:

Import multiple exported items at once
Delete multiple policies / vocabularies at once
Export multiple policies / vocabularies at once
Undeploy policies
Unpublish policies / vocabularies

Ofcourse, one should not use this tool in a production environment, as the versioning itself has a valid purpose. The tool is meant to ease development.

I previously published this tool under the name "Policy Exporter". This is an update to that tool and a name change to better reflect it's purpose. It's now called "BRE Policy and Vocabulary manager" (what's in a name ;-) ) and it's current version is 1.1.

The tool itself, including full sourcecode, can be downloaded from this location.

A scenario example

While developing a Policy, you need a Vocabulary item added to the Vocabulary used in the current Policy:

Save the Policy.
Unpublish the Vocabulary using the tool
Press F5 in the Business Rule Composer
Ammend the Vocabulary
Publish the Vocabulary
Press F5 once again
Use the new item within the Policy

Woohoo!

2006-04-04T19:53:00.000+02:00

I've been awarded the Microsoft Most Valuable Professional (MVP) Award again! Thanks to everyone who asked me questions or help this year ;-)

FWD: Retrieving context properties within a map

2006-04-03T11:20:00.000+02:00

OK, first off, I'm not sure if you really want this, but people are asking for it a lot and I ran across a solution, so read this article if you want to retrieve a context property for the current message within the mapper (using a custom functoid).

[offtopic] Windows Media Center 2005 - configuring more then 2 tuners

2006-03-30T23:09:00.000+02:00

Fyi, this Green Button Community article on installing more then 2 tuners in Media Center 2005 works. However, when trying it, I bumped onto the reply the forum moderator gave (Posted 5/2/2005 7:29 PM).

It turns out, Media Center recognizes all tuners automatically, at least in my case. Just starting up the Registry editor and copying the "User Settings" hive from a configured device under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners\{A799A800-A46D-11D0-A18C-00A02401DCD4} to the unconfigured device, altering it's properties (UserDefName, [RecordOrder]) and it worked :-D.

XPathMutatorStream

2006-03-01T21:04:00.000+01:00

It seems this little gem is hidden as almost no one I talk to knows of it's existence.

Within the Microsoft.BizTalk.XPath namespace in the (GAC) assembly Microsoft.BizTalk.XPathReader.dll, a class is contained called XPathReader. Relying on this reader it's possible to match XPath expressions in a streaming fashion. This information is out on the web and discussed on many occasions.

However, within the Microsoft.BizTalk.Streaming namespace in the (GAC) assembly Microsoft.BizTalk.Streaming.dll, another class named XPathMutatorStream is defined. The class does what it's name implies, it allows for a callback to alter the value of specified XPathExpressions (hence the reference to XPathReader first). A basic implementation could look like this:

---

MemoryStream stream = new MemoryStream();

StreamWriter sw = new StreamWriter(stream);

string xmldocInstance = "bla";

sw.WriteLine(xmldocInstance);
sw.Flush();
stream.Position = 0;

XPathCollection queries = new XPathCollection();

queries.Add(new XPathExpression("/*[local-name()='Order']/*[local-name()='OrderLines']/*[local-name()='OrderLine']"));

ValueMutator mutator = new ValueMutator(this.XPathCallBack);

XPathMutatorStream mutatorStream = new XPathMutatorStream(stream, queries, mutator);

XmlTextReader reader = new XmlTextReader(mutatorStream);

while (reader.Read());

***

private void XPathCallBack(
int matchIdx,
XPathExpression matchExpr,
string origVal,
ref string finalVal)
{
MessageBox.Show("hit!");

finalVal = "bli";
}

BizTalk Server 2006 - Pipeline Component Side-By-Side deployment

2006-01-25T23:54:00.000+01:00

In BizTalk Server 2004, it was impossible to perform a side-by-side deployment of pipeline components (if someone knows an approach to implement this without renaming the assembly, please let me know ;-) ). Every aspect of BizTalk Server components was deployed into the GAC, but the pipeline components themselfs.

It turns out, however, the team did a tremendous job in altering this 'feature' to support GAC-ed pipeline components within BizTalk Server 2006. The trick to enable this is to:

Version the pipeline component (sign the assembly)
reference the component by placing it into the "Pipeline Components" folder at development time in order to use it in your pipelines
remove it from this folder after building the pipeline or in deploying your solution to production
GAC the component

The compiled pipeline assembly will reference the correct version and the AppDomain used by BizTalk Server 2006 will search the GAC before resorting to it's private paths, amongst which is the "Pipeline Components" folder.

Using the BRE in content-based routing scenarios

2006-01-21T16:51:00.000+01:00

After writing Schema Editor extensions, I got the idea of creating a Pipeline Component which can be used in content-based routing scenarios. Also, I've attached a 'whitepaper' which explains the sample solution as well as the IExtension interface used to implement the Schema Editor extension. The idea is simple:

A message comes into / flies out of the system through a Pipeline;
The Pipeline contains a component which looks at the schema for the passing message;
The Schema contains an annotation which states the Policy that applies to the Schema;
The appropriate Policy is executed by calling the BRE;
The Policy alters the message;
The message flying by is modified to contain the altered message from the BRE.

After this, any following component or system will deal with the altered message.

One thing I haven't implemented due to the fact I haven't currently got a BizTalk 2004 VPC image is AppDomain caching of the Schema lookup. This caching would greatly improve performance of the Pipeline Component.

The associated whitepaper can be downloaded here. The source code is here.

BizTalk Server 2006 - export Orchestration as image

2006-01-15T22:10:00.000+01:00

The UK SDC BizTalk 2004 Documenter tool, which can be found over here, does an excellent job of generating documentation for BizTalk projects.

However, I often need to take snapshots of my BizTalk Orchestration and I'm currently using BizTalk Server 2006, which the tool doesn't support (yet?). I've written my own 'tool' for this, by stealing, euhm, borrowing, some code from them and throwing some of my own in as well.

It's downloadable here for anyone to use. Sourcecode is included as always.

Have fun using it and let me know if you run into trouble ;-)

Of BizTalk Server Schema Editor Extensions...

2006-01-12T12:00:00.000+01:00

Ever wanted to add your own annotations to your schemas for some purpose like the Flat File extension, property promotion, the HWS extension or the TPM extension does? You can edit the schema by hand, but that wouldn’t be any fun, right? ;-) As an exercise, I’ve created two extensions which can easily be used in Visual Studio and extend the default schema editor, like the flat file extension does.

One of the extensions allows an annotation to be added to the root schema element which can be read somewhere else in order to execute a specific BRE policy belonging to the message. This solution could be used to execute a policy from a pipeline in a content-based routing scenario.

The other extension solves an issue a guy with me on a recent course has in processing COBOL copybooks. He needs to determine whether a field should be treated (and validated) as a date field instead of the normal numeric value. The extension allows for adding a property on any field stating it’s a date field according to the COBOL copybook standards.

Figuring out how and what with schema extensibility can be a daunting task, as documentation is scarce. If you end up developing with these extensions, look into these properties, as there are not used, but very interesting (and hopefully, self-descriptive):

InstanceGenerator
InstanceValidator
SchemaValidator

Also, note the custom tab on the CopyBookExtension. It stems from a method called by the schema editor called GetCustomViews and the tab shows which fields will be interpreted as a date.

Sourcecode for the CopyBookExtension is here, sourcecode for the BRECallPolicyExtension can be found here.

Using the extension is easy:

Compile the extension. A post-build action will copy the output to the schema editor extension folder of BizTalk
Open up a schema in Visual Studio
Select the extension
In case of the CopyBook extension, select a field and note the “CopyBook” group in the PropertyGrid
In case of the BRECallPolicy extension, select the node and note the “BRE” group having a dropdown list of all published policies (latest only) or the error if nothing is found on the local host BizTalkRuleEngineDb ;-)

Let me know what you think!

Default values in maps - better option in BizTalk 2006

2006-01-11T13:03:00.000+01:00

In a course I'm following today we discussed different ways of keeping nodes from dissapearing after a map has been applied. We discovered a new option in 2006 b2, which I'm very font of. Basically, up until now, we had a few ways (maybe even more?) of ensuring output nodes were created correctly:

default value in schema (space) - drawback: not visible in editor, schema specific, where as we might want to leave some nodes out in other mappings.
space in value in map - drawback: not visible in editor.
use some string functoid to fill the value (empty argument for the functoid) - drawback: non-functional functoids in the map, warning if the datatype is not string.
scripting functoid - ability to return a null value, which works with any datatype on the right hand side. drawback: non-functional functoids in the map
map a record (any record) into the elements or attributes - drawback: clutters the map with non-functional and illegal links.

in 2006 b2, there's a 6th possibility:

The value property of the right hand side map now has a dropdown, containing '<empty>', which greatly enhances the functional aspect of the property.

Privacy and BizTalk Server 2006 - the "BizTalk Server Operators" group

2006-01-09T17:09:00.000+01:00

Coming up in BizTalk Server 2006: the "BizTalk Server Operators" group. Users within this group are unable to view privacy relevant details such as context properties for suspended messages and their message body, which "BizTalk Server Administrators" can... Kudos!

BRE PolicyExporter - v1.0 - export arbitrary policies with ease..., import without publish, etc.

2005-12-14T23:17:00.000+01:00

Ever had a Policy which failed to load in the Business Rules Composer because some object didn't exist in the GAC anymore of it's interface was chanced? Ever wanted to just select a bunch of Policies to export or delete all at once? Irritated by the fact you cannot edit a policy within the version you imported it in?

Use PolicyExporter and be on your way. It's a simple utility which allows you to make an arbitrary selection of both Policies and Vocabularies in a RuleStore and export them to whereever. Download the sourcecode here.

It's compatible with both BizTalk Server 2004 and BizTalk Server 2006 beta 2 (convert the project to VS.NET 2005 to compile)

Programmatically creating BizTalk Server Orchestrations

2005-11-24T15:48:00.000+01:00

After some digging, I found a way to programmatically create BizTalk Server Orchestrations through the use of Microsoft.BizTalk.IMS.

I've created a sample generator for your enjoyment here. It's a simple Console application which create an Orchestration receiving a message and executing a Policy using the CallRules shape inside an Atomic Scope.

I personally think this MetaModel opens possibilities for tools like best practices implementors, template processing, etc. Let me know what you think!

Note: the model works in 2006 beta 2 as well.

Schema instance generator for use within Orchestrations

2005-10-27T21:20:00.000+02:00

Ever ran into a situation where you want to initialize an instance of a used schema? Normally, lots of people use a C# method which loads a string into a XmlDocument instance and return that instance to the Orchestration engine. This is all fine and dandy, but what happens if the schema the original instance string was based on is changed?

I've created a little helper class which instantiates an example fragment of a dynamically selected schema (the Schema needs to be deployed ofcourse). It essentially performs the same task as the "Generate Instance" does within Visual Studio, but without the nodes being repeating a number of times and without serializing the instance to disk. It returns a simple XmlDocument instance containing an instance of the specified schema.

Using this class would enable better maintainability of your Orchestrations, as you need not change any code in the Orchestration itself if you refactor your schemas. Ofcourse, if you're adding or removing fields which are used in the Orchestration, you need to update those as you would normally.

It's located here. Comments are very welcomed! Find my e-mail address here

BizTalk Server Pipeline Component Wizard - 2006 edition (2.0)

2005-10-17T22:27:00.000+02:00

Being sick over the weekend allowed me some time to port the current Pipeline Component Wizard to BizTalk Server 2006.

It has been revised on some portions, improved on others due to VS.NET 2005's enhanced support for CodeDOM (although we're not quite there yet, as per this blog entry).

The wizard now implements regions and has some other neat tricks regarding VS.NET 2005. Check it out once VS.NET 2005 has been released (the current version does not work on beta builds).

Obviously, there's no longer support in this release for BizTalk Server 2004 / VS.NET 2003, as the code has been ported to .NET 2.0.

Get it here.

Let me know what you think (for those with the ability to use the product right now ;-))

Unit testing BRE policies

2005-10-11T22:05:00.000+02:00

When Unit testing BRE policies, it might be useful to be able to declaratively define which RuleStore to use, which Policy to test and which Version to load. Using the Attributes I created, doing this become pretty simple. One of the Attributes also provides for a class Type to be instantiated implementing IRuleSetTrackingInterceptor, which allows tracking what the BRE does while executing the requested Policy. Have a look at the code, as I think it might be useful for many UnitTesting scenarios with version policies.

using it is quite simple:

using MartijnHoogendoorn.BizTalk.BRE.Testing.Attributes;

[Test]
[PolicyTester("MyPolicy", typeof(MyInterceptor))]
[RuleStore("Integrated Security=SSPI;Database=BizTalkRuleEngineDb;Server=MyRuleEngineServer", "MYDOMAIN", "myusername", "mypassword")]
private void TestMyPolicy()
{
  XmlReader reader = new XmlTextReader(@"..\..\..\MyMessage.xml");
  XmlSerializer serializer = new XmlSerializer(typeof(MyMessage));
  MyMessageClass = (MyMessageClass) serializer.Deserialize(reader);

  MyInterceptor interceptor = (MyInterceptor) PolicyTesterHelper.ExecutePolicy(new object[] { "Some interesting value", someObject });
  foreach(string eventLine in interceptor.PolicyExecutionEvents)
  {
    System.Diagnostics.Trace.WriteLine(interceptor.PolicyExecutionEvents.ToString());
  }
}

See the sample code here for a real implementation

Testing remote Business Rules (BRE)

2005-10-05T21:50:00.000+02:00

When using remote Business Rules, you might encounter a situation where you might want to test a policy, stored in a remote SQL Server. I had some security issues, just a tip, here's how I resolved it:

using Microsoft.RuleEngine;

// 0 = LOGON32_PROVIDER_DEFAULT
// 2 = LOGON32_LOGON_INTERACTIVE
IntPtr token = IntPtr.Zero;

const string domain = "MYDOMAIN";
const string password = "MYPASSWORD";
const string username = "MYUSERNAME";
const string server = "MYSERVER";

if(LogonUser(username, domain, password, 2, 0, out token) != 0)
{
WindowsIdentity identity = new WindowsIdentity(token);
WindowsImpersonationContext context = identity.Impersonate();

try
{
  SqlRuleStore ruleStore = new SqlRuleStore(
   string.Format("Integrated Security=SSPI;Database=BizTalkRuleEngineDb;Server={0}", Server));

  RuleSetInfo currentRuleSetInfo = null;

  // retrieve information about the latest published RuleSet from the RuleStore
  RuleSetInfoCollection ruleSets = RuleStore.GetRuleSets(Policy, RuleStore.Filter.LatestPublished);

  if(ruleSets.Count > 0)
  {
   // select the retrieved RuleSetInfo object
   currentRuleSetInfo = ruleSets[0];

   // retrieve the latest published RuleSet from the RuleStore
   RuleSet currentRuleSet = RuleStore.GetRuleSet(currentRuleSetInfo);

   // create a PolicyTester to test the RuleSet
   PolicyTester tester = new PolicyTester(currentRuleSet);

   tester.Execute(facts, this);
  }
}
catch(Exception e)
{
  System.Diagnostics.WriteLine(e.ToString());
}
finally
{
  context.Undo();
  CloseHandle(token);
}
}

Ofcourse, as I typed this at heart, some things (like the actual facts being put into the execute action of the PolicyTester) might not compile / work when plainly copied into Visual Studio. Make sure to reference Microsoft.RuleEngine.dll from your project. Also note that this is an example and could be enhanced greatly. ;-)

The Microsoft MVP Summit - day 2

2005-09-30T14:16:00.000+02:00

So I was mistaken about the product group meetings, they'll be today and tomorrow...

Having said that, the morning started off with the keynote sessions by people like Steve Balmer, Jim Allchin, Group Vice President of the Platform Group and Kevin Johnson, Co-President Platform products & Services division.
I never saw Steve in person before and I must say, he's a huge presence which can motivate people like you've never seen before (don't let the glasses fool you :p). It was good to see him taking feedback and suggestions for product enhancements, changes to the different products and the likes. Jim Allchin presented a lot of new stuff in Windows Vista which hadn't even been on the PDC yet. After a nice lunch, we went back to Microsoft Campus to have sessions on new technology to come.

I've seen some awesome new enhancements for the post-Vista era, up until 2010. Microsoft is doing a lot to be as customer focused and feedback driven as they can be and the design of Blackcomb will actually be largely based on customer requirements. The experience with giving feedback is improving dramatically over the next year or so, as Ladybug, Betaplace, Microsoft feedback and others will be merged into one feedback system where Microsoft developers are required to let the customer know what happened to their feedback before a raised issue can be closed.
That's a huge step forwards from the current way of fire and forget feedback from a customer standpoint...

Strolling around, I landed in a session by Don Box (charasmatic dude) about Windows Workflow Foundation. It was fun to see how easy integrating workflow into your application has been made by the development team, actually being a metadata engine, you can simply hook up any code you like by deriving from Activity. You then write your application declaratively, in Xaml. You can do parallel and sequential workflows, and much more. Check out WWF ("dub-dub-ef") here.

After the different sessions, I hooked up with some great BizTalk people like Stephen W. Thomas, Allan Smith, Paul Somers, Scott Cairney, Bill Chesnut, Romualdas Stonkus, et al. It was fun to hear from other BizTalk MVPs about their experiences in the field. Joined by some people from the BizTalk product team, they were really interested in hearing from us regarding actual customer scenarios for BizTalk Server.

After the very nice dinner at the Best West cafe on campus, I went to sleep, the jetlag is really starting to kick in, I wonder how it will be going home and ariving 22 hours later... It's gonna be a long first week back at the office I think... :/

The Microsoft MVP Summit - day 1

2005-09-29T14:24:00.000+02:00

Ok, so today we had registration on the agenda from 13:00 onward, and thus had the entire morning to explore Seattle. Being the typical tourists we appearantly are, we walked through downtown Seattle to the Spaceneedle. We could have taken the monorail, but decided to get to learn Seattle a bit better from the ground first...

Walking through downtown Seattle, we concluded it's quite a nice, quiet city, where given the cars driving around (Ford FS120, Mercedes E200, BMW 760i, etc) people seem to be 'somewhat wealthy'. Seattle is very clean, has nice big sidewalks and isn't quite as crowded as let's say downtown Manhattan. The spaceneedle was a great experience, although I wish it still was the tallest building in Seattle, it does have a great lookout over the harbor, the city and even the mountains (we had some fog upstairs, so that view was somewhat blocked)

On the way back to our hotel from the Spaceneedle, we rode the Monorail, which was fun (and a lot quicker than walking :)). We had a stroll through Pike Place Market,

, drank a beer on the waterfront in the harbor and then took the bus to register for the event. We arrived at the Microsoft Campus. which seems like a complete city on it's own, within beautiful surroundings I must say, forrest every way you look :)

After registration, we went to get my coupon for the Microsoft store filled. Appearantly, a few more people felt like doing the same, as the store was literally packed full with other MVPs. They really have great stuff in the store, so I ended up using my entire coupon's worth, mostly on gifts for others (Martin, I thought of you and have your request in my bag ;O).

So we registered and shopped, it was time to start the party! We dropped the stuff I bought at the hotel and went to the Hyatt hotel in Bellevue, just outside of Seattle, were we had a nice dinner, met a lot of other MVPs, connected to some EMEA MVPs we already knew and generally had a great time the rest of the evening. Around 10:30 I got tired and went back to the hotel to get some sleep.

All in all, this was a good and relaxed start of the real summit. Tomorrow, we have meetings with the award product groups, so I finally get to meet some of the great minds behind the BizTalk Server product (and Windows Workflow Foundation, which looks very promising too, ofcourse) :-D

The Microsoft MVP Summit - day 0

2005-09-28T13:46:00.000+02:00

We've flown to Seattle with SAS a few hours ago. From Amsterdam, we first headed to Copenhagen where we waited a gruesome 4 hours for our connection, an Airbus heading to the US. A cool thing to note about SAS is the fact they have WiFi internet access onboard. It's slow as can be, but at least you can e-mail (and blog) from an altitude of 10km flying 900+ kilometers an hour... :-D (see the picture we took here)

After the dreadful 9 hour flight (on which there were 4 different movies to see, but they weren't streaming them, so hooking up the touchscreen too late means missing a chunk of the movie) where you learn to appreciate a repulsion caddle might have from being transported on a boat, we landed on SeaTac (Seattle / Tacoma) airport.

We ofcourse had to fill out quite some paperwork, one for the US Customs Service stating we had nothing to declare, and one for Visa Waiver countries stating your personal information. On top of the paperwork, we had both our index fingers digitalized and a mugshot taken by the border patrol.

After this exercise of patience, we moved on to get our luggage. We had to take it of the belt, haul it to another room, where we placed it on another bagage belt, were the security checks the European custom services performed were executed again...

Picking up our luggage from the secondary belt, we headed to Seattle by cab (the shuttle was nowhere to be found, or we didn't look hard enough :-))

Frans is staying at the W hotel, for a review of the hotel by another MVP, Sahil Malik, see this page. I'm staying in a hotel two blocks away, on 6th Avenue, called the Crown Plaza. My sentiments regarding this hotel are quite similar to Sahil's, except at least he has what appears to be a minibar and I'm stuck drinking chlorized water from the tap... :/

After walking around down-town Seattle and obviously missing all the cool restaurants (there don't appear to be a lot in the area??) we finally stumbled upon a restaurant worth mentioning. It's and Italian restaurant located next to hotel "Vintage Park" on 5th Avenue and they serve a great red wine (Cabernet Sauvignon) called "Washington" I believe and the Roasted Chicken with Risotto is great :-D

Anyways, I'm not normally in the habit of blogging about non-business stuff, but I thought it would be fun to write up my experiences in Seattle, the summit and being in Redmond... More non-NDA info to follow!

Messages disappearing in BizTalk Server 2004 when the Envelope property of the schema is set to 'Yes'

2005-07-21T13:53:00.000+02:00

I'm documenting this 'feature' in order to save some poor soul(s) the same frustration I went through.

It turns out that if your combine the normal XmlReceive pipeline with a Schema which has the 'Envelope' property set to 'Yes', the message is consumed, but never processed. It just vanishes into the void, no EventViewer error message or warning, no suspended message in HAT, nothing.

If you set the 'Envelope' property back to 'No' or '(Default)', things works as was to be expected. Ofcourse, setting the Receive Pipeline to a custom pipeline with the XmlDisassembler property 'Envelope Schemas' set to be the defined schema will split the interchange just fine.

However, I myself would expect at least a warning from BizTalk Server that it is consuming the message.

The message can be found to be parsed by the receive pipeline by looking in HAT from the 'Queries' menu / Most recent 100 service instances...

BizTalk Server 2004 in combination with VS.NET 2005 B2

2005-07-02T11:33:00.000+02:00

Please see http://wessamzeidan.net/cs/blog/archive/2005/06/08/37.aspx if your Web Publishing Wizard stops working...

TechEd 2005 - Ask The Experts

2005-07-01T12:31:00.000+02:00

In case anyone is interested, my schedule for ATE on TechEd 2005 Europe:

Tuesday - 11:30 - 14:00
Wednesday - 11:00 - 14:00
Friday - 13:15 - 15:00

Party Resolution: NorthWind Suppliers table

2005-06-20T13:57:00.000+02:00

Using the NorthWind database' Suppliers table for Party Resolution as an example might pose some interesting issues. BizTalk Explorer doesn't let you add Parties containing `~!@#$%^&*()+=[]{}|;'"<>,?

With names in the CompanyName column like Grandma Kelly's Homestead and Heli Süßwaren GmbH & Co. KG it seems to be impossible to simply add those companies as parties for creating a NorthWind order processing example orchestration....

Altering the BizTalkMgmtDb (bts_party and bts_party_alias) seems to do the trick though (not supported ofcourse ;-p). Just inserting a new party into bts_party and setting it's alias in bts_party_alias works like a charm (fully tested with simple party resolution)

Webcast recording available - Custom Pipeline Component development

2005-05-28T17:03:00.000+02:00

For anyone interested in my talk about BizTalk Pipeline Component development, please see this link: http://www.livemeeting.com/cc/microsoft/view?id=BTSBAG-62&pw=35DKTQ

Feedback is appreciated!

Webcast Sample - Disassembler component for 401k requests

2005-05-28T10:44:00.000+02:00

In the Webcast I did yesterday, I explained Disassembler functionality, promoting properties from a deployed schema, writing distinguished properties, disassembling a binary Pdf Form into an Xml message which can be used in conjunction with e.g. Party resolution, etc. If you're interested in reviewing the sample material, the component and sample solution, pipelines, etc. can be downloaded here

new version of the XmlCompleteValidator (v1.1) pipeline component

2005-05-26T11:03:00.000+02:00

I've updated the XmlCompleteValidator component, it had some minor issues resulting is less readable errors being generated. It's available here.

Upcoming webcast

2005-05-23T12:51:00.000+02:00

I'm doing a webcast this coming Friday regarding pipeline components in general, the use of the Pipeline Component Wizard to enable rapid development of pipeline components, etc. I've got demos lined up involving creating a simple pipeline component from scratch, building the same component using the wizard to show productivity increasement, building a Disassembler pipeline component, using IProbeMessage, etc. Here's the info:

Title: BizTalk Pipeline Components - abstracting external document types from your business
URL: https://www.livemeeting.com/cc/microsoft/join?id=BTSBAG&role=attend&pw=35DKTQ
Time: Friday, May 27, 2005 21:45-23:30 (GMT+1) (1-2:30pm PST)

Pipeline Component Wizard - v1.34

2005-05-23T10:14:00.000+02:00

Yet another update. This time it's to fix an anomaly in the Disassembler implementation. It's code was incorrectly generated, resulting in a compilable unit, but to not much affect. It now uses a Queue implementation in order to allow easy implementation of the Splitter Pattern.

Pipeline Component Wizard - v1.33 released

2005-05-14T17:38:00.000+02:00

An e-mail from Boudewijn vd Zwan (thanks!) got me started on re-testing the Pipeline Component Wizard and I found a few bugs, which are fixed in the 1.33 release just posted on GotDotNet:

[BUG] AssemblingSerializer generated code has wrong member reference
[BUG] Unsupported CATID values removed
[BUG] Solution always named after latest added project
[BUG] SchemaWithNone with stored incorrectly in the PropertyBag

Sourcecode is included ofcourse, and the documentation part of it is getting better (not quite there yet though..)

Enjoy, and please let me know of any issues...

"a multi-rooted schema may not define a message part; reference the roots directly"

2005-05-13T10:58:00.000+02:00

Using the "Add Generated Items" to add an "Adapter" for use with SQL Server in order to send data from BizTalk to e.g. a Stored Procedure, you might end up with a so-called multi-rooted schema.

This kind of schema is generated to enable a response coming back from SQL Server to be used in the Orchestration you need the message (using a Request-Response send port).

However, within the schema, you end up with 2 rootnodes. 1 for the Request message structure and 1 for the Response message structure.

If you want to promote a property within one of those structure, you could end up with the compiler telling you "a multi-rooted schema may not define a message part; reference the roots directly". This would occur e.g. if you would use a Message Assignment shape to set the promoted property to some value within your Business Process.

In order to resolve this error, make sure a "Root Reference" is selected in the generated schema pointing to the element containing the promoted property. This setting can be found on the "Schema" toplevel element in the Schema Editor view.

As I Walk Along I Wonder...

2005-05-11T16:07:00.000+02:00

Having an Orchestration where I perform a mapping and try to set a promoted property to some value for either correlation or business use, I sometimes encounter this error:

A failure occurred while evaluating the property MacawSchemas.PropertySchema.SomeField against the message part data. The message part data does not contain at least one of the nodes specified by the XPath expression (listed below) that corresponds to the property.

It ofcourse means BizTalk cannot find the property I'm refering to by using the XPath originally specified when the property was promoted (in the annotation within the document schema). However, I was certain I had the XPath setup correctly.

After some testing, it turned out I made the mistake of assuming the field would be available, while in fact, the mapper didn't produce the field in creating the message, as it had neither an inbound link (naturally), nor a default value.

Setting a "space" character to be it's default value in the mapping target is all I needed to get an empty element generated and for the Orchestration to work......

Can't win 'em all :-o

Note to self: cleanup xml schemas after removing promoted properties

2005-05-10T14:55:00.000+02:00

Ofcourse, I'm the only one having trouble with this, but I'll blog it never the less... :-)

I recently (just now :p) encountered some behaviour of the Visual Studio environment for editing xml schemas installed by BizTalk Server 2004 which might lead to trouble in testing and deployment (actually, just in running my Orchestrations in general).

As you probably know, BizTalk adds a nice annotation to your schemas upon promoting a property, like so:

<xs:annotation>
<xs:appinfo>
  <b:properties>
   <b:property name="ns0:CorrelationToken" xpath="/*[local-name()='Order' and namespace-uri()='http://www.macaw.nl/schemas/OrderProcessing']/@*[local-name()='CorrelationToken' and namespace-uri()='http://Macaw.General.Definitions']" />
  </b:properties>
</xs:appinfo>
</xs:annotation>

I would expect BizTalk to remove this information once the field you actually promoted is removed from the xml schema. However, it doesn't... This causes the BizTalk runtime to try and locate the field given it's XPath in order to promote the field value to the message context. Ofcourse, the field can no longer be located, indicated with an XLANG/s exception:

Uncaught exception terminated service MacawOrchestrations.Axapta.ContactPerson(85ce7703-cea4-61cc-67ed-86bfa9998e7c), instance 652d12a4-ed8e-4bca-aa58-9a065d65ca4d

A failure occurred while evaluating the property MacawSchemas.PropertySchema.CorrelationToken against the message part data.

Resolving this issue proves to be quite simple, just right-click on the rootnode, navigate to the promoted properties, select "Property Fields", and remove the promoted property manually.

Just a reminder...

Deploying from the BizTalk Explorer contextmenu...

2005-05-10T11:42:00.000+02:00

Having the option to deploy BizTalk elements from the Visual Studio environment is a very handy thing. However, I made the mistake of using this option to deploy into a test environment I was setting up.

Visual Studio confirmed a successful deployment and I went about configuring the Orchestrations I just deployed (they were nicely visible in the BizTalk Explorer).

When testing the first deployed Orchestration however, it turned out the deployment was successful, but to the end that Visual Studio had deployed the components into the GAC of the local computer while using the remotely configured database for registering the BizTalk components (BizTalkMgmtDb); obviously resulting in failing Orchestrations.

It still means the Deploy option is a good thing, but only to the end of testing the components on the local machine. Reconfiguring the Deployment database doesn't tell Visual Studio to deploy the components to the remote machine GAC (which seems normal, installing to a remote GAC isn't supported to the best of my knowledge)

Microsoft.XLANGs.Core.PersistenceException: Exception occurred when persisting state to the database.

2005-05-09T16:59:00.000+02:00

Ever ran in to this kind of exception:

Microsoft.XLANGs.Core.PersistenceException: Exception occurred when persisting state to the database. ---> Microsoft.BizTalk.XLANGs.BTXEngine.PersistenceItemException: A batch item failed persistence
Item-ID b668d473-774c-4fae-979b-90fff5444ff5
OperationType MAIO_CommitBatch
Status -1061151998
ErrorInfo The published message could not be routed because no subscribers were found. Please use the Health and Activity Monitoring (HAT) tool to diagnose this situation.

I just ran into this while testing an Orchestration I have build. I had encountered it before, but didn't document / remember the solution.

f.m.i. (for my info): It turns out this exception will occur if you're calling another Orchestration, which isn't in an Enlisted state... I'll try to remember next time... :-|

There's something about Parties

2005-05-07T10:58:00.000+02:00

Trying to implement an Order Fulfillment example BizTalk application for the Northwind database, I ran into some Party resolution issues. It turns out Parties cannot have names with any of the following characters in them:

`~!@#$%^&*()+=[]{}|;"'<>,?
As the Northwind database has a Suppliers table consisting of a default of 29 suppliers with various 'complex' names, e.g.:

Cooperativa de Quesos 'Las Cabras';
Forêts d'érables;
and Heli Süßwaren GmbH & Co. KG;

I wasn't able to create parties to fulfill my orders from the Northwind database based on the Supplier companyname.

BizTalk Performance Charateristics online

2005-05-02T13:43:00.000+02:00

Read all about BizTalk Server 2004 performance characteristics at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS_2004WP/html/04d20926-20d2-4098-b701-52238a267eba.asp

It's a great read!

Untyped Message Processing and external projects

2005-04-28T17:37:00.000+02:00

Have you ever used untyped messages send to a general Orchestration (error handling, general processing, etc)?

If you have, you've probably casted the outbound message into a variable of type System.Xml.XmlDocument and passed that to the called or started Orchestration...

However, if you factor out the general orchestration to enable reuse from other projects, you are no longer able to do this.

Instead, BizTalk wraps your Orchestration parameter into a type called Microsoft.XLANGs.RuntimeTypes.XmlDocumentSerializationProxy, which enables serialization of type System.Xml.XmlDocument, as it's missing the Serializable attribute. It does this outbound only, so you need to perform a few steps to get your Orchestration call back online:

1) reference Microsoft.XLANGs.RuntimeTypes.dll from the Microsoft BizTalk Server 2004 installation folder into both projects;

2) change your general Orchestration (the one factored out) to have a your Orchestration parameter as type Microsoft.XLANGs.RuntimeTypes.XmlDocumentSerializationProxy instead of System.Xml.XmlDocument (otherwise, the types won't match and VS.NET won't let you compile ;-) );

3) change your calling Orchestration to have a variable of type Microsoft.XLANGs.RuntimeTypes.XmlDocumentSerializationProxy instead of System.Xml.XmlDocument and set your Expression shape to do this:

GeneralConfirmation = new Microsoft.XLANGs.RuntimeTypes.XmlDocumentSerializationProxy(ConfirmAccount);

3) compile the external referenced project;

4) reconfigure your Call Orchestration or Start Orchestration shape.

Now your Orchestration will function correctly, just like before. The difference being able to reuse the general Orchestration originally contained within the same project.

Suppressing automatic message splitting

2005-04-27T12:44:00.000+02:00

If you have BizTalk automatically splitting up inbound messages and want to suppress this behaviour, set the maxOccurs property to "*" (unbounded) for any repeating structure you don't want BizTalk splitting up on:

<xs:element name="Journal">
...
</xs:element>

becomes:

<xs:element maxOccurs="unbounded" name="Journal">
...
</xs:element>

BizTalk Performance - dedicated blog from Microsoft!

2005-04-27T12:19:00.000+02:00

Finally, an answer to many customer questions....:

http://blogs.msdn.com/biztalkperformance/

Kind words from Microsoft, I'm a MVP awardee

2005-04-02T19:02:00.000+02:00

I just wanted to let everyone know I've been awarded the Microsoft Most Valuable Professional award for BizTalk Server. I couldn't be happier at the moment :-D

XmlCompleteValidator pipeline component...

2005-03-17T17:54:00.000+01:00

This simple pipeline replacement for the standard XmlValidator component enables users to aggregate validation errors within inbound messages based on defines schemas, just like the XmlValidator component out of the box in BizTalk Server 2004. Instead of the XmlValidator component, this component validates the entire message, reporting only after processing the whole. This way, you know right away of all validation errors within a message to be processed instead of fixing one error, only to discover another, and yet another, ...

A lot of the code within this component has been 'borrowed' from the original XmlValidator component.

Hope you enjoy it, it's sourcecode is included as always (only sourcecode, you need to compile it yourself). It has a readme.txt within the archive, please be sure to read it before trying to use it.

it's located here

Document contents validation

2005-03-05T09:45:00.000+01:00

Lots of customers are asking for a way to validate their document structure based on contents, not merely the structure of the document as BizTalk does by default.

Here's a simple method of achieving document validation within BizTalk Server 2004. Mind you, there is ofcourse a performance trade-off, as BizTalk needs to load the entire message into a DOM tree in order to perform this validation.

First, make sure your schemas actually define your datatypes in a correct manner. It's easy to define the structure and think you're done, but you rarely are. Another hint is to use include schemas, as the actual data definitions are quite frequently re-used, and this promotes definition re-use. An example for a dutch zipcode definition:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema
    xmlns:basetypes="http://basetypes.macaw.nl"
    xmlns="http://SchemaValidation.inputSchema"
    xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
    targetNamespace="http://schemas.macaw.nl/samples/inputSchema"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:import
    schemaLocation=".\definitions.xsd"
    namespace="http://basetypes.macaw.nl" />
  <xs:annotation>
    <xs:appinfo>
      <b:references>
        <b:reference
            targetNamespace="http://basetypes.macaw.nl" />
      </b:references>
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="zipcode">
    <xs:complexType>
      <xs:sequence>
        <xs:element
            minOccurs="0"
            ref="basetypes:dutch-zipcode" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

followed by the include schema defining:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema
    xmlns="http://basetypes.macaw.nl"
    xmlns:b="http://schemas.microsoft.com/BizTalk/2003"
    attributeFormDefault="unqualified"
    elementFormDefault="qualified"
    targetNamespace="http://basetypes.macaw.nl"
    version="200401"
    xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="dutch-zipcode">
    <xs:simpleType>
      <xs:restriction base="xs:string">
        <xs:pattern value="^[1-9]{1}[0-9]{3} ?[A-Z]{2}$" />
      </xs:restriction>
    </xs:simpleType>
  </xs:element>
</xs:schema>

As you can see, the dutch-zipcode has a pattern defined, which is a regular expression comforming to the format of the dutch zipcode. This ensures any zipcode defined within the zipcode element conforms to this specification. However, BizTalk only checks the contents of messages after specifically being instructed to do so, as it pertains a performance trade-off. Telling BizTalk Server 2004 to check the actual contents of inbound messages can easily be achieved by creating a custom pipeline within which you drop an out-of-the-box pipeline component called the XmlValidator:

Ensure you use the design-time property called "Document schemas" to designate our schema as the target for validation.

Deploy the pipeline, configure your ports, drop a correct message into the defined folder and see that it is processed.
Subsequently, drop a message into the folder not adhering to the regular expression defined within the pattern and see BizTalk Server 2004 rejecting the message and setting it's state to "Suspended - Not Resumable" combined with an event in the eventlog telling you why it was denied.

That's all, along with my apologies for being absent for such a long time!

BizTalk Explorer v1.31

2004-12-15T21:38:00.000+01:00

OK, it seems all I have time for these days (unfortunately!) is an occasional update to BizTalk Explorer. v1.31 has some bugfixes and new functionality, both reported and requested by people using the product (thanks Niklas, Richard, Hugo, et al!). This update includes:

new: delete host (DEL key and contextmenu)
new: double-click on items opens "edit" dialog

bugfix: right-clicking the host and selecting "Save config" yielded an exception
bugfix: save as XML for log didn't allow correct cancelling
bugfix: import binding files didn't skip file if it was invalid due to editing (deserializing)
bugfix: the BizTalkServerSelector didn't function correctly. fixed, now defaults to locally configured server and database if nothing is done.

As always, I hope you enjoy it and await your comments!

Download it here (GotDotNet) or here (Personal webspace)

Request: 1 person has reported editing an address of a receivelocation / send port doesn't work if Visual Studio is not installed. I removed Visual Studio and didn't encounter this problem. If anyone else has this issue, please contact me so I can look in to it. Thanks!

BizTalk Explorer v1.30

2004-11-28T17:17:00.000+01:00

So, I was reading many questions in the newsgroups about how to easily "copy" a running BizTalk environment from one place and deploy it somewhere else, without taking the running machine offline.

BizTalk Explorer v1.30 supports this option ("Deployment" -> "Copy BizTalk environment") by allowing you to specify a BizTalk database, after which it:

copies the assemblies used within that environment
copies the support libraries linked to these assemblies (non-BizTalk and non-Microsoft)
exports the bindings within the server (assembly related and non-bound)
creates a file called "deploy.bat", which performs all necessary actions to deploy the BizTalk environment somewhere else
zips up the files and places it in a folder of your choosing

Once these steps have been completed, simply copying the zip file to another server with BizTalk installed (and having gacutil/btsdeploy in the path) and running "deploy.bat" should re-create the entire BizTalk environment.

Let me know of any issues, as I think this is a very handy feature and it should be well tested.

Download the new BizTalk Explorer v1.30 from GotDotNet or from here

Enjoy!

BizTalk Explorer v1.27

2004-11-13T15:29:00.000+01:00

I've had some requests on my blog and per e-mail for some changes and got some bug reports (THANKS!)

Additions/Changes in v1.27:

layout: center initial "add database" dialog on screen
bugfix: alternate connection information not used in DatabaseSelector
new: database selector no longer tries to locate databases automatically, only local database is retrieved (from registry)
new: database selector now has a manual search button to retrieve biztalk databases
new: items are sorted alphabetically within the tree
bugfix: enlisted parties correctly removed, enlisted orchestrations commited before removing assembly

It's on GotDotNet, and here

As always, let me know of any requests, bug reports or general rants!

BizTalk Explorer v1.26 - cool changes

2004-11-08T22:27:00.000+01:00

Sorry, all I have time for these days seems to be BizTalk Explorer... If you have questions or comments regarding other BizTalk areas or would like me to spend some time explaining a given section of BizTalk Server 2004, please don't hesitate to ask...

So here's another update to the BizTalk Explorer tool. New in this release (it's undergone quite some rebuilding internally):

fixed bug with starting/stopping orchestrations (invalid parameter, no change)
multiple selection possibility for elements (deleting, starting, exporting, enlisting, killing, etc)
quick-edit feature for receive locations and sendports

I've uploaded all releases to GotDotNet as well, but you can download a copy of v1.26 here.

Enjoy! (Let me know what you think..)

BizTalk Explorer v1.25

2004-10-31T17:03:00.000+01:00

New in this release:

removing all Send Ports (Send Ports treeitem)
removing all Receive Ports (Receive Ports treeitem)
removing all Receive Locations for a given Receive Port

Thanks to Niklas Engfelt for suggesting these little gems.

Please not that it's not possible to remove the primary Receive Location for a given Receive Port so that will not be deleted.

Hope you enjoy it!

Download it here (GotDotNet is down again...)

2004-10-20T21:29:00.000+02:00

BizTalk Explorer v1.24 is here!

Major updates include:

Ordered deployment of assemblies
Starting/stopping all published orchestrations on host level (make sure you bind them first...)
some bugfixes

One of the most reported bugs was that when BizTalk Explorer started the second time, it reported "Reference not set to an instance of an object". This was due to the wrong use of Dotfuscator on my part and should be resolved as it isn't Dotfuscated anymore (sourcecode will be release soon)

The executable can be found here, due to GotDotNet not performing anymore... I will upload this release once GotDotNet is up again.

As always, I'd be interested to hear of any issues with this release!

BizTalk Explorer v1.23

2004-10-04T23:51:00.000+02:00

So version 1.23 sees the light. Some new enhancements in this release, as well as some bugfixes:

send ports didn't work as expected

exporting unbound assemblies is now possible (from the Assemblies node)

editing binding information upon importing to different servers (for each server)

exporting host configuration database xml

some more minor things

Enjoy! let me know of anything you come across!

For those who don't have the GotDotNet workspace ready, you can get the releases here.

BizTalkInstaller: a BTSICustomAction replacement

2004-10-03T14:38:00.000+02:00

In developing with BizTalk, I've had clients needing simple to use MSI packages instead of "complex" NAnt scripts. Creating MSI packages to deploy a BizTalk solution is fairly simple, but using the BTSICustomAction to deploy the contained assemblies and bindings is somewhat of a less trivial task.

The issues my clients encountered when using MSI packages created using BTSICustomAction are:

Inability to edit binding files before actual deployment, hence needing multiple MSI packages for each server configuration

Not all assemblies are removed when uninstalling, due to these assemblies being either used by active/suspended instances of Orchestrations or having bindings with Orchestrations not in unenlisted state.

No feedback within the MSI packages about the deployment of the contained components. Having to manually iterate through all logfiles created.

Acting on this information, I've decided to create a replacement Custom Action library, which allows users to edit bindings before deploying them, asks whether or not to kill running Orchestrations upon uninstall to ensure proper removal of affected assemblies and provides feedback about what went on during deployment/removal.

The Custom Action I've created is used in exactly the same manner the BTSICustomAction is used, it takes a parameter DEPLOY, which, when set to true tells the BizTalkInstaller class to actually deploy (install the assemblies and run the binding files) the components. If the parameter is missing or not set to "true", contained assemblies are installed into the GAC, but no binding files are executed.

Also, when uninstalling, all files installed without DEPLOY=true will still be checked for any bindings which might keep them from being removed.

Using the component is easy:

Create a new "Setup Project"

Add your components to the project

Add your exported bindings to the project

Add the custom action BizTalkInstaller to each section in the Custom Action Editor, setting "CustomActionData" within the Install section to "/DEPLOY=true" if you want bindings to be executed, or nothing if you don't want that.

At this stage, you're actually ready to build the Setup Project and deploy it. BTSICustomAction forced you to include two extra files within the Setup Project:

btstd.xml (contains the standard BizTalk assembly list)

BindingSchema.xsd (contains the schema information for binding files)

I've added those two files to the BizTalkInstaller as Embedded Resource, so there's no need to include those files in your Setup Project. You, however, still need to include BTSIAssemblyOrder.dll in your Setup Project.

Note: altering binding files before deployment should be used with caution, as you have full control of each aspect of the binding file.

UPDATED: NEW DOWNLOAD LOCATION

The Custom Action library can be downloaded here

As always, full sourcecode is included and feedback is very welcomed!

Testing the unzipcomponent sample

2004-09-30T21:32:00.000+02:00

Recently, some people have requested a usage example of the UnzipComponent pipeline component example posted here. I've create a sample solution, containing the UnZipComponent itself and a BizTalk project consisting of a simple receive pipeline witht the UnZipComponent in it.

The UnZipComponent uses a build action to copy itself to "C:\Program Files\Microsoft BizTalk Server 2004\Pipeline Components", so if you have installed BizTalk Server 2004 somewhere else, please alter the location by right-clicking the project, choosing "Build Events", navigating to "Post-build Event Commmand Line" and pressing the three dots button on the right.

A binding file has been included which you can import with the BizTalk Deployment Wizard or from the command-line using "btsdeploy.exe" from the BizTalk Server 2004 installation folder.

The binding file contains folder references, which probably don't match your testbed setup. Alter them after importing them by using BizTalk Explorer (whichever version :p)

The sample project does nothing other than decompressing the inbound zipfile and submitting 1 message for each file within the zipfile. I've included a file called "test.zip", which contains 2 simple XML files consisting of very interesting data, which will be put into the output folder configured. Nothing else.

UPDATED: NEW DOWNLOAD LOCATION

The sample project can be found here. (Don't forget to alter the AssemblyInfo.cs and project properties of the BizTalk project to refer to your own strong name key file.)

Please remember that the btsntsvc.exe process (the BizTalk service) locks the pipeline component when in use. So if you alter anything, stop the service, compile and start the service, otherwise the post-built event will fail)

One more thing... Pipeline components should normally not be within the same solution as the projects that use these components, as the components get loaded by the pipeline designer and locked, so compilation would become rather hard...

Hope you enjoy this!

Exporting bindings

2004-09-30T21:14:00.000+02:00

As there have been numerous requests by e-mail for this, I'll explain the ways of exporting binding information from the BizTalk Management database.

Binding files define the connection between components within the BizTalk environment, they define send/receive ports, groups, orchestrations properties, etc.

To export binding information, you have some options:

Use the BizTalk Deployment Wizard from the Microsoft Biztalk Server 2004 start menu and follow the steps (choose "Export BizTalk assembly binding file, choose the server, choose the appropriate assembly, the location of the exported binding and press Finish)

Utilize the command-line application "btsdeploy", located within the BizTalk Server 2004 installation folder, e.g.:

C:\Program Files\Microsoft BizTalk Server 2004>btsdeploy export name="UnzipSampleProject" version="1.0.0.0" culture="neutral" publickeytoken="8321b1fe6c55c81a" binding="unzipsampleproject.xml"

Use BizTalk Explorer (my version), found here. Right-click the assembly you'd like bindings to be exported for, choose a folder to export to. The binding files will be named after the assembly and postfixed with "_export" and ".log".

BizTalk Explorer v1.1

2004-09-28T17:39:00.000+02:00

I've updated my BizTalk Explorer. You can download it here. New in this release:

Deployment, easily add your BizTalk assemblies to multiple servers at once
Altered the "browse for BizTalk database" dialog
Numerous bugfixes (thanks for the feedback!)

Hope you like it, let me know!
If you encounter a diagnostics screen on startup, remove the config file from %userprofile%\local Settings\Application Data (BizTalkExplorer.config)

Pipeline Component Wizard v1.32

2004-09-24T15:21:00.000+02:00

Small update. Thanks to feedback from Aman, I discovered that SchemaWithNone and SchemaList were not functioning correctly. I fixed the issue for SchemaWithNone, it works fine now. However, for SchemaList, you need some functionality to store the selected schemas within the PropertyBag as a string, and the CodeDOM I was working on didn't quite cut it, so I decided to let it up to you to implement a simple StringBuilder to store the selected Schemas from the SchemaList.

version 1.32 can be downloaded here

BizTalk Explorer v1.0

2004-09-22T23:35:00.000+02:00

OK, so having a lot of clients asking me for a BizTalk Explorer UI without the need to install Visual Studio .NET 2003, I've (re)written BizTalk Explorer outside. It has support for:

Undeployment of assemblies, including removing any binding information for pipelines, mappings, etc.

Exporting bindings posed by Assemblies

Binding, (un)enlisting, starting, stopping and killing Orchestration(s) (instances)

Adding, removing and configuring Parties

Adding, removing, (un)enlisting, starting, stopping and configuring Send Port Groups, including send ports which belongs to those groups

Adding, removing, (un)enlisting, starting, stopping and configuring Send Ports, including filter expressions (including promoted properties for your own documents), and inbound/outbound mappings

Adding, removing, and configuring Receive Ports, again including filters and maps

Adding, removing, enabling, disabling and configuring Receive Locations

For now, expect and please report bugs, as it's an initial release. More features to come, feel free to request any! I hope to make this utility BizTalk Explorer on steroids...

The sourcecode will be made available soon, but I've got some more work to do before I think it's stable and elegant (and documented?) enough to release to the general public.

Please let me know what you think. I've spend a great deal of time on this application and would love feedback, including bugs, flames, comments, suggestions, the works!

You can download the application here

O, by the way, while I was working on this application, I got in contact with a great guy, which has also released a utility that matches mine. His name is Paul Somers, and he has won (congrats!) the BizTalk Server competition (Next time, gadget, next time!) (Scott, please adjust the Dutch gaming laws, so we can join the competition next time around...:-p). Check out his version of the tool here

Hope you enjoy!

Using the Pipeline Component Wizard

2004-09-18T08:44:00.000+02:00

Lately, I've been getting lots of e-mail from people asking me how, after installing the Pipeline Component Wizard, they can actually use it. To do so:

Open up Visual Studio .NET 2003

Press CTRL+SHIFT+N to create a new Project

The New Project dialog appears, containing a tree to the left of the screen, called "Project Types".

In the project types tree, choose "BizTalk Projects"

In the templates screen, on the right of the dialog, the available BizTalk templates appear.

Within the templates screen, choose "BizTalk Server Pipeline Component Project"

Type a name and location for your new project, and press OK

The wizard will appear, follow the screens and choose your options for the new Pipeline Component you're about to build.

After finishing the wizard, it will generate the solution, project files, resources, assemblyinfo and code for the options you selected within the configuration pages of the wizard.

You're done, now a fully functional pipeline component according to your specifications has come to life, implement the code you'd like the component to perform, compile it, etc.

Property editors & pipeline components

2004-09-10T23:58:00.000+02:00

There have been a lot of questions in the newsgroups recently regarding custom type editors within pipeline components.

I've created a sample component in which you choose which database you'd like to connect to and subsequently choose a customer the pipeline component is specific to. These kind of properties could be used to perform customer specific actions within a pipeline component.

Currently, the component implements a UITypeEditor derivement to implement it's editing functionality for choosing a customer. It uses the NorthWind database within the choosen server, and has very little error checking to see whether or not the DBMS exists, etc.

However, it demonstrates the purpose of custom editors for your properties, besides the obvious SchemaWithNone and SchemaList which are already supplied within Microsoft.BizTalk.Component.Utilities.dll and within my Pipeline Component Wizard.

Now for a screenshot, as it usually clarifies a lot (I've removed a portion of the connection string configured):

The component can be downloaded here

Please note this:

1) As BizTalk uses a component called ComponentObjectWrapper to wrap the actual object being handled within the pipeline component editor, and you simply cannot reference Microsoft.BizTalk.PipelineEditor.dll in a decent way without breaking Visual Studio integration with BizTalk, I've applied a "dirty" reflection hack in which I at runtime retrieve the actually instantiated component from this object using reflection, like this:


BindingFlags bindingFlags = BindingFlags.NonPublic | BindingFlags.Instance;
Type t = context.Instance.GetType();
FieldInfo fi = t.GetField("cachedPipelineComponent", bindingFlags);
CustomerSpecificComponent instance = fi.GetValue(context.Instance) as CustomerSpecificComponent;

It's a dirty way to get to the object, but it means I don't have to reference any other components and currently seems to be the only way to achieve reading another property from our design-time component.

2) The project has a build event specified which should deploy the component to the GAC. If it doesn't work, please alter it to represent the location of your gacutil installation.

As always, all sourcecode is included, and comments are very welcome!

Scripting functoids

2004-09-04T21:20:00.000+02:00

Ever wondered how the scripting functoid generates the code to perform what you have put into the buffer, how the checking for validity is done and the methods you define are published?

In reality, Microsoft.BizTalk.BaseFunctoids.InlineScriptCompiler compiles all scripting functoids to the target language. It generates an assembly by using CodeDOM and ICodeCompiler services, in about the same way as outlined in e.g. KB Q304655.

Let's say we have implemented an inline script functoid which defines a C# method like this one:

public string MyConcat(string param1, string param2)
{
return param1 + param2;
}

The InlineScriptCompiler would generate an assembly like this:

using System;
namespace BizTalkMapper
{
   public class FunctoidInlineScripts {
      public string MyConcat(string param1, string param2)
      {
         return param1 + param2;
      }
}

Considering the above, I would argue that combining your logic into one external assembly would be better for a lot of reasons, including code re-use, documentation purposes (try using NDoc on inline script functoids :-p), functional grouping, the ability to use resources, external assemblies, etc.

Just my 2 cents...

Writing a pipeline component in J#

2004-08-23T12:07:00.000+02:00

Ever wondered if besides C# and VB.NET, J# could be used to create pipeline components for use in BizTalk Server 2004? It can.

I've created a simple pipeline component in J#.

However, due to the nature of J#, many interfaces cannot be implemented in this language. This has to do with J#'s inability to deal with out parameters. As IPersistPropertyBag has an interface containing:

IPersistPropertyBag.GetClassID(out System.Guid classid)

, it cannot be implemented and design-time properties have no use.

Due to this and other rather big differences between C# and VB.NET on one hand and J# on the other, I won't be implementing support for J# in my Pipeline Component Wizard.

The example can be downloaded here

Update: Pipeline Component Wizard v 1.3

2004-08-21T23:22:00.000+02:00

Sorry for the delay in posting, I've been on holiday...

Another update to the Pipeline Component Wizard, this time, the following functionality has been added/changed:

1) support for code generation in both C# and VB.NET!
2) bug fixed with the defined component version not being used
3) the designer properties is now a dropdown list, no more editing
4) the designer properties have been expanded with two new members:
4a) SchemaList, which allows multiple (referenced) schemas to be associated with your component (like the XmlDisassemblerComponent has)
4b) SchemaWithNone, a dropdown listbox which allows you to associate a single referenced schema with your component (like the FlatFileDisassembler component has)

There have been reports of a missing component with the installer in previous versions of the Wizard, resulting in incorrect functioning of the wizard. This was due to me accidentaly excluding the Microsoft.BizTalk.Wizard, which is needed by the Pipeline Component Wizard. Subsequently, the CustomAction failed to register the DLL. This should be fixed now.

Support for other .net languages could easily be added if needed, as the CodeDOM has code has been updated to be language independent.

I hope you enjoy this release, please let me know of any issues, my e-mail address can be found here

The sourcecode for the updated pipeline component wizard can be downloaded at GotDotNet

Thanks to Jim Bowyer for creating a VB pipeline component for all to have a look at!

Retrieving the matching schema for an inbound message

2004-08-14T12:49:00.000+02:00

Ever wanted to know which schema matches your inbound message from within a pipeline component? Here's how:

Reference Microsoft.BizTalk.Streaming,

Use the following code within your pipeline component:

Stream orgStream =
   inmsg.BodyPart.GetOriginalDataStream();

MarkableForwardOnlyEventingReadStream helperStream =
   new MarkableForwardOnlyEventingReadStream(orgStream);

IDocumentSpec docSpec = null;

try {
  docSpec =
     pc.GetDocumentSpecByType(
         Utils.GetDocType(helperStream));
}
catch(Exception e)
{
  Trace.WriteLine(e.ToString());
}

if(docSpec != null)
{
  XmlSchemaCollection schemaCollection =
     docSpec.GetSchemaCollection();

    // use the schema collection
}
else
{
  Trace.WriteLine(
     "No schemas found matching our inbound message");
}

Now you have the schema collection which matches your document (this code is minimal, real exception handling should be added ofcourse.

Notes:

Utils.GetDocType does nothing more then read the stream using a XmlTextReader and combine any found namespace with the root element name. If you have DOCTYPE declarations, you could implement this yourself to avoid having to put your DTDs in %windir%\system32.

The MarkableForwardOnlyEventingReadStream class helps in easily resetting the position back to where the stream was once the Utils.GetDocType is done.

Hope you find this useful!

Update: undeployassembly

2004-08-14T10:00:00.000+02:00

I've updated my undeployassembly command line utility for a few reasons:

added support for /a switch, removing all assemblies from BizTalk, other than "Microsoft" namespace assemblies (be careful!)

added the ability to shoot down any existing message instances within the BizTalk queue by assembly name (killServiceInstancesByAssemblyName), otherwise assemblies would not be removed because they were in use.

cleaned up the source somewhat

What I can't figure out is how to make sure no referenced assemblies are being used, i.e. how to set up the uninstall order. the DeploymentDriver being used doesn't seem to allow me using it's methods for that. If anyone has any suggestions for this issue, please let me know!

UPDATED: The file can be found here

XmlDTDValidator pipeline component

2004-08-13T11:41:00.000+02:00

Ever tried using DOCTYPE declarations in your inbound messages? The XML Disassembler does not validate against DTDs, only schemas. If you have DOCTYPE declarations in your inbound message, BizTalk uses the standard XmlResolver for reading in the message and will log about four error messages into the EventViewer, of which one will state:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive" Source: "mscorlib" Receive Location: "c:\test\input\*.xml" Reason: Could not find file "C:\WINDOWS\system32\MyDTD.DTD".

I can imagine situations in which you can't alter the messages coming in on forehand and need the message to be both validated and processed by BizTalk, without having to store your DTD files in C:\WINDOWS\system32.

I've written a pipeline component which both validates the inbound message against it's DOCTYPE, having a design-time property in which you specify where your DTD files reside, and removes the DOCTYPE declaration in order for BizTalk to process the message. (by the way, the framework was built using my Pipeline Component Wizard)

Ofcourse, a design-time property setting the folder in which your DTDs reside, might not be the best option, it's for example use only. In a production environment, you want this value to come from the App.Config file or another place you can alter real-time without having the redeploy your pipeline.

I hope you have fun with this component and would love to hear any comments, flames, what not (see this page for my e-mail address)

Pipeline components: checking to see if you're interested in processing a message

2004-08-07T21:37:00.000+02:00

If you ever needed to see whether your component has to process a given message, try implementing the IProbeMessage interface. It gives you the message, you can analyze it and return a boolean value to indicate whether you're interested in processing the message. If you return false, the next component in the current stage will be executed in the normal manner. If you return true, your component will be executed by the messaging engine.

Example:

public bool Probe(IPipelineContext pc, IBaseMessage inMsg)
{
  // TODO: implement decision logic for the
  // message here
  return true;
}

Please note that the stage you will use the component in must be configured as FirstMatch, which in this version is only the Disassemble stage (see the documentation, the BizTalk 2004 documentation mus be installed for this to work and you have to copy the link and paste it in another window because of security constraints)

Update (2004/08/08): this interface has now been incorporated in my pipeline component wizard (just choose DisassemblingSerializer for the component stage), the pipeline component wizard can be retrieved here or here

Update: Pipeline Component Wizard

2004-08-07T21:14:00.000+02:00

The Pipeline Component Wizard has been expanded and some minor bugs were fixed. It's uploaded to gotdotnet and you can also download it here from my personal webspace.

Major issues: this new version implements the IAssemblerComponent and IDisassemblerComponent interfaces (AssemblingSerializer and DisassemblingParser) as well as the IProbeMessage interface to see if you want to process the message (see this page for more information)

I've removed the older version links from the previous article about the Wizard to ensure everyone gets the latest version.

Hope you enjoy this update! Let me know of any issues, my e-mail address can be found here

Decompression pipeline component (zip files)

2004-08-06T22:58:00.000+02:00

Ever wanted to unzip a file and use it's content in BizTalk Server? I've implemented an unzip component (disassembler stage) which uncompresses the files within the zip file and allows you to process them like you would normal messages. The sourcecode can be found here. I hope you enjoy this sample. Please let me know of any trouble with using it or any suggestions you might have!

BizTalk 2004 - Suspended Queue Listener Windows Service

2004-07-31T22:16:00.000+02:00

Quite a mouth full for the title, but as it says, I've implemented a simple Windows Service component which will watch your Suspended Queue for messages, both realtime and batch-wise.

The batch-wise implementation searches the Queue for Suspended messages upon Service startup and calls a routine which you will implement to do whatever you want with the message from the Queue.

The realtime part is realized by subscribing to a WMI Event fired, called "MSBTS_ServiceInstanceSuspendedEvent". I've supplied the sourcecode for you to have a look at, let me know if you find anything you have questions about or don't agree with. Hope you enjoy this one!

UPDATE:

it's ready for download here

E-mail address

2004-07-29T22:06:00.000+02:00

In order to make it easier for people to contact me with questions regarding my work, blogs, etc. I'll post my e-mail address here. I'll make it cryptic to try and avoid spambots from gaining my e-mail address...

I work at Macaw (www.macaw.nl). Just put my full name (martijn hoogendoorn, replace the blank with a '.' character) in front of the domain, and you'll be contacting me :-)

Pipeline Component Wizard

2004-07-18T23:08:00.000+02:00

Following Boudewijn van der Zwan and Scott Woodgate's adapter wizard (thanks guys, awesome piece of code!), I've created a Wizard that creates pipeline components as I got tired of implementing the same interfaces over and over again. I hope you enjoy this, as I know I will, please let me know of any problems, recommendations or questions you have! You can download the wizard's sourcecode here; an installer is included. After installation, start a new Visual Studio instance and press CTRL+SHIFT+N to generate a new project. Browse to "BizTalk Projects" and select "BizTalk Server Pipeline Component Project" to use the wizard.
The project uses CodeDOM instead of template files, as the number of files generated by the wizard is severely limited in regards to the aforementioned wizard from Boudewijn and Scott.

Again, I'd love to here your feedback on this project, as it couldn't possibly be bug-free :-p

UPDATE: It's also on GotDotNet now.

Feed

2004-07-07T21:57:00.000+02:00

I've been getting some requests to enable my feed. Unfortunately, this blogging site is not using RSS feeds, only Atom feeds. My Atom feed is http://martijnh.blogspot.com/atom.xml

Virtual Server 2005 Release Candidate ROCKS!

2004-07-07T20:49:00.000+02:00

WOW! I'm currently setting up 2 BizTalk Server installations to test different settings, and get a good feeling with Enterprise Single Sign-on. I'm using Microsoft Virtual Server 2005 RC - Enterprise Edition to perform this task and what a ride that's been. I've been using other virtual machine software (you know which I mean) before, but this both performs great and is easily viewed from Internet Explorer. What a breeze it was setting up the virtual machines...

Update: WOW again, you can switch CD-ROM images and network settings realtime!

ListBindings utility

2004-07-07T17:26:00.000+02:00

Within a production environment, it can be helpful to easily see your binding information without using BizTalk Explorer (as you don't have Visual Studio handy) or VBScript. This little utility is something I built for such a situation. It might be helpful to you too or alternatively you could use it as a starting point for using ExplorerOM. Download the sourcecode here.

Note: it currently only displays Send and Receive ports and their relevant information (not everything, just the most important elements). Feel free to enhance or just use it. If you have any suggestion, don't hesitate to send me feedback!

More Performance Tips

2004-07-06T21:36:00.000+02:00

Another tip from Hans-Peter Mayr at TechEd 2004, which I forgot to mention is to keep your schema small (less descriptive). Smaller schemas make for smaller message instances and a smaller impact on performance.

Don’t promote properties that are not to be used for routing purposes. Use distinguished properties if you need to use a message’s value within an orchestration.

As most of you know, BizTalk Server 2004 persists data at many points within the lifetime of a message. If you can keep persistence points down to a minimum, performance will increase. One tip I think Carlo Poli already mentioned is to move your mappings from an external format into an internal format out of your orchestrations (where they shouldn’t reside in the first place) and place the map on the receive port which will be bound to your orchestration. This saves you 1 persistence point per message, again enhancing performance.

Wrapping an atomic scope around subsequent send shapes within your orchestration will also save you persistence points. Imagine a simple orchestration in which you subsequently have 1 receive shape, followed by 2 send shapes

Moving the 2 send shapes into an atomic scope together

will save you 1 persistence point and thus performance.

A common mistake made is not having SQL Server Agent running. This will avoid cleaning up DTC packages from being run and your DBMS will grow unchecked.

Example Functoid

2004-07-06T21:32:00.000+02:00

Functoids are small pieces of code that assist you in implementing the logic which maps an incoming message to an outgoing message. They can be used from the toolbox within Visual Studio in the mapper if correctly installed. To be displayed within the toolbox in Visual Studio, a functoid needs to live inside a special folder. This folder resides within the BizTalk Server installation folder, subfolder Developer Tools\Mapper Extensions.

Any functoid you write has to derive from the BaseFunctoid Class. A functoid can expose it’s logic as inline script by overriding methods which are called to check for this script (GetInlineScriptBuffer). If you desire not to provide scripting code for the functoid, you should make sure the XSLT mapper can find your component by placing it into the Global Assembly Cache (GAC). I’ve written a very simple example functoid for you to play with and use called the "SearchReplace" functoid. It takes 1 node, a searchterm (a regular expression), and a replacement term (also a regular expression). Play around with it and let me know what you think (hint: run the testharness included and do a "test map")!

Talking to Scott Woodgate at TechEd 2004 Europe

2004-07-01T23:01:00.000+02:00

This morning I talked to Scott Woodgate, the Product Manager for BizTalk Server 2004. It was both an informative and a helpfull talk as I got to give feedback on using the product and ask questions regarding implementation choices they made for the product. This afternoon Scott presented an excellent session, EBZ315 - A .NET Developers Drilldown into Building Advanced Business Process Using BizTalk Server 2004 Orchestration (promptly abbreviated by Scott, as the title was both incorrect spelling and way too long to pronounce without restarting at least 3 times :-p).

Within the session, Scott mainly focused on correlation and convoys, interesting concepts as most of you might already know. Unfortunately, time was short as always with interesting sessions...

Another BizTalk 2004 performance hint

2004-06-30T23:21:00.000+02:00

Another BizTalk 2004 performance hint comes from TechEd 2004 as well: if you're sending multiple messages at once, wrap it in an atomic scope. This will create 1 persistence point instead of n (n being the number of messages you send). Persistence points are points at which BizTalk serializes the orchestration state into the MessageBox, draining the system from resources.

Send ports, performance issue

2004-06-30T23:09:00.000+02:00

Something I've picked up at TechEd 2004 Europe which I hadn't considered myself yet: don't use XmlTransmit on sendports unless you need to demote properties back into the message. This prevents a performance drain!

Easily undeploy an assembly

2004-06-30T23:03:00.000+02:00

Using this console application you can easily undeploy assemblies from BizTalk. It saves the current binding information into an XML file for easy restore. Things that might be improved: 1) Kill existing orchestrations running from the BizTalk environment, to ensure the assembly can actually be deployed. Currently, false positives might be generated if orchestration instances still exist. 2) bugs? 3) It's poorly documented, if anyone offers...

UPDATED: Here it is

File adapter with regular expressions

2004-06-30T22:48:00.000+02:00

I've altered the FILE adapter SDK sample so that you don't have to create a receive location for each file extension you wish to pick up. It accepts a regular expression for the receive location. It's a simple modification, put I was creating lots of receive locations, one for each filetype I had to pick up and got fed up with it. Let me know what you think!

In PickupFilesAndSubmit (DotNetFileReceiverEndpoint.cs), change the lines

DirectoryInfo di = new DirectoryInfo(this.properties.Directory);
FileInfo[] items = di.GetFiles(this.properties.FileMask);

into:

FileInfo[] items = GetFilesRegex(this.properties.Directory, this.properties.FileMask);

and add the function GetFilesRegex() where you want it:

/// <summary>
/// This method retrieves all files from a given folder, matching
/// a regular expression in their filename
/// </summary>
/// <param name="path">The folder to enumerate
/// <param name="regularExpression">the regular expression string for matching the files</param>
/// <returns>an array containing the found filesystem entries</returns>
public static FileInfo[] GetFilesRegex(string path, string regularExpression)
{
// queue == FIFO
Queue q = new Queue();

DirectoryInfo di = new DirectoryInfo(path);

foreach(FileInfo fi in di.GetFiles())
{
  // see if we match our regular expression
  if(Regex.Match(fi.Name, regularExpression).Success)
  {
   // we do, enqueue the file
   q.Enqueue(fi);
  }
}

// copy the queue back into an array of files
FileInfo[] fia = new FileInfo[q.Count];
q.CopyTo(fia, 0);

return fia;
}

You're done! Now configure the adapter (file mask) with a regular expression: (?i).*\.((dw)|(pd))f