In the wee early hours of the morning I recently got an email saying I had been awarded a MVP for another year.

This time I got awarded as an Azure MVP coming from BizTalk MVP. My love has been BizTalk for the last 12 years (and even Site Server before that for those that remember back that far…no they didn’t have punch cards ).

As always I don’t feel this is personally my award, but more of an award to you, the community with your hunger and thirst for knowledge to make a difference in your day to day.

And of course thank you to Microsoft for your belief in the MVP program and individuals such as myself.

……

The formal bits out of the way….I’m back for 2012/13!!! Should be a great year…great tour…and huge developments. Stay tuned.

My focus will be How to Integration and how-to integrate to/from Windows Azure.

Well folks – the appfabric labs have come out with a real gem recently.

In CTP we have:

• EDI + EAI processing
• AS2 http/s endpoints
• ‘Bridges’
• Transforms

and of course the latest version of

• ServcieBus, Queues and Topics.

To get the real benefit from this ‘sneak peek’ there’s a bit of setup required. To those familiar with BizTalk there’s a few EDI screens declaring parties/partners and agreements you’ll have seen before.

To get cracking:

1. Update your local bits with the latest and greatest - Installing the Windows Azure Service Bus EAI and EDI Labs - December 2011
   Part of this install is to install the Service Bus Connect component, which installs the BizTalk 2010 LOB Adapter pack.
   
   
   So this is really quite interesting. As the WCF LOB Adapter SDK provides a framework for developers to build out ‘adapters’ to connect systems/endpoints through a sync/async messaging pattern.
   
   The BizTalk Adapter Pack 2010 is the BizTalk Team set of adapters built on top of the WCF Adapter Framework. The BizTalk Adapter pack includes:
   - SQL Server Adapter. Hi performance sql work, notifications, async reads, writes etc.
   - SAP Adapter – uses the SAP Client APIs (under the hood) to talk directly to SAP. Very powerful
   - SIEBEL Adapter
   - Oracle DB Adapter
   - Oracle ES Adapter
   
   These adapters are exposed as ‘WCF Bindings’ with BizTalk or a small amount of code, allows you to expose these adapters as callable WCF Services.
   
   What does this mean in our case here?
   If you think about your on-premise Oracle system, we now have a local means of accessing Oracle and we can then push the message processing (e.g. a new order arrived) into our ‘cloud’ bridge where we have the immediate benefit of HA + Scale. Do some work there, and spit the result out any which way you want. Maybe back down to on-premise, or in a Queue or to Azure Storage.
   
   
2. Sign up to AppFabricLabs – http://portal.appfabriclabs.com and provision your ‘servicebus’ service.
   This provides your EDI/EAI relay endpoints and also provides a way for you to listen/send requests to/from the cloud.
   
3. Here I have used mickservices as my ServiceBus namespace.
   
   (I created a Queue and a couple of Topics for later use – not really needed here)
   Note: grab your HIDDEN KEY details from here – owner + <key#>
   
4. From within the Portal Create a Queue called samples/gettingstarted/queueorders
   
   
   
5. Register at the EDI Portal – http://edi.appfabriclabs.com
   Even though this says ‘EDI’ think of it as your sandpit. It’s where all your ‘widgets’ live that are to run in Azure Integration Services.
   
   The registration form had me stumped for a little bit. Here’s the details that work.
   
   
   
   Notice my servicebus namespace – just the first word. I previously had the whole thing, then variations of it.
   Issuer Name: owner
   Issuer secret: <the hidden key from above>
   
   Click save/register and you should be good here.
   
6. Once this is done – click on Settings –> AS2 and Enable AS2 message processing (which is EDI/HTTP – you might be lucky enough to get the msgs as XML, but most times no). This will create some endpoints for you b2bgateway… style endpoints.
   
   
   
7. At this stage, have a look under Resources and you’ll notice that it’s empty. But…they have Schemas, Transforms and Certificates. We’ll come back to that later.
   
8. Let’s head to Visual Studio 2010 with the updates installed and open up the Sample Order Processing project.
   
   I installed my samples under c:\samples
   
   
   If all opens well you should see:
   
   
   Note: there’s a couple of new items here: (expand out artifacts)
   *.bcs – Bridge. There’s a MSDN Article describing these – I was like ‘what???’. Basically these are a ‘processing pipe’ of which various operations can be performed on a message in stages. These stages are ‘atomic’ and they also have ‘conditions’ as to whether they *need* to be applied to the said message. So a bridge could take a message, convert it to XML and broadcast the message out to a Topic.
   
   Opening up the designer – it gets pretty cool I must say!!!
   
   
   Note the ‘operations’ on the LHS. I must have a play with these guys  
   Another thought – how extensible is this? I’d bet we could write our own widgets to throw on the design surface as well.
   
   By double clicking on the BridgeOrders component, you can see the designer surface come up with the ‘stage processing’.
   
   
   
   Here you can see the ‘bridge’ (I wonder if that term will last till the release) will accept only 2 types of message schemas – PO1 + PO2. Maps them out to a more generic PO format.
   The map – XMLTransform from my initial testing only applies one map, the first one that matches the source schema (this is the same as BizTalk).
   
   Close the bridge view down and leave the BridgeConfiguration open.
   
9. Click anywhere on the white surface of the BridgeConfiguration and set your Service Namespace property from the Properties window (this guy was hard to find!!)
   Put <your service namespace> you created originally.
   
   
10. Save and click Deploy and a Deployment window comes up – put your details in from above.
    
    
    After deployment completes, keep an eye on the Output window as this has all the URLs you’ll need for the next step. In particular the BridgeOrders.
    
    Feel free to go back to your Azure Portal –> Resources and see your deployed bits in there, Schemas, Transforms etc.
    
    
11. Running what you’ve built – sending a message to the ‘bridge’ (here I’ve borrowed info from the ‘Readme.html’ in the sample project folder)
    We don’t need to setup the whole EDI Trading partner piece. – just send messages to a restful endpoint – aka the bridge.
    

1. From the samples folder locate the Tools\MessageSender project. (you may have to build it in VS.NET first)
2. from a command prompt run messagesender.exe
   
   
   In my case it looks like this:
   
   
   
   Took me a little to get this originally, make sure all your VS.NET stuff is deployed properly.
   
   So effectively we have sent PO1.xml to our ‘Bridge’ and it’s been accepted, validated and transformed into ‘something else’ and popped onto a Queue called Samples/gettingstarted/QueueOrders.
   
   We will now get the message Reader to Read it.
   

From a command prompt at that location, run the following to Listen to the queue

Wrapping up -

Here is obviously a quick walk through of what’s possible, performance, scale and throughput are other measures that we haven’t got here – given it’s CTP/Labs we’re not quite ready for that conversation.

BizTalk adapter pack will expose out for e.g. your SAP system to a wider audience and imagine having restful WCF services to call that provide you customer data in the format you want…or better still…deliver it straight to you!
(currently in BTS 2010, the adapter pack is licensed separately, it’s part of BTS standard or enterprise. BTS2009 it *was* licensed separately for RRP $5K. Maybe we’ll see this as a separate component again.)
Or you could do like the SharePoint team and write a brand new WCF Adapter (‘connector’ in their terms) – ‘Duet’ and spend 18 months doing so.

Some things I’d like to see here is a Rules Processor or Engine – being a long long BizTalk fan, the rules engine is a massive strength of any loosely coupled solution. The majority of BizTalk solutions I come across don’t employ any rules engines…or better still, Windows Workflow 2,3+ (but not 4 or 4.5) has a rules ‘executor’ which is very powerful in it’s own right. Who’s heard or used the Policy shape?

Given that this is a sneak peak at what is on the horizon, this is definitely a space not to miss.

Get those trial accounts going and enjoy!

In particular I’d like to call out Rick’s Article (well done Rick!) for a great read on this space also.

Mick.

A recent project my team has worked hard on has come to fruition . This was a challenging project in these key areas:

• High volume – benchmarks of 20000 concurrent requests/sec through the system.
• Low latency – time is critical as price and market changes going through.
• Scalable – different data centers, different regions in the world.

Seemed like a great challenge…. and we built some great componentry through it, utilising the best of many worlds.

Centrebet have released a press release about their Microsoft Azure Cloud, Application Integration solution. This is a tremendous success story for such a well-known Australian brand.

Centrebet deploys app integration platform

http://www.newsmaker.com.au/news/11380

In recent days the Cumulative Update #2 has been released with several improvements and fixes.
(The team have done an amazing job addressing previous issues/fixes with currently *no* open issues!!! WELL DONE GUYS)

As you may well know by know – BizTalk is made by people who care for Integrators (us) who care. BizTalk has always been a quality product in the past and in my opinion will be in the future.

Here’s some details if CU2 that was passed onto me…

Biz Ta lk Ta20 10 CU 2 Details:

 Public KB Article

 BizTalk Server 2010 CU2 List

BizTalk Adaptor Pack (BAP) 2 010 CU 2 Details:

 Public KB Article

 BAP 2010 CU2 List

-------------------------------------------

Hi,

It is a pleasure to announce the release of CU2 forBizTalk Server 2010 on behalf of the BPD Sustained Engineering group.  With this CU, we have addressed all known customer issues from all branches of BizTalk (i.e. applicable to BizTalk 2010) and also available in multiplelocales.

This CU includes few serviceability improvements along with product updates based on customer requests.  Some of the notable updates in this CU are:

• Transparent CU Setup : Provides info on number of fixes applicable to feature during installation
• Enhanced EPM debug Tracing : Helps the support team isolate specific failures in a large volume scenario
• BizTalk Host Instances not coming back-on-line after SQL being off-line : Allows to restart the BizTalk service automatically when SQL connectivity is restored
• BAM Archive checks & Logging before dropping tables from BAMPI: Prevents possible data loss scenarios when archiving fails unexpectedly and also allows for a user configurable setting on the rate of archiving.
• Applications stop responding or crash when System Center Operations Manager monitors BizTalk Server applications: Issue was caused by a race condition in one of our internal class and has been fixed in this release.

Wow! I'm finally through the other side... what a quest and I thought I'd share some of the details with you.

RFC_READ_TABLE rfc can be used to call into SAP and retrieve table data - *sort of* (and that's a big sort of) like a 'DataSet'.

Using it requires a little work and understanding.

In your either BizTalk project or other project from VS.NET:

1. Add the SAP bits to your VS.NET project - ready for action
   1. select 'Add Adapter Service Reference' (for BTS projects -> Add New Generated Item->Consume Service Adapter...)
      
   2. On the Binding Wizard Screen select sapBinding and configure the appropriate connection string details such as:
      

      string

      sapUri = "sap://CLIENT=800;LANG=EN;@A/sapsrv/00?GWHOST=sapsrv&GWSERV=sapgw00&RfcSdkTrace=true";
      <You need to stick your own sapURI above - that is more or less a sample>
   3. Click on the Connect and under RFC->OTHER , select RFC_READ_TABLE (or you can type it in the box to search)
   4. Click Ok to generate the proxy and other details.
   5. Either your BizTalk Project or your non-BTS project has now all the relevant details to communicate to SAP.
      
      I tend to build out all this functionality first in a Console App just so I know what is needed within the BTS environment, also I find it much quicker to test/debug etc. here.
      
2. Ok - onto the code. I've got 2 routines for you, one that uses the Proxy Classes built by the wizard in the last step, and a routine from 'first principles'.
   
   One of the things that I really like about the BTS Adapter Pack and certainly in this case, is that depending on the shape of the XML you pass to the adapter, it determines the table and type of operation that it is to do.
   
   Both of these examples below you could wrap into a functoid/helper/whatever and use directly from code.
   
3. Proxy Code - version 1 - here I define some parameters and make a straight call to the table CSKS.
   NOTE: Use FieldNames not Field Labels (took me a few hrs on that one ;)
   
   

   using

   LOBTYPES = microsoft.lobservices.sap._2007._03.Types.Rfc;

   private

   staticvoid GetDataFromSAP()
   { 
        RfcClient clnt = newRfcClient(); //myproxy client
        string[] data = GetAppDetailsForCurrentUser("SAP");
        clnt.ClientCredentials.UserName.UserName = data[0];
        clnt.ClientCredentials.UserName.Password = data[1];
        LOBTYPES.TAB512[] rfcData = new microsoft.lobservices.sap._2007._03.Types.Rfc.TAB512[0];
        LOBTYPES.RFC_DB_OPT[] rfcOps = new microsoft.lobservices.sap._2007._03.Types.Rfc.RFC_DB_OPT[0];
        LOBTYPES.RFC_DB_FLD[] rfcFlds = new microsoft.lobservices.sap._2007._03.Types.Rfc.RFC_DB_FLD[]
        { 
            new LOBTYPES.RFC_DB_FLD() {
            FIELDNAME = "KOSTL",
            LENGTH=10
            },
            new LOBTYPES.RFC_DB_FLD() {
            FIELDNAME = "DATBI",
            LENGTH=8
            },
            new LOBTYPES.RFC_DB_FLD() {
            FIELDNAME = "DATAB",
            LENGTH=8
            }
        };
   try
   {
     clnt.Open();
     clnt.RFC_READ_TABLE(";", string.Empty, "CSKS", 50, 0,ref rfcData, ref rfcFlds, ref rfcOps);
     Console.WriteLine("RFC RESPONSE\r\n\r\nData:" + rfcData.Length.ToString());
   }
   catch (Exception ex)
    {
      Console.WriteLine("ERROR: " + ex.Message);
    }
   }
4. More from first principles so this is to give you more of a BTS picture.
   NOTE: The use of the '%' sign to get a wildcard match on a KOSTL field, despite in the SAP Client UI the users enter a '*'
   
   

   private

   staticvoid GetDataFromSAPV1()
   {
       string[] data = GetAppDetailsForCurrentUser("SAP");
       SAPBinding binding = newSAPBinding();  //A reference to Microsoft.Adapters.Sap is needed.
       //set up an endpoint address
       string sapUri = "sap://CLIENT=800;LANG=EN;@A/sapsrv/00?GWHOST=sapsrv&GWSERV=sapgw00&RfcSdkTrace=true";
       EndpointAddress address = newEndpointAddress(sapUri);
       try
       {
           ChannelFactory<IRequestChannel> fact = newChannelFactory<IRequestChannel>(binding asBinding, address);
           // add credentials
           fact.Credentials.UserName.UserName = data[0];
           fact.Credentials.UserName.Password = data[1];
           // Open client
           fact.Open();
           //get a channel from the factory
           IRequestChannel irc = fact.CreateChannel();
           //open the channel
           irc.Open();
           string inputXml = "<RFC_READ_TABLE xmlns='http://Microsoft.LobServices.Sap/2007/03/Rfc/' xmlns:ns1='http://Microsoft.LobServices.Sap/2007/03/Types/Rfc/'>"
           + "<DELIMITER>|</DELIMITER>"
           + "<QUERY_TABLE>CSKS</QUERY_TABLE>"
           + "<ROWCOUNT>10</ROWCOUNT><ROWSKIPS>0</ROWSKIPS>"
           + "<DATA /><FIELDS>"
           + "<ns1:RFC_DB_FLD><ns1:FIELDNAME>KOSTL</ns1:FIELDNAME></ns1:RFC_DB_FLD>"
           + "<ns1:RFC_DB_FLD><ns1:FIELDNAME>DATAB</ns1:FIELDNAME></ns1:RFC_DB_FLD>"
           + "<ns1:RFC_DB_FLD><ns1:FIELDNAME>DATBI</ns1:FIELDNAME></ns1:RFC_DB_FLD>"
           + "</FIELDS>"
           + "<OPTIONS>"
           + "<ns1:RFC_DB_OPT><ns1:TEXT>KOSTL LIKE '1234%' AND BUKRS EQ '63' AND KOKRS EQ 'APPL'</ns1:TEXT></ns1:RFC_DB_OPT>"
           + "</OPTIONS>"
           + "</RFC_READ_TABLE>";
           //create an XML reader from the input XML
           XmlReader reader = XmlReader.Create(newMemoryStream(Encoding.Default.GetBytes(inputXml)));
           //create a WCF message from our XML reader
           Message inputMessge = Message.CreateMessage(
                        MessageVersion.Soap11,
                        http://Microsoft.LobServices.Sap/2007/03/Rfc/RFC_READ_TABLE,
                        reader);
           //send the message to SAP and obtain a reply
           Message replyMessage = irc.Request(inputMessge);
           //create a new XML document
           XmlDocument xdoc = newXmlDocument();
           //load the XML document with the XML reader from the output message received from SAP
           xdoc.Load(replyMessage.GetReaderAtBodyContents());
           XmlNodeList nds = xdoc.DocumentElement.SelectNodes("//*[local-name()='WA']");
           foreach (XmlNode nd in nds)
           {
                   string[] parts = nd.InnerText.Split('|');
                   Console.WriteLine("CC={0} From: {1} To: {2}", parts[0], parts[1], parts[2]);
            }
            xdoc.Save(@"d:\sapout.xml");
            irc.Close();
            fact.Close();
       }
   catch (Exception ex)
   {
            Console.WriteLine("ERROR: " + ex.Message);
   }
   }
   

As far as I know there’s some good news and bad news…

Good news: the existing BizTalk SharePoint Adapter *should* work with SharePoint 2010 – you will however need to add ‘<rebinding>’ section to the existing adapters web.config *if* the SharePoint 2010 is installed locally to the BizTalk Server.

The ‘rebinding’ tells your local .NET app that even though you want V12.0.0.0 of the SharePoint APIs, V14.0.0.0 assemblies will give it to you.

Bad News: there’s no new BTS Adapter on the Horizon for this – AFAIK.

------

With SharePoint 2010 we now have the capability to involve many tighter technologies such as:

1) event notification, rather than polling for the adapter.

2) LINQ and ADO.NET Entities to query the Data.

3) SharePoint Client WCF Service – Client.svc . This is a lightweight and fast interface, where we can batch up requests, send them over the wire and get back just what we ask for.

4) Lists.ASMX web service (+ the others) for backward compatibility.

When I get some time…:D, I’m keen to develop a .NET LOB WCF Adapter.

I thought I’d share a few interesting SAP tales.

I’ve been working on a project lately of integrating with not 1 SAP Server, but 2 and the 2nd one is across the seas accessible via a SAP Router (which is similar to Proxy Servers for the internet). I’ve got to thank Rohit Singh (MS) and his team for some great feedback, as well as Scotty and Kent Weare whom were helping me to nut out where half these settings go.

Specifically I needed BTS to be a ‘remote RFC server’ for the two SAP Servers.

What does a Router string look like I hear you ask - “/H/devapp1/S/3300/H/acmesaprouter.acme/S/3300/H/sapdb01.acme”. Something like that, and the user’s type this into the SAP Client UI to connect…my chances of being able to ‘stick’ this somewhere in the BizTalk world was diminishing.

So I had to get my BizTalk box to talk through the ‘SAP Router’ out with the right credentials to another SAP Server 1200KMs away…”good luck” little adapter I thought. ("good luck” Mick I though too)

Here’s the low down:

1. Use a saprfc.ini file – scarce documentation, but do able.
2. Set a RFC_INI System Environment variable.
3. Turn on Rfc Tracing
4. Get on well with the SAP teams.
5. Get on well with the SAP teams…oh I mentioned that one already.

Here’s how you do it – after you’ve installed and setup the prereqs for the SAP Adapter (don’t forget to add the SAP Adapter property schema to BizTalk)

Starting out:

- I jumped in and used the 'Consume Service Adapter' Wizard to work out connection details and look at the IDOCs schemas.

The problem is - as time goes by, you want to see debugging and other details to tweak as trying to establish a connection. The Receive Location (WCF-Custom, sapBinding) SAP URI get's horribly long.

I was happy to put up with this when I got the first connection to the SAP Server1 (local).

This *didn't* work for SAP Server2(remote) - trust me, it's a square peg in a round hole.

Using SAPRFC.INI :(generally the MS Docs will get you started, but I found they had incomplete settings so I had to go elsewhere - a Siebel->SAP 2001 document served the purpose)

1. Create a System Environment Variable called RFC_INI  and point it to where you want your saprfc.ini file to live.
   e.g. SET RFC_INI=d:\BizTalk_Dev\SAP\saprfc.ini
   (the MS documentation doesn't say *exactly* where to put the saprfc.ini - I tried it in the bts folder, windir...many places)
2. Set the Receive Location to use the saprfc.ini - e.g. sap://client=110;lang=en;@D/SAPSERVER?LISTENERDEST=BTS_INBOUND&RfcTraceSdk=true
3. Using the SAPRFC.INI file

Sample SAPRFC.INI - for local SAP connection

DEST=SAPSERVER
TYPE=A
ASHOST=DEVAPP1
GWHOST=DEVDB1
GWSERV=sapgw00
SYSNR=00
RFC_TRACE=0
ABAP_DEBUG=0
USE_SAPGUI=0

DEST=BTS_INBOUND
TYPE=R
GWHOST=DEVDB1
GWSERV=sapgw00
PROGID=BizTalkDev_Inbound  (<-- this is allocated from SAP)
SYSNR=00
RFC_TRACE=0
ABAP_DEBUG=0
USE_SAPGUI=0

Connecting to a SAP Server via a SAP Router String - sample saprfc.ini
e.g. router string -/H/devapp1/S/3300/H/acmesaprouter.acme/S/3300/H/sapdb01.acme

ListenerURI (BTS Receive Location) = sap://client=110;lang=en;@D/ACMESAP?LISTENERDEST=ACMESAP_INBOUND&RfcTraceSdk=true

DEST=ACMESAP
TYPE=A
ASHOST=/H/devapp1/S/3300/H/acmesaprouter.acme/S/3300/H/sapdb01.acme
GWHOST/H/devapp1/S/3300/H/acmesaprouter.acme/S/3300/H/sapdb01.acme

GWSERV=sapgw00
SYSNR=00
RFC_TRACE=0
ABAP_DEBUG=0

DEST=ACMESAP_INBOUND
TYPE=R
GWSERV=sapgw00
GWHOST=/H/devapp1/S/3300/H/acmesaprouter.acme/S/3300/H/sapdb01.acme
PROGID=BizTalkDev2_Inbound
SYSNR=00
RFC_TRACE=0
ABAP_DEBUG=0

HTH folks and saves you guys some time - :)

Folks I recently came across this site on codeplex - http://btsazureadapters.codeplex.com

Where a hard working Danny(not sure of further details) has created a send and receive adapter for the cloud.

The adapter surface and publish the required information into the cloud so for e.g. your endpoints show up publically in the .NET Servicebus domain – we can do this through the apis, but it’s nice to have them all wrapped up.

There’s even some great piccys as well (just over look the ‘Microsoft Confidential’ at the bottom :-| - I did ping him over this.)

The adapters are WCF based adapters, meaning they will run in or outside of BizTalk using the .NET LOB Adapter SDK Framework.

As a Plan B to this – you can always use a Custom WCF Adapter with the basicHttpRelayBinding or netTcpRelayBinding to punch through to the cloud.

I reckon this will be a very exciting area going forward…keep your eyes peeled.

Go and grab the bits and start playing… :)

A handy couple of articles that have been updated for Biztalk 2009, full article attached at the bottom.

Enjoy

Hi - today I came across a handy Matrix that describes the supported versions for the Biztalk Adapter Pack v2.0

Handy reference I reckon :)

---snip---