Scribe Online: Replication and Integration Services for MS CRM
Scribe Software has historically been a strong player in the Microsoft CRM space, offering a configuration-based, no-code solution to both data migration and integration requirements.
In this post I have a look at their newer hosted offering: Scribe Online – where from $99 USD / month (and up) you can rent solutions for replication, data migration or integration.
This post will give you an overview of the installation experience, introduce you to the configuration process and demonstrate just how easy it is to get up and running very quickly. I was quite impressed by my first taste of this offering and would certainly table it as a potential solution in certain customer scenarios.
What is Scribe Online?
The Scribe Online solution is a Scribe hosted solution that covers 2 functional areas:
- Replication Services, and
- Synchronisation Services
The emergence of hosted CRM offerings gave rise to a new need that Scribe’s Replication Services product was introduced to meet. Scribe’s Replication Services allows you to mirror your hosted CRM database down to a local SQL Server database. This opens up ‘SQL as source’ reporting approaches and integration options. It is very easy to setup, inexpensive to license and suitably robust. Think how much easier some requirements can be met when you can do a simple SQL read rather than having too muck around writing web service calls.
Scribe’s Synchronisation Services offers similar functionality to the Scribe Insights product. It supports one off data migration scenarios and on-going integration scenarios with Microsoft CRM as either Source or Destination system. Again it’s Scribe hosted, and again it provides support for CRM Online but is equally suitable for an on premise installation of CRM.
I’ll keep this brief, I just want to give you a feel for the process. Scribe offers 14 day free trials from their website for the Scribe Online products so you can easily get your hands dirty.
You follow your nose through that trial sign up process, give them some contact details and they then email you a link like this:
That link, directs you to the Scribe Online console. You’ll be prompted to update your Silverlight if you are not on the latest (the console is a Silverlight app) and then asked to sign in:
The Console is divided horizontally with Replication Services residing in the top half of the screen and Synchronisation Services sitting at the bottom:
Now, you do need to install a local Agent in order to use Scribe Online, so it’s not 100% SAS. The Agent is required for connection to the source and destination systems. It’s installed as a Service.
I’m running on a Windows 2008 VM and I just went ahead and installed it locally:
You can see it installed and running here under my VM’s Services:
and for those that are curious, here’s what the Scribe Online architecture looks like:
That’s the install process. On to configuration…
Configuring Scribe Online Replication Services
From the Scribe Online console you kick off a wizard to add a new Replication Services solution:
You select the Agent that you installed earlier and then provide the connection details to your CRM Organisation and to your SQL Server database.
Your source can be CRM Online, a CRM on premise install or an IFD/ADFS install:
Your destination can only be a SQL Server database, and you need to create an empty database in advance:
Next, you select which CRM Entities / SQL tables you wish to replicate:
And then finally you pick when you want the replication to run. You have all the normal sort of scheduling options here, you can go daily at a certain time, you can deselect weekends, or have it only run once a month. You can also have it run every x minutes (but 15 minutes is the lowest number you can go with):
And that’s it. The Solution is now listed back on the Console screen and I can kick it off immediately if I want:
And before too long you will have a nice little replica of your CRM database sitting on your SQL Server ready for you to write reports against or to feed into a data warehouse or integrate into an ERP system:
To keep the replication going and your local database up to date the only trick is to make sure your Agent Service remains running and make sure you set your schedule correctly. And if your Agent Service stops running for a period of time Scribe Online will send you an email to let you know.
As at May 2012 the advertised price for Scribe Online Replication Services is $99 USD / month, with an upper limit of 175 million records replicated per annum. The pricing goes up in step increments from there to e.g.: $475 for 1 billion records.
Configuring Scribe Online Synchronisation Services
The Synchronisation Services is Scribe Online’s integration and data migration engine.
Here the process is pretty similar. From the console we create a Solution and work our way through a wizard:
You pick the Agent you wish to use and then define at least 1 Map records. A Map is required for each source-destination table combination.
For this demo I am going to map a local SQL staging database’s table of Contact records to the CRM Contacts entity (but I could be reading those source records from a text file or other ODBC/OLE DB source).
You pick your data source type:
And then provide the connection string details:
Similar story for the destination connection. The same connection types are offered:
Here’s my connection:
Once you have specified the connection paths to your Source and Destination systems you need to provide a few more connection details and explain how Scribe should match existing records and what to do when records are matched. Lots of options there:
Last part of the mapping is to drop down to field level and map the individual fields. It’s a drag and drop experience. If some transformation is required you can easily add formulas to each mapping through a nice formula editor:
And we’re done with the mapping. Last step is to set the schedule. We see the same options here that we saw earlier:
And that’s it. I can wait for the schedule to kick off the synchronisation or I can run it manually.
In my example I have a local SQL table containing this data:
And this is how that data looks once my Scribe solution successfully loads it into CRM:
Here’s the Job History back in Scribe Online:
That gives you a feel for what a one-off data migration would be like. My solution was created with integration in mind though. I configured a unique key between my source and destination systems and informed Scribe how I wanted it handle updates vs. inserts.
I’ve also told Scribe to refer to the ‘UpdatedLast’ date field in my source table when performing subsequent synchronisations so that it only bothers to process records where that field is ahead of the last run date (indicating a recent change).
To simulate a subsequent interface run I change the phone number on 2 records and then bump the UpdatedLast field forward:
I kick the synchronisation job off again and see those 2 source record updates successfully synchronised up to CRM:
There you go, a quick run through on the basics of Scribe Online Synchronisation services. I must say I found the whole experience very intuitive and robust. The user interface and the end-to-end configuration user experience speaks volumes for the maturity of Scribe’s development processes and quality of their application architecture.
As at May 2012 the advertised price for Scribe Online Synchronisation Services is $99 USD / month, with an upper limit of 1.5 million records processed per annum. The pricing goes up in step increments from there to e.g.: $1000 for 100 million records.
Unlike with their Scribe Insight product there is no 60 day license option here to support migration scenarios - the minimum term is 12 months. But still, $99 x 12 is not a huge outlay. Something to consider.
Update: Since writing this post I have had a look at Scribe Insight, Scribe’s traditional on-premise solution for Integration and Migration. You can read all about that here. Scribe Online’s Synchronisation services is definitely a more pleasant user interface to be working in but you also lose a lot of functionality compared to that available with Scribe Insight.
In that Scribe Insight post I ran through a more complicated scenario where I have both Accounts and Contacts to migrate and the Contacts are parented by the Accounts and the Accounts each has a Primary Contact. Plus, I have a pick list field with source values that need to be mapped. I’ve since come back to Scribe Online and worked through the same scenario, here are some notes:
For the pick list mapping I populate a Mapping table in Scribe Online:
And then when I map my source and destination fields I add a Lookup formula that references this table (similar to doing a vlookup in Excel). I’d like to show you a screenshot now but my Scribe Online has decided to stop working. It’s driving me mad just at the moment to be honest. Anyway, to link the Contacts to their parent Account records there is another type of Lookup formula available which will essentially do a vlookup into your target system. So, the capability looks to all be there.
Please feel free to share your experience and opinions via the comments below.