Condensed notes from the Build about where Microsoft technology is going over the next year

Here is a summary of what I learned at the Build. I went to a lot of trouble this year to attend this conference which was not easy considering that the agenda was empty up to the last minute. But it was certainly worth while because Microsoft reveled where it is going with its technology which included a unified user experience from mobile devices through to large television sized devices.

CNN “Microsoft unveils a radically redesigned Windows 8”

This technology will only be available in about a year from now, probably industry will adopt this in perhaps 2 years from now. This conference is important because no strategy would be complete without an idea of where Windows is going because it is our main operating system. It is necessary because our software engineers need to gather the skills needed to take advantage of this technology and it is necessary in order to make informed decisions over if and how to integrate other technology such as iPhone and iPad.

Windows 8 Metro Style

We can expect a shift towards touch base applications over the coming years. Microsoft is of the opinion that all screens will be touched enabled and the ones that are not will feel antiquated when used. To make use of the Windows 8 metro style the user will have to adapt by learning new gestures and become familiar with the concept of having a less cluttered desktop using active Tiles. This means a “Re imagine your application” when making an application in Metro style. It was emphasized that there are a number of applications where the chrome style is simply the most appropriate meaning that this style of UI will continued to be supported. The difference between metro and chrome will go along the lines of the difference between a DOS prompt and Windows ie an expert system that requires training to a intuitive system that requires minimal training. Chrome seems to be based on the philosophy that “Less is more”.

Performance and energy saving has become a key factor in the design of Windows 8. A cold start up takes about 20 seconds. To make this possible the number of running processes has been minimized and that tasks that are not currently in view are suspended. This has an impact on the design and development of applications because they only have a few seconds to persist there state before the task is suspended. It is now possible to boot a Windows 8 client directly from a USB memory stick. Also the first bleeding edge problems with VS are coming to light eg

http://blog.galasoft.ch/archive/2011/09/25/quick-tip-killing-a-metro-style-app-in-windows-8.aspx?utm_source=twitterfeed&utm_medium=twitter&utm_campaign=Feed%3A+galasoft+%28Laurent+Bugnion+%28GalaSoft%29%29

There are some features that may take more time to be adopted by the industry. Such as location aware applications that can make use of local devices or the touch sensor that perform a kind of electronic hand shake and setup a link between the 2 devices for collaboration or simply to exchange an electronic business card.

The Windows 8 user experience is based on a fast and fluid immersive and full screen user experience. Applications communicate with each other through “charms” that are implemented via shared contracts. The same intuitive interface will be used across all Windows devices from mobile devices through desk tops and televisions.

The result is a non cluttered canvas with controls. Commands that are frequently used go on the canvas, all others are revealed through the edge gestures. The metro style desktop consists of Tiles that provide simple text or images via pre defined templates and come in 2 sizes. Secondary tiles are created by pinning the content of the application and have the same capabilities as the main tile. They provide a deep link within the application. Live tiles keep people connected to your app and male it more likely the application is put on the first page.

Notifications appear via a toaster icon that appears for a short period of time and user must opt onto it. They use Windows Push Notification Service WNS and can update any time your application is running

There is a new layer that sits directly on the Windows Kernal called WinRT. This is a clean API that has no duplication of Runtime API and all obsolete or inapplicable APIs have been removed. Metro style apps for c# and VB.net can interact directly with WinRT APIs or via .Net for Metro style apps via the CLS and Win32 APIs.

To publish a metro app there is a pipeline of checks that guarantee that an application quality. This consists of Pre Processing, Security tests, Technical Compliance, Content compliance, Signing and publishing.


x86, 64, and ARM processors will all be supported. This has a big positive implication that all mobile devices running on ARM processors will support these applications.

No solid time line was given. Some guesses included perhaps it being available about this time next year. The next milestone is Beta, the RC, then RTM and then GA

Windows 8 Server

I did not attend many Windows 8 server sessions. But a lot of progress has been made on performance. I also believe that the HyperV virtualization has been improved by lessons learned in Azure. This is a way to make low cost hard disks available in a SAN. This can perform very fast because it is possible to make use of more than 1 network card through teaming. It is also possible to change a system drive on the fly.
.Net 4.5

In the key note it is really sure that Silverlight is definitely not dead. Microsoft has been working on making it possible to program the UI in languages programmers might choose. This includes the HTML5/Java script as well as C++. Microsoft is in the process of making a metro style implementation of office. From what I could determine this is going to be made using HTML5 for the view part of the presentation layer and Java script as the model art of the view model. All this sits on WinRT APIs which sits directly on Window Kernel Services. At first site this has been misinterpreted last year as a move away from Silverlight. This is not the case because the idea is to allow the programmer to express themselves in any appropriate programming language and this was demonstrated by including demonstrations in C and C++. I asked some experts whether the HTML5/JavaScript implementation would use the MVVM pattern. The answer was that this question had been asked by a lot of people and that it should be possible but there are no examples for this yet.

Entity framework EF 4.5 now includes Enums, SQL Server and Azure features such as spatial functions
In .Net 4 data manipulation was made by first starting with the data and then setting up the computation. TPL Dataflow allows the computation to be first setup and then the data

Other Parallel Computing Additions

Combinators Task.WhenAll, Task.WhenAny
Timer integration Task.Delay(TimeSpan), CancellationTokenSource.CancelAfter(TimeSpan)
Task schedulingConcurrentExclusiveSchedulerPair
Fine-grained control DenyChildAttach, HideScheduler, LazyCancellation, EnumerablePartitionerOptions
ThreadLocal.Values
PERFORMANCE (“it’s just faster”)
The garbage collector has been improved. A multi-cored JIT with pre-fetch options makes ASP.Net start 35% faster.

.Net 4.5 is an in place upgrade from .Net 4.0

VS2011

Firstly VS2011 can open VS2010 project without changing the format.

There is a 3D editor that has the capability of breaking down each transformation that is made to a 3D object. It is not designed for creating these objects but can manipulate them. Improvements have been made on the IDE experience for C++ programmers. This includes unit tests, and color intellisence pickers etc.

Visual Studio 11 is the tool for Windows 8 and supports VB.NET, C#, C++ & HTML5/JS. There is a new designer Available today built on shared architecture with Expression Blend. When shelving changes information over which windows are open are saved together with the shelf set and are re established when opening the shelve set the windows are reopened in their original position. There is also a document well that avoids each time a file is clicked upon that a new document window is created. There is a XAML editor with IntelliSense. It is now possible to use C# code directly in Jscript and have changes in the generated HTML fed back into the code behind files.

There is a static code analysis feature that allows you to look and replace chunks of repeated code.
PowerPoint templates enable the mocking of an interactive UI.

Asynchronous programming is necessary for creating responsive fast and fluent applications. Here are some new features coming with .Net 4.5

Asynchronous programming models
Windows Runtime: IAsyncOperation
.NET Framework: Task
JavaScript: Promises
All are objects representing “ongoing operations”
All use callbacks to signal completion of operation
Challenge: Callbacks turn your code inside out
Insight: Automatic transformation to callbacks is possible

Asynchronous methods automatically transform normal code into a callback state machine eg

public async Task GetXmlAsync(string url) {
var client = new HttpClient();
var response = await client.GetAsync(url);
var text = response.Content.ReadAsString();
return XElement.Parse(text);
}

.Net 5.0 aka Project Roslyn

The .Net 5.0 JIT will be written in C#. This means that it will be possible to call APIs to the pipeline used to compile code. This pipeline comprises of 4 steps

1. Parser
2. Symbols / Meta data export
3. Binder
4. Emitter

Here’s an example

ScriptEngine engine = new ScriptEngine();
Session session = Session.Create();
Engine.Execute(“using System;, session);
Engine.Execute(“for(int I = 0; I < 10; i++) Console.WriteLine(i*i);”,session); Var f = (funcengine.Execute(“new Func(Sqr)”);
For (int i=0; i<10 i++) Console.WriteLine(f(i)); This might look strange but it could be used in allowing users to use a macro style interface to your application. In my case I doubt that I would use this but it is very interesting. I could imagine that this will allow mixing of languages that could be useful in creating fast code. There is also a command line Roslyn C# Compiler that runs in a similar style as for F# and includes intellisence and code tips. To create a reference to another assembly “er” is used, eg: Er “PresentationCore” Here’s an example of translating c# into VB Public VB.SytaxNode Covert( CS.SyntaxTree syntaxTree, IDictionary identifierMap = null,
Bool convertStrings =false)
{
Var text = syntaxTree.Text;
Var mode = syntaxTree.Root;
Var vbText = Convert(text, node, identifierMap, convertStrings);
Return VB.SyntaxTree.ParseCompilationUnit(vbText).Root;
}

There is also a feature for .Net 3.5 on demand that I believe means that we don’t need to install the .Net 3.5 framework when certain obsolete functions are required.

.Net 5 will be a fresh installation

HPC

I have met up with some HPC / Azure MVPs and discussed some of the problems that I encountered when making my cat model in the cloud. The problem was that I was not able to transfer a 60 GB VM into the cloud including a local SQL Server. The suggestion was to use SQL Azure and transfer data via blob storage. In the next couple of months there will be a release of HPC Server that runs of a standard worker role meaning that we don’t need to construct and maintain our own VM. For very large quantities of data we could consider speeding up the queries by using TriadLinq aka Linq to HPC. This works by producing a sealed data block that is replicated between multiple nodes and is then queried upon in parallel. In addition to the Windows server based HPC Scheduler there is an Azure based HPC Scheduler. These are useful because it means that we don’t need to write our own schedulers.

There was a demo of a PC with 4 water cooled graphics cards that had 2500 times the processing power of a Cray. The HPC team uses these machines for numerical tasks. Currently it takes a crack programmer to correctly program such an algorithm, the hope is that this will become abstracted away at some point in the future.

Azure

Azure is a very cheap, elastic and quickly configurable source of compute power and one that we really should take advantage of. I saw some presentations on debugging and branch caching.

ALM Application Life Cycle Management

Attendees of Build where given a golden ticket to an evaluation version of the SaaS version of Team Foundation Server 2010. This contains a many of Application Lifecyle Management features that are all a part of professional software engineering. The Application life cycle is based on 2 parts . The Development cycle where a product backlog is worked upon during sprints to create a working software asset. The Operations which consists of an Ops back log and monitoring. When a bug is discovered a corresponding requirement is made that is sent back to the development backlog. In addition to the existing ALM functions a Code review function was demonstrated in the context of a development team with inline or side by side code comparison

Currently both TFS Server and TFS SaaS are capable of:
Work items, Source Control and Build
Agile Product/Project Management
Test Case Management
Heterogeneous Development

TFS SaaS has the advantage when:
Near Zero setup and administration
Collaborate with anyone from anywhere

TFS Server has the advantage when
Virtual Test Lab Management
SharePoint Integration
Data Warehouse and Reporting

User experience in an L39 Jet

This has got to be the ultimate user experience. I am sitting behind the pilot in this Russian designed L39 Jet. If anyone would like to try it is quite expensive but is something you will never forget. Here are some links:

• http://www.incredible-adventures.com/l-39-flights.html


• http://www.mach1aviation.com/varied/l-39-safety-video.htm

My pilot was Dave Riggs who is a hollywood stunt pilot for films like XXX, Iron Man, Lord of War. We flew together with David LaFaille and did some combat flying and some low level flight avoiding radar in canion, by low level I mean we flew 4m of the ground. My stomach had a problem digesting this new user experience so we finished the day with some formation flying.

First Impression of the Build

Just before the build started I had a chance to talk to a user experience MVP and to a company that has made some compelling applications for Health care and emergency services. This was quite fortunate because the majority of this conference is all about a revamp of the way that the user experiences Windows. Basically over the coming years there will be a shift towards touch based applications. The vision is that all screens will be touched enabled and the ones that are not will feel antiquated when used.

It was very interesting talking to Christian Moser before the conference. He is writing a book on User Experience where controls are represented by a kind of design pattern. For a control to be successful it’s use must be intuitive which means that it’s use should be common knowledge. This means that you should think twice before inventing a new way of interaction. In the context of Windows 8 this is going to be interesting because there are a new gestures that have been invented. Also the concept of having a less cluttered desktop with active tiles has an impact on the design of applications. I heard a lot the buzz word of “Re imagine your application”. There are a lot of sessions around this Metro style. It was also emphasized that for some applications the chrome style is simply the most appropriate UI and that this will be continued to be supported. While I was talking to Christian he mentioned that in his book he will be describing some aspects of expert users. Before the build the example on an expert user was the airline booking system which is a command line interface that requires a lot of training but is super efficient. I could imagine that the difference between metro and chrome would go down the same lines. Chrome seems to be based on the philosophy that “Less is more”, according to my friends at BlackMarble most users really appreciate something like this.

In the key note it is really sure that Silverlight is definitely not dead. In fact MS has been working on making it possible to program the UI in languages programmers might choose. This includes the HTML5/Java script as well as C++. Performance and energy saving has become a key factor in the design of Windows 8.

Before the conference I was talking to some friends at Black Marble about how an application can determine where it is. For example in a hospital application your device should know what other devices are nearby. This can be done in several ways. One way is to use the SID of the wireless network. This works quite well but there can be problems of leakage of signal from one zone to another. Another method that could be used is GPS repeaters. This is essentially like setting up a GPS satellite within your building. This could be particularly interesting for emergency services in conjunction with some mapping on a Surface 2 system to keep track of where there fire fighters are etc.

Part of the Windows 8 slate included a touch sensor. The concept is that two people are using an application and what to collaborate. So they physically touch there slates and an electronic handshake occurs that enables further communication over Bluetooth etc. This also works with sensors which I believe work by induction and need no battery. Another example is a business card that when touched opens a web browser to a website. Another example starts an application that if not present will install it first then run it.

There is really a lot going on. I have met up with some HPC / Azure MVPs and discussed some of the problems that I encounted when making my cat model in the cload. Basically my problem was that I was not able to transfer a 60 GB VM into the cloud including a local SQL Server. The suggestion was to use SQL Azure and transfer data via blob storage. Also to think about TriadLinq which is a way to spread out queries onto multiple VMs. The thing is Azure is simply way cheaper than using in house servers. So I will attend some sessions that will look at these in a little more detail. On the Slate PC is a pre beta version of Windows 8 and Visual Studio 2011. I have also got an evaluation version of the SaaS version of Team Foundation Server 2010 which contains a lot of Application Lifecyle Management features that are all a part of professional software engineering.

Some notes on the application of Cloud computing

I am just about to go on holiday. So before I forget here are ideas about the application of Cloud Computing to Catastrophe Modeling.

Cloud computing makes sense for Cat simulations for various reasons. The costs are very reasonable, for example all my tests came to about 82 CHF compared to the fixed cost of a VM in a datacenter that mounts up to several thousand franks. Microsoft can achieve this by making the administration of the hundreds of thousands of VMs as automated as possible thus getting an economy of scale that we don't have in smaller data centers. Creating the cloud VMs took a few minutes as opposed to weeks via our internal/external processes. I could imagine that WMWare will improve the provisioning of virtual machines and that one day this may be as simple as filling out a web site. But we don't have such a system yet and if we did we would have a much more limited pool of servers and therefore higher costs than if the whole thing was in the cloud. One last aspect was that surprisingly the end to end time to process the bench mark cat model in the cloud took less time than on our on-premis servers.

There are some different possibilities how number crunching processes can be implemented. I have seen solutions that can call an executable, if this executable needs something to be installed on the machine it is possible to configure setup tasks that install software as the VM is being instantiated. In the context of my modeling platform I think I would implement the job submission in the same way as in the prototype I described earlier. By this I mean I would transfer the data to be simulated as a blob and once completed add a reference to this data in a Queue that the work roles pole for work. The difference would be instead of poling the results queue I would expose an on premise WCF service over http and call this from the cload using claims based tokens. I would use Queues for status information because they have an intrinsic order, but I would send this information via one way calls to the on premise web services. Since security is made using claims based security the role of the firewall changes slightly. The reason is that cloud apps need to connect via http or https to internal services and that the services and not the firewall will carry out authentication. So there is a shift of responsibility from the firewall the services

From my experience of uploading custom VMs into the cloud I am not sure how well HPC with burst into Azure works in practice. I will follow up on this at the Build conference.

Thinking about how Cloud computing can be integrated into an organization there are 3 aspects to consider Network, Storage and Compute.

A means of synchronizing data between office and mobiles devices brings substantial benefits. Some years ago I tried a CTP of Microsoft Mesh which enabled the synchronization of data between devices. Apple recently released a similar cloud service. There was a demo at the PDC which showed a business man losing his PC but because all his configuration and working data is continuously in synch it was possible to take a new PC and carry on where he left off. Since the CTP I have seen a version of Mesh in Hotmail and Office 360. Although it is difficult to come up with a single business case where this is useful it does add to productivity because it improves the functionality of the environment that we work in.

On the network idea there are datacenters around the world making a global presence much easier to maintain. On the other hand there is latency in getting data to and from the cloud. This makes the SaaS Team Foundation Server offering very interesting.

I think cloud computing will bring about a shift from relational databases to a more object orientated data storage model. Relational databases are not intrinsically more performant. At present I believe the performance of properly dimensioned on premise sql servers out-perform the cloud sql servers. This will probably change SQL Azure developes and as technologies like TriadLinq become available enabling the distribution of the workload needed in carrying out database queries. The reason is that a central sql server is a bottle neck, where as in a cloud the compute needed to carry out queries is by design made for scalability.

With this in mind I think it's worthwhile to encourage the development of web based apps or apps that can be deployed from the web because these fit easily into the Cloud. We would need to manage the scope of the broad set of mobile devices that may or may not be supported in our environment. In particular Rapid Application Development in the context of Microsoft orientated companies which would favor Windows mobile devices even though these devices have not been on the market long enough to really take a large market share. To develop iPad and iPhone applications means needing an Apple workstation learning C and a whole new API.

When looking at data It is difficult to categorize data into security groups. The reason is that depending on the application the data either be sensitive or non sensitive depending on it's context. It is therefore more practical to make security groups for applications

Parallel.For seems to have a large overhead compared to a simpler implementation

I implemented the following alternative to the parallel for using threads:

        static void MyParallelFor(int fromInclusive, int toExclusive, Action<int> body)
        {
            int numProcs = Environment.ProcessorCount;
            using (CountdownEvent ce = new CountdownEvent(numProcs))
            {
              int rangeSize = (toExclusive - fromInclusive) / numProcs;
              for (int p = 0; p< numProcs; p++)
              {
                  int start = rangeSize * p;
                 //int end = start + rangeSize;
                 int end = p == numProcs-1 ? toExclusive : start + rangeSize;
                 ThreadPool.QueueUserWorkItem(delegate
                   {
                      for (int i = start; i < end; i++) body(i);
                      ce.Signal();
                    });
              }
              ce.Wait();
            }
        }

I then tested this on the benchmark earthquake model as described in my last blog entry. Since the signature of the parallel for and myParallelFor are identical the comparison can be made by commenting out the implementation that is not under test.

            for (i = 1; i <= ns; i++) // Outer loop going through lots of lines of exposure
            {
                loss[i] = 0;

                    //Parallel.For(1, nr,
                    MyParallelFor(1, nr, // Inner loop going through many events
                         (jj) =>
                         {
                            // Some maths to determine the mean damage ratio (see previous blog entry for details)
                            localsum[jj] = mdr * value[jj];
                         }

                         });
                 loss[i] = localsum.Sum();
            }

I then ran the above code on a physical machine with ProLiant DL580 G5 12 Cores 2.66 GHz Intel Xeon(R) X7460 with 8188 MB RAM

Parallel.For took about 114 seconds with all 12 CPUs running evenly at about 70%.
MyParallelFor took about 71 seconds with 11 CPUs at 30% and 1 CPU at 80%,

I found this difference quite surprising as it seems that the parallel.for implementation in System.Threading.Tasks seems to have a very large overhead. Not only does it take longer but the cpu’s are working harder. Using reflector I could see that the implementation of private static ParallelLoopResult ForWorker was quite extensive. When I have some time I will add some buffering in the localsum to check whether the problem is caused by cache invalidation. 

Results of runing a simplified earthquake model in the cloud

Here are some preliminary results of how long it takes to upload and run 10000 iterations of a cat model under various worker role configurations

Extra Large with 8 cores took 26s
Large with 4 cores took 34s
Medium with 2 cores took 49s
Small with 1 core took 82s

For comparison here are some run times on a D20 desktop

VB.NET on a D20 compiled with VS2010 running on 1 thread  took  104s
F90 on a D20 compiled with Intel running on 1 thread took 162s
C++ on a D20 compiled with Intel with optimization on 1 thread took 58 s

The time needed to upload exposure and events took about 100s Within the cloud it took about 1s to download from blob storage

The initial flat file text files had the following sizes and number of points

Exposure file 7.3MB    385560 Points
Event Catalogue 4.8MB  194510 Events

To reduce the run times we reduced the number of events from 194510 to 10000. So this means the actual run times are 20 times longer.

I have not received the bill for these tests but I think it is probably quite cheap. I think there is a lot to be said for cloud computing because I only had to worry about my code, the infrastructure was completely abstracted away. In fact setting up a new node takes about 8 minutes with a very friendly billing terms that you pay only for the time that the role is active.

Numerical calculations in Azure continued

After some help from Steve Spencer I figured out the best way to debug the WorkerRole was not to do the whole thing in unit tests but to pressed the play button on the WorkerRole and use a unit test to feed it with data. It turned out the problem was within the blob storage code. I still have some work to refactor the worker role but I want to get some results so I am going to leave it as it is for the moment:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Threading;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;
using Microsoft.WindowsAzure.StorageClient;
using AzureHelper;
using System.Configuration;
using System.IO;
using System.Text;
using CatModel;
using Contracts;
using Entities;
using Infrastructure;

namespace WorkerRole1
{
    public class WorkerRole : RoleEntryPoint
    {
        private String localPath;
        private String calcsimExePath;

        private EQModel eQModel { get; set; }

        private DateTime Start { get; set; }
        private DateTime Finish { get; set; }
        private TimeSpan Stopwatch { get; set; }
        public ILog Log { set; get; }

        private AzQueueHandler jobInputQueue;
        private AzQueueHandler jobOutputQueue;
        private AzQueueHandler jobLogQueue;

        private AzBlobHandler jobInputBlog;

        private AzBlobHandler dataStoreExposureX;
        private AzBlobHandler dataStoreExposureY;
        private AzBlobHandler dataStoreExposureV;

        private AzBlobHandler dataStoreCatalogSm;
        private AzBlobHandler dataStoreCatalogSx;
        private AzBlobHandler dataStoreCatalogSy;
        private AzBlobHandler dataStoreCatalogSd;

        private AzBlobHandler jobLossBlob;
        private AzBlobHandler jobLogBlob;
        private AzBlobHandler jobOutputBlog;

        private int idleCount;
        private int idleMax;
        private int idleSleepTime;

        public override bool OnStart()
        {
            // Set the maximum number of concurrent connections
            ServicePointManager.DefaultConnectionLimit = 12;

            // For information on handling configuration changes
            // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

            return base.OnStart();
        }

        private void Init()
        {
            AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
            idleCount = 0; // how many times the instance have been idle
            idleMax = 10;   // After 10 times (5 minutes = 30 secs *10) being idle, then die
            idleSleepTime = 30 * 1000; // sleep for this number of seconds between each queue poll

            var accountName = ConfigurationManager.AppSettings["AzureAccountName"];
            var accountKey = ConfigurationManager.AppSettings["AzureAccountKey"];

            jobInputQueue = new AzQueueHandler(ConfigurationManager.AppSettings["JobInputQueue"], accountName, accountKey);
            jobOutputQueue = new AzQueueHandler(ConfigurationManager.AppSettings["JobOutputQueue"], accountName, accountKey);
            jobLogQueue = new AzQueueHandler(ConfigurationManager.AppSettings["JobLogQueue"], accountName, accountKey);
            jobInputBlog = new AzBlobHandler(ConfigurationManager.AppSettings["JobInputBlob"], accountName, accountKey);

            jobLogBlob = new AzBlobHandler(ConfigurationManager.AppSettings["jobLogBlob"], accountName, accountKey);
            dataStoreExposureX = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreExposureX"], accountName, accountKey);
            dataStoreExposureY = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreExposureY"], accountName, accountKey);
            dataStoreExposureV = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreExposureV"], accountName, accountKey);

            dataStoreCatalogSm = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreCatalogSm"], accountName, accountKey);
            dataStoreCatalogSx = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreCatalogSx"], accountName, accountKey);
            dataStoreCatalogSy = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreCatalogSy"], accountName, accountKey);
            dataStoreCatalogSd = new AzBlobHandler(ConfigurationManager.AppSettings["dataStoreCatalogSd"], accountName, accountKey);

            jobLossBlob = new AzBlobHandler(ConfigurationManager.AppSettings["JobLossBlob"], accountName, accountKey);
            jobOutputBlog = new AzBlobHandler(ConfigurationManager.AppSettings["JobOutputBlob"], accountName, accountKey);

            eQModel = new EQModel();
            this.Log = new Log("WorkerRole");

            localPath = Environment.CurrentDirectory;

            // pull EXE file from blob storage to local file system
            calcsimExePath = System.IO.Path.Combine(localPath, "Startup\\calcsim.exe");
        }

        public override void Run()
        {
            // This is a sample worker implementation. Replace with your logic.
            Trace.WriteLine("WorkerRole1 entry point called", "Information");
            Init();

            TraceInfo("AzJobHost::Run() Azure Instance ID {0}, DeploymentId {1}", RoleEnvironment.CurrentRoleInstance.Id, RoleEnvironment.DeploymentId);

            bool more = true;
            //Stopwatch swWrkRoleLifetime = Stopwatch.StartNew();

            // msg pump loop
            string id = ""; string popId = "";
            while (more)
            {
                id = ""; popId = "";
                string msg = jobInputQueue.GetMessage(ref id, ref popId, false);

                if (msg == null)
                {
                    idleCount++;
                    if (idleCount >= idleMax)
                        more = false;
                    else Thread.Sleep(idleSleepTime);
                }
                else
                {
                    ProcessMsg(id, popId, msg);
                }
            }

            //swWrkRoleLifetime.Stop();
            //TraceInfo("AzJobHost::Exit(). Execution time {0}", swWrkRoleLifetime.Elapsed);
        }

        private bool ProcessMsg(string id, string popId, string msg)
        {
            bool rc = true;

            //Stopwatch sw = Stopwatch.StartNew();
            TraceInfo("AzJobHost::ProcessMsg( '{0}', '{1}') - Azure Instance Id: {2}", id, msg, RoleEnvironment.CurrentRoleInstance.Id);

            List<double> x = new List<double>();
            List<double> y = new List<double>();
            List<double> v = new List<double>();
            List<double> sm = new List<double>();
            List<double> sx = new List<double>();
            List<double> sy = new List<double>();
            List<double> sd = new List<double>();

            Start = DateTime.Now;
            dataStoreExposureX.RecieveDataFromStorage<double>(msg, out x);
            dataStoreExposureY.RecieveDataFromStorage<double>(msg, out y);
            dataStoreExposureV.RecieveDataFromStorage<double>(msg, out v);
            Finish = DateTime.Now;
            Stopwatch = Finish.Subtract(Start);
            this.Log.LogMessage(String.Format("Time to Upload Exposure to model {0} milliseconds",  Stopwatch.TotalMilliseconds), Stopwatch.TotalMilliseconds);

            Start = DateTime.Now;
            dataStoreCatalogSm.RecieveDataFromStorage<double>(msg, out sm);
            dataStoreCatalogSx.RecieveDataFromStorage<double>(msg, out sx);
            dataStoreCatalogSy.RecieveDataFromStorage<double>(msg, out sy);
            dataStoreCatalogSd.RecieveDataFromStorage<double>(msg, out sd);

            Finish = DateTime.Now;
            Stopwatch = Finish.Subtract(Start);
            this.Log.LogMessage(String.Format("Time to Upload Catalog to model {0} milliseconds", Stopwatch.TotalMilliseconds), Stopwatch.TotalMilliseconds);

            List<double> losses = new List<double>();

            Start = DateTime.Now;
            eQModel.RunModel(x, y, v, sm, sx, sy, sd, out losses);
            Finish = DateTime.Now;
            Stopwatch = Finish.Subtract(Start);
            this.Log.LogMessage(String.Format("Time to Run Japan Earthquake {0} milliseconds", Stopwatch.TotalMilliseconds), Stopwatch.TotalMilliseconds);

            this.Log.LogMessage("=== Appending Model Log to Server Log ====");

            List<LogEvent> ModelLogs = eQModel.log.GetLogs();
            this.Log.Add(ModelLogs);

            jobLossBlob.SendDataToStorage<double>(msg, losses);
            jobLogBlob.SendDataToStorage<LogEvent>(msg, this.Log.GetLogs());

            jobOutputQueue.PutMessage(msg);

            jobInputQueue.DeleteMessage(id, popId);
            return rc;
        }

        private void TraceInfo(string format, params object[] args)
        {
            string msg = string.Format(format, args);
            Trace.WriteLine(msg, "Information");
        }

        private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)
        {
            // If a configuration setting is changing
            if (e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange))
            {
                // Set e.Cancel to true to restart this role instance
                e.Cancel = true;
            }
        }

        void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
        {
            TraceInfo("AzJobHost::UnhandledException: " + (Exception)e.ExceptionObject);
            RoleEnvironment.RequestRecycle();
        }

    }
}