Cross platform applications with HTML5/JavaScript and how this ties in with SOLID OO principles

I have just started to think about writing applications that are cross platform impendent ie that can run on devices like iPad. Here are some notes about this subject
Over the last few years I have made extensive use of SOLID. OO programmin principles as I developed C# applications in .Net.

Where
SRP: The Single Responsibility Principle A class should have one, and only one, reason to change.
OCP: The Open Closed Principle You should be able to extend a classes behavior, without modifying it.
LSP: The Liskov Substitution Principle Derived classes must be substitutable for their base classes.
ISP: The Interface Segregation Principle Make fine grained interfaces that are client specific.
DIP: The Dependency Inversion Principle Depend on abstractions, not on concretions.


My first reaction was to reapply these principles in JavaScript. So I started to think of interfaces, dependency injection containers, mocking frameworks and Test Driven development. It turns out that there is a fundamental difference between C# and JavaScript in the sence that c# is class based and JavaScript is prototypical. So it is difficult to apply the notion of interfaces and abstraction in the world of JavaScript.

So the question is what principles are relevant to a Prototypical language over a Class based OO language. As far as I can see :

S Applicable
O Not Applicable
L Applicable
I Not Applicable
D Not Applicable

One principle that applys to both programming paradigms is DRY which stands for Don't Repeat Yourself

JavaScript Object Oriented Programming include classes, inheritance, and scope that can be used encapsulate, support namespaces, and avoid collisions. Actually JavaScript does not have a class entity BUT it implements the pattern of classes. The difference is in the inheritance model. In other object oriented languages, class is an actual data type that represents the blueprint for creating objects. In JavaScript, although we can use Functions to simulate an object blueprint, they are just in fact objects themselves. These objects are then used as models (aka prototypes) for other objects. Applying the concept of prototypal inheritance allows us to create “subclasses”, or objects that inherit the properties of another object. This becomes particularly useful when we want to use the methods of another object with some slight modifications.


The next thing I was thinking about is that after programming a while in Xaml I preffer to use the MVVM model over the MVC. This is in MVC the Controller is tightly coupled to the view. In practical terms this means that every time you need to change a control on the view the controller needs to change things like how this effects all the other controls. Where as in MVVM the View is completely separate from the VM and thus a change to a control results in minimal update to the ViewModel. It turns out that there is a JavaScript library called KnockOut.js that makes the MVVM pattern possible

Here are some examples of JavaScript. It can be seen that JavaScript is far from just a begineers language. JavaScript's prototypal OOP is flexible and it does not limit the programmer to things like static typing.


Here is an example of how to create your own foreach function:

Array.prototype.foreach.function(callback)
{
for (var i=0; i= companrand:}
"=" : function(comporand){return function(e){return e= companrand:}
}
}

a.set(op[">="](5).foreach(alert);

This is called Curring after Mr Curry.
btw If you uise this in DOS you will need toescape the > to ^>

----
Here is a way tobiuld up the arrays:
Number.protype.inputModule
{
UpTo : function(Upeer,step)
{
var a[]
for(var i = this.valueof(Upper); j return a;
}
}

(1).Upto(20,2).....

This is called High Order Programming

Here's a link to some further examples: http://www.w3schools.com/js/

The next version of Visual Studio will have a lot more support for JavaScript development, so this should reduce the barrier to trying this out...

Condensed notes from the Build about where Microsoft technology is going over the next year

Here is a summary of what I learned at the Build. I went to a lot of trouble this year to attend this conference which was not easy considering that the agenda was empty up to the last minute. But it was certainly worth while because Microsoft reveled where it is going with its technology which included a unified user experience from mobile devices through to large television sized devices.

CNN “Microsoft unveils a radically redesigned Windows 8”

This technology will only be available in about a year from now, probably industry will adopt this in perhaps 2 years from now. This conference is important because no strategy would be complete without an idea of where Windows is going because it is our main operating system. It is necessary because our software engineers need to gather the skills needed to take advantage of this technology and it is necessary in order to make informed decisions over if and how to integrate other technology such as iPhone and iPad.

Windows 8 Metro Style

We can expect a shift towards touch base applications over the coming years. Microsoft is of the opinion that all screens will be touched enabled and the ones that are not will feel antiquated when used. To make use of the Windows 8 metro style the user will have to adapt by learning new gestures and become familiar with the concept of having a less cluttered desktop using active Tiles. This means a “Re imagine your application” when making an application in Metro style. It was emphasized that there are a number of applications where the chrome style is simply the most appropriate meaning that this style of UI will continued to be supported. The difference between metro and chrome will go along the lines of the difference between a DOS prompt and Windows ie an expert system that requires training to a intuitive system that requires minimal training. Chrome seems to be based on the philosophy that “Less is more”.

Performance and energy saving has become a key factor in the design of Windows 8. A cold start up takes about 20 seconds. To make this possible the number of running processes has been minimized and that tasks that are not currently in view are suspended. This has an impact on the design and development of applications because they only have a few seconds to persist there state before the task is suspended. It is now possible to boot a Windows 8 client directly from a USB memory stick. Also the first bleeding edge problems with VS are coming to light eg

http://blog.galasoft.ch/archive/2011/09/25/quick-tip-killing-a-metro-style-app-in-windows-8.aspx?utm_source=twitterfeed&utm_medium=twitter&utm_campaign=Feed%3A+galasoft+%28Laurent+Bugnion+%28GalaSoft%29%29

There are some features that may take more time to be adopted by the industry. Such as location aware applications that can make use of local devices or the touch sensor that perform a kind of electronic hand shake and setup a link between the 2 devices for collaboration or simply to exchange an electronic business card.

The Windows 8 user experience is based on a fast and fluid immersive and full screen user experience. Applications communicate with each other through “charms” that are implemented via shared contracts. The same intuitive interface will be used across all Windows devices from mobile devices through desk tops and televisions.

The result is a non cluttered canvas with controls. Commands that are frequently used go on the canvas, all others are revealed through the edge gestures. The metro style desktop consists of Tiles that provide simple text or images via pre defined templates and come in 2 sizes. Secondary tiles are created by pinning the content of the application and have the same capabilities as the main tile. They provide a deep link within the application. Live tiles keep people connected to your app and male it more likely the application is put on the first page.

Notifications appear via a toaster icon that appears for a short period of time and user must opt onto it. They use Windows Push Notification Service WNS and can update any time your application is running

There is a new layer that sits directly on the Windows Kernal called WinRT. This is a clean API that has no duplication of Runtime API and all obsolete or inapplicable APIs have been removed. Metro style apps for c# and VB.net can interact directly with WinRT APIs or via .Net for Metro style apps via the CLS and Win32 APIs.

To publish a metro app there is a pipeline of checks that guarantee that an application quality. This consists of Pre Processing, Security tests, Technical Compliance, Content compliance, Signing and publishing.


x86, 64, and ARM processors will all be supported. This has a big positive implication that all mobile devices running on ARM processors will support these applications.

No solid time line was given. Some guesses included perhaps it being available about this time next year. The next milestone is Beta, the RC, then RTM and then GA

Windows 8 Server

I did not attend many Windows 8 server sessions. But a lot of progress has been made on performance. I also believe that the HyperV virtualization has been improved by lessons learned in Azure. This is a way to make low cost hard disks available in a SAN. This can perform very fast because it is possible to make use of more than 1 network card through teaming. It is also possible to change a system drive on the fly.
.Net 4.5

In the key note it is really sure that Silverlight is definitely not dead. Microsoft has been working on making it possible to program the UI in languages programmers might choose. This includes the HTML5/Java script as well as C++. Microsoft is in the process of making a metro style implementation of office. From what I could determine this is going to be made using HTML5 for the view part of the presentation layer and Java script as the model art of the view model. All this sits on WinRT APIs which sits directly on Window Kernel Services. At first site this has been misinterpreted last year as a move away from Silverlight. This is not the case because the idea is to allow the programmer to express themselves in any appropriate programming language and this was demonstrated by including demonstrations in C and C++. I asked some experts whether the HTML5/JavaScript implementation would use the MVVM pattern. The answer was that this question had been asked by a lot of people and that it should be possible but there are no examples for this yet.

Entity framework EF 4.5 now includes Enums, SQL Server and Azure features such as spatial functions
In .Net 4 data manipulation was made by first starting with the data and then setting up the computation. TPL Dataflow allows the computation to be first setup and then the data

Other Parallel Computing Additions

Combinators Task.WhenAll, Task.WhenAny
Timer integration Task.Delay(TimeSpan), CancellationTokenSource.CancelAfter(TimeSpan)
Task schedulingConcurrentExclusiveSchedulerPair
Fine-grained control DenyChildAttach, HideScheduler, LazyCancellation, EnumerablePartitionerOptions
ThreadLocal.Values
PERFORMANCE (“it’s just faster”)
The garbage collector has been improved. A multi-cored JIT with pre-fetch options makes ASP.Net start 35% faster.

.Net 4.5 is an in place upgrade from .Net 4.0

VS2011

Firstly VS2011 can open VS2010 project without changing the format.

There is a 3D editor that has the capability of breaking down each transformation that is made to a 3D object. It is not designed for creating these objects but can manipulate them. Improvements have been made on the IDE experience for C++ programmers. This includes unit tests, and color intellisence pickers etc.

Visual Studio 11 is the tool for Windows 8 and supports VB.NET, C#, C++ & HTML5/JS. There is a new designer Available today built on shared architecture with Expression Blend. When shelving changes information over which windows are open are saved together with the shelf set and are re established when opening the shelve set the windows are reopened in their original position. There is also a document well that avoids each time a file is clicked upon that a new document window is created. There is a XAML editor with IntelliSense. It is now possible to use C# code directly in Jscript and have changes in the generated HTML fed back into the code behind files.

There is a static code analysis feature that allows you to look and replace chunks of repeated code.
PowerPoint templates enable the mocking of an interactive UI.

Asynchronous programming is necessary for creating responsive fast and fluent applications. Here are some new features coming with .Net 4.5

Asynchronous programming models
Windows Runtime: IAsyncOperation
.NET Framework: Task
JavaScript: Promises
All are objects representing “ongoing operations”
All use callbacks to signal completion of operation
Challenge: Callbacks turn your code inside out
Insight: Automatic transformation to callbacks is possible

Asynchronous methods automatically transform normal code into a callback state machine eg

public async Task GetXmlAsync(string url) {
var client = new HttpClient();
var response = await client.GetAsync(url);
var text = response.Content.ReadAsString();
return XElement.Parse(text);
}

.Net 5.0 aka Project Roslyn

The .Net 5.0 JIT will be written in C#. This means that it will be possible to call APIs to the pipeline used to compile code. This pipeline comprises of 4 steps

1. Parser
2. Symbols / Meta data export
3. Binder
4. Emitter

Here’s an example

ScriptEngine engine = new ScriptEngine();
Session session = Session.Create();
Engine.Execute(“using System;, session);
Engine.Execute(“for(int I = 0; I < 10; i++) Console.WriteLine(i*i);”,session); Var f = (funcengine.Execute(“new Func(Sqr)”);
For (int i=0; i<10 i++) Console.WriteLine(f(i)); This might look strange but it could be used in allowing users to use a macro style interface to your application. In my case I doubt that I would use this but it is very interesting. I could imagine that this will allow mixing of languages that could be useful in creating fast code. There is also a command line Roslyn C# Compiler that runs in a similar style as for F# and includes intellisence and code tips. To create a reference to another assembly “er” is used, eg: Er “PresentationCore” Here’s an example of translating c# into VB Public VB.SytaxNode Covert( CS.SyntaxTree syntaxTree, IDictionary identifierMap = null,
Bool convertStrings =false)
{
Var text = syntaxTree.Text;
Var mode = syntaxTree.Root;
Var vbText = Convert(text, node, identifierMap, convertStrings);
Return VB.SyntaxTree.ParseCompilationUnit(vbText).Root;
}

There is also a feature for .Net 3.5 on demand that I believe means that we don’t need to install the .Net 3.5 framework when certain obsolete functions are required.

.Net 5 will be a fresh installation

HPC

I have met up with some HPC / Azure MVPs and discussed some of the problems that I encountered when making my cat model in the cloud. The problem was that I was not able to transfer a 60 GB VM into the cloud including a local SQL Server. The suggestion was to use SQL Azure and transfer data via blob storage. In the next couple of months there will be a release of HPC Server that runs of a standard worker role meaning that we don’t need to construct and maintain our own VM. For very large quantities of data we could consider speeding up the queries by using TriadLinq aka Linq to HPC. This works by producing a sealed data block that is replicated between multiple nodes and is then queried upon in parallel. In addition to the Windows server based HPC Scheduler there is an Azure based HPC Scheduler. These are useful because it means that we don’t need to write our own schedulers.

There was a demo of a PC with 4 water cooled graphics cards that had 2500 times the processing power of a Cray. The HPC team uses these machines for numerical tasks. Currently it takes a crack programmer to correctly program such an algorithm, the hope is that this will become abstracted away at some point in the future.

Azure

Azure is a very cheap, elastic and quickly configurable source of compute power and one that we really should take advantage of. I saw some presentations on debugging and branch caching.

ALM Application Life Cycle Management

Attendees of Build where given a golden ticket to an evaluation version of the SaaS version of Team Foundation Server 2010. This contains a many of Application Lifecyle Management features that are all a part of professional software engineering. The Application life cycle is based on 2 parts . The Development cycle where a product backlog is worked upon during sprints to create a working software asset. The Operations which consists of an Ops back log and monitoring. When a bug is discovered a corresponding requirement is made that is sent back to the development backlog. In addition to the existing ALM functions a Code review function was demonstrated in the context of a development team with inline or side by side code comparison

Currently both TFS Server and TFS SaaS are capable of:
Work items, Source Control and Build
Agile Product/Project Management
Test Case Management
Heterogeneous Development

TFS SaaS has the advantage when:
Near Zero setup and administration
Collaborate with anyone from anywhere

TFS Server has the advantage when
Virtual Test Lab Management
SharePoint Integration
Data Warehouse and Reporting

User experience in an L39 Jet

This has got to be the ultimate user experience. I am sitting behind the pilot in this Russian designed L39 Jet. If anyone would like to try it is quite expensive but is something you will never forget. Here are some links:

• http://www.incredible-adventures.com/l-39-flights.html


• http://www.mach1aviation.com/varied/l-39-safety-video.htm

My pilot was Dave Riggs who is a hollywood stunt pilot for films like XXX, Iron Man, Lord of War. We flew together with David LaFaille and did some combat flying and some low level flight avoiding radar in canion, by low level I mean we flew 4m of the ground. My stomach had a problem digesting this new user experience so we finished the day with some formation flying.

First Impression of the Build

Just before the build started I had a chance to talk to a user experience MVP and to a company that has made some compelling applications for Health care and emergency services. This was quite fortunate because the majority of this conference is all about a revamp of the way that the user experiences Windows. Basically over the coming years there will be a shift towards touch based applications. The vision is that all screens will be touched enabled and the ones that are not will feel antiquated when used.

It was very interesting talking to Christian Moser before the conference. He is writing a book on User Experience where controls are represented by a kind of design pattern. For a control to be successful it’s use must be intuitive which means that it’s use should be common knowledge. This means that you should think twice before inventing a new way of interaction. In the context of Windows 8 this is going to be interesting because there are a new gestures that have been invented. Also the concept of having a less cluttered desktop with active tiles has an impact on the design of applications. I heard a lot the buzz word of “Re imagine your application”. There are a lot of sessions around this Metro style. It was also emphasized that for some applications the chrome style is simply the most appropriate UI and that this will be continued to be supported. While I was talking to Christian he mentioned that in his book he will be describing some aspects of expert users. Before the build the example on an expert user was the airline booking system which is a command line interface that requires a lot of training but is super efficient. I could imagine that the difference between metro and chrome would go down the same lines. Chrome seems to be based on the philosophy that “Less is more”, according to my friends at BlackMarble most users really appreciate something like this.

In the key note it is really sure that Silverlight is definitely not dead. In fact MS has been working on making it possible to program the UI in languages programmers might choose. This includes the HTML5/Java script as well as C++. Performance and energy saving has become a key factor in the design of Windows 8.

Before the conference I was talking to some friends at Black Marble about how an application can determine where it is. For example in a hospital application your device should know what other devices are nearby. This can be done in several ways. One way is to use the SID of the wireless network. This works quite well but there can be problems of leakage of signal from one zone to another. Another method that could be used is GPS repeaters. This is essentially like setting up a GPS satellite within your building. This could be particularly interesting for emergency services in conjunction with some mapping on a Surface 2 system to keep track of where there fire fighters are etc.

Part of the Windows 8 slate included a touch sensor. The concept is that two people are using an application and what to collaborate. So they physically touch there slates and an electronic handshake occurs that enables further communication over Bluetooth etc. This also works with sensors which I believe work by induction and need no battery. Another example is a business card that when touched opens a web browser to a website. Another example starts an application that if not present will install it first then run it.

There is really a lot going on. I have met up with some HPC / Azure MVPs and discussed some of the problems that I encounted when making my cat model in the cload. Basically my problem was that I was not able to transfer a 60 GB VM into the cloud including a local SQL Server. The suggestion was to use SQL Azure and transfer data via blob storage. Also to think about TriadLinq which is a way to spread out queries onto multiple VMs. The thing is Azure is simply way cheaper than using in house servers. So I will attend some sessions that will look at these in a little more detail. On the Slate PC is a pre beta version of Windows 8 and Visual Studio 2011. I have also got an evaluation version of the SaaS version of Team Foundation Server 2010 which contains a lot of Application Lifecyle Management features that are all a part of professional software engineering.

Some notes on the application of Cloud computing

I am just about to go on holiday. So before I forget here are ideas about the application of Cloud Computing to Catastrophe Modeling.

Cloud computing makes sense for Cat simulations for various reasons. The costs are very reasonable, for example all my tests came to about 82 CHF compared to the fixed cost of a VM in a datacenter that mounts up to several thousand franks. Microsoft can achieve this by making the administration of the hundreds of thousands of VMs as automated as possible thus getting an economy of scale that we don't have in smaller data centers. Creating the cloud VMs took a few minutes as opposed to weeks via our internal/external processes. I could imagine that WMWare will improve the provisioning of virtual machines and that one day this may be as simple as filling out a web site. But we don't have such a system yet and if we did we would have a much more limited pool of servers and therefore higher costs than if the whole thing was in the cloud. One last aspect was that surprisingly the end to end time to process the bench mark cat model in the cloud took less time than on our on-premis servers.

There are some different possibilities how number crunching processes can be implemented. I have seen solutions that can call an executable, if this executable needs something to be installed on the machine it is possible to configure setup tasks that install software as the VM is being instantiated. In the context of my modeling platform I think I would implement the job submission in the same way as in the prototype I described earlier. By this I mean I would transfer the data to be simulated as a blob and once completed add a reference to this data in a Queue that the work roles pole for work. The difference would be instead of poling the results queue I would expose an on premise WCF service over http and call this from the cload using claims based tokens. I would use Queues for status information because they have an intrinsic order, but I would send this information via one way calls to the on premise web services. Since security is made using claims based security the role of the firewall changes slightly. The reason is that cloud apps need to connect via http or https to internal services and that the services and not the firewall will carry out authentication. So there is a shift of responsibility from the firewall the services

From my experience of uploading custom VMs into the cloud I am not sure how well HPC with burst into Azure works in practice. I will follow up on this at the Build conference.

Thinking about how Cloud computing can be integrated into an organization there are 3 aspects to consider Network, Storage and Compute.

A means of synchronizing data between office and mobiles devices brings substantial benefits. Some years ago I tried a CTP of Microsoft Mesh which enabled the synchronization of data between devices. Apple recently released a similar cloud service. There was a demo at the PDC which showed a business man losing his PC but because all his configuration and working data is continuously in synch it was possible to take a new PC and carry on where he left off. Since the CTP I have seen a version of Mesh in Hotmail and Office 360. Although it is difficult to come up with a single business case where this is useful it does add to productivity because it improves the functionality of the environment that we work in.

On the network idea there are datacenters around the world making a global presence much easier to maintain. On the other hand there is latency in getting data to and from the cloud. This makes the SaaS Team Foundation Server offering very interesting.

I think cloud computing will bring about a shift from relational databases to a more object orientated data storage model. Relational databases are not intrinsically more performant. At present I believe the performance of properly dimensioned on premise sql servers out-perform the cloud sql servers. This will probably change SQL Azure developes and as technologies like TriadLinq become available enabling the distribution of the workload needed in carrying out database queries. The reason is that a central sql server is a bottle neck, where as in a cloud the compute needed to carry out queries is by design made for scalability.

With this in mind I think it's worthwhile to encourage the development of web based apps or apps that can be deployed from the web because these fit easily into the Cloud. We would need to manage the scope of the broad set of mobile devices that may or may not be supported in our environment. In particular Rapid Application Development in the context of Microsoft orientated companies which would favor Windows mobile devices even though these devices have not been on the market long enough to really take a large market share. To develop iPad and iPhone applications means needing an Apple workstation learning C and a whole new API.

When looking at data It is difficult to categorize data into security groups. The reason is that depending on the application the data either be sensitive or non sensitive depending on it's context. It is therefore more practical to make security groups for applications

Parallel.For seems to have a large overhead compared to a simpler implementation

I implemented the following alternative to the parallel for using threads:

        static void MyParallelFor(int fromInclusive, int toExclusive, Action<int> body)
        {
            int numProcs = Environment.ProcessorCount;
            using (CountdownEvent ce = new CountdownEvent(numProcs))
            {
              int rangeSize = (toExclusive - fromInclusive) / numProcs;
              for (int p = 0; p< numProcs; p++)
              {
                  int start = rangeSize * p;
                 //int end = start + rangeSize;
                 int end = p == numProcs-1 ? toExclusive : start + rangeSize;
                 ThreadPool.QueueUserWorkItem(delegate
                   {
                      for (int i = start; i < end; i++) body(i);
                      ce.Signal();
                    });
              }
              ce.Wait();
            }
        }

I then tested this on the benchmark earthquake model as described in my last blog entry. Since the signature of the parallel for and myParallelFor are identical the comparison can be made by commenting out the implementation that is not under test.

            for (i = 1; i <= ns; i++) // Outer loop going through lots of lines of exposure
            {
                loss[i] = 0;

                    //Parallel.For(1, nr,
                    MyParallelFor(1, nr, // Inner loop going through many events
                         (jj) =>
                         {
                            // Some maths to determine the mean damage ratio (see previous blog entry for details)
                            localsum[jj] = mdr * value[jj];
                         }

                         });
                 loss[i] = localsum.Sum();
            }

I then ran the above code on a physical machine with ProLiant DL580 G5 12 Cores 2.66 GHz Intel Xeon(R) X7460 with 8188 MB RAM

Parallel.For took about 114 seconds with all 12 CPUs running evenly at about 70%.
MyParallelFor took about 71 seconds with 11 CPUs at 30% and 1 CPU at 80%,

I found this difference quite surprising as it seems that the parallel.for implementation in System.Threading.Tasks seems to have a very large overhead. Not only does it take longer but the cpu’s are working harder. Using reflector I could see that the implementation of private static ParallelLoopResult ForWorker was quite extensive. When I have some time I will add some buffering in the localsum to check whether the problem is caused by cache invalidation. 

Results of runing a simplified earthquake model in the cloud

Here are some preliminary results of how long it takes to upload and run 10000 iterations of a cat model under various worker role configurations

Extra Large with 8 cores took 26s
Large with 4 cores took 34s
Medium with 2 cores took 49s
Small with 1 core took 82s

For comparison here are some run times on a D20 desktop

VB.NET on a D20 compiled with VS2010 running on 1 thread  took  104s
F90 on a D20 compiled with Intel running on 1 thread took 162s
C++ on a D20 compiled with Intel with optimization on 1 thread took 58 s

The time needed to upload exposure and events took about 100s Within the cloud it took about 1s to download from blob storage

The initial flat file text files had the following sizes and number of points

Exposure file 7.3MB    385560 Points
Event Catalogue 4.8MB  194510 Events

To reduce the run times we reduced the number of events from 194510 to 10000. So this means the actual run times are 20 times longer.

I have not received the bill for these tests but I think it is probably quite cheap. I think there is a lot to be said for cloud computing because I only had to worry about my code, the infrastructure was completely abstracted away. In fact setting up a new node takes about 8 minutes with a very friendly billing terms that you pay only for the time that the role is active.