Some notes on F#

About a year ago I was looking at how to convert a second order polynomial fit written in R to F#. The function looked like

polyfit <- function(x, y, filename = 'fit_out.csv') {
x2 <- x^2
## make linear model fit
lm.fit <- lm(y ~ x + x2)
write('Intercept,x,x^2', filename, append = FALSE)
cat(lm.fit$coefficients, file = filename, sep = ',', append = TRUE)
}

Unfortunately at this time there where not many mathematical libraries available for F# and I would have had to write my own. So I put F# to one side

At the PDC 2008 there where many sessions on how to make the most of multi core processors. To do this in C# is very challenging and hard to maintain. Here are some notes I made during the F# session on the last day of the PDC:

F# is a non imperative programming language. In imperative languages x = x + 1 makes sense, however from a mathematical stand point it is nonsence. I F# you first explore the domain by writing code and executing it with alt enter. Once finished you can compile the code and expose it to other .net languages. Since F# uses immutable variables it does not have the problem of updating shared values when running in parallel. It declative style lends itself well to parallel processing and is used by some users of the htc server.

#light
Let y = o
Does not mean assign 0 to y. Instead it means bind the value 0 to the symbol assigned to y. In this way y is read-only and immutable. The same principle holds for functions eg
Let sqr x = x * x

Writing a function in F# as if it where imperative
Let sqr x = x * x
Let SumOfSquareI = nums =
Let mutable acc = 0.0
For x in nums do
Acc <- acc + sqr x
Acc

This is not the way to program F#. Notice the genric approach to variable typing. In this case the return type was determined by the compiler working backward through the code to 0.0 which is a double.

A mathematician would do
Let SumOfSqrF =
Match nums with
[] -> 0.0
h::f -> sqr h + sumof: sumofsquaresf t

Match is a switch on steroids. Its a branch mechanism that binds. This has the effect of making the calculation parrellizeable

The F# way would be
Let sumofsquares nums =
Nums
> seq.map sqr
> seq.sum

The map function applies the function sqr to each element of nums. The > is a pipeline operator that works in the same way as a pipe in unix or dos

The flow has been encapsulated. There are fewer places to screwup. It lets the bar tender make the coffee.

There is a parallel extension framework that can be used to make this multi-core multi server enabled.

Raising the level of abstraction enables parallisation without restructuring code.

#light
Open system.net
Open system.io
Let ticker = "msft"
Let url = "http....
Let req = webrequest.create(url)
Let resp = req.getresponse()
Let stream = resp.getresponsestream()
Let reader = newstreamreader(stream)
Let csv = reader.readtoend()
Let prices = csv.split([','])
>seq.skip 1
>seq.map(fun line -> line.split([',']))
>seq.filter(fun values -> values > seq.length 7)
>seq.map(fun values -> system.datetimeparse(values.[0]) float values.[6])

The results can be visualized by the FlyingFrog library and the following commands

Grid prices;;
Plot prices ;;

To give up a thread so as not to block tasks when waiting

Let! Req = req.AsynchGetResponse

To make a static method asynchronous

Let internal loadprices ticker = async {.....

To make a pipeline asynchronous

Tickers
>seq.map loadprices
>async.parallel
>async.run
>.map(fun prices -> new stockanalyser(prices,days)

The flow of logic would be completely overwhelmed by the code needed to setup delegates and threads.

In 2009 there will be a supported release of F#
For further details see fsharp.net

Adding icons to WPF applications

I had some problems inserting icons into my wpf application. I was getting errors like "The image format is unrecognised". Here is the solution:

1. Got the properties of the WPF sub project
2. In the application tab select the [...] button next to the Icon text box
3. Browse to your icon4. When you start in debug mode the icon will not appear. It's a bug in VS2005 and VS2008
5. Under the debug drop down select "Start Without Debugging" and the icon will appear

“Cannot generate SSPI context”

This is another interesting error message that recently popped up when connecting to an SQL2005 server. It reminding me of some problems that I had installing SQL2005 3 years ago. Then the error was that the service account that the SQL Server was using had insuficient access rights. This time our friends in IT made the IP addresses of our servers DHCP. Unfortunately they made a mistake making a mixup of IP and server name which caused this problem.

Using enterprize storage drives to host database files

It can make sence to use network storage devices to host database files. I believe this works for small administration databases. But if the database needs to handle some significant load you can end up in some performance problems. Here is an interesting story demonstrating limitations of network storage

22.10 5:30pm GMT+1
There are serious time out issues in our Eastern US offices that stop a database application from working. CPU is about 70-90% PF Rate is at a stable 80%

23.10 7am GMT+1
CPU is about 0-10% after applying artificial loadPF Rate remains stable 80%
Since it is 3am in Eastern US this is a good indication that cause if from responsiveness of Network storage

23.20 8am GMT+1
I move the database files from Network storage and put them on D:

23.10 1pm GMT+1
The CPU is around 30% a significant improvement.

23.10 5:30pm GMT+1
The performance has improved but still lots of time outs

24.10 7am GMT+1
I move the TempDB to D:
At about 11am EUS time and one of my collegues increased the paging memorySince there where no performance issues that day I believe the TempDB was the main cause of the problem and not the Paging memory. Tests carried out at 1pm GMT+1 and the fact that there where no performance issues between 9-11 EUS time support this

Lessons learned
Putting database files on the netapp share can produce a significant drop performance.The drop in performance is only apparent during the day as the Netap share is being used.

"Could not create an instance of type"

This is an interesting error message. Imagine you have been biulding up a wpf application using lots of user controls and databinding. Copying code from one control to another to save time on the typing. Then it is time to use 2 user controls together in the same window and then you get the error "Could not create an instance of type". Looking at this closely it happens during the initialization of one of the controls. Looking on the internet shows some articles about the problems of making wpf controls inherit from one another. It turned out to be much simpler. I only had forgotten to modify the depency object to be of the same type of the control.

This is my first blog entry

Just comming back from the PDC with a lot of ideas in how to optimize code to run on multi core cpu on a server farm. I have made an architectural model for migrating from Windows Forms to WPF applications using the recently released WPF Toolkit