SSH on Windows

Reminder...

Make sure your .ssh folder is in your home path. If your home path is wrong or not set try to set the HOME environment variable. From cmd you can check if your home is set by running the set home command. If your in a domain and the HOME path is set by active directory, you can do the following in a startup script:

set HOME=c:\users\...

As long as the ssh keys are correct, they should be used.

This isn't magic.

Web.config Transformations Reminder

To save me more future frustration, Web.config transformations from Web.Release.config and/or Web.Debug.config don't seem to work when ran locally within VS. At one point I even think I saw a reference confirming this somewhere but I can't seem to find it now.

You can test your transformations in a couple of ways.

• Use the right click context menu that pops up when you click on one of the transformations and select "Preview Transform" to test your transformation.
• Publish your web app to a local folder as explained here and take a look at the resulting Web.config.

Show what's going on when running TeamCity web tests

We run automated web tests from within a browser

I say this very matter of factually almost as if this statement needs no clarifying. The reality is though that running fully automated tests using a browser has proven to be very difficult. Of all the singular problems we've applied our time to, automating web tests has been one of the most frustrating and time consuming. Our current solution includes a mash up of the following:

• Selenium
• WatiN
• JS Test Driver
• QUnit

Now when we add TeamCity to the mix and have these test run on a remote machine we don't even get to see what's going on as the service is running in the background. This makes troubleshooting almost impossible.

The point of this post is to remind me that when we need to see why these web tests are failing on a remote machine we can get a little help by doing the following.

1. Stop the TeamCity Build Agent service
2. Open a command prompt and run the build agent manually.

C:\TeamCity\buildAgent\bin>agent.bat start

Helpful Links

http://stackoverflow.com/questions/488443/running-watin-on-teamcity

CacheRepository Use Case - Reading From (Somewhat) Unstructured Text File

Yesterday I was given a text file with a rather weird format that contained several hundred lines like this.



I needed to pull out the first sequence of numbers after the "work place name". I was able to use CacheRepository for this quite easily. I created an entity with the same name as the text file. Then it was just a matter of parsing a string (line) to get out the data I wanted.

CacheRepository Introduction (Why I Made It)

From time to time I have the need to write ETL programs to move data from one data source (sometimes sql server) to another data source (often times sql server). If things are going good this usually is because I'm migration data from a legacy app to one of our newer applications. Performance tends to be very high on my list of requirements. Ensuring a quick turn around time between making a change and seeing the result is very important in these types of apps.

Although these applications are typically .net console apps that are created to ultimately run one time the reality is that they can turn into pretty complex programs and I feel they must be coded with "production" code. I don't typically create unit test for them but that has not always been out of the question.

With that said though I don't want to use "heavy" tooling like NHibernate or Entity Framework to extract and load data from the data sources. When I first started writing these programs I used ADO .Net but quickly fell victim to loads of duplication from copy and pasting. When .net Micro ORM's started to become more popular it seemed like these would be a perfect fit and for some time I felt they were.

Switching to a micro ORM help my code writing and reading significantly which allowed me to focus on other aspects of these programs. Performance bottlenecks from querying and inserting data became more and more frustrating. I found myself writing caches for my query results which helped but often seemed clumsy and inconsistent. Caching my query results did speed up my queries but it then became very apparent that I needed to start bulk inserting data because my inserts were taking way too long.

That's when I created CacheRepository. The tool is essentially just a wrapper over top the micro orm Dapper .Net and it extension Dapper-Extensions. The truth is though I could have wrote this using just about any micro orm.

I've tried to stress usability as much as possible so go try it out.

https://nuget.org/packages/CacheRepository

Sending Anonymous Emails

I'm creating an application within Appharbor and wanted to configure Elmah to send emails to me when an application error occurs. Configuration for this is done from the errorMail tag within the Web.config file. The problem is that this tag expects a user name and password and I obviously don't want to set these in plain text.

While researching this I discovered this article which stated that you didn't need to authenticate when sending an email to a Google apps account.

I really didn't believe that this was possible but I gave it a try and it worked. I happen to have a Goggle Apps account so I was able to use the same MX record that was used in the article.

I looked into this more and made the following observations:

1. Anonymous emails can be sent to all gmail and Google apps users. When I say anonymous I mean that you don't have to supply credentials at all. All you need is a valid MX record which can be looked up quite easily.

2. Not only can anonymous emails be sent, you can actually send an email with a from address of gmail.com or of another Google apps domain (or any domain for that matter). This is interesting because when I tried this the Google gravatar of the fake address was actually displayed right in my gmail app. I would have thought that Google would have at least authenticated its' own users.

3. You can't use the smtp server "smtp.gmail.com" without enabling ssl and authenticating yourself with a valid user name and password. This is true for either relaying emails to outside domains (not gmail or a Google apps account) which makes perfect sense but also for sending a an email within the domain.

I haven't tried this with other popular email hosts but I suspect that this behavior is not unusual. It was just surprising to me. Use the following code to try this on your own:

Helpful Links

http://blog.dantup.com/2011/05/configuring-elmah-to-send-emails-without-putting-your-password-in-the-config-file

Coding Beliefs #1

Code is read far more often than it is written. Creating code that is easy to read and easy to understand gives it a higher likelihood of being easier to change. That is far more important than how quickly you can write the code the first time.

Application Startup Tasks in FubuMVC

Within my applications I often have a need to perform some kind of initialization whenever the application starts up. I typically call these application startup tasks. Because I use Structuremap as my dependency injection tool it is really easy to register and scan for specific implementations of an interface. So I usually create the following:

public interface IApplicationStartupTask
{
    void Execute();
}

Now when I want all of the implementation of this interface I can just inject an enumeration into some type using constructor injection.

IEnumerable<IApplicationStartupTask> tasks

But here's the problem. The only way I know how to instantiate an object on application startup without new-ing it up explicitly is by doing this:

ObjectFactory.GetInstance<SomeStartupType>()

Really what I end up doing is this though:

ObjectFactory.GetAllInstances<IApplicationStartupTask>().Each(x => x.Execute);

Throughout my application I try very hard not to use ObjectFactory explicitly but I've never found a way around it when trying to execute these startup tasks. Because this hard dependency has "just worked" for me I've just moved on and not worried about it. But today I changed the way I register Structuremap within FubuMVC and my hard dependency no longer worked. I usually configure FubuMVC like this:

FubuApplication.For<ConfigureFubuMVC>().StructureMap(ObjectFactory.Container)

But I changed it to this



Now a new Container is used per unit of work instead of the global Container. I went out searching for the "right way" of doing this and with some help from Jeremy Miller I discovered that what I was trying to do is already built into FubuMVC!

Jeremy says..." Use the IActivator interface from Bottles and register that with your container. Or really, just convert your IApplicationStartupTask's to IActivator's and you're good to go."

Auto-Start ASP .NET Applications

Reminder!

I always forget that since ASP.NET 4 (IIS 7.5 - Windows 7 / Windows Server 2008 R2) we can enable the web application to auto-start which makes start up times less of an issue if the app is not used frequently throughout the day. This tends to be important for me as we do a lot of application initialization on app start up.

I pulled the following from Scott Guthrie...

Configuring an ASP.NET 4 Application to Auto-Start

To use the ASP.NET 4 auto-start feature, you first configure the IIS “application pool” worker process that the application runs within to automatically startup when the web-server first loads.  You can do this by opening up the IIS 7.5 applicationHost.config file (C:\Windows\System32\inetsrv\config\applicationHost.config) and by adding a startMode=”AlwaysRunning” attribute to the appropriate <applicationPools> entry:

<applicationPools>

     <add name="MyAppWorkerProcess" managedRuntimeVersion="v4.0" startMode="AlwaysRunning" />

</applicationPools>

If you load up the Windows task manager, click the “show processes from all users” checkbox, and then hit save on a startMode attribute change to the applicationHost.config file, you’ll see a new “w3wp.exe” worker process immediately startup as soon as the file is saved.

A single IIS application pool worker process can host multiple ASP.NET applications.  You can specify which applications you want to have automatically start when the worker process loads by adding a serviceAutoStartEnabled="true" attribute on their <application> configuration entry:

<sites>

     <site name="MySite" id="1">

          <application path="/" serviceAutoStartEnabled="true" serviceAutoStartProvider="PreWarmMyCache" />

     </site>

</sites>

<serviceAutoStartProviders>

     <add name="PreWarmMyCache" type="PreWarmCache, MyAssembly" />

</serviceAutoStartProviders>

The serviceAutoProvider="PreWarmMyCache" attribute above references a provider entry within the config file that enables you to configure a custom class that can be used to encapsulate any "warming up" logic for the application.  This class will be automatically invoked as soon as the worker process and application are preloaded (before any external web requests are received), and can be used to execute any initialization or cache loading logic you want to run before requests are received and processed:

public class PreWarmCache : System.Web.Hosting.IProcessHostPreloadClient {

    public void Preload(string[] parameters) {

        // Perform initialization and cache loading logic here...

    }

}

IIS will start the application in a state during which it will not accept requests until your "warming up" logic has completed.  After your initialization code runs in the Preload method and the method returns, the ASP.NET application will be marked as ready to process requests. 

You can optionally combine the new auto-start "warming up" feature with the load-balancing capabilities of the IIS7 Application Request Routing (ARR) extension, and use it to signal to a load-balancer once the application is initialized and ready to accept HTTP traffic – at which point the server can be brought into the web farm to process requests.

Helpful Links

http://weblogs.asp.net/scottgu/archive/2009/09/15/auto-start-asp-net-applications-vs-2010-and-net-4-0-series.aspx

Sync SVN Repository to a Bitbucket Git Repository

At my current job we have multiple Subversion repositories that are hosted internally with VisualSVN Server. This has worked great for us for many years now. Before subversion (and many many years ago) we used CVS as our version control tool and WinCVS for our client.

Well just as it was previously obvious that we needed to switch from CVS to Subversion, I'm starting to get a similar feeling with switching from Subversion to Git. So we are now interested in the possibilities of hosting private repositories in Bitbucket. We aren't experts in the GIT ecosystem and we run windows throughout here. Because of this I want to start with baby steps. I figured a good start would be to just get one of our internal repositories out into Bitbucket so the following are the steps I took (minus the many hours) to do just that.

Notice: I'm assuming there are many ways to do what I was trying to do. This is just the way I did it :) Also, these steps are unfortunately not complete. I did not record everything that I did here. I just had some notes after my success and thought I would share.

Step 1- Install Git for Windows

This project provides a fork of Git on Windows. I actually installed this months ago so I can't remember any specific issues that I may have had.

Step 2- Install Ruby

I can't remember if Git for Windows installs Ruby already for you or not. It probably doesn't but then I'm just not sure as I installed Ruby separately years ago but for some reason think that maybe Git for Windows has a dependency on it. Like I said though, it probably doesn't. None the less, Ruby must be installed.

Step 3- Install the Ruby gem svn2git

gem install svn2git

That should do it. I looked into other tools that supposedly migrated Subversion repositories to Git but this one seemed the best for me.

I tried several times to get svn2git to work but just couldn't. Then I realized that git already has git-svn built into it so I should just use that! At this point I'm not even sure that we needed Ruby installed. Like I stated earlier, these steps aren't compete.

Step 4- Clone the SVN Repository using git svn

Start up Git bash and type:

git svn clone -s https://server.local/svn/someProject

This clones the SVN repository (including the trunk, tags, and branches) into the directory you are in. The -s flag states that the repo structure is in the standard format. Be warned that if you are cloning an old/active repository that this will take a long time to complete. I actually recommend doing this once and then making a copy of the resulting folder. This way if you end up messing up something in the clone (like I did several times) you can just delete it and use the copy you created instead of having to run this command again.
Note: If you have a repository that you just can't do a complete clone of because it is too big you can run the following command:

git svn clone -s -r HEAD https://server.local/svn/someBigProject

The -r HEAD means to just grab the most recent version of the trunk so this will not include any history. If you do this then you can skip Step 5 below.

Step 5- Execute a Git Checkout

git checkout -b trunk remotes/trunk

This is needed because if you look at the folder that was created from the clone, you'll notice it is empty except for a .git folder. This command is checking out the trunk to the working tree. After this you'll see all of the code that was in your trunk.

Step 6- Change name of branch from trunk to master

git branch -m master

I don't think I had to do this but I thought it was a good idea as master seems to be the convention for Git.

Step 7- Update the Working Tree

git svn rebase

The documentation states that this fetches revisions from SVN and "rebases" the current tree and that it is similar to an svn update. The thing is though is that when I did this I expected nothing really to happen because I just pulled from svn and there weren't any commits to it but for some reason this acted like it was doing a bunch of modifications.
Now things get even weirder. I ran the same command again:

git svn rebase

This seemed to do even more updates and took a little bit of time to complete. At this point when I ran the same command again I received the following response:

'Current branch trunk is up to date.'

That's what I was expecting the first time I ran that command. I don't understand but this is what worked for me.

Step 8- Create a New Bitbucket Repository

I created a private repository.

Step 9- Generate & Install SSH Keys

I would assume that you can following this documentation to do this. I used it to install the keys that I created but I didn't use PuTTygen to generate them. Instead I used ssh-keygen. I basically followed this Github documentation except for step 4. A good way to test that you have the keys set up properly is to execute the following command:

ssh -T hg@bitbucket.org

If you receive a message similar to the following then you are good to go:

conq: logged in as gbrunton.
You can use git or hg to connect to Bitbucket. Shell access is disabled.

Step 10- Prepare repository for Bitbucket

git remote add origin ssh://git@bitbucket.org/gbrunton/someProject.git

This adds a remote for the repository. Whatever that means :) I noticed though that it updates the config file within the .git folder.

Step 11- Push to Bitbucket

git push -u origin master

This command pushes the local repo into Bitbucket with all of the trunk master history!

Step 12- Schedule Daily Task

Because all I want right now is to just copy what gets committed into SVN over to Bitbucket, I created a batch file that gets ran by the scheduler.

Conclusion

This took quite a bit of time to figure out and really isn't that useful as is. In order to completely remove our SVN usage we would have to be able to migrate our Bugzilla bugs over to Bitbucket's Issues. We would also have to find a replacement for the current commit hook we have between SVN and Bugzilla. At this point I think this was a good first step.

Helpful Links

http://stackoverflow.com/a/3240146/384853
http://trac.parrot.org/parrot/wiki/git-svn-tutorial

Sql Server 2012 FileTable & Nhibernate

In the project I'm currently working on we're taking advantage of Sql Server's 2012 FileTable feature. Windows API compatibility allows our users to use windows explorer to batch "upload" literally hundreds of documents at a time and allows for easy/familiar file management. We also need a way to link to files from what we are calling DocumentExtensions. The general use case is a user uploads files/folders to the application using the virtual network share that sql server creates. Then they use a web application to link selected files/folders to DocumentExtensions.

Call me old school if you like but we still rely heavily on Nhibernate as our ORM. I just haven't found a compelling reason to switch to Entity Framework. So here's the problem, we want to use Nhibernate to link DocumentExtensions to files/folders that have already been uploaded.

Here's an integration test to help explain what is needed:



First off, this is an integration test that interacts with a real database. Also, I'm using the Builder patter to create real services instead of mocking things out. This test is not the fastest but it gives me a great deal of confidence that what I've implemented actually works all the way from the top to the bottom. I find this very comforting especially when working on techniques that are new to me.

So the ApplicationDocumentsFileTable is an abstraction that allows me to easily gain access to the DirectoryInfo object at the root of the FileTable. I find myself creating a bunch of little abstractions on top of the FileTable features. I'm using this to create a temp file at the root of the FileTable.

Next is the PathLocatorResolver is yet another one of these FileTable abstractions I'll get into in a bit.

Once I create the temp file, I create a DocumentExtension entity that I then want to have linked to the temp file. After the link is created I Save, Flush, and clear the Nhibernate session. To prove that everything was persisted correctly I grab the link by it's id and check to make sure the properties equal the properties from the link that was created above. I'm excluding the PathToPhysicalDocument property because as you'll see, this is a property based off an Formula.

Alright, on to the implementation:



There is a lot going on here that I'm not going to explain now. The interesting thing here is the IPathLocatorResolver. If you remember I built one up in the test and explicitly passed it into the AddLink method. Well I stick to the thought that it is bad to inject services into domain entities but when you must use the Double Dispatch pattern. This service is responsible for taking a path to a document and resolving it to a FileTable path locator hierarchyid.



This just leaves the DocumentExtensionLink entity:

Observable Set Implementation for NHibernate

I want to guarantee that some defined actions always take place within my domain whenever an item is added to one of my domain entity collections (a many to one relationship).

Here's the setup

Assume we have the typical Order (parent) that contains OrderItems (children) entities.

    1 public class Order

    2 {

    3     private Iesi.Collections.Generic.ISet<OrderItem> orderItems;

    4 

    5     public Order()

    6     {

    7         this.orderItems = new HashedSet<OrderItem>();

    8     }

    9 

   10     public int Id { get; protected set; }

   11 

   12     public virtual IEnumerable<OrderItem> OrderItems

   13     {

   14         get { return this.orderItems; }

   15     }

   16 

   17     public void AddOrderItem()

   18     {

   19         this.orderItems.Add(new OrderItem());

   20     }

   21 }

As you can see, I don't expose the internal orderItems collection type. Instead I return the collection as an IEnumerable. This actually gives my domain some control over the collection as it effectively makes it immutable (only the collection, not the items in the collection). I like doing this and will probably continue but making the collection observable would give me some flexibility on this in the future.

What I'm interested in doing right now is just making it observable for internal usage as I have multiple methods within the Order entity that can add order items. If I can get the collection to be observable I could just register for the add event and do whatever needed to be done at that point.

Lets see some tests to show what I'm trying to do

    1 [Test]

    2 public void When_an_orderItem_is_added_an_one_internal_event_should_be_fired_off()

    3 {

    4     // Arrange

    5     var order = new Order();

    6     var numberOfTimesAddEventWasCalled = 0;

    7     ((INotifyCollectionChanged)order.OrderItems).CollectionChanged += (x, y) =>

    8     {

    9         if (y.Action.Equals(NotifyCollectionChangedAction.Add))

   10             numberOfTimesAddEventWasCalled++;

   11     };

   12 

   13     // Act

   14     order.AddOrderItem();

   15 

   16     // Assert

   17     Assert.AreEqual(1, numberOfTimesAddEventWasCalled);

   18 }

   19 

   20 [Test]

   21 public void When_an_orderItem_is_added_to_a_non_empty_Order_that_is_associated_to_a_session_one_internal_event_should_be_fired_off()

   22 {

   23     // Arrange

   24     var repo = new NHRepository<Order>(base.activeSession);

   25     var order = new Order();

   26     order.AddOrderItem();

   27 

   28     repo.Save(order);

   29     repo.Flush();

   30     repo.Clear();

   31 

   32     var fromDb = repo.Get(order.Id);

   33 

   34     var numberOfTimesAddEventWasCalled = 0;

   35     ((INotifyCollectionChanged)fromDb.OrderItems).CollectionChanged += (x, y) =>

   36     {

   37         if (y.Action.Equals(NotifyCollectionChangedAction.Add))

   38             numberOfTimesAddEventWasCalled++;

   39     };

   40 

   41     // Act

   42     fromDb.AddOrderItem();

   43 

   44     // Assert

   45     Assert.AreEqual(1, numberOfTimesAddEventWasCalled);

   46 }

Both of these test are not typical of tests that I usually write because they are way to knowledgeable and dependent on the implementation. These weren't developed as test first. I created them when I was trying to figure out how to implement this observable collection.

The first test is just there to verify that my solution worked without persistence. The second test was created to see how things worked when I actually applied persistence.

Onto the Implementation

In order to get this test to pass I created the ObservableHashSet. I came across this idea when I found this article by Gary DeReese. Specifically within the comments of that blog were references to this article by Adrian Alexander. Adrian created a project called "ObservableCollections" that I pretty much copied from (with some slight changes). Essentially, there are three new types that need to be created.

    1 public class ObservableHashedSet<T> : HashedSet<T>, System.Collections.Specialized.INotifyCollectionChanged

    2 {

    3     public event NotifyCollectionChangedEventHandler CollectionChanged;

    4 

    5     public override bool Add(T item)

    6     {

    7         var wasAdded = base.Add(item);

    8         if (wasAdded) invokeCollectionChanged(NotifyCollectionChangedAction.Add, item);

    9         return wasAdded;

   10     }

   11 

   12     public override bool Remove(T item)

   13     {

   14         var isChanged = base.Remove(item);

   15         if (isChanged) invokeCollectionChanged(NotifyCollectionChangedAction.Remove, item);

   16         return isChanged;

   17     }

   18 

   19     public override void Clear()

   20     {

   21         base.Clear();

   22         invokeCollectionChanged(NotifyCollectionChangedAction.Reset, null);

   23     }

   24 

   25     private void invokeCollectionChanged(NotifyCollectionChangedAction action, object changedItem)

   26     {

   27         if (CollectionChanged != null) CollectionChanged(this, new NotifyCollectionChangedEventArgs(action, changedItem));

   28     }

   29 }

The first type is the ObservableHashedSet that inherits from the Iesi.Collections.Generic.HashSet class and the System.Collections.Specialized.INotifyCollectionChanged inteface. I like to use sets for my NHibernate collections and the HashSet is the typical implementation to use. The INotifyCollectionChanged interface is what allows us to catch different events regarding the collection.

    1 public class ObservableHashedSetType<T> : NHibernate.UserTypes.IUserCollectionType

    2 {

    3     public object Instantiate(int anticipatedSize)

    4     {

    5         return new ObservableHashedSet<T>();

    6     }

    7 

    8     public IPersistentCollection Instantiate(ISessionImplementor session, ICollectionPersister persister)

    9     {

   10         return new PersistentObservableSet<T>(session);

   11     }

   12 

   13     public IPersistentCollection Wrap(ISessionImplementor session, object collection)

   14     {

   15         return new PersistentObservableSet<T>(session, (ObservableHashedSet<T>)collection);

   16     }

   17 

   18     public bool Contains(object collection, object entity)

   19     {

   20         return ((ISet<T>)collection).Contains((T)entity);

   21     }

   22 

   23     public object IndexOf(object collection, object entity)

   24     {

   25         return -1;

   26     }

   27 

   28     public IEnumerable GetElements(object collection)

   29     {

   30         return (IEnumerable)collection;

   31     }

   32 

   33     public object ReplaceElements(object original, object target, ICollectionPersister persister, object owner, IDictionary copyCache, ISessionImplementor session)

   34     {

   35         var result = (ISet<T>)target;

   36         result.Clear();

   37         foreach (var item in ((IEnumerable)original))

   38         {

   39             result.Add((T) item);

   40         }

   41         return result;

   42     }

   43 }

The second type is the ObservableHashedSetType that inherits from the NHibernate.UserTypes.IUserCollectionType interface. This interface is NHibernates extension point for creating custom collection types (which is exactly what we are trying to do)

    1 public class PersistentObservableSet<T> : NHibernate.Collection.Generic.PersistentGenericSet<T>, INotifyCollectionChanged

    2 {

    3     public event NotifyCollectionChangedEventHandler CollectionChanged;

    4 

    5     public PersistentObservableSet(ISessionImplementor session) : base(session) { }

    6     public PersistentObservableSet(ISessionImplementor session, ISet<T> coll)

    7         : base(session, coll)

    8     {

    9         if (coll != null) ((INotifyCollectionChanged)coll).CollectionChanged += onCollectionChanged;

   10     }

   11 

   12     public override void BeforeInitialize(ICollectionPersister persister, int anticipatedSize)

   13     {

   14         base.BeforeInitialize(persister, anticipatedSize);

   15         ((INotifyCollectionChanged)gset).CollectionChanged += onCollectionChanged;

   16     }

   17 

   18     private void onCollectionChanged(object sender, NotifyCollectionChangedEventArgs args)

   19     {

   20         if (CollectionChanged != null) CollectionChanged(this, args);

   21     }

   22 }

The third and final type is the PersistentObservableSet that also inherits from the INotifyCollectionChanged interface as well as from the NHibernate.Collection.Generic.PersistentGenericSet type. Truthfully I don't fully get the total gist of this. Obviously it has to do with persistence but... I would probably have to look into the source of NHibernate to understand this more. I probably should but not today.

After creating these types I was able to update the Order entity to use the new ObservableHashSet.

    1 public class Order

    2 {

    3     private Iesi.Collections.Generic.ISet<OrderItem> orderItems;

    4 

    5     public Order()

    6     {

    7         this.orderItems = new ObservableHashedSet<OrderItem>();

    8     }

    9 

   10     public int Id { get; protected set; }

   11 

   12     public virtual IEnumerable<OrderItem> OrderItems

   13     {

   14         get { return this.orderItems; }

   15         protected set

   16         {

   17             this.orderItems = (Iesi.Collections.Generic.ISet<OrderItem>)value;

   18             ((INotifyCollectionChanged)this.orderItems).CollectionChanged += orderItemChangedCallback;

   19         }

   20     }

   21 

   22     private void orderItemChangedCallback(object sender, NotifyCollectionChangedEventArgs e)

   23     {

   24         //

   25     }

   26 

   27     public void AddOrderItem()

   28     {

   29         this.orderItems.Add(new OrderItem());

   30     }

   31 }

There was just one more thing to do and that was create the mapping file. Seems like a lot of people are using Fluent NHibernate nowadays but I'm still old school and use hbm.xml mappings.

    1 <?xml version="1.0" encoding="utf-8" ?>

    2 <hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="Sample" namespace="Sample">

    3     <class name="Order">

    4         <id name="id" access="field">

    5             <generator class="hilo">

    6                 <param name="column">OrderNextHi</param>

    7                 <param name="max_lo">100</param>

    8             </generator>

    9         </id>

   10 

   11         <set name="OrderItems" table="OrderItem" inverse="true" cascade="all-delete-orphan"

   12              collection-type="Sample.Data.UserTypes.ObservableHashedSetType`1[[OrderItem]], TrustFundAEA.Data">

   13             <key column="OrderID" on-delete="cascade" />

   14             <one-to-many class="OrderItem" />

   15         </set>

   16     </class>

   17 </hibernate-mapping>

The Test Results

Huh??? Why did the second test fail. It failed because the event fired off twice as I was only expecting it to fire once. Looking into this I see that after retrieving the Order entity from the db and once the lazy OrderItems is accessed, NHibernate fetches the persistent OrderItem and then ADDs it to the orderItems collection. Do you see where I'm going here? When the persisted OrderItem is added, the NotifyCollectionChangedAction event is fired off. Then when a new OrderItem is added the event is fired off again incrementing the numberOfTimesAddEventWasCalled variable both times. Well that was not what I was expecting. I'm only interested in the event when the entity itself adds to the orderItems collection, not when NHibernate does.

Getting this second test to pass proved to be a little difficult. I looked around Google and couldn't find a solution which surprised me because it seems to me that the typical usage would not want this event fired off when NHibernate is populating the collection.

I went looking for a solution within the PersistentObservableSet. I realized that the BeforeInitialize method was essentially the impetus for this unwanted event so I tried removing it with no luck. Doing so surprisingly resulted in the second test failing because no events were being fired off. My next attempt was to keep the BeforeInitialize method removed and add the AfterInitialize method with the same code as what was in the BeforeInitialize method. This seemed quite reasonable as I only wanted this event to be fired off after the entities had been initialized. Unfortunately this change still didn't fix the test as the events in test 2 still weren't being fired off. So it appears that the BeforeInitialize method must stay. Now I was really starting to think I should just open up NHibernate and try to track down another option but wanted to try one more thing.

    1 private void onCollectionChanged(object sender, NotifyCollectionChangedEventArgs args)

    2 {

    3     if (CollectionChanged != null && base.WasInitialized) CollectionChanged(this, args);

    4 }

I'm not sure if this is the best way of handling this but guess what

Well after all of this I've decided that I don't think this implementation is worth it if I'm only going to use it internally. Looking at long this took me to figure out, this conclusion stings a little. But at least I have a new tool that I can use in the future.

So today I was trying to get the following test to pass:

    [TestFixture]
    public class When_getting_a_step_property_value
    {
        [Test]
        public void a_value_set_can_be_retreived()
        {
            // Arrange
            var model = new FieldMappingModel { DateFormat = "test" };
 
            // Act
            var valueFound = new StepValueService().Get(x => x.DateFormat);
 
            // Assert
            Assert.AreEqual(model.DateFormat, valueFound);
        }
    }

Where the StepValueService looked like this:

    public class StepValueService
    {
        public dynamic Get(Expression<Func<dynamic, dynamic>> publicGetter)
        {
            var memberExpression = (MemberExpression)publicGetter.Body;
            var propertyName = memberExpression.Member.Name;
 
            // Doing some funky stuff with reflectoin using the properyName
            return null;
        }
     }

This wouldn't compile failing with the following error:

An expression tree may not contain a dynamic operation

Changing the service to the following fixed it but it was rather slow:

         public class StepValueService
         {
                 public dynamic Get(Func<dynamic, dynamic> publicGetter)
                 {
                          var propertyName = publicGetter(new DynamicMemberNameExpression());
 
                          // Doing some funky stuff with reflection using the properyName
                          return null;
                 }
 
                 private class DynamicMemberNameExpression : DynamicObject
                 {
                          public override bool TryGetMember(GetMemberBinder binder, out object result)
                          {
                                   result = binder.Name;
                                   return true;
                          }
                 }                
         }

Ended up not using this but might be interesting in the future...

NHibernate Observation

I ran three tests of an automated process and learned an interesting thing. Here is the set up:

Test 1

Setup

- Commented out explicit transactions from NHibernate's Session

   1: //this.contextTransactions.Add(databaseName, session.BeginTransaction());

- Used a repository that retrieved entities by a natural key

   1: var reporting = base.NHibernateSession()

   2: .CreateQuery("from Reporting where identification = :identification")

   3: .SetString("identification", identification)

   4: .UniqueResult<Reporting>();

   5:  

Results

Process ran in under 1 minute. The Sql profiler logged 14,631 rows. This ran very fast but using implicit transactions in NHibernate is not a best practice. Also, anytime you see a process generating 14,000 rows you probably aren't doing something correctly. Here is a sample of the logged rows from the profiler:

Notice that a lot of the extra rows exist because of the Audit Login lines. These must have something to do with the fact that we are using implicit transactions because as you'll see later when we use explicit transactions, you don't see all of these lines.

Test 2

Setup

- Used explicit transaction

   1: this.contextTransactions.Add(databaseName, session.BeginTransaction());

- Used a repository that retrieved entities by a natural key (Same as Test 1)

   1: var reporting = base.NHibernateSession()

   2: .CreateQuery("from Reporting where identification = :identification")

   3: .SetString("identification", identification)

   4: .UniqueResult<Reporting>();

Results

Process ran in 9 minutes. Interestingly enough, the Sql Profiler only logged 3,676 rows as compared to the 14,621 rows observed from Test 1. This was completely unexpected to me because typically when something takes longer to run it also generates more queries. But take a look at the follow screen capture of the profiler:

The reason there are so fewer rows is because you don't see all of those Audit Login lines we saw in Test 1. So why did this test take so much longer? Let's go onto the next test and we'll get back to this answer later.

Test 3

Setup

- Used explicit transaction (Same as Test 2)

   1: this.contextTransactions.Add(databaseName, session.BeginTransaction());

- Provided some caching in the repository that would only create queries to the database if the natural key that was being searched for hadn't already been searched.

Results

Process ran in under 1 minute. The Sql profiler logged 1,192 rows that look much like what happened in Test2 but without all of the queries that were cached now in the repository (explaining why we have less rows than Test 2). Note, I understand that having 1,192 rows logged here is still quite a large amount. I could probably make this number better with a better fetching strategy.

Conclusion

So now the number of log entries all make sense but it still isn't completely clear as to why Test 1 with implicit transactions ran so quickly even without performing any caching within the repository. I tracked it down to one thing by comparing the profiler logs. The difference is that Test 1 and Test 3 are performing all inserts/updates at the end of the unit of work and all select statements are done at the beginning of the unit of work. Test 2 has inserts and update statements inline / throughout the entire unit of work.

I can explain how this happened although I can't fully explain what the difference is (but I do have an idea). For test one, none of the inserts/updates were performed until a flush command was issued and the flush command was issued at the end of the unit of work. In fact if this flush command is not issued at all then no data is persisted (no insert and/or update queries are ran).

In Test 3 I believe that the inserts and updates statements were at the end because it appears that (from Test 2) performing queries on the database (not using primary keys) interrupts things and forces the 1st level cache to be cleared (somewhat anyways). The caching in the repository include in Test 3 made it so that the queries didn't have to be ran. Using id generators besides the Identity ones that we are using (such as guid or hi/lo) may have solved this problem as well.

Ultimately I believe it all comes down to the fact that the 1st level cache can keep everything in memory and then open up just one connection (or something) and fire off all of the inserts/updates all at once. This is the part that I can't fully explain.