Show what's going on when running TeamCity web tests

We run automated web tests from within a browser

I say this very matter of factually almost as if this statement needs no clarifying. The reality is though that running fully automated tests using a browser has proven to be very difficult. Of all the singular problems we've applied our time to, automating web tests has been one of the most frustrating and time consuming. Our current solution includes a mash up of the following:

• Selenium
• WatiN
• JS Test Driver
• QUnit

Now when we add TeamCity to the mix and have these test run on a remote machine we don't even get to see what's going on as the service is running in the background. This makes troubleshooting almost impossible.

The point of this post is to remind me that when we need to see why these web tests are failing on a remote machine we can get a little help by doing the following.

1. Stop the TeamCity Build Agent service
2. Open a command prompt and run the build agent manually.

C:\TeamCity\buildAgent\bin>agent.bat start

Helpful Links

http://stackoverflow.com/questions/488443/running-watin-on-teamcity

CacheRepository Use Case - Reading From (Somewhat) Unstructured Text File

Yesterday I was given a text file with a rather weird format that contained several hundred lines like this.



I needed to pull out the first sequence of numbers after the "work place name". I was able to use CacheRepository for this quite easily. I created an entity with the same name as the text file. Then it was just a matter of parsing a string (line) to get out the data I wanted.

CacheRepository Introduction (Why I Made It)

From time to time I have the need to write ETL programs to move data from one data source (sometimes sql server) to another data source (often times sql server). If things are going good this usually is because I'm migration data from a legacy app to one of our newer applications. Performance tends to be very high on my list of requirements. Ensuring a quick turn around time between making a change and seeing the result is very important in these types of apps.

Although these applications are typically .net console apps that are created to ultimately run one time the reality is that they can turn into pretty complex programs and I feel they must be coded with "production" code. I don't typically create unit test for them but that has not always been out of the question.

With that said though I don't want to use "heavy" tooling like NHibernate or Entity Framework to extract and load data from the data sources. When I first started writing these programs I used ADO .Net but quickly fell victim to loads of duplication from copy and pasting. When .net Micro ORM's started to become more popular it seemed like these would be a perfect fit and for some time I felt they were.

Switching to a micro ORM help my code writing and reading significantly which allowed me to focus on other aspects of these programs. Performance bottlenecks from querying and inserting data became more and more frustrating. I found myself writing caches for my query results which helped but often seemed clumsy and inconsistent. Caching my query results did speed up my queries but it then became very apparent that I needed to start bulk inserting data because my inserts were taking way too long.

That's when I created CacheRepository. The tool is essentially just a wrapper over top the micro orm Dapper .Net and it extension Dapper-Extensions. The truth is though I could have wrote this using just about any micro orm.

I've tried to stress usability as much as possible so go try it out.

https://nuget.org/packages/CacheRepository

Sending Anonymous Emails

I'm creating an application within Appharbor and wanted to configure Elmah to send emails to me when an application error occurs. Configuration for this is done from the errorMail tag within the Web.config file. The problem is that this tag expects a user name and password and I obviously don't want to set these in plain text.

While researching this I discovered this article which stated that you didn't need to authenticate when sending an email to a Google apps account.

I really didn't believe that this was possible but I gave it a try and it worked. I happen to have a Goggle Apps account so I was able to use the same MX record that was used in the article.

I looked into this more and made the following observations:

1. Anonymous emails can be sent to all gmail and Google apps users. When I say anonymous I mean that you don't have to supply credentials at all. All you need is a valid MX record which can be looked up quite easily.

2. Not only can anonymous emails be sent, you can actually send an email with a from address of gmail.com or of another Google apps domain (or any domain for that matter). This is interesting because when I tried this the Google gravatar of the fake address was actually displayed right in my gmail app. I would have thought that Google would have at least authenticated its' own users.

3. You can't use the smtp server "smtp.gmail.com" without enabling ssl and authenticating yourself with a valid user name and password. This is true for either relaying emails to outside domains (not gmail or a Google apps account) which makes perfect sense but also for sending a an email within the domain.

I haven't tried this with other popular email hosts but I suspect that this behavior is not unusual. It was just surprising to me. Use the following code to try this on your own:

Helpful Links

http://blog.dantup.com/2011/05/configuring-elmah-to-send-emails-without-putting-your-password-in-the-config-file

Coding Beliefs #1

Code is read far more often than it is written. Creating code that is easy to read and easy to understand gives it a higher likelihood of being easier to change. That is far more important than how quickly you can write the code the first time.

Application Startup Tasks in FubuMVC

Within my applications I often have a need to perform some kind of initialization whenever the application starts up. I typically call these application startup tasks. Because I use Structuremap as my dependency injection tool it is really easy to register and scan for specific implementations of an interface. So I usually create the following:

public interface IApplicationStartupTask
{
    void Execute();
}

Now when I want all of the implementation of this interface I can just inject an enumeration into some type using constructor injection.

IEnumerable<IApplicationStartupTask> tasks

But here's the problem. The only way I know how to instantiate an object on application startup without new-ing it up explicitly is by doing this:

ObjectFactory.GetInstance<SomeStartupType>()

Really what I end up doing is this though:

ObjectFactory.GetAllInstances<IApplicationStartupTask>().Each(x => x.Execute);

Throughout my application I try very hard not to use ObjectFactory explicitly but I've never found a way around it when trying to execute these startup tasks. Because this hard dependency has "just worked" for me I've just moved on and not worried about it. But today I changed the way I register Structuremap within FubuMVC and my hard dependency no longer worked. I usually configure FubuMVC like this:

FubuApplication.For<ConfigureFubuMVC>().StructureMap(ObjectFactory.Container)

But I changed it to this



Now a new Container is used per unit of work instead of the global Container. I went out searching for the "right way" of doing this and with some help from Jeremy Miller I discovered that what I was trying to do is already built into FubuMVC!

Jeremy says..." Use the IActivator interface from Bottles and register that with your container. Or really, just convert your IApplicationStartupTask's to IActivator's and you're good to go."

Auto-Start ASP .NET Applications

Reminder!

I always forget that since ASP.NET 4 (IIS 7.5 - Windows 7 / Windows Server 2008 R2) we can enable the web application to auto-start which makes start up times less of an issue if the app is not used frequently throughout the day. This tends to be important for me as we do a lot of application initialization on app start up.

I pulled the following from Scott Guthrie...

Configuring an ASP.NET 4 Application to Auto-Start

To use the ASP.NET 4 auto-start feature, you first configure the IIS “application pool” worker process that the application runs within to automatically startup when the web-server first loads.  You can do this by opening up the IIS 7.5 applicationHost.config file (C:\Windows\System32\inetsrv\config\applicationHost.config) and by adding a startMode=”AlwaysRunning” attribute to the appropriate <applicationPools> entry:

<applicationPools>

     <add name="MyAppWorkerProcess" managedRuntimeVersion="v4.0" startMode="AlwaysRunning" />

</applicationPools>

If you load up the Windows task manager, click the “show processes from all users” checkbox, and then hit save on a startMode attribute change to the applicationHost.config file, you’ll see a new “w3wp.exe” worker process immediately startup as soon as the file is saved.

A single IIS application pool worker process can host multiple ASP.NET applications.  You can specify which applications you want to have automatically start when the worker process loads by adding a serviceAutoStartEnabled="true" attribute on their <application> configuration entry:

<sites>

     <site name="MySite" id="1">

          <application path="/" serviceAutoStartEnabled="true" serviceAutoStartProvider="PreWarmMyCache" />

     </site>

</sites>

<serviceAutoStartProviders>

     <add name="PreWarmMyCache" type="PreWarmCache, MyAssembly" />

</serviceAutoStartProviders>

The serviceAutoProvider="PreWarmMyCache" attribute above references a provider entry within the config file that enables you to configure a custom class that can be used to encapsulate any "warming up" logic for the application.  This class will be automatically invoked as soon as the worker process and application are preloaded (before any external web requests are received), and can be used to execute any initialization or cache loading logic you want to run before requests are received and processed:

public class PreWarmCache : System.Web.Hosting.IProcessHostPreloadClient {

    public void Preload(string[] parameters) {

        // Perform initialization and cache loading logic here...

    }

}

IIS will start the application in a state during which it will not accept requests until your "warming up" logic has completed.  After your initialization code runs in the Preload method and the method returns, the ASP.NET application will be marked as ready to process requests. 

You can optionally combine the new auto-start "warming up" feature with the load-balancing capabilities of the IIS7 Application Request Routing (ARR) extension, and use it to signal to a load-balancer once the application is initialized and ready to accept HTTP traffic – at which point the server can be brought into the web farm to process requests.

Helpful Links

http://weblogs.asp.net/scottgu/archive/2009/09/15/auto-start-asp-net-applications-vs-2010-and-net-4-0-series.aspx