Logging with log4net and NUnit

This is a reoccurring problem that I forget how to solve:

The problem

• A class under test contains useful logging.
• Your class configures logging using an assembly attribute:
  

[assembly: log4net.Config.XmlConfigurator(Watch = true)]

• It is being tested via the nunit-console or gui and you want to see the output of the log messages.

What's going on
When using assembly attributes to configure log4net, logging is configured by the first such attribute that is loaded (more in the log4net faq). As NUnit uses log4net, it gets there first. By default, NUnit has logging switched off, hence no log messages are emitted.

The solution
As hinted in the log4net faq, you can also load configure logging programatically, whenever and wherever you wish. So...

1. Include a log4net configuration file in your Unit test project (as app.config, or a separate log4net.config).
2. Load this file at the start of your test run. See my post on the NUnit SetupFixture for more details.
   

XCopy install of ReportViewer assemblies

A recent project at work was a Console C# application that used .Net 2.0's LocalReport class to generate letters. To ease deployment, I prefer to distribute an application via copying instead of using an installer. A frequent problem for users of the LocalReport class is that it requires three assemblies that are not part of the .Net 2.0 Framework (but confusingly, are part of VS 2005). Whilst these assemblies can be installed (to the GAC) via Microsoft's ReportViewer re-distributable (SP1), this breaks the principle of an xcopy install. The solution of course is to locate the required assemblies and mark them as CopyLocal in the project that requires them.

This should be no problem of course, as the first two assemblies show up in the .Net reference list:

• Microsoft.ReportViewer.WinForms.dll
• Microsoft.ReportViewer.Common.dll

Inspection of the second using Reflector reveals the third to be:

• Microsoft.ReportViewer.ProcessingObjectModel.dll

which Reflector fails to locate - but why, if has been successfully installed? Even a search of the System partition fails to locate it.

The answer is the platform agnostic GAC (%SystemRoot%\assembly\GAC_MSIL). Its contents are seemingly invisible to searching: the only way to access them is:

1. Via the command-shell, e.g. via a command-prompt, cd/explorer into %SystemRoot%\assembly\GAC_MSIL.
2. Via Microsoft's gacutil.exe.
   

Doing this allows you to locate the third assembly at:

• %SystemRoot%\assembly\GAC_MSIL\Microsoft.ReportViewer.ProcessingObjectModel\8.0.0.0__b03f5f7f11d50a3a
  

It can then be copied somewhere visible to the project (e.g. the application's lib directory) and added to the list of CopyLocal dependencies.

How long to change a culture?

A current project I have on the go is writing a C# console app to replace an ageing VB5 application.

For many on the project, this is the first time writing production C# instead of VB. In addition we're aiming at a SOLID design, aided by NUnit, Spring.Net and of course Cruise Control.Net.

A reoccurring challenge we've faced on this project is re-learning how to write software. Its more than just learning a new language, tool or framework, its more like loosing the shackles of apartheid. This analogy may seem a little strong, but it was quite clear to me after discussing with an Afrikaans South African (in his 20's) just how long it would take for his country to be free of apartheid's effects: we concluded at least three more generations.

Whilst I know that the team won't have to be great-great-great-grandparents before we loose the shackles of VB and procedural programming, I need to remember that cultural of change of any sort doesn't just happen overnight!

This was further driven home to me today when ThoughtWorks met with us to discuss an upcoming project. They shared how to aid culture change, they send newbies on a boot-camp that lasts 6-8 weeks!

Leadership advice from Bear Grylls - Facing the Frozen Ocean

For more on the book visit Amazon. Picked up a copy from my local library as it was the only title by Bear Grylls they had. Having heard a bit about this chap I wanted to learn more and was blown away by this book.

As well as being a gripping read, it contains sound advice on building and leading a team. The problem Bear set out to solve was the crossing of the North Atlantic in an open boat (a RHIB). Points of note were:

• The team he assembled had the required mix of expertise, but instead of seeking out the very best in each field, Bear looked for people who were passionate about achieving his goal and excellent team-players.

• Bear took a pure leadership / management role in solving the problem: he left the team use their own skills to solve problems to the best of their ability. He "merely" made strategic decisions and more importantly served the team as a servant-leader whenever possible.
  

External images in a local report

We've been playing with Reporting Services at work lately, in particular the use of Local Reports.

Adding external images to these reports is a bit fiddly. The "handling images" section of this devx article sheds some useful light on what the path must look like, and how to handle relative paths.

As local reports require an absolute path, one means of doing this to use an embedded VB.NET function:

Public Function GetAppPath() As String
   Return System.AppDomain.CurrentDomain.BaseDirectory
End Function

Using this function, here's how you can set up the Value property of an image to point to the external image file.

=System.IO.Path.Combine(Code.GetAppPath(),"image.jpg")

I'm a DHTC programmer

Interesting programmer personality test. My programming personality breakdown is below. Do you agree with this? What kind of a programmer are you?

DHTC

You're a Doer.
You are very quick at getting tasks done. You believe the outcome is the most important part of a task and the faster you can reach that outcome the better. After all, time is money.

You like coding at a High level.
The world is made up of objects and components, you should create your programs in the same way.

You work best in a Team.
A good group is better than the sum of it's parts. The only thing better than a genius programmer is a cohesive group of genius programmers.

You are a Conservative programmer.
The less code you write, the less chance there is of it containing a bug. You write short and to the point code that gets the job done efficiently.

A user story story

Although I've known about user stories for about five years, I've only now been in the position to practise what I preach.

Having promoted them within my team (particularly following QCon 2007), I'm trying to use them on every project I work on and have to say I am blown away by their effectiveness! Although we captured them onto a wiki, their power immediately became apparent when we started using the real paper/card versions.

First off, estimating (complexity) became immediately easier: we analysed each card and divided them into three piles (easy, medium and complex). Unknowns and design notes were made direct onto each card, for later reference. The cards could also be simply arranged in dependency groups. Pieces of paper divided into piles made the shuffling back and forth incredibly easy and low cost.

We then assigned effort estimates (weeks) to each story and grouped them into possible iterations. Again, the ease with which changes could be made, and the effects so easily seen gave the analysis extra power. I've always previously used Excel to do this kind of work, but whilst it "looks" professional and does the sums for you, it lacks the speed and imprecision of paper!

The power of paper was further demonstrated when we met with a system commissioner, and talked them through our analysis. Each piece of paper could be referred to in turn, to address unknowns and discuss implementation considerations. The story groups could be simply seen, but also easily rearranged.

The ironic twist is that we kept our wiki versions of the story up-to-date as we changed the paper ones. However, we came to the conclusion that whilst the paper stories looked unprofessional (my writing is not the neatest :-), the real paper ones were just so much more powerful.

Keeping CruiseControl DRY

I recently set up CruiseControl.Net to build a number of VB applications. This post describes how I kept the CC configuration DRY (Don't Repeat Yourself), whilst allowing more applications to be easily added and reported on.

My initial set up consisted of a single project that built a set of applications defined by a text file. Whilst this made adding a new application easy, it had a number of problems:

1. Any source control change caused all apps to be rebuilt.
2. The reporting was not at the application level - if one failed to build, it showed up as all had failed.

Version two led to a separate project for each application, but this quickly became messy, as the same elements (e.g. filtertriggers, source control details) were repeated time and time again. Also, editing one project configuration would cause all projects to be rebuilt, as the one file contained all configurations.

This led to version three, which stayed DRY thanks to the use of DTD entities to separate out:

1. Duplicate elements.
2. Projects into separate config files.

Although each project was now defined in a separate config file, it could reuse the entities defined in the main ccnet.config file, thus staying DRY. Adding a new project was still simple, as it just took two steps:

1. Create a modified copy of an existing project sub-config file.
2. Modify the main ccnet.config file so it references the new sub-config file.
   

An alternative approach to this is to use CI Factory. Whilst this does streamline the setup of CruiseControl.Net, I found that once this was done, adding new projects (as above) was easy.

Testing CI Factory did have its benefits though. It led me to consider splitting the Cruise Control service into multiple services, so as to separate the main ccnet.config file into one per group of applications. This may well provide performance gains, as well as remove the problem of changes to one project config causes all to be rebuilt.

Watch this space!

QCon 2007 - What I learnt about process - Part 2

Going through my notes I discovered this useful map of an agile life-cycle. I think it came from Rachel Davies's session on user stories.



As text it reads:

Bird's eye view of life cycle

Exploration

Explore

• business domain
• technology
• process
• scope

Keep as short as possible (< 2 weeks)

Release planning - create roadmap of releases

• May not happen until some initial iterations
• Flesh out plan as you progress
• Deliver high risk / high value items early
• Set concrete release dates but vary features

Iterations

Production - aim is to arrive here ASAP

Planning phases

Customer needs

Implementation considerations

Implementation tasks

Estimated task list

Customer reviews effort vs. necessity

UI Design is often done an iteration ahead,
to help users & team understand what is required

QCon 2007 - What I learnt about process

A post in a series that summarises what I learnt at QCon 2007.

This post focuses on development processes.

For a mind map version, click here

Inspiration came from the following excellent sessions:

• Agile Leadership Tutorial - Diana Larsen
• Usage-Centered Design Tutorial - Larry Constantine
• Modifiability - Martin Fowler
• Operational Manageability - Dan Prichett, ebay Technical Fellow
• User stories and release planning - Rachel Davies
• Test driven development - Steve Freeman
• Agile PM: The Googley Way - Jeff Sutherland
• The Yawning Crevasse of Doom - Martin Fowler & Dan North

Implementing change

• Have retrospective to establish SWOT's to address
• Prioritise cultural changes each iteration / project
• Do simple things well, then focus on clever/hard things

Agree on principles as early as possible, e.g.

• design
• unit testing

Key success ingredients are

• Encapsulation
• Classes should "keep secrets"
• Tell, don't ask
• Prefer private over public
• Leads to more readble code
• Keep the domain model clean of non-domain code/entities (e.g. UI, DB)

• Do the simplest thing that will work (not the same as doing the easiest thing)
• Making code test driven: lowers cost of change
• Test for required behaviour not implementation
• Refactoring is the second leg of TDD (first being testing)
• Growing better developers is best investment that can be made in a team
  

Use risk maps

Bridges and ferries

• Bridges:
• Promote activities that build bridges between developers and the business
• Encourage activities that have feedback
• Minimise "distance between" developers and the business
• Consider co-locating developers and end-users
• Don't build "ferries", that distance or separate developers from the business

User stories

Less is more

• Tool for collaborative planning, i.e. purpose is to encourage communication between stakeholders
• Facilitates collabrative dynamics, via the 3 C's:
• Card
• Conversation
• Confirmation
• Can exist alongside more formal and comprehensive documents
• Aim is to write less sooner, to minimise cost of change
• Sometimes prior analysis should be done to understand problem domain, before moving onto stories

Why's and how's of a story

• System "commisioner" provides "why's"
• End users can provide "how's"

Format

• As a I need so that .
• e.g. As a book buyer I want to find a book by entering the ISBN number, so I can locate it quickly
• OR simply state user goal
  

Scrum

Minimise "work in progress" - "Done" means system and acceptance tests pass

Minimal process

• Artifacts
• Product backlog
• Spring backlog
• Burndown chart
• Meetings
• Sprint planning
• Daily meeting
• Sprint review
• People
• Product owner
• Scrum master
• Team

Builds

Include rollout script for use by Operations.

QCon 2007 - References to follow up on

Various resources that I haven't got round to looking at yet, but are worth doing sooner...

• Behind closed doors - people management
• Don't make me think - user interface design
• Maverick - business management
• How buildings learn - architecture
• thinktecture.com - resouces for WCF
• fitnesse.org - web based acceptance test framework
  

Adding tagging to dokuwiki

This wasn't as straight forward as I'd hoped, but we achieved it in < 1 hour.

We used the tag plugin for dokuwiki. Installing this (and its pre-requisite plugins, feed and pagelist) was easy.

Initially I couldn't get the topic feature (that shows pages with a tag) to work - it resulted in a blank page. This was traced to curl not being installed. Doh!

The easy fix was (detailed at the curl site):

1. Get the required binaries (php_curl.dll, libeay32.dll and ssleay32.dll).
2. Ensure the latter two binaries are in the PATH.
3. Uncomment the curl extension in php.ini.

QCon 2007 - What I learnt about architecture

This is the first post in a series that summarises what I learnt at QCon 2007.

This post focuses on solution architectures.

For a mind map version, click here

Inspiration came from the following excellent sessions:

• Amazon Technology Platform - Werner Vogels (CTO)
• Putting qualities into architecture - Kevlin Henney
• Modifiability - Martin Fowler
• Operational Manageability - Dan Prichett, ebay Technical Fellow
• Agile Architecture is not Fragile Architecture - Kevlin Henney and James Coplien

Replacing Amazon's architecture

• Monolithic application to SOA
• Took 6 years
• Cost $2 billion
• Only 30% of cost delivered visible business value
• 70% was on the "heavy-lifting"

Architecture relates to

• how it runs
• how it fails
• how its developed
• how it changes during
• development time
• run-time

Architecture is interwoven with the development process

Operational manageability

• Design for
• operational configuration
• operational monitoring
• failure (no SPOF's)
• Maintain clear dependencies
  

Design up-front the parts that are costly to change

• Reduce the significance of remaining decisions
• Defer decisions to the last responsible moment
• Metric is cost of reversing decision
• e.g.

• testability
• security
• maintainability
  
• performance
• Its like putting stones in a bucket.

How much up front design (UFD)?

• BUFD - Big
• RUFD - Rough
• NUFD - None
• Time spent should be inversely proportional to
• Knowledge of domain
• Experience in design

Quotes

• "A stable architecure is not the same as a frozen architecture"
• "Doing the simplest thing that works is not same as doing the easiest thing"

Setting up dokuwiki on IIS with integrated authentication

Having enjoyed the benefits of MediaWiki, at my previous employer, I've been keen for some time to set one up a my new place. A friend of my recommended dokuwiki, which stacks up well against others.

Setting up was fairly easy, but as there were a few gotcha's, here's what worked for me.

The mix

• IIS on XP Pro SP1
• PHP (latest version at time of writing)
• dokuwiki (latest version at time of writing)
• Active Directory

The recipe

IIS + PHP

1. Install PHP (used the Windows installer, despite recommendations against it). During the install process, selected to use the ISAPI server version and just the GD2 extension.
2. Followed Peter Guy's excellent instructions for remaining configuration and testing.
3. If you can, put php.ini under source control - it does require tweaking, which is worth tracking, i.e. in addition to the changes in step 2, the SMTP server needs pointing to the desired mail server.
4. If you want to use integrated authentication, remember to uncheck anonymous access and check integrated authentication (see below)!
5. NB: Don't skip the testing step - this may well save you grief later!

dokuwiki

1. Unzip the download and point the default web site at the dokuwiki folder.
2. Follow the dokuwiki instructions.
3. Remember to move/rename the install.php script once you've set it up as you like it.
4. Place all of the dokuwiki under source control, noting that conf/local.php will require to be editable if you configure dokuwiki from the configuration web page. Note also that using the web page can overwrite any manual changes to local.php, though it does attempt to include a second configuration file (local.protected.php) that such changes should be placed in.
   

Getting integration authentication to work

1. Setup basic authentication so that you have a handle on how this works - this is a useful fall back position. Dokuwiki instructions are excellent.
2. Using the ACL configuration page, create two acls: one for editors, one for readers. These are written to conf/acl.auth.php. The editor group name can then be used to match against the Active Directory group that you wish to give edit rights to. E.g. if you just want your developers to have edit rights and they exist in an AD group called "Development", create an ACL entry granting all access to a new group named Development, and change the ALL group access to read-only.
   
3. Follow James Van Lommel's excellent instructions on getting LDAP to work. Note that you may require to set the credentials that are used to connect to ldap, should anonymous authentication be disabled.
   
4. local.conf (or rather local.protected.php, as this is not overwritten) needs the following added to pick up the users credentials (taken from the dokuwiki ldap page):

if (isset($_SERVER['AUTH_USER']) and !isset($_SESSION[$conf['title']]['auth']['info'])) {

 list($d, $username) = split("\\\\", strtolower($_SERVER['AUTH_USER']), 2);
$_REQUEST['u'] = $username;
}

4. inc/auth/ntlm.php requires the following change (also taken from the dokuwiki ldap page):

function auth_checkPass($user, $pass){
// verify that IIS has authenticated this person via NTLM
if(isset($_SERVER['AUTH_USER']) and isset($_SERVER['AUTH_TYPE'])) {
return true;
}else{
return false;
}
}

If you're using Firefox to access the wiki, then my earlier post about NTLM authentication and Firefox may be of interest.

Implementing nice url's

dokuwiki provides instuctions for displaying nice url's with Apache, but none for IIS. Fortunately James van Lommel has already figured out using mod_rewrite for IIS. At the time of writing, the mod_rewrite website was unavailable, so I stuck with James's configuration, but used the free version of ISAPI_Rewrite instead. Having dokuwiki as the default website meant the following httpd.ini settings worked for me (note the addition of the idx handler, as James's configuration didn't handle indexes for namespaces).

[ISAPI_Rewrite]
RewriteRule ^/_media/(.*)\?(.*) /lib/exe/fetch.php?media=$1&$2 [L]
RewriteRule ^/_detail/(.*)\?(.*) /lib/exe/detail.php?media=$1&$2 [L]
RewriteRule ^/_detail/(.*) /lib/exe/detail.php?media=$1 [L]
RewriteRule ^/$ /doku.php [L]

RewriteRule ^/lib/(.*) /lib/$1 [L]
RewriteRule ^/(.*)\?do=(.*) /doku.php?id=$1&do=$2 [L]
RewriteRule ^/(.*)\?idx=(.*) /doku.php?idx=$2 [L]
RewriteRule ^/doku.php\?id=(.*) /doku.php?id=$1 [L]
RewriteRule ^/(.*) /doku.php?id=$1 [L]

Diffing .Net assemblies

As we're about to commence a major .Net phase at work, I know that this is something I'll be needing to do.

Scott Hanselman has provided a great post (as ever) on the subject, which I'm posting here so I don't loose track of it: Managing Change with .NET Assembly Diff Tools

MDAC 2.7 broken by another installer

Encountered this knarly problem recently and its solution took some arriving at, so worth publishing.

A system with MDAC 2.7 received the install of an application that uses MDAC 2.1. In theory both versions of MDAC can live happily side-by-side, but for some reason the install replaced sqloledb.dll with an incompatible version.

Re-installing MDAC 2.7 didn't detect the change, so didn't fix this issue. The only resolution was to install MDAC 2.8.

Software development is like ... telling a story

Went North last weekend for a fantastic baptism in Glossop. Whilst waiting to join the M6 from the M6 Toll road, had an interesting conversation with my sons (aged seven and nine). They had been listening in on some work calls regarding some NAnt scripts that were playing up, and that combined with Vista's $10 billion development cost, led them to ask (for the nth time) what, exactly, I did at work.

Whilst trying to provide an understandable explanation, it struck me that the answer given can help measure the depth of someone's development background and experience.

The answer I gave (and sub-sequent cross-examination I endured) proceeded something like this:

• Child: So why did Vista cost so much to make?
• Developer: Because its a very complex system, that requires lots of (expensive) people to create.
• Child: Why is it so complex?
• Developer: <Sigh - here we go> Well let's imagine a story. A person (called a customer), tells you a story about what they want the software to do.
• Child: Uh-huh
• Developer: Software developers take that story and re-write it in a language that a computer understands.
• Child: What if the story is in French?
  
• Developer: That doesn't matter - developers use a "software language", like Ruby, which the computer understands and is written in English.
  
• Child: So what is a software language?
• Developer: Its like a normal language, with two differences: firstly, the language has a very limited vocabulary; secondly, it must be used exactly - no spelling, punctuation or grammar mistakes allowed.
• Child: So what happens if you make a mistake?
• Developer: The computer won't understand what you've told it.
• Child: Wow! Does that mean you have to get the story exactly correct, before the computer can read it?
• Developer: Weeeeell.... kinda.... Its more like the computer will do something different from what we asked it.
• Child: So why did Vista cost so much to make?
• Developer: <Sigh - lets try a different tack> You remember the story that the developer is translating?
• Child: Uh-huh
• Developer: Well, although the story is simple to the customer; to the computer, its hideously complex. So complex that no single person can understand it all. Therefore the story is divided amongst lots of people, who each attempt to understand bits of it. Its about managing complexity.
• Child: Uh-huh. Are we nearly there yet ...

Principles of software development

My blog reading (Jeff Atwood, Scott hanselman) the past few weeks has turned up some excellent articles on principles of software development. For ease of reference, I've grouped them together here:

• The DRY principle (Don't Repeat Yourself). Published originally in The Pragmatic Programmer, an abbreviated version is available in this pdf.
• Robert Martin's SOLID principles:
• Single Responsibility Principle (SRP) - pdf.
• Open Closed Principle (OCP) - pdf.
• Liskov Substitution Principle - pdf.
• Inversion of Control - martin fowler's article, msdn article.
• Dependency Inversion Principle - pdf.
• Principles of package design - an excellent pdf that covers:
• Principles of package cohesion:
• Reuse-release Equivalence Principle
• Common Reuse Principle
• Common Closure Principle
• Principles of package couping:
• Acyclic Dependencies Principle
• Stable Dependencies Principle
• Stable Abstractions Principle

The next thing I need to mull over is how best to learn, apply and pass on these principles. Watch this space.

NTLM authentication and Firefox

Firefox is great, but using it with sites that use NTLM (e.g. SharePoint) can be a right pain. Unless you have Firefox set right, you'll always be prompted for your Windows credentials.

The solution to this can be found by googling, but I've repeatedly struggled to track it down, so here it is for future reference:

1. At the Firefox address bar, enter about:config and filter for network.automatic-ntlm-auth.trusted-uris. Select the setting and enter the domain name in question, e.g. my-sharepoint.
2. Restart Firefox and you should not longer be prompted for your Windows credentials.

Converting a file's encoding

Handling files that are in the wrong encoding (e.g. Unicode, little-endian) can be a major pain, especially on Windows systems.

Unix comes to the rescue here with the iconv tool, which ships with Ruby.

e.g.
iconv.exe -c -s -f UCS-2LE -t UTF-8 wrong-enc.txt > correct-enc.txt