A few weeks into a TDD style project

Programming

So, as you may have read in my previous rant regarding Test Driven Development, I am one of two developers participating in a test of TDD. The project is relatively small in scope and is mostly in a stand-alone “engine” so really, I think it's a pretty ideal test subject. If you were paying attention, your spidey sense started tingling when you read “…an is *mostly* in a stand-alone engine…” Yes, well, as one might guess the mostly is now causing us some grief. The subsystem I am working on interfaces with the main application so that it knows when it has work to do. In the final design of the subsystem, it is a stand-alone windows service consuming application events from the event queue. Now, not only is this a test TDD project, but also trying to incorporate some Agile/Scrum techniques. I think maybe this was a mistake. Agile makes more sense when you have a team of greater then two. So really I think we are suffering an issue stemming from Iteration grouping. Our 1st iteration is quite large and provides the framework for the rest. However, we are not able to take advantage of smaller implementation items that can be pushed to later interactions. Which causes us to call my sub-system from within the application that writes to the event queue, instead of just reading the event queue.

Now, here's where it really gets ugly. Due to how we have our name-spacing and library hierarchies, I have to move my subsystem completely out of the main project library and add it to the library code it interfaces with. And later I will have to move it back. The sub-system isn't huge, but that's a pain. 

And why didn't we see this coming? Because we are implementing from what the tests dictate we need, not what we will need for the entire project. Some may argue that the tests apparently were not complete and I agree. But I think it's a little unreasonable to know that from the test perspective until it's discovered by actual implementation. And these happen late in the game so a change is not only a change to the implementation but to the entire underlying tests which defeat the purpose of starting with the tests.

I guess one might further argue that TDD is not a replacement for detailed design. But I have to wonder, if the design is indeed so detailed as to reveal this issue before any coding (and I have seen designs that would fit the bill, as well as designed some of that ilk for prior employers) that one would not need to start with the tests as testing techniques would have been built into the requirements and design. Of course if you can , why not as it prevents people from getting time crunched and not implementing tests at all. But isn't the point of TDD to speed up and allow for iterative design (as well as pushing testing to the forefront, which I praise it for) ?

Regards!