Episode 186: Martin Fowler and Pramod Sadalage on Agile Database Development

Filed in Episodes by on June 1, 2012 11 Comments

Recording Venue: Skype

Guest: Martin Fowler and Pramod Sadalage

Martin FowlerPramod SadalageIn this episode, we talk with Pramod Sadalage and Martin Fowler about database evolution and agile database development. We discuss the basic challenges for working with a database in an agile development culture and how to include database design and most of all, database evolution, in the usually short development cycles of modern software development methods. We talk about continuous integration for database-related code changes using scripted database schema changes, the usage of multiple database schemas to overcome centralized deployments for development teams, how to migrate data in incremental steps, and what tools can help in this agile environment. We discuss how to break down database refactorings in small and incremental steps, one of the most important parts of integrating database changes into an agile development flow.

Tags: , , , ,

Comments (11)

Trackback URL | Comments RSS Feed

  1. Maybe the best episode since the software archeology episode – maybe even best ever.

    I would like to add a link to a good read to getting started with some of the most basic techniques mentioned in the podcast. In a series of posts K. Scott Allan describes some basic versioning practices: http://odetocode.com/Blogs/scott/archive/2008/01/31/three-rules-for-database-work.aspx
    A very good read for the beginner on this topic.

  2. Solid episode, well done! I believe – and you sort of mentioned it – that the crucial point is the undo path, i.e. if you want to migrate back to an older version. With these tools (I used dbdeploy) you need to write your undo scripts manually and as database constraints might only catch you once the data has moved forward.
    I guess the only way to make sure you haven’t screwed up the scripts is to set up your continuous integration server to downgrade to a far earlier version and make sure that still works..

  3. Hal Arnold says:

    Nice beginning article.
    I was hoping for a description of how one would migrate or modify the production application’s database when that application has continuous transactions.
    We’d like to complete a continuous delivery environment, all the way to actual production release. But we haven’t been able to figure out how to roll back migration or mod changes to the db because in the time that the modified scripts are ‘out there’ there, even for a short time, there can be literally thousands of writes and writes based on the changing db.
    We’ve thought about having a parallel schema, and keeping both up, where we queue up all the changes, so that we could roll back and then run the changes to the ‘rolled back to’ old schema, but haven’t had the courage to try.
    Any ideas on this score?

    • Dan says:

      In our production environment we are doing just what Martin had mentioned, snippets of incremental schema changing SQL and it is working just fine. Whilst we could advance any older db to any arbitrary point in time, the issue of (incremental) rollback we have decided is too scary to consider. Full restore is the remedy for that edge-case. All I can say is: enough testing so you feel that edge-case is unlikely.

  4. I’ve experienced both worlds and undoubtedly, the best way to approach database evolution incrementally is by using the cited techniques.
    Unfortunately I’ve seen companies demanding the delivery of Erwin files (or another database maintanance tool) which in most of ocasions cannot be automatically generated based on the migrations/patches.
    These situations might generate extra efforts if we fail to convince the customer that keeping database under version control is a much better way of working.

  5. Chris says:

    I’m going to have to agree with Steen on this one, best episode ever.

  6. ross says:

    Good episode.

    However, I’d be surprised to work on a project that didn’t roughly use the proposed approaches. Every project I have worked on for 10+ years has used similar techniques.

  7. Artie says:

    What strategies are people using to handle branches? In the past, we’ve come up with namespaces for sets of scripts and all we had to do was make sure that the namespaces were executed in the correct order. I know that rails uses timestamps now. Have people tried anything else that worked?

Leave a Reply

Your email address will not be published. Required fields are marked *