6 months down the line how do I know which order to apply those changes?

I wrote a blog post last year on the very same topic: http://shashivelur.com/blog/2008/07/hibernate-db-migration

@Jeroen: Haha. Nice one :)

@Eggsy84: If you are really serious you can include undo scripts for each migration. A database backup before applying the changes is helpful too, although that depends on the size of the database.

We have implemented this way tracking our SQL changes. Currently up to over 130 ;)

One thing we have noticed is that it become very hard to roll back the changes after you have applied them.

If (and lets face it when) an install goes wrong and you have already performed your SQL update what do people tend to do?

You can of course take a DB backup before the update but for large databases this might not be feasible due to restore times.

Any other suggestions?

Eggsy

It needs the organisation to mandate and enforce proper change control where every change to a production system gets handled formally and creates a log record. Really, structural changes should only be doable from a controlled admin user and preferably on the machine hosting the database. Of course, this may not work with all RDBMSs :(

1) http://www.Liquibase.org
2) http://dbdeploy.com

Both use more information in their table than this to spot other types of errors and don’t require you to produce a custom solution. Liquibase is especially good as it can deal with the going backwards in time problem found when doing branching and automated database releases.

@Mike Marcos: OK, that works until you release your software and you enter the next development cycle. You need a new script then.

@Rob: Not sure how GIT solves the problem.

@Tiago Albineli Motta: Thanks for the info, will definitelly try it out. My proposal has 0 learning curve though.

@The B.: Interesting addition in case your environment requires it.

rake db:migrate

You make changes to the db? Commit that build script.

You sit down to start working? Update your project.

Uh, oh, merge conflict? Oh, look, we both tried to change this table, let’s figure out how to change it to both our advantages.

A sound strategy it is, though. I personally wouldn’t be comfortable with the application being able to run ALTER commands (that is what we have DBAs and DBA automation for), but that’s as much personal preference as anything else.

Cheers!

http://bitbucket.org/stepancheg/mysql-diff/

to compare database schemas. It outputs difference as sequence of ALTER TABLE statements. It is not always reliable, but very helpful.

Each ticket (read: project) has associated files and revisions. Some of these files are in the /schema_updates/ directory. We can push, or pull, tickets from production using a script I wrote. One of the steps is to apply the schema_update. To avoid name collisions we use the format [ticket_number]_up.sql for the code to apply the schema, and [ticket_number]_down.sql if the ticket needs to be pulled from production (or QA, for that matter, which has its own database).

The same push/pull (well, mostly push) script will also tell you the status of a ticket (is it in QA? production? not even that far? in between?) so it can tell you if any particular schema update has been applied.

We were doing API version tracking when I was working in S/390 assembler using fixed-width flat files with a header and a footer. Everything old is new again.

201012081445_CreateSomeTable
201012081617_AddIndexToFoo

Rails got away from integer organization a long time ago and its my preferred method on my PHP migrations framework: https://github.com/ruckus/ruckusing-migrations

I also like to use vlad the deployer inside non-rails projects, to handle automating ssh for deploying my app to multiple servers / environments from source control. this can run the migrations for you automatically.

http://msdn.microsoft.com/en-us/library/ms180047.aspx

Use these extended properties on the schema level, with property name matching your schema migration script that has been applied. Use the value to track when schema migration was applied.

Cheers

Adam Żochowski

@Stuart Grimshaw: Great question. We usually work on trunk, and db changes on branches or local dev experiments do not go into the migration script. Although I didn’t mention it in the post, the migration is a 1:1 match of a releasable project version. So if something is only exists in a branch then it’s sql delta scripts do not (yet) belong into the version migration script.

I create 008-create-this-table.sql and someone else creates 008-create-that-table.sql it’s fairly simple I guess to rename one of them to 009 when the first hits the master branch, but at our company we can have 3 or 4 projects on the go, not to mention branches for bug fixes too. With that much going on, it would quickly become unmanageable.

We’ve been looking for a solution to this problem for a long time, and it’s not easy :-)

If you’re interested in helping to make your process available to anyone and would like to improve MP as well, we’d love to have your help with the project!

@Price: I keep in a folder all sql scripts which take my database from point A to point B.
Regarding rollback, yes I could have undo scripts as well but it’s an overhead. I know (from a podcast) that Amazon does that. I will usually grab a database backup just before running those on production. Of course anything that will be run on production has already been tested on staging.

I recently took over a database which has no versioning whatsoever. All changes take place directly on production. This will help in finding a good way of stopping this madness :)

A couple of questions:

- Barry Coughlan mentioned keeping the ALTER TABLE scripts in a seperate folder. Why would you want to do that? I can’t think of any scenario where that would help…

- How do you handle rollback actions in case something goes wrong and the new version must be rolled back to the previous one?

ActiveRecord in Rails and LiquiBase both do that with schema_information and DATABASECHANGELOG tables respectively. Why not utilise that instead of creating your own?

Another improvement that could be done is to do a check in th ebeginning of the script that it is applying to the correct version. eg 007 refuses to run if current version is not 006.

http://code.google.com/p/c5-db-migration/

Automatic upgrade of the schema via the application is out of the question in large and high traffic projects where the schema migration (with its 3 phases: expand, release, cleanup) may last days do to alters and data fixes on huge tables.