New project: Denormalizer


I was working on South integration for a project the other day and realized that I wanted to denormalize some things.

Like, keep the number of pages in the site update in the update record, the number of errors, title in the site update page record rather than sucking it from the saved HTML, that kind of thing.

And, I realized that, when things go wrong, there should be an automated way of fixing/updating the denormalized data.

Hence, the Denormalizer.

The basic idea is to specify the denormalized pieces of data in your database in a format resembling a South migration. You provide rules that say “run this query on that database, and stick the results over here.” Every denormalized piece of data has a rule for creating it.

Then, whenever things get hosed, or you’re just feeling insecure about the state of the data, put it into “maintenance mode” and have the Denormalizer go and count things up, extract other things (like the title, above) and fix up the denormalized fields.

Any time you want to make a new denormalized data chunk, just specify the rules for it, and off you go…it’ll be just like it was always there being dutifully updated along with the “normal” fields.

Pretty cool idea; wonder if I’ll ever get to implement it…