Just about every other management consulting firm out there claims to have some “proprietary” framework or methodology that somehow makes them better. Having worked as a management consultant and gotten an MBA I’ve been exposed to my share of frameworks, and despite the criticism they often get I’ll admit they have their place. So I thought I could put forward a simple framework I often use (subconsciously) to get things done (if you got into Web 2.0 early on this might seem pretty basic):
(1) Identify and learn your data sources – this could include getting the OWL ontology for an RDF data source, the XML schema for data returned from an API or table fields for CSV files or online tables like Yahoo DataTables. You should do exploratory queries to ensure you can retrieve the data you want.
(2) Identify common values – often, you will be using one value to obtain data from multiple sources about something (for example, you use the name of the company to get the location of its HQ from one API and the names of its subsidiaries from another), or you will be feeding values obtained from one data source to get information from another source (you get the address of the HQ from one API, then feed it into another to get its coordinates). Identify those values and test.
(3) Figure out how you will process the data – you may want to do something simple like aggregation or sorting, or something more sophisticated like feeding it through algorithms. Again perform some tests, preferably with local copies of sample data.
(4) Map data values (or derived data) to elements of your visualization or interface – this is the part where you get to be creative and is usually the hardest part.
The time spent on each step can vary widely. I will try to put more meat around each of these steps in further posts.