With each new year the world resets, we take a little time off to spend with our families, think about what was done and then it’s time to think about the year ahead and what we need to achieve. I thought I would take a moment to talk about some evolution that I am seeing in the year past. For those of you that do not know, I picked up the family and moved them to Seattle last Christmas. It has been a big change for the family, we are all settled in and the little girls are speaking with American accents, that’s evolution. So what does the world look like from an analytics perspective ?
I am working with eBusiness companies on the West Coast, every business has a different business model and the data volumes are truly amazing, so it is fair to say that the world is exciting and evolution is the norm. When I arrived, the world looked a little glum unless you worked in open source Hadoop. The 12 months interceding have been a great learning for all of us in analytics, the Hadoop people are starting to realise that the world will not just be Map-Reduce and the relational people are realising that Hadoop is here to stay. The big conversation today is how do we make all these pieces fit together and work nicely ?
So today, we find ourselves in a world of Architects, which is to say we are not short of the “right approach” to do this, incidentally all the approaches are different.
The great challenge is that there is no single perfect architectural model, each application will have specific data and processing needs and subsequently architectural model. The architects are going to have to find a small number of design patterns that support the permutations around the application architecture models. And then work with the development teams to roll out these patterns, the great fear will be the supportability of multiple platforms analytic environments if these design patterns are not forthcoming soon.
Some work that I have been doing recently has been around identifying what we should be building patterns for in our new multi-system environments. The answer has been simple enough, analytics business needs has not changed, the business still has all the same requirements around latency, traceability, quality, etc. The only bit that has changed is that we are using multiple systems and we need to figure out how these ideas will work across the multiple platforms.
If you are having these discussions then I hope you involve some of the smart people that I have worked with over the years as they are providing some great insight into these same discussion all around the world.
I hope you all are looking forward to the evolution as much as I am.
