I recently attended a Microsoft seminar on BI (business intelligence) and who should I meet there but a former colleague, who worked with me in the field of what was then called decision support (in the days before the term BI was coined by Howard Dresner).
The first interesting point made was that, over 20 years since I started in BI, one of the continuing challenges is that the ability to access data is limited. That may be explained by the next point. An interesting diagram was shown during the session which highlighted some of the issues facing BI (I have adapted it here):
My era was very much that of Traditional IT, whose prime concern was control. Data was extracted from the source (usually operational systems), massaged and finally provided in a different form to the user. This ensured that the data came from the right sources, followed the approved definitions, and used the right formulas. If you follow this approach, access to data is going to be limited.
The problem with the traditional approach is that it creates other challenges, which again were issues we faced 20 years ago, namely:
- IT expends a lot of effort in providing information that is often used by one person, once;
- information requirements change too fast for the technology and approach to keep up.
The solution from the Microsoft perspective is to encourage self-service BI, with the emphasis on agility. It helps to understand that Microsoft advocate their PowerPivot product as the route to enable self-service BI. One unquestionable fact is that the goal of the universal BI front-end tool has been reached – and that tool is Microsoft Excel.
The problem with self-service BI is that the issue of proper understanding of the source data is not managed. It actually gets us back to the days of fourth-generation languages, such as SAS and FOCUS, when users with some technical understanding could pull data from mainframe databases without asking for the help of a programmer. This led to situations in which two people could produce information in a meeting, using the same source, but with differing results. That created the demand for a ‘single source of the truth’, something which the data warehouse was supposed to address, which brought the control of data back into the realm of IT.
So it seems that BI’s problems continue today in much the same way as they have done in the past. How can we solve these ongoing problems? If you have a suggestion, please let me know.
Notes from the UK version of the BI seminar can be found here.