Inmon versus Kimball : the agile development of a data warehouse

Abstract

The aim of this study was to develop a new data warehouse development approach known as the agile data warehouse methodology using either Inmon's or Kimball’s approach as the foundation along with the best practices associated with three agile system development methodologies (extreme programming, SCRUM, and feature driven development). The agile data warehouse methodology aims to provide guidelines for a development team that tackles the unique technical aspects of data warehouse development along with the issues associated with it, using agile best practice in order to develop a data warehouse in an agile environment. In order to achieve this aim, an in-depth literature study on agile system development methodologies was undertaken. The literature study started off by identifying the reasons behind the birth of agile system development methodologies, followed by a definition and discussion of an agile system development methodology. This was followed by a brief explanation and discussion of extreme programming, SCRUM, and feature driven development. In addition to the literature study done on agile system development methodologies, a literature study was carried out on data warehousing to reveal the history of data warehouses. The discussion on the history of data warehouses was followed by the discussion of business intelligence in order to resolve the uncertainty between the terms, data warehousing and business intelligence, and to clarify the relationship between the two terms. This was followed by the justification of a data warehouse system and the literature study on data warehouses was completed by discussing the approaches of the two giants (Inmon and Kimball) in data warehouse development. Following the literature study, two questionnaires were developed and distributed to determine the best suited candidates for the development of the agile data warehouse methodology. The data warehouse questionnaire was targeted at revealing whether Inmon's or Kimball's approach is best suited for the agile development of a data warehouse system and revealed that Kimball's approach is best suited in this regard. The agile system development methodology questionnaire was focused on determining which of the three agile system development methodologies (extreme programming, SCRUM, and feature driven development) is best suited for each major phase of development for Kimball's approach and revealed that SCRUM is best suited for the planning and the implementation phases, feature driven development is best suited for the design phase, and extreme programming is best suited for the development phase. Based on the knowledge gained from the data warehouse and agile system development methodology questionnaires, version 0 of the agile data warehouse methodology was developed and tested in collaboration with an industry partner by implementing the agile data warehouse (version 0) methodology in the actual development of a data warehouse system in a real-world environment. In order to evaluate the effectiveness of the implementation of the agile data warehouse (version 0) methodology, the version 0 questionnaire was developed and distributed and revealed that the agile data warehouse (version 0) methodology succeeded in the successful and agile development of a data warehouse system. Based on the implementation and evaluation of the agile data warehouse (version 0) methodology, version 1 was developed to result in an improved version of the agile data warehouse methodology. Version 1 of the agile data warehouse methodology was evaluated by developing and distributing the version 1 questionnaire, which aimed to verify the usefulness of the agile data warehouse (version 1) methodology which consisted of Kimball's approach as the foundation along with the best practices of extreme programming, SCRUM, and feature driven development. The final verdict of the study is - The agile data warehouse (version 1) will provide assistance and support to a project team in the successful and agile development of a data warehouse system in a development environment characterised by uncertainty and changing requirements. In conclusion, the agile data warehouse (version 1) methodology introduces a new innovative approach to data warehouse development to provide assistance and support to project teams for the successful and agile development of a data warehouse system. Therefore, the agile data warehouse methodology is recommended to project teams who desire to shift their data warehouse development process towards a more agile environment or for data warehouse development projects faced with uncertainty and changing requirements.