The total cost of operations (TCO) of Business Intelligence (BI) systems is often measured in three categories: time-to-completion of projects, on-budget completion of projects, and cost per user of BI applications. There is a key process in every project that impacts all three categories: Business Requirements Engineering.
An effective requirements methodology ensures that project scope is clearly understood and costs accurately estimated. At the same time, when we deliver what users want, usage and adoption of the solution increase the user base. Why then do so many programs not take a closer look and the effectiveness of their approach to this key part of the process?
Requirements gathering methodologies have been refined for decades, but we consistently see inadequacies and gaps with our end users. The “Business Requirements” deliverable has become a checkbox for most Project Managers and a nightmare for Business Analysts not familiar with BI.
The biggest issues with traditional methodologies are:
Agile methodologies are becoming more and more popular in some types of projects. We are seeing higher user engagement in the entire process and of course, quicker deliveries (albeit with iterative functionality). But, how do we improve the requirements process for traditional BI methodologies? How do we deliver a world class product that attracts a large user base? How do we deliver functionality faster and at a lower cost?
I believe it starts with formulaic requirements tied to particular project types. Too many times, we hear Business Analysts say, “OK business units, what do you want?” It usually starts with a report mockup and they begin documenting all the fields and metrics. As page filler, we see things like:
There is a place for these type of requirements, but they certainly lack the detail necessary to build a successful solution. Instead, we must consider the type of project we’re undertaking first.
This project could be the introduction of an enterprise data warehouse (EDW) and BI platform where none existed before, or replacing existing infrastructure. Most commonly, the project involves the creation of a new BI application, report suite, and/or the integration of new data sources into an existing solution. Other types of projects exist, but are generally IT-driven such as platform upgrades, application simplification, and so on.
These projects are usually driven from the consolidation of disparate departmental solutions. Ralph Kimball and others have done a great job of providing the recipe for kicking off new programs:
The key activities in the requirements stage involve a deep partnership with executive sponsors and steering committees. User interviews are conducted but the requirements are kept at a much higher (conceptual) level than specific applications. The goal is to gather enough information to present both a business and technical roadmap for the program.
Both program and project level initiatives lead to new infrastructure. External forces such as procurement requirements, product support deprecation, or organizational changes often impact platform change decisions as well.
The requirements process is very different for this type of project. Consider organizing the specifics into big buckets, including:
Most BI platform vendors have similar approaches when sizing your system and licensing needs. You can leverage this during your requirements sessions.
We frequently see philosophical “chicken or the egg” debates with these projects. Consider a scenario where a new transactional system is being developed or a new third party extract is available for consumption. Do you proactively go after this source and integrate it into your environment at an atomic level following your existing best practices? Or do you wait for business funding and a specific BI application need to tie it back to? Said another way, do you build it and hope they will come knowing that your architecture will support most business requirements?
I’m a big fan of integrating as much data as possible as quickly as possible. Too many times source systems purge data limiting the types of analytical solutions that can leverage it. More data means better data profiling. This approach mitigates this risk and reduces integration costs. Don’t get me wrong, there will be enhancements and rework needed once the BI application requirements start flowing in, but hopefully you are 80% there.
More often than not, funding wins this battle and the integration must wait for official projects. Regardless of the inception method, the project will be most successful with hardened non-functional requirements centered around:
Start by building frameworks and guidelines around these types of requirements. When the business project hits, you can easily work through variances.
Before starting the detailed data requirements, architects need some cycles with the data itself. This involves profiling, looking for relationships, and business scenarios. This puts them in a good position to take decision points to requirements sessions rather than rework requirements after design as started.
Functional requirements are more centered on the specifics of the source and desired state. Consider these areas in your framework or requirements solicitation:
These could be reports, dashboards, mobile applications, or any other presentation of the data requirements to the users. These have their own non-functional areas such as compliance, security, and performance, but most of the time they will inherit overall constraints driven from the platform.
Once a good understanding of the data source for the application is achieved, requirements should focus more around functionality. We classify them into:
The major gap we see in BI application requirements is a lack of communication. These are living documents that must be allowed to evolve throughout the project and as the solution is showcased to the users. During the process, be careful to protect scope but be mindful to the wins you can achieve by deviating from the exact mockup to provide additional business value.
Enable your Business Analysts and Architects to be more effective during requirements engineering sessions by providing frameworks and guidelines targeted for specific project types. The requirements you gather are the key to delivery success, perceived or otherwise.