The ‘Nature’ of Data Ecosystems

There have been a number of studies in the last few years about the success rate of data projects, and the results are disconcerting. Depending on whom you ask, 70-90 percent of data initiatives fail to deliver on their promises. Why is this the case, and what can we do to prevent this? Last week, we co-hosted a workshop on “Building a Data-Driven Culture” with our friends at Innovate Newport to discuss ways to prepare for a data implementation that can set you up for success.

At the beginning of our discussion, I mentioned some key differences between business intelligence projects and other technology initiatives. The fact is, BI projects cannot be looked at as merely technical undertakings. It is not something you can hand off to your IT team to implement and return. We used the term “ecosystem” to capture the inherent complexity of a strong, enterprise-wide data pipeline.

Lessons from Nature

We looked at ecosystems in nature, from the perspective of the monarch caterpillar. Along its journey from egg to adult butterfly, it interacts with several different components within its ecosystem. As a caterpillar, its only food source is milkweed leaves, which has led to an interesting co-evolution of monarchs and milkweed. Milkweed leaves produce a poison that is harmful to monarchs, to which monarchs have adopted a resistance. In turn, milkweed plants have made their poison more powerful, and monarchs have responded by developing stronger resistance. It’s an evolutionary arms race with two components of the ecosystem constantly responding to one another through generations. Additionally, milkweed produces a separate chemical that, when ingested by monarchs, attracts wasps that are deadly to caterpillars—introducing another species into this relationship.

Of course, the monarch caterpillars that survive emerge as butterflies, pollinating a range of flowers that provide important nutritional components for animals all the way up to us humans at the top of the food chain. It is intricate, it is complex, and any disruption in this tight integration can cause ripple effects throughout the environment.

The Business Intelligence Ecosystem

Like monarchs in their natural ecosystem, your data will touch a number of different components on its journey towards providing valuable insight. The technical components alone can be intimidating. Source data applications, extract-transform-load (ETL) processes, data warehouse, data marts, and of course, reporting and analytics applications like Power BI, Qlik, and Tableau, all integrate tightly to convert data into information. But the technology is just a small part of it. I identified a “three-legged stool” of the modern data ecosystem:

  • Technical. These are the systems and software mentioned previously, but also infrastructure—where data will live and how it will be processed—and technical people, including analysts, report developers, data engineers, and anyone else who will be managing or supporting these systems.
  • Business. For your data ecosystem to provide actual insight, it needs context from business leaders, not just strong technology. It needs to align closely to your over-arching strategic objectives.
  • Management. When we think of management in a BI context, we must consider three different angles: people management, to ensure that your team is making data an integral part of their day-to-day; project management, which is particularly important for data-related initiatives given their complexity, and management of culture, as adoption of a data-driven mindset into an organization’s culture is key to the success of any large data project.

Each of these factors creates a potential point of failure. However, unlike monarchs and milkweed, your business can’t afford to adapt over several generations. You need to get it right the first time, or you’ll be left behind. How can we ensure success when the odds are stacked against us?

A Roadmap for Success

So how can you transform your data from a humble caterpillar into a noble butterfly? We identified several tips for building, maintaining, and sustaining a data-driven culture, but we highlighted two key components that ought to be part of any data strategy.

  1. A comprehensive, detailed plan, with flexibility to adapt for the future. It seems somewhat contradictory, but you need to plan your implementation with as much detail as possible, while understanding that your ecosystem will never actually be “done.” It will constantly evolve. Every time you add a new software application, you will need to consider what you want to do with the data and how you’ll incorporate it into the ecosystem. A well-designed data architecture will consider all of your current data sources, but also be flexible enough to easily incorporate new data sources as they arise.
  2. Quick wins. Once your project plan is complete, step back and take a look at it. Can you identify any small projects that can be done earlier in the process to build momentum and keep enthusiasm high? What are some pain points that data can help with right now? You don’t have to wait until the full ecosystem is in place to start realizing an impact. Tweak your plan to facilitate early deliverables wherever you can, and move those tasks up in your timeline if you can do so without making things more difficult later on in the project.

The complexity of a data ecosystem can be humbling, but don’t let it intimidate you. If you’re ready to take the next step in evolving your approach to data, we’re here to help!

Wondering where you stand on your data journey? Click here to take our BI Maturity Quiz. Or, if you’re ready to take the next step, learn more about our data strategy session.

You May Also Like:

Share this: