Innovation’s Black Box: How Corporate Information Deficit Holds Them Back
Salesforce.com transformed the world of customer information, turning a previously fragmented process into an integrated, accessible system of truth. While not always well implemented by companies, Salesforce has made customer relationship management (CRM) a bona fide business discipline. A single source of customer truth is now an achievable reality for corporations. Could we do the same for corporate innovation?
Innovation Black Box
Large organizations generate thousands of ideas each week, run hundreds of experiments to test them, and may implement a handful of them as new products, services, or operational improvements. Over five years, that’s millions of ideas and insights about how businesses can improve. Plus, years of accumulated experience about what works and what doesn’t. The potential for harnessing insights from these activities is at least equal to what salesforce has done for CRM.
However, today information on innovation is hidden in different corners of the organization, accessible only through the collective memory of the people who worked on the projects involved. The problem is particularly acute for large, global, and often siloed organizations, with people are spread out across locations and time zones.
This is what my friend Simon Hill, CEO at Wazoku, calls the “Innovation Black Box” – a void at the center of a company’s innovation strategy where knowledge is lost, inaccessible to the rest of the organization.
This problem has been around for a very long time. I remember working on it in the late 1990s as a consultant. However, the urgency about opening the innovation black box has increased because of the potential of Generative AI. Imagine AI having the ability to scan millions of past ideas, experiments, and decisions, identifying patterns and unsolved high-value problems. This could transform an organization’s ability to innovate, turning missed opportunities into breakthrough solutions.
This is a fabulous vision, but it’s one that has three big obstacles to overcome – data access, explicit versus tacit knowledge, and the use case problem. Here is my understand of these challenges and the potential to overcome them. I would love to hear how companies are solving these problems today.
Data Access Problem
Today, data on innovation projects remains fragmented. It is trapped in disconnected tools like Excel, PowerPoint, Miro, Figma, and various portfolio management systems. Innovation data, unlike customer data, is rarely consolidated into a single source of truth.
Having data locked up in multiple systems makes it very difficult to interrogate. Our ability to use GenAI to identify insights from an organization’s institutional knowledge is effectively blocked if if we cannot access these data. My understand is that CIOs and Enterprise Architects see this problem. Many corporations have initiatives to aggregate information in data lakes, etc. This should allow more of the enterprise’s knowledge can be harnessed.
However, this alone may be insufficient to put innovation on the same level as CRM, unless we can include the tacit as well as explicit knowledge.
Tacit v. Explicit Knowledge
Knowledge management experts draw a distinction between tacit and explicit knowledge. Explicit knowledge is deliberately created artefacts – like project reports or idea competition submissions – whereas tacit knowledge is the implicit know-how of a group.
A colleague at IBM used to tell the story of an effort to get water company engineers in the North-East of England to capture maintenance information in a shared database. They preferred to keep a book under the counter of a roadside café than enter it into a shared system. The data existed in the community; it was just locked away from management. The engineers resisted all attempts to adopt the new technology as it didn’t solve a problem that they experienced.
This is like many innovation projects. Most of the day-to-day experience and activities is not written down or shared formally. It is relayed informally through anecdotes and advice from colleagues. It is written on a white board or a series of Miro boards. It is not recorded in an IT system. The real innovation black box is this tacit knowledge. It is rich in context, and more valuable to a person seeking to learn from the experiences of others.
There are ways of doing this. I think Simon’s team have a tremendous asset in the vast number of open innovation challenges from the Wazoku Crowd. They have been running challenges inside and outside companies for a decade. Another approach is to capture tacit knowledge in a narrative database. Cognitive Edge has such a tool also.
A solution to the innovation black box must be about tacit, as well as explicit knowledge, because that’s where the real value lies.
Use case problem
Problems like the “innovation black box” are maddening because they are universally recognized but difficult to motivate people to solve. What mobilizes action in an organization is a clear, demonstrable problem and a solution that can generate a measurable improvement. The promise to connect all your data in a single repository and make it easy to interrogate is exciting, but non-specific. Its a promise of better outcomes, not an outcome of itself.
What we need is specific high-value customer problem to solve that can generate measurable impact. This is far more likely to mobilize resources to link together explicit data sets and capture tacit knowledge. Our starting point needs to be: what is the problem this will solve for customers and why is our solution better at solving it than any alternative, including non-consumption (in other words what they are doing today)?
Use cases are most likely related to specific opportunities to serve customers, improve service or reduce costs. A generalized desire to get better at innovation is rarely persuasive. CIOs are under a lot of pressure to demonstrate value from Gen AI. They know fragmented IT systems hinder AI’s potential. Finding specific innovation use cases to solve is the right place to start.