Will Things Fall Apart if the Centre Cannot Hold (All the Data)?

Okay starting with a title shamelessly ripped off from W.B Yeats may not be cool, but it does capture the essence of this post. We are in a period when decentralised approaches to data storage and access is in vogue. Blockchain and other Peer-to-Peer (P2P) database designs are two obvious examples, but that else is out there? Why is decentralisation so attractive? Is there a hidden downside? What’s wrong with keeping everything in one place? What is the business problems we are trying to solve? I’ll try to answer some of those questions here (and probably open up a few more as I go along).

For me, this debate in consumer goods over centralisation versus decentralisation comes down to a long list of expensive, unsatisfactory attempts to create end-to-end transparency and a perceived lack of trust (in data quality, people and organisations). Are we as an industry unique in these challenges? When it comes to the scale and diversity of our value chain the answer is probably “yes.” I struggle to think of another industry that embraces everything from a farmer with a few acres in the remotest territory in the world and some of the most sophisticated and automated processing plants.

For many years our industry has relied on two approaches to solving these transparency and trust problems: seek to enforce ever-increasing centralisation (e.g. global processes and company data warehouses) and utilisation of a large number of intermediaries for data pooling and exchange (e.g. GS1 GDSN, E2Open, 1WorldSync, Nielsen Brandbank etc).

Despite demonstrable successes, the result can be an increase in bureaucracy, high data costs, a proliferation of content APIs and a sense that the journey may never end. For smaller value chain players there can be a sense of frustration as they struggle with multiple systems and protocols mandated by their customers and suppliers. What is clearly needed are more elegant process and technical solutions that provides freedom to operate and retain data-ownership while maximising data quality and visibility for all stakeholders. Democratisation of data is trending.

Enter decentralised, P2P solutions that promise to address issues of data privacy, commercial trust and ease of data exchange, without the need for onerous governance or costly technology fixes. In consumer goods, blockchain ‘consortia’ are the most visible examples at the moment (e.g. the IBM Food Trust), but the DataPorts initiative from CGF illustrates an alternative approach that uses more established technical standards to provide discovery and trusted exchange of content.

These new approaches are not without their criticisms. What exactly is the problem we are trying to solve and for whom? For example, in the major consumer goods blockchain initiatives pursuing traceability, smaller agricultural businesses still complain to me about complexity and there being no obvious benefit beyond keeping their retail or manufacturing customers happy.

Then there are technical objections. Take blockchain. As with many new technological developments the transactional performance is typically lower than the established approaches, but it is improving rapidly as competing organisations seek to seize market share. Supply chain may end up being the killer app for blockchain there is still some way to go before a dominant set of technical design and standards appear.

And blockchain isn’t the only Distributed Ledger (DLT) technical approach. The IOTA Project gets away from blocks and sequential chains and replaces them with streams of entangled individual transactions based on a technology called the Tangle. This seeks to address scaling and throughput limitations of blockchain.

Of course, there are also more traditional approaches such as distributed and graph database schemas that can deliver some of the required functionality. Or simply more fully exploiting established protocols for APIs and URIs to connect parties together without substantive technology change.

I think we are over the initial hype of decentralisation but we are still on a steep learning curve. More and more value chain use cases are being tested: recall management, supplier qualification and audit, certification management, traceability, quality testing, sustainability reporting, and B2B payments to name a few. More and more use cases feature advanced analytics and machine learning.

The IBM Food Trust (blockchain) and companies such as Transparency-One (graph technology and blockchain) are already demonstrating that decentralised approaches can scale in consumer goods value chains if you have the right focus, governance and business partners.

The central versus decentralised discussion will continue and sometimes a well-designed central approach will be the answer. However, democratisation of data is trending for the many reasons I outlined at the start of this post. As the various technical approaches mature we need to be clearer on the problems that decentralised approaches solve. Then, and only then, will we be able to make the right choices.

Trevor Davis 
Consumer Products Industry Expert