The valuable insights from data gold mining are often suspected, or even known to exist, yet they remain buried.
The fundamental challenge of an organization’s data is transforming it efficiently and effectively into physical products and services that maximize revenue and maintain competitiveness. Same old, same old, yes? But a host of new tools are helping product developers dig deeper into their terabytes of data to uncover value and actionable insights that result in better products and organizational performance.
In turn, these tools and the techniques needed to use them cost-effectively are spurring product developers to find and maximize real-world value and are working to incorporate them into their products and services.
The resulting gold being mined by these new tools and techniques is not by itself some long-sought breakthrough but an intermediate step in end-to-end product lifecycle management (PLM). And the counterpart is just as critical. Before gold is forged into something of value, it’s meticulously assayed to determine its purity.
The valuable insights from data gold mining are often suspected, or even known to exist, yet they remain buried. Managers are unwilling or unprepared to sift through this data and put it to use. Amid resource scarcities and time pressures, few organizations know how to do this effectively or affordably—the perennial headaches of data mining.
Mining for the data gold is often seen as too iffy to promise an ROI. And yet this gold is often related to the biggest challenges facing every product and system: obsolescence and failure, ever-evolving user wants and needs, marketplace disruptions, competition, sourcing, service, and rising costs.
The challenge in data gold mining is no longer just about getting data; it’s about the strategic application of new technologies to transform that data into actionable insights and value. The following areas are key to this challenge:
- Tightly integrating the most common forms of generative AI and agentic AI into PLM platforms
- Better techniques for generating AI queries
- Better analytical tools…descriptive, predictive, and prescriptive…to make sense of the returns from AI queries.
These tools and techniques seek to find gold in the form of hidden similarities among seemingly unconnected and unrelated phenomena. This includes feedback from customers who have little or nothing in common and are using dissimilar products and services.
Gold is also buried in the incongruities in sales orders and rejections, field service, warranty claims, manufacturing stoppages, and supply chain disruptions. Whether the data is structured or unstructured no longer matters. Ditto for whether the data is internal or external. For example, incorporating parts of the Industrial Internet of Things (IIoT) and their connected data-generating devices into the Large Language Models (LLMs) on which AI is trained.
What is also new is the size and depth of databases searched, as well as how these new tools and techniques overcome the disconnects that plague every organization’s data. These disconnects include bad and useless data formats; errors, ambiguities, and inaccuracies; data buried in departmental silos; legacy data with unsuspected value; and data that is misplaced or lost.
All this is aided by digital transformation in all its myriad forms. Digital transformation is increasingly vital to gold mining because it helps users gather terabytes of data into more complete, more accurate, more manageable, and more focused LLMs. Digital transformation can also help users pinpoint what is (still) needed for timely/effective decision making (e.g., data that did not get into a given LLM and should be for subsequent queries).
CIMdata itself is adapting by:
- Broadening its PLM focus to work with clients’ AI projects, creating an AI Practice with Diego Tamburini as Practice Director and Executive Consultant. He has held key positions at Microsoft, Autodesk, SDRC (now part of Siemens Digital Industries Software), and the Georgia Tech Manufacturing Research Center.
- Expanding its work and capabilities in digital transformation that enables PLM—and is enabled by PLM in turn. The ultimate goal is to bring together engineering technology (ET), information technology (IT), and operations technology (OT) at the top of the enterprise.
Managers and staffers who receive these AI and analytics findings have a similarly daunting agenda. They must learn how to discern and understand what the gold is telling them. And they must learn how to weed out what has already been simulated, designed, or engineered for production. And they must learn how to choose the most viable of these tools and techniques and how to manage them.
Effective data governance is crucial to gold mining. I strongly recommend a review of an AI Governance webinar written by Janie Gurley, CIMdata’s Data Governance Director, and me. Posted July 8, 2025, by Engineering.com, available at https://www.engineering.com/ai-governance-the-unavoidable-imperative-of-responsibility/
Further insight is available in my most recent Engineering.com posting, available at https://www.engineering.com/in-the-rush-to-digital-transformation-it-might-be-time-for-a-rethink/
Getting the gold into product development offers many potential benefits by uncovering:
- Causes of product and system failures
- Unexpected obsolescence in products and systems
- Users’ new wants and changing needs
- Early indications of marketplace disruptions
- Insights into competitors’ strategies
- Pending shortfalls in sourcing…and alternatives
- Likely cost increases and finding options
- Unrecognized service needs
- Better methods for production and operations…thanks to AI’s new ability to handle streaming data in LLMs.
Over time, diligent searchers may turn up dozens of connections and correspondences in these nine bullet points. Some will be simple, random coincidences. But many will turn into gold that reveals both opportunities and challenges.
Managing resistance
I urge readers to push the envelope, to find new approaches to everyday tasks, and try new things. While there is always pushback from staff and managers who are already overworked, without encouragement, change will never happen.
The usual response is, “Yes, you’re right, we’ll get to it later.” We caution that they are letting routine tasks get in the way of potential game changers. Ignoring these issues will not make them go away.Yes, obsolescence and failure, user wants and needs, marketplace disruptions, and all the rest (see above) will eventually surface, gain urgency, and become tasks that everyone must address.
Inevitably, this new gold will have to be dug out from data we assumed was useless and then painstakingly engineered into products, services, and forecasts. These values will mandate changes to the organization’s facilities, systems, processes, business partners, and suppliers. And they will have to be communicated to the sales force and distributors.
And there will be resistance to change and its huge costs, which are the underlying themes of this article. Those costs are part of addressing newly uncovered digital gold in a mad scramble amid fears that errors and oversights will place profitability, competitiveness, and job security at risk.
In summary, the quest for digital gold is our modern-day equivalent of the myth of Jason and the Argonauts and their search for the Golden Fleece. Like Jason, we must embark on a perilous journey and overcome countless challenges to seize a prize that promises untold value—transforming raw data into profitable products and services.
