I remember when I started my career in tech, many of my customers were moving away from mainframes to distributed computing for those applications that required more human involvement. As a percentage, traditional mainframe programs shrunk and the skills necessary to maintain them became scarce. I vividly recall a conversation with my customer at the time, Citibank, who shared their concern about updating a particularly important COBOL program because “the guy who wrote it is dead and the guy maintaining it is holding a gun to our head”.
Fast forward to today. Data warehouses are still by and large built by hand; very little automation except for some graphical interfaces and scripts for the design tools. I recently met with a prospect who spoke fondly about their data warehouse as if it were an elderly relative; about how much it had helped them when they first wrote it 12 years ago, but how, over time they had lost the original model, had hired a 3rd party who did not document the work and that a recent attempt to change it had ended in disaster, actually keeping the firm from closing their books for the quarter. In essence, it was now in “assisted living” mode but he made it clear it had a DNR (do not resuscitate) order in place. He then introduced me to the one programmer left who had any knowledge of the system. He had an eerie calm about him and at $175/hr. on a two year contract I am pretty sure he was designing his retirement home.
This is the cycle; as more productive technologies emerge, older technologies become “non-interesting” to those who want to be contributing as much as possible to the business. Just last week, Oracle signaled a clear shift by discontinuing Oracle Warehouse Builder in favor of Oracle Data Integrator. Yep, you guessed it, OWB used to be a free tool; ODI is now fee-based. It’s essentially a new tool that does the same thing as the old one with some front-end visual updates and more emphasis on data integration for applications instead of warehouses.
What’s really required here is a new approach that eliminates the need for ETL in the warehouse. Our product teams have come as close as you can get with the Kalido Information Engine v9. Its Unified Load Controller automatically decides how best to handle data and move it from source to staging. Transformations are carried out along with loading, all without human intervention. This addresses the area that typically takes 80% of the time and effort in building a data warehouse.
You should care about this because it allows you to deliver value to the business faster but more importantly, it institutionalizes the process so you will never be held hostage. Our very first customers are still productively using our products some 14 years later and getting good value from them, despite the expected turnover in staff from year to year. That’s a great ROI story. Even better, with the new functionality, they can eliminate more tooling and save more money than they were already saving.
So I suggest you get ahead of the curve now and explore the new features of KIE v9. You won’t be disappointed.
