There are two opposing forces on the earth of knowledge: an general consolidation throughout the trendy knowledge stack & an enormous enlargement pushed by AI capabilities. AI is rewriting each rule about what’s potential with knowledge in 2025.
Listed here are Idea’s High Themes in Knowledge in 2025 with the complete presentation on the backside.
-
The Nice Consolidation. After a decade of increasing complexity within the trendy knowledge stack, corporations want to dramatically simplify their architectures to drive higher outcomes. Patrons we communicate to say, “Don’t promote me one other instrument.”
In consequence, we’re seeing consolidation on particular person cloud knowledge warehousing platforms Snowflake & Databricks, the place most enterprises have picked their dominant structure. There may be additionally a wave of consolidation inside BI favoring collaborative BI instruments that stability centralized & decentralized management like Omni. There’s a race to command extra & extra compute inside these consolidated platforms as a result of nearly all of income & in the end income resides there.
The workplace of the CFO continues to use strain to drive extra ROI on core knowledge & AI, which is a shock given 2010-2022 knowledge budgets grew unabated & the fervent curiosity in AI. Specifically, this has pressured the cloud knowledge warehouses, & clients are searching for novel architectures to dramatically cut back cloud knowledge warehouse spend the place potential. 50% price financial savings are potential with newer transformation architectures like SQLMesh / Tobiko Knowledge.
-
Scale-Up Architectures. New cloud knowledge storage codecs are growing in significance though slower than anticipated by way of adoption as a result of they lack the related enterprise instruments. Nonetheless, in the long run, this creates an increase in workload-specific question engines like MotherDuck & Datafusion.
These question engines are sometimes scale up reasonably than scale out. This implies builders can begin on their native machines & reap the benefits of the exceptional computing energy of their MacBooks to deal with all of the overwhelming majority of workloads.
-
Agentic Knowledge. If the IT division sooner or later is the HR division for AI brokers, we must always anticipate that knowledge can be remodeled simply as a lot as each different group. Traditionally, there was a divide between software program engineering & AI/ML groups, & that can change. Lots of the software program design ideas, like digital environments, will come to knowledge.
We should always anticipate the overwhelming majority of SQL queries to be executed by AI. To make these queries correct, knowledge modeling turns into a fully important know-how to eradicate hallucinations & assure high quality. As well as, knowledge observability instruments like Monte Carlo will turn out to be more and more vital as knowledge not solely feeds the core BI & analytics layers but in addition the AI programs which can be key elements of each manufacturing software, each inner & exterior.
That AI ought to drive some fairly vital efficiencies that parallel the 25% to 50% productiveness positive aspects throughout Google, Microsoft, & ServiceNow, or the $275 million in price financial savings of Amazon migrating from one model of Java to a different. This could free budgets for brand spanking new initiatives.
Smaller fashions will dominate throughout the enterprise with the candy spot someplace between 10 billion to 70 billion parameters, due to a 600X distinction in inference price & very comparable ranges of accuracy. If the wave of innovation from DeepSeek is any indication, there’s considerably higher efficiency & a continued deflation & inference price over the brief & intermediate time period.
Additionally on a enjoyable observe, this presentation was the one I synthesized utilizing AI. When you have suggestions, let me know, & the complete AI speaker notes can be found right here.
