Snowflake Innovates on Performance & Efficiency While Reducing Costs

Posted on by

At Snowflake, our product philosophy is centered on a single unified platform, and thus, a single core engine. To support all your workload needs with unmatched performance and efficiency, we continuously enhance and deliver performance improvements on this core engine. This approach allows you to benefit from enhanced performance effortlessly, without incurring additional costs or requiring migrations, upgrades or manual intervention.

In this blog, we will cover some of the most recently launched improvements for the Snowflake platform. To deliver the most optimal performance at the best price for your workloads, we continually invest in two key areas:

  1. Automatic performance improvements are delivered seamlessly to customers, with no knobs to tune or actions to take.
  2. Intelligent workload optimization features allow customers to improve query performance, access insights and optimize storage and computing.

Latest improvements to the Snowflake engine and platform

Automatic performance improvements

Recently, we announced many platform performance improvements, and we continue to invest in making data ingestion into Snowflake even faster, making the compiler faster and more efficient; making the optimizer more intelligent; and improving core query execution performance.

For example:

  • Ingest performance: We improved the ingest performance of both JSON and Parquet files with case-insensitive data up to 25%. 
  • Core infrastructure improvements: We continuously enhance Snowflake’s core infrastructure to achieve better price for performance. For example, we rolled out Adaptive Network Optimization, which improved throughput between the nodes in a warehouse by introducing faster intra-execution node communication and better network compression, and made improvements to aggregation placement. This resulted in up to a 40% improvement in query efficiency. We also continued to migrate Microsoft Azure workloads to faster ARM processors without customers needing to take any action. Some customers are seeing up to 10% price performance benefit due to this improvement. 
  • Intelligent optimizations: We continue to make Snowflake more intelligent — to choose the best possible optimizations, such as introducing more granular selectivity estimations, which helps Snowflake make better decisions on join orders. Additionally, we optimized query performance for common query patterns, such as improving memory management with holistic and adaptive broadcast join decisions, which improves the average execution time for all affected queries by up to 10%.

These performance improvements can be observed in the Snowflake Performance Index (SPI), an aggregate index for measuring real-world improvements in Snowflake performance experienced by customers over time. Since we started tracking this metric, query duration for customers’ recurring workloads has improved by 27%.*

Performance and efficiency improvements to Automatic Clustering, Search Optimization and Materialized Views

In addition to automatic performance improvements, we also continue to invest heavily in improving features that help customers better optimize their workloads. Automatic Clustering, Materialized Views and Search Optimization are major examples of this, and they all accelerate your queries via intelligent data-processing techniques. These features were designed from the ground up to be easy to configure with zero maintenance, so you can focus on getting results from your data as fast and efficiently as possible.

Automatic Clustering helps optimize tables where queries repeatedly filter, aggregate or join on the same columns. Simply specify a clustering key, and Snowflake automatically maintains your table in a well-clustered state. Since the start of 2024, we have delivered several significant performance and efficiency improvements under the hood to the core execution engine of Automatic Clustering that, altogether, have reduced Automatic Clustering maintenance costs by more than 10% on average. One of our biggest customers saw cost reductions of over 30%, and we observed up to 50% cost reductions for certain tables.

Likewise, we have been making substantial investments in the performance and efficiency of the Search Optimization Service and Materialized Views. Search Optimization is a robust feature that improves the performance of point lookup queries, highly selective queries and queries involving large volumes of semi-structured data, such as application, network or infrastructure logs by an order of magnitude. This optimization works seamlessly with Snowflake’s new SEARCH function, which allows users to search for exact characters or text within specific columns or across multiple tables. Together, these capabilities are particularly valuable in the observability domain, especially for log analytics and cybersecurity, where billions of rows need to be analyzed in near real time. A Snowflake Materialized View precomputes (“materializes”) a data set from a query — and automatically maintains the results to be up-to-date and consistent — to improve the performance of frequent or complex queries (like BI dashboard views of sales or usage data, or analyses of semistructured data). 

As a result of our investments, we are thrilled to pass on the cost reductions to customers: As of Aug. 1, we have reduced the price for maintaining Search Optimization and Materialized Views by 80% in all deployments. (See the updated credit rates in Table 5 of the Consumption Table on the Snowflake Pricing page.)

What’s next

At Snowflake, we’re on a continuous quest to enhance price for performance for customers, with a particular focus on accelerating the core database engine, delivered through our weekly release cadence. While some vendors continue to use synthetic benchmarks like TPC-DS to claim performance gains over their competitors, we remain obsessively focused on improving real-world performance for our customers, using the Snowflake Performance Index as our North Star metric.

To learn how Snowflake measures and prioritizes performance improvements, please read more about the Snowflake Performance Index here. For a list of key performance improvements by year and month, visit Snowflake Documentation.

*Based on internal Snowflake data, query duration for customers’ stable workloads improved by 27% from Aug. 25, 2022 to April 30, 2024. To calculate SPI, we identify a group of customer workloads that are stable and comparable in both amount of queries and data processed over the period presented. Reduction in query duration resulted from a combination of factors, including hardware and software improvements and customer optimizations. Improvement in query duration metrics are rounded to the nearest hundredth.