TechPrufer
Get started
Hex
Hex logoHex · Feature trail

Snowpark Integration

6 posts · Jun 2022 – May 2024

What this trail covers

Hex has deepened its Snowpark integration, allowing users to seamlessly switch between SQL, Python, and R cells while leveraging Snowpark dataframes and container services. This enhances security through Snowflake OAuth and simplifies session creation, enabling applications like Hex to run entirely within Snowflake's managed infrastructure. It supports advanced use cases like data pipelines with CDC and ML workloads using Snowpark ML, including GPU compute for accelerated ML training. The integration now also supports Snowflake ML functions, enabling users to train and call ML models using only SQL within Hex, with support for partitioned model training.

All Hex trails
  1. Building ML Models In Minutes With Hex and Snowflake ML FunctionsThis post details how Hex integrates with Snowflake ML functions, specifically the `SNOWFLAKE.ML.FORECAST()` function, to enable users to train and call ML models using only SQL. It demonstrates using `SYSTEM$QUERY_REFERENCE()` for data preparation and highlights the ease of partitioned model training with the `SERIES_COLNAME` parameter. The post shows how Hex returns results as Pandas dataframes, allowing for further Python-based analysis.
  2. Deepening Our Snowpark IntegrationThis post introduces the ability to use Snowpark dataframes across SQL, Python, and visual cells within Hex, building upon the existing Snowpark integration. It highlights the 'Snowpark Easy Button' for simplified session creation and mentions support for Snowflake OAuth. The post also outlines advanced use cases for data pipelines and machine learning.
  3. Snowpark + Hex Ask Me AnythingThis post is a Q&A summary from a joint Snowflake and Hex AMA session. It highlights how to initiate a Snowpark instance in Hex, manage packages within Hex and Snowpark, outlines Hex's advantages over Jupyter (ease of setup, direct Snowflake integration, collaboration, reactive cells, Hex Magic), clarifies the Hex-Snowflake partnership, details CI/CD integration via Git sync and native versioning, and discusses orchestration/scheduling options. It also explains compute pushdown to Snowflake, the comparison with PySpark, and the use of Snowpark Optimized Warehouses for memory-intensive operations. Finally, it touches on Snowpark ML for feature engineering and the distinction between Hex Magic and Snowpark, as well as model versioning and hyperparameter tuning in Snowflake ML.
  4. Hex's Partnership with Snowflake: Behind the MagicThis post details Hex's partnership with Snowflake, focusing on the technical implementation of features that integrate Hex with Snowflake's capabilities. It explains how Hex leverages Snowflake's data warehouse scale through features like Query Mode, which automatically builds CTEs to keep computation at the warehouse level. It also elaborates on the polyglot workflow, detailing the technical approach to seamlessly move between SQL and Snowpark dataframes, including how SQL queries are wrapped in sessions to create Snowpark dataframes and how Snowpark dataframes referenced in SQL queries are handled by creating temporary tables. Furthermore, it discusses the deployment of Hex within Snowflake using Snowpark Container Services, enabling data, compute, and user management to reside within the Snowflake account.
  5. Hex + Snowpark Container Services: Secure Analytics and ML for EnterprisesThis post announces the ability to deploy Hex entirely within Snowflake using Snowpark Container Services, enabling data analysis and ML training without data leaving the Snowflake account. It highlights GPU compute availability within these containers for accelerated ML. Additionally, it details deeper Snowflake integrations including R support via ODBC pushdown and the Snowpark ML API for feature engineering and model training.
  7. Hex + Snowflake: Notebooks in the CloudThis post details how Hex's integration with Snowflake allows for faster SQL execution by pushing queries to Snowflake, avoiding data copying. It also highlights the use of Snowpark for Python, enabling Python code to execute natively within Snowflake for data transformations and model training. The combination of SQL and Python within Hex notebooks, leveraging Snowflake's compute, is presented as a key enabler for faster and more scalable data analysis workflows.
  8. Introducing Snowpark for Python in HexThis post introduces first-class support for Snowpark for Python within Hex. It details how Snowpark DataFrames are executed lazily on Snowflake, improving efficiency and avoiding memory limits compared to traditional Pandas DataFrames. The post highlights Hex's role as an 'easy button' for Snowpark by auto-magically installing the API and dependencies, simplifying setup. It also demonstrates a workflow for a business analyst to find top customers using Snowpark and Hex, and mentions the integration with Snowflake Partner Connect for easy trial setup.