Snowflake - Build & Architect Data pipelines using AWS
- Description
- Curriculum
- FAQ
- Reviews
Snowflake is the next big thing and it is becoming a full blown data eco-system . With the level of scalability & efficiency in handling massive volumes of data and also with a number of new concepts in it ,this is the right time to wrap your head around Snowflake and have it in your toolkit . This course not only covers the core features of Snowflake but also teaches you how to deploy python/pyspark jobs in AWS Glue and Airflow that communicate with Snowflake , which is one of the most important aspects of building pipelines .
Anyone who has a basic understanding of cloud and belong to one of the below backgrounds can benefit from this course :
– Data Scientists / Analysts
– Data Engineers / Software Developers
– SQL Programmers or DBA’s
– Aspiring Data analysts and scientists who are learning SQL and Python
This Course covers :
- What is Snowflake
- Most Crucial Aspects of Snowflake in a very practical manner
- Writing Python/Spark Jobs in AWS Glue Jobs for data transformation
- Real Time Streaming using Kafka and Snowflake
- Interacting with External Functions & use cases
- Security Features in Snowflake
Prerequisites for this course are :
- Knowing SQL or at least some prior knowledge in writing queries
- Scripting in Python (or any language )
- Willingness to explore ,learn and put in the extra effort to succeed
- An active AWS Account & know-how of basic cloud fundamentals
Important Note – You need to have an active AWS Account in order to perform the sections related to Python and PySpark . For the rest of the course , a free trial snowflake account should suffice .
Some Tips :
- Try to watch the videos at 1.2X speed
- Read the reference links and the official documentation of Snowflake as much as possible
-
5What is a data-warehouse ?Video lesson
-
6Two Aspects of a Data EcosystemVideo lesson
-
7Lab - Setup Snowflake Trial AccountVideo lesson
-
8Snowflake ArchitectureVideo lesson
-
9Snowflake Object HeirarchyVideo lesson
-
10Snowflake - Virtual WarehousesVideo lesson
-
11Snowflake - Different Billing ComponentsVideo lesson
-
12Snowflake - Track your consumptionVideo lesson
-
13Snowflake- Resource MonitorsVideo lesson
-
14Introduction - Different Tables in SnowflakeVideo lesson
-
15Lab - Create Tables in SnowflakeVideo lesson
-
16Snowflake - Views , Materialized Views and Secure ViewsVideo lesson
-
17Lab - Create Views in SnowflakeVideo lesson
-
18Lab - Create Secure Views in SnowflakeVideo lesson
-
19More about Views in SnowflakeVideo lesson
-
20Section OverviewVideo lesson
-
21Introduction to partitions and clustering keysVideo lesson
-
22Lab - Micropartitions and Clustering keysVideo lesson
-
23Benefits of Micro-partitions and ClusteringVideo lesson
-
24Understanding Clustering Depth and Cluster OverlapVideo lesson
-
25Lab - Selecting your clustering KeysVideo lesson
-
26Lab - Check Query Profile and historyVideo lesson
-
27Lab - Query Processing and CachingVideo lesson
-
28Search Optimization FeatureVideo lesson
-
29Section OverviewVideo lesson
-
30Data Ingestion - Real World Use CasesVideo lesson
-
31Lab - Create an Integration Object to Connect Snowflake with AWS S3Video lesson
-
32Lab - Ingest CSV from S3 to SnowflakeVideo lesson
-
33Lab - Ingest JSON from S3 to SnowflakeVideo lesson
-
34Introduction to Continuous Data Ingestion in SnowflakeVideo lesson
-
35Lab - Create and implement SnowpipeVideo lesson
-
36Snowpipe - Billing Estimation and Key Considerations for Data IngestionVideo lesson
-
37Lab - Extracting/Unload Data from Snowflake to S3Video lesson
-
42Section OverviewVideo lesson
-
43Introduction to StreamsVideo lesson
-
44Lab - Implement Standard StreamsVideo lesson
-
45Lab - Implement Append-Only StreamsVideo lesson
-
46Lab - Streams in a TransactionVideo lesson
-
47Streams - Data Retention and StalenessVideo lesson
-
48Lab - Change Tracking using "Changes"Video lesson
-
49Project OverviewVideo lesson
-
50Lab - Create Streams - Project Solution Part-1Video lesson
-
51Lab - Create Streams - Part-1 ContinuationVideo lesson
-
52Lab - End to End Pipeline in ActionVideo lesson
-
53Introduction to User Defined Functions and UDF TypesVideo lesson
-
54Lab - Write and implement a Scalar UDFVideo lesson
-
55Lab - Write Tabular UDF in SQLVideo lesson
-
56Lab - Implement Javascript UDFsVideo lesson
-
57What is Pushdown in UDF ?Video lesson
-
58Lab - How can pushdown expose the underlying data ?Video lesson
-
59Lab - Write Secure UDFsVideo lesson
-
60Section OverviewVideo lesson
-
61Introduction to External FunctionsVideo lesson
-
62Lab - Write Deploy AWS Lambda FunctionVideo lesson
-
63Create IAM RoleVideo lesson
-
64Lab - Create API GatewayVideo lesson
-
65Lab - Securing and Deploy API GatewayVideo lesson
-
66Lab - Create External Function in SnowflakeVideo lesson
-
67Section OverviewVideo lesson
-
68Lab - Connect Python with Snowflake in your local machineVideo lesson
-
69Introduction to AWS GlueVideo lesson
-
70Lab - Deploy and execute python script to AWS GlueVideo lesson
-
71Lab - Parameterize your python script on AWS GlueVideo lesson
-
72Lab - Python Pandas with Snowflake on AWS GlueVideo lesson
-
73What is Pushdown in Spark 3.1 ?Video lesson
-
74Lab - Deploy a Pyspark script using AWS GlueVideo lesson
-
75Lab - Setup Managed Airflow Cluster on AWSVideo lesson
-
76Lab - Configure Snowflake Connectivity in AirflowVideo lesson
-
77Lab - Deploy a PySpark Transformation job in AWS GlueVideo lesson
-
78Lab - Setup Airflow DAGVideo lesson
-
79Section OverviewVideo lesson
-
80Lab - Download the necessary JAR FilesVideo lesson
-
81Lab - Setup Kafka in your local systemVideo lesson
-
82Lab - Setup Kafka Snowflake ConnectorVideo lesson
-
83Lab - Setup Encryption Keys for Kafka-Snowflake ConnectivityVideo lesson
-
84Lab - Streaming Data in ActionVideo lesson
External Links May Contain Affiliate Links read more