Skip to content
Navigation menu
Search
Search
Log in
Create account
DEV Community
Close
DEV Community
is a community of 862,249 amazing developers
We're a place where coders share, stay up-to-date and grow their careers.
Create account
Log in
Home
Listings
Podcasts
Videos
Tags
FAQ
Forem Shop
Sponsors
About
Contact
Other
Code of Conduct
Privacy Policy
Terms of use
Twitter
Facebook
Github
Instagram
Twitch
#
dataengineering
Follow
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
What is the Lakehouse, the latest Direction of Big Data Architecture?
qazmkop
qazmkop
qazmkop
Follow
May 14
What is the Lakehouse, the latest Direction of Big Data Architecture?
#
opensource
#
dataengineering
#
bigdata
#
database
Reactions
8
reactions
Comments
Add Comment
10 min read
Save
Saved
Correlation With Rapid Miner
AnuoluwapoDS
AnuoluwapoDS
AnuoluwapoDS
Follow
May 4
Correlation With Rapid Miner
#
dataengineering
#
datascience
#
data
Reactions
6
reactions
Comments
Add Comment
1 min read
Save
Saved
SQL FUNCTION WHERE(How to get the rows you want)
AnuoluwapoDS
AnuoluwapoDS
AnuoluwapoDS
Follow
May 4
SQL FUNCTION WHERE(How to get the rows you want)
#
dataengineering
#
data
#
database
Reactions
7
reactions
Comments
Add Comment
1 min read
Save
Saved
Data Preparation using Rapid miner
AnuoluwapoDS
AnuoluwapoDS
AnuoluwapoDS
Follow
Apr 27
Data Preparation using Rapid miner
#
rapidminer
#
datascience
#
data
#
dataengineering
Reactions
4
reactions
Comments
Add Comment
1 min read
Save
Saved
SQL Practice Exercises
AnuoluwapoDS
AnuoluwapoDS
AnuoluwapoDS
Follow
Apr 27
SQL Practice Exercises
#
dataengineering
#
datascience
#
database
#
microsoftsqlserver
Reactions
10
reactions
Comments
Add Comment
1 min read
Save
Saved
How to prepare for the GCP Professional Data Engineer certification
Gabriel Luz
Gabriel Luz
Gabriel Luz
Follow
May 2
How to prepare for the GCP Professional Data Engineer certification
#
googlecloud
#
dataengineering
#
gcp
#
bigdata
Reactions
16
reactions
Comments
Add Comment
8 min read
Save
Saved
Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Apr 4
Making Data Engineering Easier: Operational Analytics With Event Streaming and Reverse ETL
#
datascience
#
eventstreaming
#
dataengineering
#
customerdataplatform
Reactions
7
reactions
Comments
Add Comment
6 min read
Save
Saved
Using dbt for Transformation Tasks on BigQuery
Cem Keskin
Cem Keskin
Cem Keskin
Follow
May 2
Using dbt for Transformation Tasks on BigQuery
#
dezoomcamp
#
dataengineering
Reactions
10
reactions
Comments
1
comment
4 min read
Save
Saved
Considerations when performing ETL
Raphael Gutierrez
Raphael Gutierrez
Raphael Gutierrez
Follow
Mar 25
Considerations when performing ETL
#
dataengineering
#
etl
Reactions
4
reactions
Comments
Add Comment
3 min read
Save
Saved
Docker and Kubernetes
Abdullah Paracha
Abdullah Paracha
Abdullah Paracha
Follow
for
AWS Community Builders
Apr 27
Docker and Kubernetes
#
devops
#
dataengineering
#
softwareengineer
#
programming
Reactions
5
reactions
Comments
Add Comment
3 min read
Save
Saved
How to Use Apache Airflow to Get 1000+ Files From a Public Dataset
Cem Keskin
Cem Keskin
Cem Keskin
Follow
Apr 24
How to Use Apache Airflow to Get 1000+ Files From a Public Dataset
#
dezoomcamp
#
dataengineering
#
airflow
#
publicdatasets
Reactions
7
reactions
Comments
Add Comment
10 min read
Save
Saved
ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)
Rubens Barbosa
Rubens Barbosa
Rubens Barbosa
Follow
Apr 24
ELT Data Pipeline with Kubernetes CronJob, Azure Data Lake, Azure Databricks (Part 1)
#
etl
#
dataengineering
#
python
#
azure
Reactions
7
reactions
Comments
Add Comment
5 min read
Save
Saved
What is Azure Synapse Analytics?
Matt Eland
Matt Eland
Matt Eland
Follow
Apr 22
What is Azure Synapse Analytics?
#
data
#
dataengineering
#
azure
#
cloud
Reactions
4
reactions
Comments
Add Comment
7 min read
Save
Saved
Design concept of a best opensource project about big data and data lakehouse
qazmkop
qazmkop
qazmkop
Follow
Apr 16
Design concept of a best opensource project about big data and data lakehouse
#
opensource
#
dataengineering
#
bigdata
#
datascience
Reactions
9
reactions
Comments
Add Comment
9 min read
Save
Saved
Details of 4 best opensource projects about big data you should try out(Ⅰ)
qazmkop
qazmkop
qazmkop
Follow
Apr 7
Details of 4 best opensource projects about big data you should try out(Ⅰ)
#
opensource
#
dataengineering
#
bigdata
#
spark
Reactions
7
reactions
Comments
Add Comment
5 min read
Save
Saved
Debezium Change Data Capture without Kafka Connect
Ludovic DEHON
Ludovic DEHON
Ludovic DEHON
Follow
for
Kestra
Apr 5
Debezium Change Data Capture without Kafka Connect
#
database
#
dataengineering
#
opensource
#
etl
Reactions
7
reactions
Comments
Add Comment
8 min read
Save
Saved
Building GCS Buckets and BigQuery Tables with Terraform
Cem Keskin
Cem Keskin
Cem Keskin
Follow
Apr 3
Building GCS Buckets and BigQuery Tables with Terraform
#
dezoomcamp
#
dataengineering
#
bigquery
#
gcp
Reactions
6
reactions
Comments
Add Comment
4 min read
Save
Saved
Released yuniql v1.2.25. Multi-tenant support, Oracle and largest set of bug fixes
Rodel E. Dagumampan
Rodel E. Dagumampan
Rodel E. Dagumampan
Follow
Feb 22
Released yuniql v1.2.25. Multi-tenant support, Oracle and largest set of bug fixes
#
yuniql
#
database
#
devops
#
dataengineering
Reactions
5
reactions
Comments
Add Comment
4 min read
Save
Saved
Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment
qazmkop
qazmkop
qazmkop
Follow
Mar 25
Quick use of CDC: A new demo from lakesoul makes it easier to set up the environment
#
opensource
#
dataengineering
#
bigdata
#
spark
Reactions
8
reactions
Comments
Add Comment
5 min read
Save
Saved
4 best opensource projects about big data you should try out
qazmkop
qazmkop
qazmkop
Follow
Mar 24
4 best opensource projects about big data you should try out
#
opensource
#
dataengineering
#
bigdata
#
spark
Reactions
15
reactions
Comments
3
comments
3 min read
Save
Saved
Preparing for Professional Cloud Data Engineer Certification (March 2022)
David Cox
David Cox
David Cox
Follow
Mar 23
Preparing for Professional Cloud Data Engineer Certification (March 2022)
#
googlecloud
#
cloud
#
dataengineering
Reactions
2
reactions
Comments
2
comments
12 min read
Save
Saved
A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions
qazmkop
qazmkop
qazmkop
Follow
Mar 17
A new unified streaming and batch table storage solution similar to iceberg/hudi/delta lake but with several new functions
#
dataengineering
#
opensource
#
bigdata
#
programming
Reactions
8
reactions
Comments
Add Comment
2 min read
Save
Saved
Introducting Myself
AnuoluwapoDS
AnuoluwapoDS
AnuoluwapoDS
Follow
Mar 11
Introducting Myself
#
datascience
#
programming
#
dataanalysis
#
dataengineering
Reactions
7
reactions
Comments
2
comments
1 min read
Save
Saved
[OPINIÃO] Construindo uma Carreira como Data Engineer
Lis R. Barreto
Lis R. Barreto
Lis R. Barreto
Follow
Mar 9
[OPINIÃO] Construindo uma Carreira como Data Engineer
#
bigdata
#
dataengineering
#
tips
Reactions
2
reactions
Comments
Add Comment
2 min read
Save
Saved
What is Data Profiling?
Preeti Hemant
Preeti Hemant
Preeti Hemant
Follow
Mar 5
What is Data Profiling?
#
data
#
dataengineering
#
shortread
#
analyticsengineering
Reactions
2
reactions
Comments
Add Comment
1 min read
Save
Saved
Data architecture models
Preeti Hemant
Preeti Hemant
Preeti Hemant
Follow
Feb 23
Data architecture models
#
datamesh
#
data
#
dataengineering
#
etl
Reactions
2
reactions
Comments
Add Comment
6 min read
Save
Saved
[ARTIGO] Data Warehouse, Data Lake e Data Lakehouse: Conceitos e Diferenças
Lis R. Barreto
Lis R. Barreto
Lis R. Barreto
Follow
Feb 18
[ARTIGO] Data Warehouse, Data Lake e Data Lakehouse: Conceitos e Diferenças
#
bigdata
#
data
#
dataengineering
Reactions
4
reactions
Comments
Add Comment
3 min read
Save
Saved
Enabling the Customer Data Stack: RudderStack Series B Funding
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Feb 14
Enabling the Customer Data Stack: RudderStack Series B Funding
#
seriesb
#
cdp
#
rudderstack
#
dataengineering
Reactions
2
reactions
Comments
Add Comment
1 min read
Save
Saved
Kestra, infinitely scalable open source orchestration and scheduling platform.
Ludovic DEHON
Ludovic DEHON
Ludovic DEHON
Follow
for
Kestra
Feb 2
Kestra, infinitely scalable open source orchestration and scheduling platform.
#
dataengineering
#
data
#
etl
#
opensource
Reactions
3
reactions
Comments
Add Comment
6 min read
Save
Saved
Modern data warehouse patterns: ELT with Snowflake variants
biellls
biellls
biellls
Follow
Jan 29
Modern data warehouse patterns: ELT with Snowflake variants
#
snowflake
#
dataengineering
#
etl
#
elt
Reactions
9
reactions
Comments
Add Comment
6 min read
Save
Saved
Data Engineering in Julia
Logan Kilpatrick
Logan Kilpatrick
Logan Kilpatrick
Follow
Jan 29
Data Engineering in Julia
#
machinelearning
#
julia
#
dataengineering
#
julialang
Reactions
3
reactions
Comments
Add Comment
1 min read
Save
Saved
Standing on the shoulders of giants. Part one: Airflow
biellls
biellls
biellls
Follow
Jan 21
Standing on the shoulders of giants. Part one: Airflow
#
airflow
#
dataengineering
#
python
#
data
Reactions
7
reactions
Comments
Add Comment
5 min read
Save
Saved
How Engineering Teams Use RudderStack to Support Marketing
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Jan 3
How Engineering Teams Use RudderStack to Support Marketing
#
dataengineering
#
marketing
#
dataanalytics
#
datawarehouse
Reactions
6
reactions
Comments
Add Comment
7 min read
Save
Saved
Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster
Tomas
Tomas
Tomas
Follow
Dec 30 '21
Data Engineering Pipeline with AWS Step Functions, CodeBuild and Dagster
#
aws
#
dagster
#
dataengineering
#
python
Reactions
6
reactions
Comments
3
comments
10 min read
Save
Saved
Why It’s Hard for Engineering to Support Marketing
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Dec 28 '21
Why It’s Hard for Engineering to Support Marketing
#
dataengineering
#
etl
#
marketing
#
dataanlytics
Reactions
2
reactions
Comments
Add Comment
3 min read
Save
Saved
Extract csv data and load it to PostgreSQL using Meltano ELT
Jorge PM
Jorge PM
Jorge PM
Follow
Dec 22 '21
Extract csv data and load it to PostgreSQL using Meltano ELT
#
python
#
meltano
#
dataengineering
#
elt
Reactions
5
reactions
Comments
Add Comment
6 min read
Save
Saved
What Is Event-Driven Machine Learning?
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Dec 17 '21
What Is Event-Driven Machine Learning?
#
eventdriven
#
machinelearning
#
cdp
#
dataengineering
Reactions
6
reactions
Comments
Add Comment
4 min read
Save
Saved
Host a fully persisted Apache NiFi service with docker
Cribber
Cribber
Cribber
Follow
Nov 10 '21
Host a fully persisted Apache NiFi service with docker
#
docker
#
apache
#
nifi
#
dataengineering
Reactions
3
reactions
Comments
Add Comment
1 min read
Save
Saved
Relational data models
Barbara
Barbara
Barbara
Follow
Nov 29 '21
Relational data models
#
sql
#
dataengineering
#
learn
#
sketchnote
Reactions
5
reactions
Comments
Add Comment
2 min read
Save
Saved
Implementing Graceful Shutdown in Go
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Nov 26 '21
Implementing Graceful Shutdown in Go
#
go
#
dataengineering
#
sre
#
database
Reactions
14
reactions
Comments
5
comments
14 min read
Save
Saved
Open Source Analytics Stack: Bringing Control, Flexibility, and Data-Privacy to Your Analytics
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Nov 25 '21
Open Source Analytics Stack: Bringing Control, Flexibility, and Data-Privacy to Your Analytics
#
privacy
#
analytics
#
dataanalytics
#
dataengineering
Reactions
3
reactions
Comments
Add Comment
10 min read
Save
Saved
RudderStack + Blendo: Better Together
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Nov 25 '21
RudderStack + Blendo: Better Together
#
blendo
#
dataengineering
#
etl
#
cdp
Reactions
2
reactions
Comments
Add Comment
7 min read
Save
Saved
Web Scraping Sprott U Fund with BS4 in 10 Lines of Code
CincyBC
CincyBC
CincyBC
Follow
Nov 24 '21
Web Scraping Sprott U Fund with BS4 in 10 Lines of Code
#
python
#
beautifulsoup
#
dataengineering
Reactions
29
reactions
Comments
Add Comment
3 min read
Save
Saved
RudderStack’s Licensing Explained
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Nov 24 '21
RudderStack’s Licensing Explained
#
cdp
#
dataengineering
#
rudderstack
#
privacy
Reactions
3
reactions
Comments
Add Comment
4 min read
Save
Saved
Introducing RudderStack's New, High-performance JavaScript SDK
Team RudderStack
Team RudderStack
Team RudderStack
Follow
for
RudderStack
Nov 22 '21
Introducing RudderStack's New, High-performance JavaScript SDK