Short guide: How to use PostgresOperator in Apache Airflow?

SQLAlchemy inside

Set up connection to PostgreSQL

Edit postgres_default connection in Airflow Connections

DAG

from datetime import datetime
from airflow import DAG
from airflow.operators.postgres_operator import PostgresOperator

dag_params = {
'dag_id': 'PostgresOperator_dag',
'start_date': datetime(2019, 10, 7),
'schedule_interval': None
}

with DAG(**dag_params) as dag:

create_table = PostgresOperator(
task_id='create_table',
sql='''CREATE TABLE new_table(
custom_id integer NOT NULL, timestamp TIMESTAMP NOT NULL, user_id VARCHAR (50) NOT NULL
);''',
)
PostgreSQL task success
table exists issue
INSERT INTO new_table VALUES(%s, %s, %s)
insert_row = PostgresOperator(
task_id='insert_row',
sql='INSERT INTO new_table VALUES(%s, %s, %s)',
trigger_rule=TriggerRule.ALL_DONE,
parameters=(uuid.uuid4().int % 123456789, datetime.now(), uuid.uuid4().hex[:10])
)

create_table >> insert_row
full DAG body
success insert row to PostgreSQL

--

--

--

Python Dev \\ https://github.com/xnuinside \\ https://twitter.com/xnuinside. Currently open to remote https://www.linkedin.com/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Obvious Competitive Coding things from non obvious angles …….

Multi-Cloud Migration | UpTech Solution

All About Debugging: The Techniques

Will the programming job disappear?

Learn Git & GitHub in Under 6 Minutes

Introduction to Google Cloud Platform

Ventoy 1.0.75 Crack + Keygen Free Download

Why programming can seem so hard — and what to do about it

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Iuliia Volkova

Iuliia Volkova

Python Dev \\ https://github.com/xnuinside \\ https://twitter.com/xnuinside. Currently open to remote https://www.linkedin.com/

More from Medium

How to Make Your Data Pipelines More Dynamic Using Parameters in Prefect

AWS vs Azure anomaly detection

String filters in pandas: you’re doing it wrong

Apache Airflow for Data Science — How to Install Airflow Locally