Kimuksung
Kimuksung ์•ˆ๋…•ํ•˜์„ธ์š”. ๋ถ„์‚ฐ์ฒ˜๋ฆฌ์— ๊ด€์‹ฌ์ด ๋งŽ์€ ์ƒ๊ฐํ•˜๋Š” ์ฃผ๋‹ˆ์–ด Data Enginner์ž…๋‹ˆ๋‹ค.

Airflow BashOperator

Airflow BashOperator
Bash Operator

  • ํŒŒ์ด์ฌ ์ฝ”๋“œ๊ฐ€ ์•„๋‹Œ ์Šคํฌ๋ฆฝํŠธ๋ฅผ ์ •๊ธฐ์ ์œผ๋กœ ์‹คํ–‰์ด ๊ฐ€๋Šฅํ•˜๋‹ค.
1
2
3
4
5
6
7
8
classairflow.operators.bash.BashOperator(
    *, 
    bash_command: str, 
    env: Optional[Dict[str, str]] = None, 
    output_encoding: str = 'utf-8', 
    skip_exit_code: int = 99, 
    cwd: str = None, **kwargs
)
  • output_encoding : bash command ์ธ์ฝ”๋”ฉ ๋ฐฉ์‹
  • skip_exit_code : None - 0์ด ์•„๋‹Œ ๋ชจ๋“  ๊ฐ’์€ ์‹คํŒจ / 99 - task exit ํ•œ ๊ฒฝ์šฐ์— skip ์ฒ˜๋ฆฌ
  • cwd : command๊ฐ€ ์‹คํ–‰๋œ ์œ„์น˜ ( cd ์™€ ๊ฐ™์€ ๋ฒˆ๊ฑฐ๋กœ์›€์„ ์ค„์—ฌ์คŒ )
1
from airflow.operators.bash import BashOperator
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
om airflow.operators.dummy import DummyOperator
from datetime import datetime

dag = DAG(
    'dag name',
    description='testing for bashoperator',
    schedule_interval=None,
    start_date=datetime(2023, 10, 23),
    catchup=False
)

start_task = DummyOperator(task_id='start_task', dag=dag)

task1= BashOperator(
    task_id='task1',
    bash_command='python -c "print(\'Hello, world!\')"',
    output_encoding= 'utf-8'
    dag=dag
)

end_task = DummyOperator(task_id='end_task', dag=dag)

start_task >> task1 >> end_task 

์ฐธ์กฐ

https://medium.com/@agusmahari/airflow-getting-started-with-the-bashoperator-in-airflow-a-beginners-guide-to-executing-bash-1f80d25c4602