{"id":4218,"date":"2025-12-11T16:58:15","date_gmt":"2025-12-11T08:58:15","guid":{"rendered":"https:\/\/aws-oncloudai.com\/?p=4218"},"modified":"2025-12-11T16:58:15","modified_gmt":"2025-12-11T08:58:15","slug":"a-comprehensive-analysis-of-mwaa-the-efficient-data-orchestration-approach-of-aws-hosting-apache-airflow","status":"publish","type":"post","link":"https:\/\/aws-oncloudai.com\/zh_hk\/a-comprehensive-analysis-of-mwaa-the-efficient-data-orchestration-approach-of-aws-hosting-apache-airflow\/","title":{"rendered":"MWAA \u5168\u89e3\u6790\uff1aAWS \u8a17\u7ba1Apache Airflow \u7684\u9ad8\u6548\u8cc7\u6599\u7de8\u6392\u4e4b\u9053"},"content":{"rendered":"<p data-start=\"144\" data-end=\"294\">\u5728\u4f01\u696d\u6578\u4f4d\u5316\u52a0\u901f\u7684\u7576\u4e0b\uff0c\u8cc7\u6599\u5de5\u7a0b\u3001\u6a5f\u5668\u5b78\u7fd2\u8207\u6279\u6b21\u4efb\u52d9\u65e5\u76ca\u8907\u96dc\uff0c\u5de5\u4f5c\u6d41\u7a0b\u7684\u81ea\u52d5\u5316\u8207\u53ef\u9760\u6027\u6210\u70ba\u696d\u52d9\u57fa\u790e\u80fd\u529b\u3002 Apache Airflow \u4f5c\u70ba\u696d\u754c\u4e3b\u6d41\u7684\u5de5\u4f5c\u6d41\u7a0b\u7de8\u6392\u6846\u67b6\uff0c\u88ab\u5ee3\u6cdb\u7528\u65bcETL \u8abf\u5ea6\u3001\u8cc7\u6599\u7ba1\u9053\u7ba1\u7406\u8207\u6a5f\u5668\u5b78\u7fd2\u6d41\u7a0b\u7de8\u6392\u3002\u7136\u800c\uff0c\u50b3\u7d71\u81ea\u5efaAirflow \u74b0\u5883\u9762\u81e8\u90e8\u7f72\u8907\u96dc\u3001\u64f4\u5c55\u56f0\u96e3\u3001\u7dad\u4fee\u6210\u672c\u9ad8\u7b49\u6311\u6230\u3002<\/p>\n<p data-start=\"296\" data-end=\"444\"><strong data-start=\"296\" data-end=\"349\">Amazon Managed Workflows for Apache Airflow\uff08MWAA\uff09<\/strong> \u6b63\u662fAWS \u9762\u5411\u4f01\u696d\u63a8\u51fa\u7684\u8a17\u7ba1\u7d1a\u89e3\u6c7a\u65b9\u6848\uff0c\u65e8\u5728\u5e6b\u52a9\u4f7f\u7528\u8005\u4ee5\u66f4\u4f4e\u6210\u672c\u3001\u66f4\u9ad8\u6548\u7387\u5efa\u7f6e\u3001\u8abf\u5ea6\u8207\u76e3\u63a7Airflow \u5de5\u4f5c\u6d41\u7a0b\u3002\u672c\u7bc7\u6587\u7ae0\u5c07\u5f9e\u6280\u8853\u539f\u7406\u3001\u6838\u5fc3\u512a\u52e2\u3001\u61c9\u7528\u5834\u666f\u8207\u4f01\u696d\u50f9\u503c\u7b49\u7dad\u5ea6\u9032\u884c\u5168\u9762\u89e3\u6790\u3002<\/p>\n<h4 data-start=\"451\" data-end=\"482\"><strong data-start=\"453\" data-end=\"482\">\u50b3\u7d71\u4f5c\u696d\u6392\u7a0b\u7684\u75db\u9ede\uff1a\u70ba\u4f55\u9700\u8981Airflow\uff1f<\/strong><\/h4>\n<p data-start=\"484\" data-end=\"539\">\u5728\u672a\u4f7f\u7528Airflow \u4e4b\u524d\uff0c\u4f01\u696d\u901a\u5e38\u6703\u4f9d\u8cf4CRON \u6216\u8173\u672c\u8abf\u5ea6\u9032\u884c\u81ea\u52d5\u5316\u4efb\u52d9\u3002\u7136\u800c\uff0c\u9019\u5957\u7cfb\u7d71\u6709\u660e\u986f\u7f3a\u9677\uff1a<\/p>\n<h5 data-start=\"541\" data-end=\"560\"><strong data-start=\"545\" data-end=\"560\">1. \u4f9d\u8cf4\u95dc\u4fc2\u96e3\u4ee5\u7ba1\u7406<\/strong><\/h5>\n<p data-start=\"561\" data-end=\"592\">CRON \u7121\u6cd5\u8996\u89ba\u5316\u4efb\u52d9\u4f9d\u8cf4\uff0c\u4e5f\u96e3\u4ee5\u7406\u89e3\u4efb\u52d9\u4e4b\u9593\u7684\u4e0a\u4e0b\u6e38\u95dc\u4fc2\u3002<\/p>\n<h5 data-start=\"594\" data-end=\"617\"><strong data-start=\"598\" data-end=\"617\">2. \u7f3a\u4e4f\u53ef\u9760\u7684\u91cd\u8a66\u8207\u56de\u586b\u6a5f\u5236<\/strong><\/h5>\n<p data-start=\"618\" data-end=\"643\">\u4efb\u52d9\u5931\u6557\u5f8c\u591a\u6578\u7121\u6cd5\u81ea\u52d5\u91cd\u8a66\uff0c\u66f4\u4e0d\u5b58\u5728\u300c\u88dc\u6578\u300d\u80fd\u529b\u3002<\/p>\n<h5 data-start=\"645\" data-end=\"662\"><strong data-start=\"649\" data-end=\"662\">3. \u64f4\u5c55\u80fd\u529b\u6709\u9650<\/strong><\/h5>\n<p data-start=\"663\" data-end=\"686\">CRON \u4f5c\u696d\u5927\u591a\u904b\u4f5c\u5728\u55ae\u6a5f\u4e0a\uff0c\u7121\u6cd5\u81ea\u7136\u64f4\u5145\u3002<\/p>\n<h5 data-start=\"688\" data-end=\"704\"><strong data-start=\"692\" data-end=\"704\">4. \u76e3\u63a7\u80fd\u529b\u5f31<\/strong><\/h5>\n<p data-start=\"705\" data-end=\"729\">\u4efb\u52d9\u904b\u884c\u6642\u9593\u3001\u932f\u8aa4\u65e5\u8a8c\u3001\u57f7\u884c\u8da8\u52e2\u7b49\u7f3a\u4e4f\u7d71\u4e00\u8996\u5716\u3002<\/p>\n<h5 data-start=\"731\" data-end=\"746\"><strong data-start=\"735\" data-end=\"746\">5. \u4e92\u52d5\u56f0\u96e3<\/strong><\/h5>\n<p data-start=\"747\" data-end=\"778\">\u8a31\u591a\u4efb\u52d9\u9700\u8981SSH \u767b\u5165\u4f3a\u670d\u5668\u624d\u80fd\u8abf\u8a66\u6216\u6aa2\u8996\u8f38\u51fa\uff0c\u7dad\u904b\u7e41\u96dc\u3002<\/p>\n<p data-start=\"780\" data-end=\"798\">\u56e0\u6b64\uff0c\u4f01\u696d\u5e38\u5e38\u7121\u6cd5\u56de\u7b54\u4ee5\u4e0b\u95dc\u9375\u554f\u984c\uff1a<\/p>\n<ul data-start=\"800\" data-end=\"865\">\n<li data-start=\"800\" data-end=\"813\">\n<p data-start=\"802\" data-end=\"813\">\u4efb\u52d9\u4ec0\u9ebc\u6642\u5019\u5931\u6557\uff1f<\/p>\n<\/li>\n<li data-start=\"814\" data-end=\"830\">\n<p data-start=\"816\" data-end=\"830\">\u57f7\u884c\u901f\u5ea6\u70ba\u4ec0\u9ebc\u7a81\u7136\u8b8a\u6162\uff1f<\/p>\n<\/li>\n<li data-start=\"831\" data-end=\"850\">\n<p data-start=\"833\" data-end=\"850\">\u54ea\u4e9b\u4efb\u52d9\u6b63\u5728\u904b\u884c\u3001\u6392\u968a\u6216\u5361\u4f4f\uff1f<\/p>\n<\/li>\n<li data-start=\"851\" data-end=\"865\">\n<p data-start=\"853\" data-end=\"865\">\u6709\u6c92\u6709\u53ef\u91cd\u8907\u4f7f\u7528\u7684\u8abf\u5ea6\u7d44\u4ef6\uff1f<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"867\" data-end=\"890\">\u9019\u4e9b\u554f\u984c\u6b63\u662fAirflow \u89e3\u6c7a\u7684\u6839\u672c\u75db\u9ede\u3002<\/p>\n<h4 data-start=\"897\" data-end=\"931\"><strong data-start=\"899\" data-end=\"931\">Apache Airflow\uff1a\u73fe\u4ee3\u8cc7\u6599\u7de8\u6392\u7684\u6838\u5fc3\u6846\u67b6<\/strong><\/h4>\n<p data-start=\"933\" data-end=\"989\">Apache Airflow \u662f\u4e00\u500b\u958b\u6e90\u3001\u57fa\u65bcPython \u7684\u4efb\u52d9\u7de8\u6392\u5e73\u53f0\uff0c\u53ef\u7528\u4f86\u5efa\u69cb\u8907\u96dc\u7684\u8cc7\u6599\u8207\u81ea\u52d5\u5316\u6d41\u7a0b\u3002<\/p>\n<h5 data-start=\"991\" data-end=\"1005\"><strong data-start=\"994\" data-end=\"1005\">\u6838\u5fc3\u80fd\u529b\u5305\u62ec\uff1a<\/strong><\/h5>\n<h5 data-start=\"1007\" data-end=\"1030\"><strong data-start=\"1011\" data-end=\"1030\">1. DAG\uff08\u6709\u5411\u7121\u74b0\u5716\uff09\u7ba1\u7406<\/strong><\/h5>\n<p data-start=\"1031\" data-end=\"1063\">\u6240\u6709\u4efb\u52d9\u900f\u904eDAG \u7d44\u7e54\uff0c\u8996\u89ba\u5316\u5c55\u793a\u4efb\u52d9\u4f9d\u8cf4\u3001\u57f7\u884c\u8def\u5f91\u8207\u72c0\u614b\u3002<\/p>\n<h5 data-start=\"1065\" data-end=\"1092\"><strong data-start=\"1069\" data-end=\"1092\">2. \u5b8c\u5168Python \u5316\u7684\u4efb\u52d9\u5b9a\u7fa9<\/strong><\/h5>\n<p data-start=\"1093\" data-end=\"1120\">\u5de5\u4f5c\u6d41\u7a0b\u900f\u904ePython \u7de8\u5beb\uff0c\u9748\u6d3b\u3001\u53ef\u64f4\u5c55\u3001\u6613\u65bc\u7dad\u8b77\u3002<\/p>\n<h5 data-start=\"1122\" data-end=\"1142\"><strong data-start=\"1126\" data-end=\"1142\">3. \u52d5\u614b\u751f\u6210\u8207\u53ef\u91cd\u8907\u4f7f\u7528\u6027<\/strong><\/h5>\n<p data-start=\"1143\" data-end=\"1164\">\u652f\u63f4\u8907\u96dc\u908f\u8f2f\u751f\u6210\u4efb\u52d9\u3001\u53c3\u6578\u5316\u8207\u591a\u74b0\u5883\u5fa9\u7528\u3002<\/p>\n<h5 data-start=\"1166\" data-end=\"1190\"><strong data-start=\"1170\" data-end=\"1190\">4. \u8c50\u5bcc\u7684Operators<\/strong><\/h5>\n<p data-start=\"1191\" data-end=\"1193\">\u5982\uff1a<\/p>\n<ul data-start=\"1194\" data-end=\"1278\">\n<li data-start=\"1194\" data-end=\"1210\">\n<p data-start=\"1196\" data-end=\"1210\">BashOperator<\/p>\n<\/li>\n<li data-start=\"1211\" data-end=\"1229\">\n<p data-start=\"1213\" data-end=\"1229\">PythonOperator<\/p>\n<\/li>\n<li data-start=\"1230\" data-end=\"1247\">\n<p data-start=\"1232\" data-end=\"1247\">EmailOperator<\/p>\n<\/li>\n<li data-start=\"1248\" data-end=\"1278\">\n<p data-start=\"1250\" data-end=\"1278\">AWS\u3001GCP\u3001Spark \u7b49\u5927\u91cf\u6574\u5408Operator<\/p>\n<\/li>\n<\/ul>\n<h5 data-start=\"1280\" data-end=\"1303\"><strong data-start=\"1284\" data-end=\"1303\">5. \u5f37\u5927\u7684UI \u8207\u76e3\u63a7\u80fd\u529b<\/strong><\/h5>\n<p data-start=\"1304\" data-end=\"1327\">\u67e5\u770b\u4efb\u52d9\u904b\u884c\u6b77\u53f2\u3001\u65e5\u8a8c\u3001\u5931\u6557\u91cd\u8a66\u3001\u57f7\u884c\u8017\u6642\u7b49\u3002<\/p>\n<h5 data-start=\"1329\" data-end=\"1346\"><strong data-start=\"1333\" data-end=\"1346\">6. \u591a\u57f7\u884c\u5668\u6a21\u5f0f<\/strong><\/h5>\n<ul data-start=\"1347\" data-end=\"1429\">\n<li data-start=\"1347\" data-end=\"1369\">\n<p data-start=\"1349\" data-end=\"1369\">SequentialExecutor<\/p>\n<\/li>\n<li data-start=\"1370\" data-end=\"1387\">\n<p data-start=\"1372\" data-end=\"1387\">LocalExecutor<\/p>\n<\/li>\n<li data-start=\"1388\" data-end=\"1406\">\n<p data-start=\"1390\" data-end=\"1406\">CeleryExecutor<\/p>\n<\/li>\n<li data-start=\"1407\" data-end=\"1429\">\n<p data-start=\"1409\" data-end=\"1429\">KubernetesExecutor<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"1431\" data-end=\"1446\">\u652f\u63f4\u5f9e\u8f15\u91cf\u5230\u4f01\u696d\u7d1a\u7684\u904b\u4f5c\u5f62\u614b\u3002<\/p>\n<h5 data-start=\"1448\" data-end=\"1465\"><strong data-start=\"1452\" data-end=\"1465\">7. \u5ee3\u6cdb\u61c9\u7528\u5834\u666f<\/strong><\/h5>\n<ul data-start=\"1466\" data-end=\"1531\">\n<li data-start=\"1466\" data-end=\"1480\">\n<p data-start=\"1468\" data-end=\"1480\">ETL \/ \u8cc7\u6599\u7ba1\u9053<\/p>\n<\/li>\n<li data-start=\"1481\" data-end=\"1491\">\n<p data-start=\"1483\" data-end=\"1491\">\u8cc7\u6599\u5009\u5132\u8f09\u5165<\/p>\n<\/li>\n<li data-start=\"1492\" data-end=\"1504\">\n<p data-start=\"1494\" data-end=\"1504\">\u6a5f\u5668\u5b78\u7fd2\u8a13\u7df4\u7de8\u6392<\/p>\n<\/li>\n<li data-start=\"1505\" data-end=\"1519\">\n<p data-start=\"1507\" data-end=\"1519\">DevOps \u81ea\u52d5\u5316<\/p>\n<\/li>\n<li data-start=\"1520\" data-end=\"1531\">\n<p data-start=\"1522\" data-end=\"1531\">\u6279\u6b21\u6d41\u7a0b\u7ba1\u7406<\/p>\n<\/li>\n<\/ul>\n<h4 data-start=\"1538\" data-end=\"1573\"><\/h4>\n<h4 data-start=\"1538\" data-end=\"1573\"><strong data-start=\"1540\" data-end=\"1573\">MWAA\uff1aAWS \u8a17\u7ba1\u5f0fAirflow \u7684\u4f01\u696d\u7d1a\u5347\u7d1a<\/strong><\/h4>\n<p data-start=\"1575\" data-end=\"1613\">\u5118\u7ba1Airflow \u5f37\u5927\uff0c\u4f46\u81ea\u884c\u642d\u5efa\u751f\u7522\u7d1aAirflow \u74b0\u5883\u6975\u70ba\u7e41\u7463\uff1a<\/p>\n<ul data-start=\"1615\" data-end=\"1735\">\n<li data-start=\"1615\" data-end=\"1642\">\n<p data-start=\"1617\" data-end=\"1642\">\u9700\u8981\u8a2d\u5b9a\u8abf\u5ea6\u5668\u3001Web \u4f3a\u670d\u5668\u3001Workers<\/p>\n<\/li>\n<li data-start=\"1643\" data-end=\"1673\">\n<p data-start=\"1645\" data-end=\"1673\">\u7ba1\u7406\u5143\u8cc7\u6599\u5eab\uff08PostgreSQL \/ MySQL\uff09<\/p>\n<\/li>\n<li data-start=\"1674\" data-end=\"1699\">\n<p data-start=\"1676\" data-end=\"1699\">\u90e8\u7f72\u5230Kubernetes \u6216\u5206\u6563\u5f0f\u67b6\u69cb<\/p>\n<\/li>\n<li data-start=\"1700\" data-end=\"1713\">\n<p data-start=\"1702\" data-end=\"1713\">\u624b\u52d5\u64f4\u5bb9\u8207\u7cfb\u7d71\u512a\u5316<\/p>\n<\/li>\n<li data-start=\"1714\" data-end=\"1735\">\n<p data-start=\"1716\" data-end=\"1735\">\u9700\u8981DevOps \u5718\u968a\u9577\u6642\u9593\u7dad\u8b77<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"1737\" data-end=\"1821\"><strong data-start=\"1737\" data-end=\"1790\">Amazon Managed Workflows for Apache Airflow\uff08MWAA\uff09<\/strong> \u89e3\u6c7a\u4e86\u6240\u6709\u9019\u4e9b\u554f\u984c\uff0c\u8b93\u4f01\u696d\u53ef\u4ee5\u300c\u958b\u7bb1\u5373\u7528\u300d Airflow\u3002<\/p>\n<h4 data-start=\"1828\" data-end=\"1846\"><\/h4>\n<h4 data-start=\"1828\" data-end=\"1846\"><strong data-start=\"1830\" data-end=\"1846\">MWAA \u7684\u6838\u5fc3\u512a\u52e2<\/strong><\/h4>\n<h4 data-start=\"1848\" data-end=\"1870\"><strong data-start=\"1851\" data-end=\"1870\">1. \u5168\u8a17\u7ba1\uff0c\u7121\u9700\u7dad\u8b77\u57fa\u790e\u8a2d\u65bd<\/strong><\/h4>\n<p data-start=\"1871\" data-end=\"1878\">AWS \u8ca0\u8cac\uff1a<\/p>\n<ul data-start=\"1879\" data-end=\"1929\">\n<li data-start=\"1879\" data-end=\"1887\">\n<p data-start=\"1881\" data-end=\"1887\">\u74b0\u5883\u90e8\u7f72<\/p>\n<\/li>\n<li data-start=\"1888\" data-end=\"1896\">\n<p data-start=\"1890\" data-end=\"1896\">\u64f4\u5bb9\u7e2e\u5bb9<\/p>\n<\/li>\n<li data-start=\"1897\" data-end=\"1905\">\n<p data-start=\"1899\" data-end=\"1905\">\u88dc\u4e01\u66f4\u65b0<\/p>\n<\/li>\n<li data-start=\"1906\" data-end=\"1915\">\n<p data-start=\"1908\" data-end=\"1915\">\u9ad8\u53ef\u7528\u67b6\u69cb<\/p>\n<\/li>\n<li data-start=\"1916\" data-end=\"1929\">\n<p data-start=\"1918\" data-end=\"1929\">\u670d\u52d9\u7a69\u5b9a\u6027\u8207\u5b89\u5168\u6027<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"1931\" data-end=\"1947\">\u4f01\u696d\u53ea\u9700\u5c08\u6ce8DAG \u908f\u8f2f\u672c\u8eab\u3002<\/p>\n<h4 data-start=\"1954\" data-end=\"1976\"><strong data-start=\"1957\" data-end=\"1976\">2. \u8207AWS \u670d\u52d9\u6df1\u5ea6\u96c6\u6210<\/strong><\/h4>\n<p data-start=\"1977\" data-end=\"1985\">\u958b\u7bb1\u5373\u7528\u5730\u652f\u63f4\uff1a<\/p>\n<ul data-start=\"1986\" data-end=\"2107\">\n<li data-start=\"1986\" data-end=\"1999\">\n<p data-start=\"1988\" data-end=\"1999\">Amazon S3<\/p>\n<\/li>\n<li data-start=\"2000\" data-end=\"2019\">\n<p data-start=\"2002\" data-end=\"2019\">Amazon Redshift<\/p>\n<\/li>\n<li data-start=\"2020\" data-end=\"2034\">\n<p data-start=\"2022\" data-end=\"2034\">AWS Lambda<\/p>\n<\/li>\n<li data-start=\"2035\" data-end=\"2047\">\n<p data-start=\"2037\" data-end=\"2047\">AWS Glue<\/p>\n<\/li>\n<li data-start=\"2048\" data-end=\"2062\">\n<p data-start=\"2050\" data-end=\"2062\">Amazon EMR<\/p>\n<\/li>\n<li data-start=\"2063\" data-end=\"2081\">\n<p data-start=\"2065\" data-end=\"2081\">Step Functions<\/p>\n<\/li>\n<li data-start=\"2082\" data-end=\"2107\">\n<p data-start=\"2084\" data-end=\"2107\">SageMaker<br data-start=\"2093\" data-end=\"2096\" \/>\u7b49\u6578\u64da\u8207AI \u670d\u52d9\u3002<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"2109\" data-end=\"2137\">\u9019\u8b93\u5efa\u7f6eETL \/ ML Pipeline \u66f4\u6709\u6548\u7387\u3002<\/p>\n<h4 data-start=\"2144\" data-end=\"2160\"><strong data-start=\"2147\" data-end=\"2160\">3. \u81ea\u52d5\u6c34\u5e73\u64f4\u5c55<\/strong><\/h4>\n<p data-start=\"2161\" data-end=\"2200\">MWAA \u6703\u6839\u64da\u5de5\u4f5c\u8ca0\u8f09\u81ea\u52d5\u8abf\u6574Worker \u6578\u91cf\uff0c\u907f\u514d\u8cc7\u6e90\u6d6a\u8cbb\u6216\u53e2\u96c6\u904e\u8f09\u3002<\/p>\n<h4 data-start=\"2207\" data-end=\"2224\"><strong data-start=\"2210\" data-end=\"2224\">4. \u4f01\u696d\u7d1a\u5b89\u5168\u8a2d\u8a08<\/strong><\/h4>\n<ul data-start=\"2225\" data-end=\"2286\">\n<li data-start=\"2225\" data-end=\"2238\">\n<p data-start=\"2227\" data-end=\"2238\">VPC \u5167\u5b89\u5168\u9694\u96e2<\/p>\n<\/li>\n<li data-start=\"2239\" data-end=\"2251\">\n<p data-start=\"2241\" data-end=\"2251\">KMS \u8cc7\u6599\u52a0\u5bc6<\/p>\n<\/li>\n<li data-start=\"2252\" data-end=\"2267\">\n<p data-start=\"2254\" data-end=\"2267\">IAM \u7d30\u7c92\u5ea6\u6b0a\u9650\u63a7\u5236<\/p>\n<\/li>\n<li data-start=\"2268\" data-end=\"2286\">\n<p data-start=\"2270\" data-end=\"2286\">PrivateLink \u652f\u6301<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"2288\" data-end=\"2304\">\u9069\u5408\u91d1\u878d\u3001\u653f\u4f01\u7b49\u9ad8\u7b49\u7d1a\u5b89\u5168\u5834\u666f\u3002<\/p>\n<h4 data-start=\"2311\" data-end=\"2327\"><strong data-start=\"2314\" data-end=\"2327\">5. \u5b8c\u6574\u76e3\u63a7\u9ad4\u7cfb<\/strong><\/h4>\n<p data-start=\"2328\" data-end=\"2360\">\u6240\u6709\u65e5\u8a8c\u81ea\u52d5\u50b3\u9001\u5230CloudWatch\uff0c\u76e3\u63a7\u6307\u6a19\u8cc7\u6599\u900f\u660e\u53ef\u8996\u3002<\/p>\n<h4 data-start=\"2367\" data-end=\"2394\"><strong data-start=\"2370\" data-end=\"2394\">6. \u652f\u63f4\u81ea\u8a02\u5916\u639b\u7a0b\u5f0f\u8207Python \u51fd\u5f0f\u5eab<\/strong><\/h4>\n<p data-start=\"2395\" data-end=\"2446\">\u53ef\u900f\u904eS3 \u4e0a\u50b3\u5916\u639b\u7a0b\u5f0f\u3001requirements.txt\uff0c\u5be6\u73fe\u81ea\u8a02Operator \u6216\u7b2c\u4e09\u65b9\u4f9d\u8cf4\u3002<\/p>\n<p>&nbsp;<\/p>\n<h4 data-start=\"2453\" data-end=\"2473\"><strong data-start=\"2455\" data-end=\"2473\">MWAA \u7684\u7cfb\u7d71\u67b6\u69cb\u89e3\u6790<\/strong><\/h4>\n<p data-start=\"2475\" data-end=\"2493\">\u4e00\u500bMWAA \u74b0\u5883\u7531\u4e0b\u5217\u7d44\u4ef6\u6240\u69cb\u6210\uff1a<\/p>\n<ul data-start=\"2495\" data-end=\"2649\">\n<li data-start=\"2495\" data-end=\"2525\">\n<p data-start=\"2497\" data-end=\"2525\"><strong data-start=\"2497\" data-end=\"2518\">Airflow Scheduler<\/strong>\uff08\u8abf\u5ea6\u5668\uff09<\/p>\n<\/li>\n<li data-start=\"2526\" data-end=\"2547\">\n<p data-start=\"2528\" data-end=\"2547\"><strong data-start=\"2528\" data-end=\"2545\">Web Server UI<\/strong><\/p>\n<\/li>\n<li data-start=\"2548\" data-end=\"2569\">\n<p data-start=\"2550\" data-end=\"2569\"><strong data-start=\"2550\" data-end=\"2561\">Workers<\/strong>\uff08\u81ea\u52d5\u64f4\u5c55\uff09<\/p>\n<\/li>\n<li data-start=\"2570\" data-end=\"2596\">\n<p data-start=\"2572\" data-end=\"2596\"><strong data-start=\"2572\" data-end=\"2594\">\u8a17\u7ba1PostgreSQL \u5143\u8cc7\u6599\u5eab<\/strong><\/p>\n<\/li>\n<li data-start=\"2597\" data-end=\"2627\">\n<p data-start=\"2599\" data-end=\"2627\"><strong data-start=\"2599\" data-end=\"2625\">S3 \u5132\u5b58DAG\/\u63d2\u4ef6\/Python \u4f9d\u8cf4<\/strong><\/p>\n<\/li>\n<li data-start=\"2628\" data-end=\"2649\">\n<p data-start=\"2630\" data-end=\"2649\"><strong data-start=\"2630\" data-end=\"2649\">CloudWatch \u65e5\u8a8c\u7cfb\u7d71<\/strong><\/p>\n<\/li>\n<\/ul>\n<p data-start=\"2651\" data-end=\"2669\">\u74b0\u5883\u53ef\u90e8\u7f72\u5728\u591a\u53ef\u7528\u5340\uff0c\u78ba\u4fdd\u9ad8\u53ef\u7528\u6027\u3002<\/p>\n<h4 data-start=\"2676\" data-end=\"2698\"><\/h4>\n<h4 data-start=\"2676\" data-end=\"2698\"><strong data-start=\"2678\" data-end=\"2698\">MWAA \u7684\u5178\u578b\u4f01\u696d\u61c9\u7528\u5834\u666f<\/strong><\/h4>\n<h5 data-start=\"2700\" data-end=\"2718\"><strong data-start=\"2704\" data-end=\"2718\">1. \u5927\u898f\u6a21\u8cc7\u6599\u5de5\u7a0b<\/strong><\/h5>\n<ul data-start=\"2719\" data-end=\"2758\">\n<li data-start=\"2719\" data-end=\"2743\">\n<p data-start=\"2721\" data-end=\"2743\">S3 \u2192 Glue \u2192 Redshift<\/p>\n<\/li>\n<li data-start=\"2744\" data-end=\"2758\">\n<p data-start=\"2746\" data-end=\"2758\">Spark \u4efb\u52d9\u8abf\u5ea6<\/p>\n<\/li>\n<\/ul>\n<h5 data-start=\"2760\" data-end=\"2789\"><strong data-start=\"2764\" data-end=\"2789\">2. AI\/ML Pipeline \u81ea\u52d5\u5316<\/strong><\/h5>\n<ul data-start=\"2790\" data-end=\"2824\">\n<li data-start=\"2790\" data-end=\"2798\">\n<p data-start=\"2792\" data-end=\"2798\">\u8cc7\u6599\u6e96\u5099<\/p>\n<\/li>\n<li data-start=\"2799\" data-end=\"2815\">\n<p data-start=\"2801\" data-end=\"2815\">SageMaker \u8a13\u7df4<\/p>\n<\/li>\n<li data-start=\"2816\" data-end=\"2824\">\n<p data-start=\"2818\" data-end=\"2824\">\u6a21\u578b\u90e8\u7f72<\/p>\n<\/li>\n<\/ul>\n<h5 data-start=\"2826\" data-end=\"2845\"><strong data-start=\"2830\" data-end=\"2845\">3. \u8de8\u90e8\u9580\u4efb\u52d9\u81ea\u52d5\u5316<\/strong><\/h5>\n<ul data-start=\"2846\" data-end=\"2880\">\n<li data-start=\"2846\" data-end=\"2854\">\n<p data-start=\"2848\" data-end=\"2854\">\u65e5\u5831\u4efb\u52d9<\/p>\n<\/li>\n<li data-start=\"2855\" data-end=\"2863\">\n<p data-start=\"2857\" data-end=\"2863\">\u6578\u64da\u532f\u7e3d<\/p>\n<\/li>\n<li data-start=\"2864\" data-end=\"2880\">\n<p data-start=\"2866\" data-end=\"2880\">DevOps \u81ea\u52d5\u5316\u8173\u672c<\/p>\n<\/li>\n<\/ul>\n<h5 data-start=\"2882\" data-end=\"2900\"><strong data-start=\"2886\" data-end=\"2900\">4. \u6279\u6b21\u4efb\u52d9\u7ba1\u7406<\/strong><\/h5>\n<p data-start=\"2901\" data-end=\"2918\">\u5927\u91cf\u8cc7\u6599\u6279\u91cf\u6e05\u6d17\u3001\u8f49\u63db\u8207\u5206\u6790\u64cd\u4f5c\u3002<\/p>\n<h4 data-start=\"2925\" data-end=\"2959\"><\/h4>\n<h4 data-start=\"2925\" data-end=\"2959\"><strong data-start=\"2927\" data-end=\"2959\">MWAA vs \u81ea\u8a17\u7ba1Airflow\uff1a\u4f01\u696d\u9078\u64c7\u6307\u5357<\/strong><\/h4>\n<div class=\"TyagGW_tableContainer\">\n<div class=\"group TyagGW_tableWrapper flex w-fit flex-col-reverse\" tabindex=\"-1\">\n<table class=\"w-fit min-w-(--thread-content-width)\" data-start=\"2961\" data-end=\"3226\">\n<thead data-start=\"2961\" data-end=\"2993\">\n<tr data-start=\"2961\" data-end=\"2993\">\n<th style=\"text-align: center;\" data-start=\"2961\" data-end=\"2967\" data-col-size=\"sm\">\u5c0d\u6bd4\u9805<\/th>\n<th style=\"text-align: center;\" data-start=\"2967\" data-end=\"2978\" data-col-size=\"sm\">MWAA\uff08\u8a17\u7ba1\uff09<\/th>\n<th style=\"text-align: center;\" data-start=\"2978\" data-end=\"2993\" data-col-size=\"sm\">\u81ea\u8a17\u7ba1Airflow<\/th>\n<\/tr>\n<\/thead>\n<tbody data-start=\"3035\" data-end=\"3226\">\n<tr data-start=\"3035\" data-end=\"3070\">\n<td style=\"text-align: center;\" data-start=\"3035\" data-end=\"3042\" data-col-size=\"sm\">\u90e8\u7f72\u96e3\u5ea6<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3042\" data-end=\"3057\">\u2605\u2606\u2606\u2606\u2606\uff08\u5e7e\u4e4e\u96f6\u8a2d\u5b9a\uff09<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3057\" data-end=\"3070\">\u2605\u2605\u2605\u2605\u2605\uff08\u6975\u9ad8\uff09<\/td>\n<\/tr>\n<tr data-start=\"3071\" data-end=\"3095\">\n<td style=\"text-align: center;\" data-start=\"3071\" data-end=\"3078\" data-col-size=\"sm\">\u7dad\u904b\u6210\u672c<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3078\" data-end=\"3082\">\u4f4e<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3082\" data-end=\"3095\">\u9ad8\uff08\u9700\u8981\u5c08\u9580\u5718\u968a\uff09<\/td>\n<\/tr>\n<tr data-start=\"3096\" data-end=\"3132\">\n<td style=\"text-align: center;\" data-start=\"3096\" data-end=\"3103\" data-col-size=\"sm\">\u64f4\u5c55\u80fd\u529b<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3103\" data-end=\"3110\">\u81ea\u52d5\u64f4\u5bb9<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3110\" data-end=\"3132\">\u9700\u624b\u5de5\u8a2d\u8a08+ Kubernetes<\/td>\n<\/tr>\n<tr data-start=\"3133\" data-end=\"3164\">\n<td style=\"text-align: center;\" data-start=\"3133\" data-end=\"3139\" data-col-size=\"sm\">\u5b89\u5168\u6027<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3139\" data-end=\"3151\">AWS \u4f01\u696d\u7d1a\u5b89\u5168<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3151\" data-end=\"3164\">\u9700\u81ea\u884c\u5efa\u69cb\u5b89\u5168\u9ad4\u7cfb<\/td>\n<\/tr>\n<tr data-start=\"3165\" data-end=\"3196\">\n<td style=\"text-align: center;\" data-start=\"3165\" data-end=\"3170\" data-col-size=\"sm\">\u76e3\u63a7<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3170\" data-end=\"3186\">CloudWatch \u96c6\u6210<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3186\" data-end=\"3196\">\u9700\u81ea\u5efa\u76e3\u63a7\u5806\u758a<\/td>\n<\/tr>\n<tr data-start=\"3197\" data-end=\"3226\">\n<td style=\"text-align: center;\" data-start=\"3197\" data-end=\"3205\" data-col-size=\"sm\">\u6210\u672c\u53ef\u63a7\u6027<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3205\" data-end=\"3212\">\u6309\u9700\u4ed8\u8cbb<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"3212\" data-end=\"3226\">\u9700\u9577\u671f\u7dad\u8b77\u6a5f\u5668\/\u96c6\u7fa4<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<p data-start=\"3228\" data-end=\"3259\"><strong data-start=\"3228\" data-end=\"3259\">\u4f01\u696d\u898f\u6a21\u8d8a\u5927\uff0c\u5c0d\u9ad8\u53ef\u7528\u9700\u6c42\u8d8a\u9ad8\uff0c\u8d8a\u9069\u5408\u9078\u64c7MWAA\u3002<\/strong><\/p>\n<h4 data-start=\"3266\" data-end=\"3292\"><strong data-start=\"3268\" data-end=\"3292\">\u5728\u96f2\u7aef\u4e0a<\/strong><\/h4>\n<p data-start=\"3294\" data-end=\"3335\">\u4f5c\u70ba <strong data-start=\"3297\" data-end=\"3308\">AWS \u4ee3\u7406\u5546<\/strong>\uff0c\u6211\u5011\u53ef\u70ba\u4f01\u696d\u63d0\u4f9b\u5f9e\u65b9\u6848\u8a2d\u8a08\u3001\u9077\u79fb\u5230\u843d\u5730\u7684\u5168\u6d41\u7a0b\u652f\u63f4\uff1a<\/p>\n<h5 data-start=\"3337\" data-end=\"3361\"><strong data-start=\"3341\" data-end=\"3359\">1.MWAA \u4e0a\u96f2\u53ef\u884c\u6027\u8a55\u4f30<\/strong><\/h5>\n<p data-start=\"3362\" data-end=\"3389\">\u6839\u64da\u73fe\u6709Airflow \u74b0\u5883\u8a55\u4f30\u9077\u79fb\u8def\u5f91\u8207\u6210\u672c\u6548\u76ca\u3002<\/p>\n<h5 data-start=\"3391\" data-end=\"3417\"><strong data-start=\"3395\" data-end=\"3415\">2.DAG \u6539\u9020\u8207\u63d2\u4ef6\u76f8\u5bb9\u6027\u6aa2\u67e5<\/strong><\/h5>\n<p data-start=\"3418\" data-end=\"3430\">\u78ba\u4fdd\u6240\u6709\u4efb\u52d9\u53ef\u5e73\u6ed1\u904b\u884c\u3002<\/p>\n<h5 data-start=\"3432\" data-end=\"3455\"><strong data-start=\"3436\" data-end=\"3453\">3.\u67b6\u69cb\u8a2d\u8a08\u8207\u5b89\u5168\u898f\u7bc4\u843d\u5730<\/strong><\/h5>\n<p data-start=\"3456\" data-end=\"3491\">\u5305\u62ecVPC\u3001IAM\u3001\u5b89\u5168\u7fa4\u7d44\u3001KMS\u3001PrivateLink \u7b49\u914d\u7f6e\u3002<\/p>\n<h5 data-start=\"3493\" data-end=\"3511\"><strong data-start=\"3497\" data-end=\"3509\">4.\u6210\u672c\u6700\u4f73\u5316\u65b9\u6848<\/strong><\/h5>\n<p data-start=\"3512\" data-end=\"3530\">\u7d50\u5408\u5be6\u969b\u5de5\u4f5c\u8ca0\u8f09\uff0c\u7d66\u4e88\u6700\u4f73\u8cc7\u6e90\u914d\u7f6e\u3002<\/p>\n<h5 data-start=\"3532\" data-end=\"3562\"><strong data-start=\"3536\" data-end=\"3560\">5.\u6301\u7e8c\u6280\u8853\u652f\u63f4\u8207AWS \u5b98\u65b9\u8cc7\u6e90\u5c0d\u63a5<\/strong><\/h5>\n<p data-start=\"3563\" data-end=\"3582\">\u53d6\u5f97\u6700\u65b0AWS \u6280\u8853\u66f4\u65b0\u8207\u65b9\u6848\u6307\u5c0e\u3002<\/p>\n<h4 data-start=\"3589\" data-end=\"3597\"><\/h4>\n<h4 data-start=\"3589\" data-end=\"3597\"><strong data-start=\"3591\" data-end=\"3597\">\u603b\u7ed3<\/strong><\/h4>\n<p data-start=\"3599\" data-end=\"3699\"><strong data-start=\"3599\" data-end=\"3645\">MWAA \u5e36\u4f86\u4e86Airflow \u7684\u9748\u6d3b\u6027+ AWS \u7684\u7a69\u5b9a\u6027\u3001\u5b89\u5168\u6027\u8207\u53ef\u64f4\u5c55\u6027\u3002<\/strong><br data-start=\"3645\" data-end=\"3648\" \/>\u5c0d\u65bc\u5e0c\u671b\u5feb\u901f\u5efa\u7f6e\u73fe\u4ee3\u8cc7\u6599\u7ba1\u9053\u3001\u964d\u4f4e\u7dad\u904b\u6210\u672c\u3001\u63d0\u5347\u4efb\u52d9\u53ef\u9760\u6027\u7684\u4f01\u696d\u4f86\u8aaa\uff0c\u5b83\u662f\u76ee\u524d\u6700\u512a\u7684\u8cc7\u6599\u7de8\u6392\u65b9\u6848\u4e4b\u4e00\u3002<\/p>\n<p data-start=\"3701\" data-end=\"3765\">\u5982\u679c\u4f60\u7684\u5718\u968a\u6b63\u5728\u4f7f\u7528Apache Airflow \u6216\u8a08\u5283\u5efa\u7acbETL\/ML \u5de5\u4f5c\u6d41\u7a0b\uff0cMWAA \u5c07\u662f\u503c\u5f97\u512a\u5148\u8a55\u4f30\u7684\u8a17\u7ba1\u670d\u52d9\u9078\u64c7\u3002<\/p>","protected":false},"excerpt":{"rendered":"<p>\u5728\u4f01\u696d\u6578\u4f4d\u5316\u52a0\u901f\u7684\u7576\u4e0b\uff0c\u8cc7\u6599\u5de5\u7a0b\u3001\u6a5f\u5668\u5b78\u7fd2\u8207\u6279\u6b21\u4efb\u52d9\u65e5\u76ca\u8907\u96dc\uff0c\u5de5\u4f5c\u6d41\u7a0b\u7684\u81ea\u52d5\u5316\u8207\u53ef\u9760\u6027\u6210\u70ba\u696d\u52d9\u57fa\u790e\u80fd\u529b\u3002 Apache Airflow \u4f5c\u70ba\u696d\u754c\u4e3b\u6d41\u7684\u5de5\u4f5c\u6d41\u7a0b\u7de8\u6392\u6846\u67b6\uff0c\u88ab\u5ee3\u6cdb\u7528\u65bcETL \u8abf\u5ea6\u3001\u8cc7\u6599\u7ba1\u9053\u7ba1\u7406\u8207\u6a5f\u5668\u5b78\u7fd2\u6d41\u7a0b\u7de8\u6392\u3002\u7136\u800c\uff0c\u50b3\u7d71\u81ea\u5efaAirflow \u74b0\u5883\u9762\u81e8\u90e8\u7f72\u8907\u96dc\u3001\u64f4\u5c55\u56f0\u96e3\u3001\u7dad\u4fee\u6210\u672c\u9ad8\u7b49\u6311\u6230\u3002<\/p>","protected":false},"author":1,"featured_media":4219,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[65],"tags":[],"class_list":["post-4218","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technical-sharing"],"_links":{"self":[{"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/posts\/4218","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/comments?post=4218"}],"version-history":[{"count":2,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/posts\/4218\/revisions"}],"predecessor-version":[{"id":4224,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/posts\/4218\/revisions\/4224"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/media\/4219"}],"wp:attachment":[{"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/media?parent=4218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/categories?post=4218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aws-oncloudai.com\/zh_hk\/wp-json\/wp\/v2\/tags?post=4218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}