{"id":3338,"date":"2025-06-20T17:24:26","date_gmt":"2025-06-20T09:24:26","guid":{"rendered":"https:\/\/aws-oncloudai.com\/?p=3338"},"modified":"2025-06-20T17:24:26","modified_gmt":"2025-06-20T09:24:26","slug":"comprehensive-analysis-of-aws-emr","status":"publish","type":"post","link":"https:\/\/aws-oncloudai.com\/ko\/aws-emr\uc5d0-\ub300\ud55c-\uc885\ud569\uc801\uc778-\ubd84\uc11d\/","title":{"rendered":"AWS EMR"},"content":{"rendered":"<p>\uc624\ub298\ub0a0 \ub370\uc774\ud130 \uc911\uc2ec\uc740 \uae30\uc5c5\uc758 \ud575\uc2ec \uacbd\uc7c1\ub825\uc73c\ub85c \uc790\ub9ac \uc7a1\uc558\uc2b5\ub2c8\ub2e4. PB \ub2e8\uc704\uc758 \ub370\uc774\ud130\ub97c \ud6a8\uc728\uc801\uc774\uace0 \uc800\ub834\ud558\uac8c \ucc98\ub9ac\ud558\ub294 \ubc29\ubc95\uc740 \ub9ce\uc740 \uae30\uc5c5\uc758 IT \uc804\ub7b5\uc5d0\uc11c \uc911\uc694\ud55c \ubd80\ubd84\uc744 \ucc28\uc9c0\ud558\uac8c \ub418\uc5c8\uc2b5\ub2c8\ub2e4. AWS \uae30\ubc18 \uad00\ub9ac\ud615 \ube45\ub370\uc774\ud130 \ucc98\ub9ac \ud50c\ub7ab\ud3fc\uc778 Amazon Elastic MapReduce(EMR)\ub294 \uc720\uc5f0\uc131, \ud655\uc7a5\uc131, \uadf8\ub9ac\uace0 \ub192\uc740 \ube44\uc6a9 \ud6a8\uc728\uc131\uc73c\ub85c \uae08\uc735, \uc804\uc790\uc0c1\uac70\ub798, \uac8c\uc784, \uad11\uace0, \uacfc\ud559 \uc5f0\uad6c \ub4f1 \ub2e4\uc591\ud55c \ubd84\uc57c\uc758 \uace0\uac1d\uc5d0\uac8c \ucd5c\uace0\uc758 \uc120\ud0dd\uc774 \ub418\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>\n<p>AWS \uacf5\uc778 \uc5d0\uc774\uc804\ud2b8\ub85c\uc11c, \uc6b0\ub9ac\ub294 \uadc0\ud558\uc5d0\uac8c AWS EMR\uc758 \uae30\uc220\uc801 \uc6d0\ub9ac, \ud575\uc2ec\uc801 \uc7a5\uc810, \uc77c\ubc18\uc801\uc778 \uc801\uc6a9 \uc2dc\ub098\ub9ac\uc624\uc5d0 \ub300\ud55c \uc2ec\uce35\uc801\uc778 \uc774\ud574\ub97c \uc81c\uacf5\ud558\uace0, \uae30\uc5c5 \uad6c\ud604\uc5d0 \ub300\ud55c \uc81c\uc548\uacfc \ub9ac\uc18c\uc2a4 \uc9c0\uc6d0\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<h4>AWS EMR\uc774\ub780 \ubb34\uc5c7\uc778\uac00\uc694?<\/h4>\n<p>Amazon EMR(Elastic MapReduce)\uc740 Apache Hadoop, Apache Spark, Presto, Hive, Flink, Trino \ub4f1\uacfc \uac19\uc740 \uc778\uae30 \uc788\ub294 \ube45\ub370\uc774\ud130 \ud504\ub808\uc784\uc6cc\ud06c\ub97c \uc27d\uac8c \uc2e4\ud589\ud560 \uc218 \uc788\ub294 \ud074\ub77c\uc6b0\ub4dc \uae30\ubc18\uc758 \uad00\ub9ac\ud615 \ube45\ub370\uc774\ud130 \ucc98\ub9ac \uc11c\ube44\uc2a4\uc785\ub2c8\ub2e4.<\/p>\n<p>EMR\uc740 \uae30\uc5c5\uc774 \ub85c\uadf8 \ubd84\uc11d, ETL(\ucd94\ucd9c, \ubcc0\ud658, \ub85c\ub529), \uba38\uc2e0 \ub7ec\ub2dd \ubaa8\ub378\ub9c1, \uc2e4\uc2dc\uac04 \ub370\uc774\ud130 \uc2a4\ud2b8\ub9bc \ucef4\ud4e8\ud305 \ub4f1\uacfc \uac19\uc740 \ub300\uaddc\ubaa8 \ub370\uc774\ud130 \uc791\uc5c5\uc744 \ucc98\ub9ac\ud558\ub294 \ubd84\uc0b0 \ud074\ub7ec\uc2a4\ud130\ub97c \uad6c\ucd95\ud560 \uc218 \uc788\ub3c4\ub85d \uc9c0\uc6d0\ud569\ub2c8\ub2e4. \uae30\uc874\uc758 \ub85c\uceec \ud074\ub7ec\uc2a4\ud130\uc640 \ube44\uad50\ud588\uc744 \ub54c EMR\uc740 \uc131\ub2a5, \ud0c4\ub825\uc131, \uc720\uc9c0 \uad00\ub9ac \uc6a9\uc774\uc131 \ubc0f \ube44\uc6a9 \ucd5c\uc801\ud654 \uce21\uba74\uc5d0\uc11c \uc0c1\ub2f9\ud55c \uc774\uc810\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<h4>\ud575\uc2ec \uc7a5\uc810 \ubd84\uc11d<\/h4>\n<h5>1. \ud0c4\ub825\uc801\uc778 \ud655\uc7a5\uc131\uacfc \uc720\uc5f0\ud55c \uacb0\uc81c<\/h5>\n<ul>\n<li>\ud074\ub7ec\uc2a4\ud130 \ud06c\uae30\ub294 \uc791\uc5c5 \ubd80\ud558\uc5d0 \ub530\ub77c \uc790\ub3d9\uc73c\ub85c \ud655\uc7a5\ub418\uac70\ub098 \ucd95\uc18c\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li>\uc628\ub514\ub9e8\ub4dc \uc778\uc2a4\ud134\uc2a4, \uc2a4\ud31f \uc778\uc2a4\ud134\uc2a4(\ucd5c\ub300 90% \uc808\uc57d), \uc800\ucd95 \uacc4\ud68d\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4.<\/li>\n<li>\ubd84 \ub2e8\uc704\ub85c \uccad\uad6c\uac00 \uac00\ub2a5\ud574 \uc720\ud734 \ub9ac\uc18c\uc2a4 \ube44\uc6a9\uc774 \ud06c\uac8c \uc904\uc5b4\ub4ed\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h5>2. AWS \uc5d0\ucf54\uc2dc\uc2a4\ud15c\uacfc\uc758 \uc6d0\ud65c\ud55c \ud1b5\ud569<\/h5>\n<ul>\n<li>\ub124\uc774\ud2f0\ube0c \ud1b5\ud569 <strong>\uc544\ub9c8\uc874 S3<\/strong>,<strong>Glue \ub370\uc774\ud130 \uce74\ud0c8\ub85c\uadf8<\/strong>,<strong>\ud074\ub77c\uc6b0\ub4dc\uc6cc\uce58<\/strong>,<strong>\uadf8\ub798\uc694<\/strong> \uae30\ub2e4\ub9ac\ub2e4;<\/li>\n<li>\uadf8\ub9ac\uace0 <strong>\uc544\ud14c\ub098<\/strong>,<strong>\uc801\uc0c9\ud3b8\uc774<\/strong>,<strong>\uc138\uc774\uc9c0\uba54\uc774\ucee4<\/strong> \ub370\uc774\ud130 \ubd84\uc11d\uacfc AI \ub3c4\uad6c\ub294 \uc644\ubcbd\ud558\uac8c \uc5f0\uacb0\ub429\ub2c8\ub2e4.<\/li>\n<li>\uc9c0\uc6d0\ub418\ub294 \uc0ac\uc6a9 <strong>AWS \ub808\uc774\ud06c \ud615\uc131<\/strong> \ud1b5\ud569\ub41c \ub370\uc774\ud130 \ub808\uc774\ud06c\ub97c \uad6c\ucd95\ud558\uc138\uc694.<\/li>\n<\/ul>\n<h5>3. \ub2e4\uc591\ud55c \ube45\ub370\uc774\ud130 \uc5d4\uc9c4 \uc9c0\uc6d0<\/h5>\n<ul>\n<li>Spark, Hive, Presto, Flink, HBase, Trino \ub4f1 \uc778\uae30 \uc788\ub294 \ub3c4\uad6c\ub97c \uc120\ud0dd\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li>\ub2e4\uc591\ud55c \uc791\uc5c5 \ubd80\ud558\uc5d0 \ub530\ub77c \uc5ec\ub7ec \ucef4\ud4e8\ud305 \ubaa8\ub378(\uc77c\uad04 \ucc98\ub9ac, \ub300\ud654\ud615 \ubd84\uc11d, \uc2a4\ud2b8\ub9bc \ucc98\ub9ac)\uc744 \uad6c\uc131\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li>Hadoop \uc0dd\ud0dc\uacc4\uc640 \ud638\ud658\ub418\uba70 \uae30\uc874 \uc791\uc5c5\uc758 \ub9c8\uc774\uadf8\ub808\uc774\uc158\uc744 \uc9c0\uc6d0\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h5>4. \uace0\uac00\uc6a9\uc131 \ubc0f \uc790\ub3d9 \uc7a5\uc560 \ud5c8\uc6a9<\/h5>\n<ul>\n<li>\ub2e4\uc911 \uac00\uc6a9\uc131 \uc601\uc5ed\uc5d0 \ub300\ud55c \uc7a5\uc560 \ud5c8\uc6a9 \uae30\ub2a5\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<li>\ud074\ub7ec\uc2a4\ud130 \ub178\ub4dc\uc758 \uc790\ub3d9 \uad50\uccb4 \ubc0f \uc791\uc5c5 \uc7ac\uc2dc\ub3c4\ub97c \uc9c0\uc6d0\ud569\ub2c8\ub2e4.<\/li>\n<li>CloudWatch\uc640 \uacb0\ud569\ud558\uc5ec \uc2e4\uc2dc\uac04 \ubaa8\ub2c8\ud130\ub9c1\uacfc \uc54c\ub78c \uc124\uc815\uc744 \ud65c\uc6a9\ud558\uc138\uc694.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h4>\uc77c\ubc18\uc801\uc778 \uc751\uc6a9 \ud504\ub85c\uadf8\ub7a8 \uc2dc\ub098\ub9ac\uc624<\/h4>\n<h5>1. \ub370\uc774\ud130\uc6e8\uc5b4\ud558\uc6b0\uc2a4 \ubc0f BI \ubd84\uc11d<\/h5>\n<p>EMR\uc740 Hive\/Presto\uc640 \ud611\ub825\ud558\uc5ec \ub300\uaddc\ubaa8 SQL \ucffc\ub9ac \uc791\uc5c5\uc744 \ud6a8\uc728\uc801\uc73c\ub85c \uc2e4\ud589\ud558\uace0, \ub370\uc774\ud130\uc6e8\uc5b4\ud558\uc6b0\uc2a4\ub97c \uad6c\ucd95\ud558\uace0, \uc2dc\uac01\uc801 BI \ub3c4\uad6c(\uc608: QuickSight, Tableau, Superset)\uc5d0 \ub300\ud55c \uc561\uc138\uc2a4\ub97c \uc9c0\uc6d0\ud569\ub2c8\ub2e4.<\/p>\n<h5>2. ETL \ub370\uc774\ud130 \ucc98\ub9ac<\/h5>\n<p>S3, RDS, DynamoDB \ub4f1\uc758 \ub370\uc774\ud130 \uc18c\uc2a4\uc5d0\uc11c \ub370\uc774\ud130\ub97c \ucd94\ucd9c\ud558\uace0 Spark\/Hadoop\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc815\ub9ac \ubc0f \ubcc0\ud658\ud55c \ub2e4\uc74c \uacb0\uacfc\ub97c S3\uc5d0 \ub2e4\uc2dc \uc4f0\uac70\ub098 \ub370\uc774\ud130 \ub808\uc774\ud06c\uc5d0 \ub85c\ub4dc\ud569\ub2c8\ub2e4.<\/p>\n<h5>3. \uba38\uc2e0\ub7ec\ub2dd \ubaa8\ub378\ub9c1<\/h5>\n<p>Spark MLlib \ubc0f XGBoost\uc640 \uac19\uc740 \ub3c4\uad6c\ub97c \ud1b5\ud574 \uae30\ub2a5 \uc5d4\uc9c0\ub2c8\uc5b4\ub9c1 \ubc0f \ubaa8\ub378\ub9c1\uc744 \uc218\ud589\ud558\uace0, \ud559\uc2b5 \uacb0\uacfc\ub294 SageMaker\uc640 \ud568\uaed8 \ubc30\ud3ec\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<h5>4. \uc2e4\uc2dc\uac04 \ub85c\uadf8 \ubc0f \uc2a4\ud2b8\ub9ac\ubc0d \ucef4\ud4e8\ud305<\/h5>\n<p>\ub85c\uadf8 \ubaa8\ub2c8\ud130\ub9c1, \ub3d9\uc791 \ubd84\uc11d, \uc0ac\uae30 \uac10\uc9c0 \ub4f1\uc758 \uc694\uad6c \uc0ac\ud56d\uc744 \ucda9\uc871\ud558\ub294 \uc2e4\uc2dc\uac04 \ucef4\ud4e8\ud305 \ud30c\uc774\ud504\ub77c\uc778\uc744 \uad6c\ucd95\ud558\ub824\uba74 Flink \ub610\ub294 Spark Streaming\uc744 \uc0ac\uc6a9\ud558\uc138\uc694.<\/p>\n<h5>5. \ube44\uc6a9\uc5d0 \ubbfc\uac10\ud55c \ub300\uaddc\ubaa8 \ubd84\uc11d<\/h5>\n<p>Spot \uc778\uc2a4\ud134\uc2a4\ub97c \uc0ac\uc6a9\ud558\uba74 \uc784\uc2dc \ucef4\ud4e8\ud305 \uc791\uc5c5\uc744 \uc608\uc57d\ud558\uace0, \uc608\uc0b0\uc744 \ud06c\uac8c \uc904\uc774\uace0, \ube44\uc6a9 \ud6a8\uc728\uc801\uc778 \ub370\uc774\ud130 \ucc98\ub9ac \uc194\ub8e8\uc158\uc744 \uad6c\ud604\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<h4>\uae30\uc874 \ud074\ub7ec\uc2a4\ud130 \ubc0f \uae30\ud0c0 \ud074\ub77c\uc6b0\ub4dc \uc81c\ud488\uacfc\uc758 \ube44\uad50<\/h4>\n<table class=\"w-fit min-w-(--thread-content-width)\" data-start=\"1612\" data-end=\"2001\">\n<thead data-start=\"1612\" data-end=\"1678\">\n<tr data-start=\"1612\" data-end=\"1678\">\n<th style=\"text-align: center;\" data-start=\"1612\" data-end=\"1619\" data-col-size=\"sm\">\ube44\uad50 \ucc28\uc6d0<\/th>\n<th style=\"text-align: center;\" data-start=\"1619\" data-end=\"1629\" data-col-size=\"sm\">AWS EMR<\/th>\n<th style=\"text-align: center;\" data-start=\"1629\" data-end=\"1644\" data-col-size=\"sm\">\ub85c\uceec Hadoop \ud074\ub7ec\uc2a4\ud130<\/th>\n<th style=\"text-align: center;\" data-start=\"1644\" data-end=\"1659\" data-col-size=\"sm\">GCP \ub370\uc774\ud130\ud504\ub85c\ud06c<\/th>\n<th style=\"text-align: center;\" data-start=\"1659\" data-end=\"1678\" data-col-size=\"sm\">Azure HDInsight<\/th>\n<\/tr>\n<\/thead>\n<tbody data-start=\"1756\" data-end=\"2001\">\n<tr data-start=\"1756\" data-end=\"1797\">\n<td style=\"text-align: center;\" data-start=\"1756\" data-end=\"1763\" data-col-size=\"sm\">\ubc30\uce58 \uc2dc\uac04<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1763\" data-end=\"1773\">\uba87 \ubd84 \uc548\uc5d0 \uc790\ub3d9 \ubc30\ud3ec<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1773\" data-end=\"1784\">\uba70\uce60\uc5d0 \uac78\uce5c \uc218\ub3d9 \ubc30\ud3ec<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1784\" data-end=\"1791\">\uc2e0\uc18d\ud55c \ubc30\uce58<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1791\" data-end=\"1797\">\uc911\uac04<\/td>\n<\/tr>\n<tr data-start=\"1798\" data-end=\"1841\">\n<td style=\"text-align: center;\" data-start=\"1798\" data-end=\"1805\" data-col-size=\"sm\">\uc6b4\uc601 \ubc0f \uc720\uc9c0 \ubcf4\uc218 \ubd80\ub2f4<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1805\" data-end=\"1820\">AWS \ud638\uc2a4\ud305, \uae30\ubcf8\uc801\uc73c\ub85c \uc720\uc9c0 \uad00\ub9ac \ubd88\ud544\uc694<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1820\" data-end=\"1830\">\uc6b4\uc601 \ubc0f \uc720\uc9c0\uad00\ub9ac\ud300\uc758 \uc720\uc9c0\uad00\ub9ac\uac00 \ud544\uc694\ud569\ub2c8\ub2e4.<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1830\" data-end=\"1835\">\ud638\uc2a4\ud305<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1835\" data-end=\"1841\">\ud638\uc2a4\ud305<\/td>\n<\/tr>\n<tr data-start=\"1842\" data-end=\"1892\">\n<td style=\"text-align: center;\" data-start=\"1842\" data-end=\"1847\" data-col-size=\"sm\">\ube44\uc6a9<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1847\" data-end=\"1864\">\uc720\uc5f0\ud55c \uacb0\uc81c + \uc989\uc2dc \ud560\uc778<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1864\" data-end=\"1875\">\uc720\ud734 \uc790\uc6d0\uc758 \uc2ec\uac01\ud55c \ub0ad\ube44<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1875\" data-end=\"1884\">\uc608\uc57d \ud560\uc778 \uc9c0\uc6d0<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1884\" data-end=\"1892\">\uc57d\uac04 \ub354 \ub192\uc740 \ube44\uc6a9<\/td>\n<\/tr>\n<tr data-start=\"1893\" data-end=\"1931\">\n<td style=\"text-align: center;\" data-start=\"1893\" data-end=\"1900\" data-col-size=\"sm\">\ud655\uc7a5\uc131<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1900\" data-end=\"1909\">\uc2e4\uc2dc\uac04 \ud0c4\ub825\uc801 \uc2a4\ucf00\uc77c\ub9c1<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1909\" data-end=\"1916\">\ud655\uc7a5\uc758 \uc5b4\ub824\uc6c0<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1916\" data-end=\"1923\">\uc911\uac04 \ud0c4\ub825\uc131<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1923\" data-end=\"1931\">\uc911\uac04 \ud0c4\ub825\uc131<\/td>\n<\/tr>\n<tr data-start=\"1932\" data-end=\"2001\">\n<td style=\"text-align: center;\" data-start=\"1932\" data-end=\"1941\" data-col-size=\"sm\">\ub370\uc774\ud130 \ub808\uc774\ud06c\uc640\uc758 \ud1b5\ud569<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1941\" data-end=\"1968\">S3\/Lake Formation\uacfc\uc758 \uae30\ubcf8 \ud1b5\ud569<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1968\" data-end=\"1977\">\uc678\ubd80 \ud1b5\ud569\uc740 \ubc88\uac70\ub86d\ub2e4<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1977\" data-end=\"1988\">GCS\uc640\uc758 \ud1b5\ud569<\/td>\n<td style=\"text-align: center;\" data-col-size=\"sm\" data-start=\"1988\" data-end=\"2001\">ADLS\uc640\uc758 \ud1b5\ud569<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><strong>\uc694\uc57d:<\/strong> \ub85c\uceec Hadoop \ud074\ub7ec\uc2a4\ud130\ub97c \uc0ac\uc6a9\ud558\uace0 \uc788\uac70\ub098 \ub370\uc774\ud130 \uc99d\uac00, \ube44\uc6a9 \uc0c1\uc2b9, \ubcf5\uc7a1\ud55c \uc6b4\uc601\uacfc \uac19\uc740 \ubb38\uc81c\uc5d0 \uc9c1\uba74\ud574 \uc788\ub2e4\uba74 AWS EMR\ub85c \ub9c8\uc774\uadf8\ub808\uc774\uc158\ud558\ub294 \uac83\uc774 \ud6a8\uc728\uc131\uacfc \uc720\uc5f0\uc131\uc744 \uac1c\uc120\ud558\ub294 \ud604\uba85\ud55c \uc120\ud0dd\uc785\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<h4>\uace0\uac1d \uc0ac\ub840 \uc5f0\uad6c<\/h4>\n<h5>1. \uae08\uc735\uc5c5 : \uc2e0\uc6a9\uc810\uc218 \uacc4\uc0b0<\/h5>\n<p>\ud55c \ub300\ud615 \uae08\uc735 \uae30\uad00\uc740 EMR\uc744 \ud1b5\ud574 \ub9e4\uc77c TB \ub2e8\uc704\uc758 \uc0ac\uc6a9\uc790 \ud589\ub3d9 \ub85c\uadf8\ub97c \ucc98\ub9ac\ud558\uace0, Spark MLlib\uc744 \uc0ac\uc6a9\ud558\uc5ec \uc2e0\uc6a9 \ud3c9\uac00 \ubaa8\ub378\uc744 \ud559\uc2b5\ud558\uba70, \uc2e4\uc2dc\uac04 \uac70\ub798 \uc704\ud5d8\uc744 \ubd84\uc11d\ud569\ub2c8\ub2e4. \uae30\uc874 \ub370\uc774\ud130 \ud50c\ub7ab\ud3fc\uacfc \ube44\uad50\ud588\uc744 \ub54c, \ubaa8\ub378 \ud559\uc2b5 \uc18d\ub3c4\ub294 60%(1000\ubd84\uc758 1\ucd08) \ud5a5\uc0c1\ub418\uace0 \ucef4\ud4e8\ud305 \ube44\uc6a9\uc740 40%(1000\ubd84\uc758 1\ucd08) \uac10\uc18c\ud588\uc2b5\ub2c8\ub2e4.<\/p>\n<h5>2. \uc804\uc790\uc0c1\uac70\ub798 \uc0b0\uc5c5: \uc2e4\uc2dc\uac04 \ucd94\ucc9c \uc5d4\uc9c4<\/h5>\n<p>\uad6d\uacbd \uac04 \uc804\uc790\uc0c1\uac70\ub798 \ud50c\ub7ab\ud3fc\uc740 EMR\uacfc Flink\ub97c \ud65c\uc6a9\ud558\uc5ec \uc0ac\uc6a9\uc790 \ud074\ub9ad \ud589\ub3d9\uc758 \uc2a4\ud2b8\ub9ac\ubc0d \ucc98\ub9ac\ub97c \uad6c\ud604\ud558\uace0, 5\ucd08 \uc774\ub0b4\uc5d0 \uc81c\ud488 \ucd94\ucc9c \uac31\uc2e0\uc744 \uc644\ub8cc\ud558\uba70, \uc0ac\uc6a9\uc790 \uc804\ud658\uc728\uc744 15% \ud5a5\uc0c1\uc2dc\ucf30\uc2b5\ub2c8\ub2e4. EMR\uc740 AWS Lambda \ubc0f DynamoDB\uc640 \uc644\ubcbd\ud558\uac8c \ud1b5\ud569\ub418\uc5b4 \uc11c\ubc84\ub9ac\uc2a4 \ucd94\ucc9c \uc544\ud0a4\ud14d\ucc98\ub97c \uad6c\ucd95\ud569\ub2c8\ub2e4.<\/p>\n<h5>3. \uac8c\uc784 \uc0b0\uc5c5: \ub85c\uadf8 \ubd84\uc11d \ubc0f \uc624\ub958 \uc704\uce58 \ud30c\uc545<\/h5>\n<p>\ud55c \uac8c\uc784 \ud68c\uc0ac\uc5d0\uc11c\ub294 EMR\uc744 \uc0ac\uc6a9\ud558\uc5ec \uac8c\uc784 \uc11c\ubc84 \ub85c\uadf8\ub97c \ubd84\uc11d\ud558\uace0, \ube44\uc815\uc0c1\uc801\uc778 IP \uc8fc\uc18c, \uc790\uc8fc \ucda9\ub3cc\ud558\ub294 \ub178\ub4dc, \uc0ac\uc6a9\uc790 \uc774\ud0c8 \ucd94\uc138\ub97c \ud30c\uc545\ud558\uace0, \uc774\ub97c CloudWatch\uc640 \uacb0\ud569\ud558\uc5ec \ub85c\uadf8 \uc54c\ub9bc\uacfc \uc6b4\uc601 \ucd5c\uc801\ud654\ub97c \uad6c\ud604\ud569\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<h4>AWS \uc5d0\uc774\uc804\ud2b8\ub85c\uc11c<\/h4>\n<p>EMR \ubc30\ud3ec \ubc0f \ucd5c\uc801\ud654\ub294 \ud3b8\ub9ac\ud558\uc9c0\ub9cc, \uc2e4\uc81c\ub85c \uc81c\ub300\ub85c \ud65c\uc6a9\ud558\ub824\uba74 \ud074\ub7ec\uc2a4\ud130 \uc2a4\ucf00\uc904\ub9c1, \uc2a4\ud1a0\ub9ac\uc9c0 \uacc4\uce35\ud654, \ube44\uc6a9 \uad00\ub9ac, \uc791\uc5c5 \uc7a5\uc560 \ud5c8\uc6a9 \ub4f1\uc5d0 \ub300\ud55c \uae4a\uc740 \uc774\ud574\uac00 \ud544\uc694\ud569\ub2c8\ub2e4. AWS \uacf5\uc2dd \uacf5\uc778 \uc5d0\uc774\uc804\ud2b8\ub85c\uc11c, \uc800\ud76c\ub294 \ub2e4\uc74c\uacfc \uac19\uc740 \uc11c\ube44\uc2a4\ub97c \uc81c\uacf5\ud574 \ub4dc\ub9b4 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<h5>1. \uac74\ucd95 \uc124\uacc4 \uc9c0\uc6d0<\/h5>\n<ul>\n<li>\uadc0\ud558\uac00 EMR\uc744 \uc0ac\uc6a9\ud558\uae30\uc5d0 \uc801\ud569\ud55c\uc9c0 \ud3c9\uac00\ud558\ub294 \ub370 \ub3c4\uc6c0\uc744 \ub4dc\ub9bd\ub2c8\ub2e4.<\/li>\n<li>Spark, Hive, Flink \ub4f1\uc758 \ud504\ub808\uc784\uc6cc\ud06c \uc120\ud0dd\uc5d0 \ub300\ud55c \uc81c\uc548\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<li>\ub2e4\uc911 \uc9c0\uc5ed, \ud0c4\ub825\uc801\uc73c\ub85c \uc608\uc57d\ub418\uace0 \ube44\uc6a9\uc774 \ucd5c\uc801\ud654\ub41c \ud074\ub7ec\uc2a4\ud130 \uc544\ud0a4\ud14d\ucc98\ub97c \uc124\uacc4\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h5>2. \ube44\uc6a9 \ucd5c\uc801\ud654 \uc81c\uc548<\/h5>\n<ul>\n<li>\uadf9\uc2ec\ud55c \ube44\uc6a9 \uc808\uac10\uc744 \ub2ec\uc131\ud558\uae30 \uc704\ud574 \uc2a4\ud31f \ud50c\ub79c\uacfc \uc800\ucd95 \ud50c\ub79c\uc744 \uacb0\ud569\ud558\uc138\uc694.<\/li>\n<li>S3 \uacc4\uce35\ud615 \uc2a4\ud1a0\ub9ac\uc9c0\uc640 \uc9c0\ub2a5\ud615 \uc555\ucd95\uc758 \uc0ac\uc6a9\uc744 \uc548\ub0b4\ud569\ub2c8\ub2e4.<\/li>\n<li>\ub9ac\uc18c\uc2a4 \ub0ad\ube44\ub97c \ubc29\uc9c0\ud558\uae30 \uc704\ud574 \uc790\ub3d9 \uc885\ub8cc \uba54\ucee4\ub2c8\uc998\uc744 \uad6c\uc131\ud558\ub294 \ub370 \ub3c4\uc6c0\uc774 \ub429\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h5>3. \uc911\uad6d \uae30\uc220 \uc9c0\uc6d0 + \ud68c\uc0ac \uc1a1\uc7a5 + \uacf5\uc2dd \ubc14\uc6b0\ucc98<\/h5>\n<ul>\n<li>\uad6c\uc131\uc744 \ub3d5\uae30 \uc704\ud55c \ubb34\ub8cc \uae30\uc220 \ucee8\uc124\ud134\ud2b8;<\/li>\n<li>AWS \uc5d4\ud130\ud504\ub77c\uc774\uc988 \uc218\uc900\uc758 \uc6d4\ubcc4 \uccad\uad6c\uc11c\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<li>AWS \ubb34\ub8cc \uccb4\ud5d8\ud310 \ud560\ub2f9\ub7c9\uacfc \uc774\ubca4\ud2b8 \ubc14\uc6b0\ucc98\ub97c \uc2e0\uccad\ud560 \uc218 \uc788\ub294 \ucc44\ub110\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h4>\uacb0\ub860<\/h4>\n<p>Amazon EMR\uc740 \ub2e8\uc21c\ud55c \ub3c4\uad6c\uac00 \uc544\ub2c8\ub77c &quot;\ud074\ub77c\uc6b0\ub4dc \ub124\uc774\ud2f0\ube0c \ube45\ub370\uc774\ud130 \uc0ac\uace0\ubc29\uc2dd&quot;\uc744 \uc804\ub2ec\ud558\ub294 \ub9e4\uac1c\uccb4\uc785\ub2c8\ub2e4. \uae30\uc5c5\uc740 \ub9ac\uc18c\uc2a4 \ubc0f \uc6b4\uc601 \uc81c\uc57d\uc5d0\uc11c \ubc97\uc5b4\ub098 \ub370\uc774\ud130 \uac00\uce58 \ub9c8\uc774\ub2dd\uc5d0 \uc9d1\uc911\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p>\uadc0\ud558\uc758 \uc0ac\uc5c5\uccb4\uc5d0\uc11c \ub2e4\uc74c\uacfc \uac19\uc740 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud558\ub294 \uacbd\uc6b0:<\/p>\n<ul>\n<li>\ub85c\uadf8, \uac70\ub798 \ubc0f \ubd84\uc11d \ub370\uc774\ud130\uc758 \uc591\uc774 \ub04a\uc784\uc5c6\uc774 \uc99d\uac00\ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li>\ub300\ub7c9\uc758 \ub370\uc774\ud130 \uc815\ub9ac \ubc0f \ubcf4\uace0\uc11c \uc0dd\uc131 \uc791\uc5c5<\/li>\n<li>\ub192\uc740 \ub370\uc774\ud130 \ucc98\ub9ac \uc18c\ud504\ud2b8\uc6e8\uc5b4 \ubc0f \ud558\ub4dc\uc6e8\uc5b4 \uc6b4\uc601 \ubc0f \uc720\uc9c0 \uad00\ub9ac \ube44\uc6a9<\/li>\n<\/ul>\n<p>\uadf8\ub807\ub2e4\uba74 \ub370\uc774\ud130 \ucc98\ub9ac \ud50c\ub7ab\ud3fc\uc744 \ud074\ub77c\uc6b0\ub4dc\ub85c \uc774\uc804\ud558\ub294 \uac83\uc744 \uace0\ub824\ud574 \ubcfc \ub54c\uc785\ub2c8\ub2e4. Amazon EMR\uc740 &quot;\ub370\uc774\ud130 \uc18c\ube44&quot;\uc5d0\uc11c &quot;\ub370\uc774\ud130 \uc911\uc2ec&quot;\uc73c\ub85c \uc804\ud658\ud558\ub294 \ub370 \uc911\uc694\ud55c \ub2e8\uacc4\uc785\ub2c8\ub2e4.<\/p>\n<p>\ud398\uc774\uc9c0 \ud558\ub2e8\uc758 QR \ucf54\ub4dc\ub97c \uc2a4\uce94\ud558\uc5ec \ubb34\ub8cc \uc544\ud0a4\ud14d\ucc98 \uc870\uc5b8, \uae30\uc220 \uc194\ub8e8\uc158 \ud3c9\uac00 \ubc0f AWS \uc5d4\ud130\ud504\ub77c\uc774\uc988 \uacc4\uc815 \uac1c\uc124 \uc9c0\uc6d0\uc744 \ubb38\uc758\ud558\uc138\uc694.<\/p>","protected":false},"excerpt":{"rendered":"<p>\uc624\ub298\ub0a0 \ub370\uc774\ud130 \uc911\uc2ec\uc740 \uae30\uc5c5\uc758 \ud575\uc2ec \uacbd\uc7c1\ub825\uc73c\ub85c \uc790\ub9ac \uc7a1\uc558\uc2b5\ub2c8\ub2e4. PB \ub2e8\uc704\uc758 \ub370\uc774\ud130\ub97c \ud6a8\uc728\uc801\uc774\uace0 \uc800\ub834\ud558\uac8c \ucc98\ub9ac\ud558\ub294 \ubc29\ubc95\uc740 \ub9ce\uc740 \uae30\uc5c5\uc758 IT \uc804\ub7b5\uc5d0\uc11c \uc911\uc694\ud55c \ubd80\ubd84\uc744 \ucc28\uc9c0\ud558\uac8c \ub418\uc5c8\uc2b5\ub2c8\ub2e4. AWS \uae30\ubc18 \uad00\ub9ac\ud615 \ube45\ub370\uc774\ud130 \ucc98\ub9ac \ud50c\ub7ab\ud3fc\uc778 Amazon Elastic MapReduce(EMR)\ub294 \uc720\uc5f0\uc131, \ud655\uc7a5\uc131, \uadf8\ub9ac\uace0 \ub192\uc740 \ube44\uc6a9 \ud6a8\uc728\uc131\uc73c\ub85c \uae08\uc735, \uc804\uc790\uc0c1\uac70\ub798, \uac8c\uc784, \uad11\uace0, \uacfc\ud559 \uc5f0\uad6c \ub4f1 \ub2e4\uc591\ud55c \ubd84\uc57c\uc758 \uace0\uac1d\uc5d0\uac8c \ucd5c\uace0\uc758 \uc120\ud0dd\uc774 \ub418\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>","protected":false},"author":1,"featured_media":3339,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[65],"tags":[],"class_list":["post-3338","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technical-sharing"],"_links":{"self":[{"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/posts\/3338","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/comments?post=3338"}],"version-history":[{"count":0,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/posts\/3338\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/media\/3339"}],"wp:attachment":[{"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/media?parent=3338"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/categories?post=3338"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aws-oncloudai.com\/ko\/wp-json\/wp\/v2\/tags?post=3338"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}