{"cells":[{"cell_type":"markdown","source":["# 모든 암호화폐의 일(Day) 캔들 가져오기\n","\n","- 우리는 4, 5주차에서 암호화폐 데이터 분석 및 가격 예측 모델을 만들어 볼 것입니다 \n","- 이를 위해 각 암호화폐의 데이터가 필요하니 가져와보도록 합시다 \n","- [힌트]를 잘 참고하여 구현해주세요 "],"metadata":{"id":"EQ8s060DUlzO"}},{"cell_type":"markdown","source":["# 필요한 라이브러리 import 하기"],"metadata":{"id":"QukHLekmVO0K"}},{"cell_type":"code","execution_count":37,"metadata":{"id":"RUoX5t2ZUYTL","executionInfo":{"status":"ok","timestamp":1705828117231,"user_tz":-540,"elapsed":249,"user":{"displayName":"chioh song","userId":"05482793768581925652"}}},"outputs":[],"source":["import time\n","import requests\n","import pandas as pd\n","import json\n","from tqdm import tqdm"]},{"cell_type":"markdown","source":["# API로 마켓 코드 조회하여 `market_code_df` 에 저장하기"],"metadata":{"id":"voq4YeihcKCb"}},{"cell_type":"code","source":["url = \"https://api.upbit.com/v1/market/all?isDetails=true\"\n","\n","headers = {\"accept\": \"application/json\"}\n","\n","response = requests.get(url, headers=headers)\n","\n","market_code_df = pd.DataFrame(json.loads(response.text))\n","market_code_df"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":424},"id":"KWBnUR6NcNFi","executionInfo":{"status":"ok","timestamp":1705827909240,"user_tz":-540,"elapsed":851,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"6aed9479-cff5-4c3c-f4b3-ce1d40a600b2"},"execution_count":25,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" market_warning market korean_name english_name\n","0 NONE KRW-BTC 비트코인 Bitcoin\n","1 NONE KRW-ETH 이더리움 Ethereum\n","2 NONE BTC-ETH 이더리움 Ethereum\n","3 NONE BTC-XRP 리플 Ripple\n","4 NONE BTC-ETC 이더리움클래식 Ethereum Classic\n",".. ... ... ... ...\n","299 NONE BTC-ID 스페이스아이디 SPACE ID\n","300 NONE BTC-AXL 엑셀라 Axelar\n","301 NONE KRW-CTC 크레딧코인 Creditcoin\n","302 NONE KRW-ASTR 아스타 Astar\n","303 NONE BTC-SPURS 토트넘홋스퍼 Tottenham Hotspur\n","\n","[304 rows x 4 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
market_warningmarketkorean_nameenglish_name
0NONEKRW-BTC비트코인Bitcoin
1NONEKRW-ETH이더리움Ethereum
2NONEBTC-ETH이더리움Ethereum
3NONEBTC-XRP리플Ripple
4NONEBTC-ETC이더리움클래식Ethereum Classic
...............
299NONEBTC-ID스페이스아이디SPACE ID
300NONEBTC-AXL엑셀라Axelar
301NONEKRW-CTC크레딧코인Creditcoin
302NONEKRW-ASTR아스타Astar
303NONEBTC-SPURS토트넘홋스퍼Tottenham Hotspur
\n","

304 rows × 4 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"]},"metadata":{},"execution_count":25}]},{"cell_type":"markdown","source":["# 원화로 거래 가능한 마켓 코드만 `market_code_df` 에남겨놓기"],"metadata":{"id":"TAYa3hiPcYIC"}},{"cell_type":"code","source":["market_code_df = market_code_df[market_code_df[\"market\"].str.contains(\"KRW\")]\n","market_code_df"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":424},"id":"pRFmjdMHcVrv","executionInfo":{"status":"ok","timestamp":1705827909240,"user_tz":-540,"elapsed":5,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"7b98a6e6-c894-4b3b-922b-44a19ff62bad"},"execution_count":26,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" market_warning market korean_name english_name\n","0 NONE KRW-BTC 비트코인 Bitcoin\n","1 NONE KRW-ETH 이더리움 Ethereum\n","28 NONE KRW-NEO 네오 NEO\n","29 NONE KRW-MTL 메탈 Metal\n","30 NONE KRW-XRP 리플 Ripple\n",".. ... ... ... ...\n","292 NONE KRW-IMX 이뮤터블엑스 Immutable X\n","295 NONE KRW-SEI 세이 Sei\n","298 NONE KRW-MINA 미나 Mina\n","301 NONE KRW-CTC 크레딧코인 Creditcoin\n","302 NONE KRW-ASTR 아스타 Astar\n","\n","[118 rows x 4 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
market_warningmarketkorean_nameenglish_name
0NONEKRW-BTC비트코인Bitcoin
1NONEKRW-ETH이더리움Ethereum
28NONEKRW-NEO네오NEO
29NONEKRW-MTL메탈Metal
30NONEKRW-XRP리플Ripple
...............
292NONEKRW-IMX이뮤터블엑스Immutable X
295NONEKRW-SEI세이Sei
298NONEKRW-MINA미나Mina
301NONEKRW-CTC크레딧코인Creditcoin
302NONEKRW-ASTR아스타Astar
\n","

118 rows × 4 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"]},"metadata":{},"execution_count":26}]},{"cell_type":"markdown","source":["# to에 넣을 과거 시각들을 담은 list 형태의 datetimes 변수 선언하기"],"metadata":{"id":"b5q9LxVlVRu0"}},{"cell_type":"code","execution_count":27,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"kMOwtH-BUYTM","executionInfo":{"status":"ok","timestamp":1705827909240,"user_tz":-540,"elapsed":4,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"8028aab2-3d0f-467b-fa7d-08ad940ab194"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['2018-01-01T00%3A00%3A00%2B09%3A00',\n"," '2018-07-01T00%3A00%3A00%2B09%3A00',\n"," '2019-01-01T00%3A00%3A00%2B09%3A00',\n"," '2019-07-01T00%3A00%3A00%2B09%3A00',\n"," '2020-01-01T00%3A00%3A00%2B09%3A00',\n"," '2020-07-01T00%3A00%3A00%2B09%3A00',\n"," '2021-01-01T00%3A00%3A00%2B09%3A00',\n"," '2021-07-01T00%3A00%3A00%2B09%3A00',\n"," '2022-01-01T00%3A00%3A00%2B09%3A00',\n"," '2022-07-01T00%3A00%3A00%2B09%3A00',\n"," '2023-01-01T00%3A00%3A00%2B09%3A00',\n"," '2023-07-01T00%3A00%3A00%2B09%3A00',\n"," '2024-01-01T00%3A00%3A00%2B09%3A00',\n"," '2024-07-01T00%3A00%3A00%2B09%3A00']"]},"metadata":{},"execution_count":27}],"source":["datetimes = []\n","for year in range(2018,2025):\n"," datetimes.append(f\"{year}-01-01T00%3A00%3A00%2B09%3A00\")\n"," datetimes.append(f\"{year}-07-01T00%3A00%3A00%2B09%3A00\")\n","datetimes\n"]},{"cell_type":"markdown","source":["# 반년 단위로 캔들 가져와서 list 형태의 `dfs` 변수에 계속 추가해주기"],"metadata":{"id":"DizudNH35PSf"}},{"cell_type":"code","execution_count":42,"metadata":{"id":"Xqf0WTO_UYTN","executionInfo":{"status":"ok","timestamp":1705830451034,"user_tz":-540,"elapsed":12227,"user":{"displayName":"chioh song","userId":"05482793768581925652"}}},"outputs":[],"source":["headers = {\"accept\": \"application/json\"}\n","dfs = []\n","for datetime in datetimes:\n"," url = f\"https://api.upbit.com/v1/candles/days?market=KRW-BTC&to={datetime}&count=200\"\n"," response = requests.get(url, headers=headers)\n"," dfs.append(pd.DataFrame(json.loads(response.text)))\n"," time.sleep(0.04)"]},{"cell_type":"code","execution_count":43,"metadata":{"id":"8JMS2JcxUYTN","executionInfo":{"status":"ok","timestamp":1705830451034,"user_tz":-540,"elapsed":1,"user":{"displayName":"chioh song","userId":"05482793768581925652"}}},"outputs":[],"source":["df = pd.concat(dfs)"]},{"cell_type":"code","execution_count":30,"metadata":{"id":"tyqgvl2TUYTN","executionInfo":{"status":"ok","timestamp":1705827922202,"user_tz":-540,"elapsed":7,"user":{"displayName":"chioh song","userId":"05482793768581925652"}}},"outputs":[],"source":["df = df.drop(columns=[\"candle_date_time_utc\", \"timestamp\", \"prev_closing_price\", \"change_price\"])\n","df.columns = [\n"," \"market\",\n"," \"datetime\",\n"," \"open\",\n"," \"high\",\n"," \"low\",\n"," \"close\",\n"," \"trade_price\",\n"," \"trade_volume\",\n"," \"change_rate\"\n","]"]},{"cell_type":"code","execution_count":31,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":424},"id":"nbY7ZEPwUYTO","executionInfo":{"status":"ok","timestamp":1705827922202,"user_tz":-540,"elapsed":7,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"c67594f1-6e9a-4cfc-f1c0-8fcdb2747df6"},"outputs":[{"output_type":"execute_result","data":{"text/plain":[" market datetime open high low \\\n","0 KRW-BTC 2017-09-25T09:00:00 4201000.0 4333000.0 4175000.0 \n","1 KRW-BTC 2017-09-26T09:00:00 4317000.0 4418000.0 4311000.0 \n","2 KRW-BTC 2017-09-27T09:00:00 4322000.0 4677000.0 4318000.0 \n","3 KRW-BTC 2017-09-28T09:00:00 4657000.0 4772000.0 4519000.0 \n","4 KRW-BTC 2017-09-29T09:00:00 4586000.0 4709000.0 4476000.0 \n","... ... ... ... ... ... \n","2693 KRW-BTC 2024-01-17T09:00:00 59218000.0 59300000.0 58269000.0 \n","2694 KRW-BTC 2024-01-18T09:00:00 58776000.0 58900000.0 56754000.0 \n","2695 KRW-BTC 2024-01-19T09:00:00 57281000.0 58180000.0 55935000.0 \n","2696 KRW-BTC 2024-01-20T09:00:00 57738000.0 57796000.0 57242000.0 \n","2697 KRW-BTC 2024-01-21T09:00:00 57550000.0 57655000.0 57350000.0 \n","\n"," close trade_price trade_volume change_rate \n","0 4322000.0 5.602146e+08 132.484755 0.028803 \n","1 4321000.0 9.950724e+07 22.788340 -0.000231 \n","2 4657000.0 1.448276e+08 32.269662 0.077760 \n","3 4586000.0 3.721860e+08 80.588243 -0.015246 \n","4 4657000.0 2.724558e+08 59.352373 0.015482 \n","... ... ... ... ... \n","2693 58776000.0 1.666847e+11 2837.334490 -0.007095 \n","2694 57284000.0 2.472746e+11 4268.883840 -0.025385 \n","2695 57739000.0 3.199353e+11 5634.881771 0.007943 \n","2696 57517000.0 1.224661e+11 2127.464542 -0.003845 \n","2697 57525000.0 4.179776e+10 726.897959 0.000139 \n","\n","[2310 rows x 9 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
marketdatetimeopenhighlowclosetrade_pricetrade_volumechange_rate
0KRW-BTC2017-09-25T09:00:004201000.04333000.04175000.04322000.05.602146e+08132.4847550.028803
1KRW-BTC2017-09-26T09:00:004317000.04418000.04311000.04321000.09.950724e+0722.788340-0.000231
2KRW-BTC2017-09-27T09:00:004322000.04677000.04318000.04657000.01.448276e+0832.2696620.077760
3KRW-BTC2017-09-28T09:00:004657000.04772000.04519000.04586000.03.721860e+0880.588243-0.015246
4KRW-BTC2017-09-29T09:00:004586000.04709000.04476000.04657000.02.724558e+0859.3523730.015482
..............................
2693KRW-BTC2024-01-17T09:00:0059218000.059300000.058269000.058776000.01.666847e+112837.334490-0.007095
2694KRW-BTC2024-01-18T09:00:0058776000.058900000.056754000.057284000.02.472746e+114268.883840-0.025385
2695KRW-BTC2024-01-19T09:00:0057281000.058180000.055935000.057739000.03.199353e+115634.8817710.007943
2696KRW-BTC2024-01-20T09:00:0057738000.057796000.057242000.057517000.01.224661e+112127.464542-0.003845
2697KRW-BTC2024-01-21T09:00:0057550000.057655000.057350000.057525000.04.179776e+10726.8979590.000139
\n","

2310 rows × 9 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"]},"metadata":{},"execution_count":31}],"source":["df = df.sort_values(by=\"datetime\").reset_index(drop=True)\n","df = df.drop_duplicates()\n","df"]},{"cell_type":"markdown","source":["# 원화로 거래 가능한 암호화폐들을 각각의 csv로 저장해보아요"],"metadata":{"id":"0OPtIwl4u5b_"}},{"cell_type":"code","source":["from google.colab import drive\n","drive.mount('/content/drive')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"pQSCO9A5Raji","executionInfo":{"status":"ok","timestamp":1705828020320,"user_tz":-540,"elapsed":14983,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"d6f03317-9a6b-4ea8-fbe5-01a18a3b4556"},"execution_count":35,"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}]},{"cell_type":"code","execution_count":46,"metadata":{"id":"o_jTfeanUYTO","executionInfo":{"status":"ok","timestamp":1705830517649,"user_tz":-540,"elapsed":309,"user":{"displayName":"chioh song","userId":"05482793768581925652"}}},"outputs":[],"source":["def get_coin(market_code, datetimes):\n"," headers = {\"accept\": \"application/json\"}\n"," dfs = []\n"," for datetime in datetimes:\n"," url = f\"https://api.upbit.com/v1/candles/days?market={market_code}&to={datetime}&count=190\"\n"," response = requests.get(url, headers=headers)\n"," dfs.append(pd.DataFrame(json.loads(response.text)))\n"," time.sleep(0.034)\n","\n"," df = pd.concat(dfs)\n","\n"," df = df.drop(columns=[\"candle_date_time_utc\", \"timestamp\", \"prev_closing_price\", \"change_price\"])\n"," df.columns = [\n"," \"market\",\n"," \"datetime\",\n"," \"open\",\n"," \"high\",\n"," \"low\",\n"," \"close\",\n"," \"trade_price\",\n"," \"trade_volume\",\n"," \"change_rate\"\n"," ]\n","\n"," df = df.sort_values(by=\"datetime\").reset_index(drop=True)\n"," df = df.drop_duplicates()\n"," return df\n"]},{"cell_type":"code","source":["base_path = \"/content/drive/MyDrive/coin_data/days\"\n","\n","for market_code in tqdm(market_code_df[\"market\"]):\n"," df = get_coin(market_code, datetimes)\n"," if len(df) >= 730:\n"," df.to_csv(base_path + f\"/{market_code}.csv\", index=False)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lsnkpJw_bBjQ","executionInfo":{"status":"ok","timestamp":1705831870462,"user_tz":-540,"elapsed":1346150,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"02489e64-c360-4c80-ef0d-e6bf199f3f0d"},"execution_count":47,"outputs":[{"output_type":"stream","name":"stderr","text":["100%|██████████| 118/118 [22:25<00:00, 11.40s/it]\n"]}]},{"cell_type":"markdown","source":["# days에 몇개의 암호화폐가 저장됐는지 살펴보기"],"metadata":{"id":"nblTXCGDZjCh"}},{"cell_type":"code","source":["import os\n","len(os.listdir(base_path))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"5cRAhePjRCpC","executionInfo":{"status":"ok","timestamp":1705830126540,"user_tz":-540,"elapsed":273,"user":{"displayName":"chioh song","userId":"05482793768581925652"}},"outputId":"8c411292-755e-442b-fe73-28485faee66d"},"execution_count":41,"outputs":[{"output_type":"execute_result","data":{"text/plain":["101"]},"metadata":{},"execution_count":41}]},{"cell_type":"markdown","source":["- KRW로 구매 가능한 암호화폐는 총 118개였으나 저장된 파일은 101개인 것을 확인할 수 있습니다"],"metadata":{"id":"PC0fvyvBZm97"}}],"metadata":{"kernelspec":{"display_name":"chart_studio","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.2"},"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":0}