{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[{"file_id":"1gBOX9iMPVpqEGbkT3RSeH4jTQBzUykdF","timestamp":1729997779610},{"file_id":"1mdhOdWbnqwfk5RalwNqEj6IBnV7_LUia","timestamp":1729881752200}]},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["# CUHK STAT5106 Mid-Term Take Home Exam\n","## 25 Oct 2024 (Fri), 2200 - 27 Oct 2024 (Sun), 2159\n","48 hours are given, but 9 hours are expected to be finished.
\n","This is open-book; open-notes; and open-web exam. Welcome to utilize any resources.
\n","Please follow the instructions step-by-step:\n","- Please access into Blackboard \\> STAT5106 \\> Course Content \\> Mid-Term Take Home Exam\n","- Access the link below, copy the ipynb to anywhere such as local computer, or personal cloud drive, so this becomes savable.
\n","For the best experience - using **colab and your google drive account**.\n","- Code and complete all questions in this ipynb.\n","- Please keep on saving for protecting what you have input.
\n","Finally you need to upload this to Blackboard account.\n","- Upload this in Mid-Term Take Home Exam page, liked how you have submitted assignments.\n","\n","For any queries, please submit your questions on BLACKBOARD called **MID-TERM discussion room**. During the mid-term exam period - in the 48 hours. For fair the publicity to all students.\n","\n","Alan will answer most questions around the following time point:\n","- 1000, 26 Oct 2024\n","- 2200, 26 Oct 2024\n","- 1000, 27 Oct 2024\n","\n","Some of them I would not answer if your question is not suitable - such as asking the full answers.\n","\n","DON'T email to tutors or me - we won't answer in the period.\n","\n","Thanks and good luck,
\n","Alan"],"metadata":{"id":"hm20a-J7nLQL"}},{"cell_type":"markdown","source":["# Warm up Question (20%)\n","\n","What is the name of your lecturer ? (Adding prefix if necessary)"],"metadata":{"id":"c5OJIueQuyPh"}},{"cell_type":"code","source":["Your_Answer = \"ALAN HO\" #@param {type:\"string\"}"],"metadata":{"cellView":"form","id":"GRfIGwWYvTAh"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Finding the nearest available parking space\n","\n","
\n","
\n","\n","You need to park your car. It can be difficult to find an available parking space during busy times. Let's look at the data behind the parking space availability app.\n"],"metadata":{"id":"kAKvAWCOotH4"}},{"cell_type":"markdown","source":["# Task 1. Get Carpark location from data.gov.hk API (15%)\n","\n","The API page is here. ([In Traditional Chinese](https://data.gov.hk/tc-data/dataset/hk-dpo-datagovhk1-carpark-info-vacancy/resource/f4c792c6-071c-4a64-888b-afeea33d5ad7), [In English](https://data.gov.hk/en-data/dataset/hk-dpo-datagovhk1-carpark-info-vacancy/resource/01752c62-a6b6-4ddc-bf2d-25efccadc143))\n","\n","and the data dictionary is [here](https://resource.data.one.gov.hk/opendata/carpark/Parking_Vacancy_Data_Specification.pdf).\n","\n","**Write a script for calling API** to get all 471 (up to 2024-10-18) carparks.\n","- In calling API, set the \"data\" parameter to be \"vacancy\"\n","- set the \"vehicleTypes\" parameter to be \"privateCar\". We focus on private carpark spaces only.\n","\n","---\n","\n","Expected Results for a single carpark:\n","\n","```\n","# js_row = js_result[0]\n","\n","{'park_Id': '10',\n"," 'name': 'Kai Tak Cruise Terminal Car Park 1',\n"," 'nature': 'commercial',\n"," 'carpark_Type': 'multi-storey',\n"," 'address': {'buildingName': 'Kai Tak Cruise Terminal',\n"," 'streetName': 'Shing Fung Road',\n"," 'buildingNo': '33',\n"," 'floor': '1',\n"," 'subDistrict': 'Kowloon Bay',\n"," 'dcDistrict': 'Kwun Tong District',\n"," 'region': 'KLN'},\n"," 'displayAddress': '1st floor, Kai Tak Cruise Terminal, 33 Shing Fung Road, Kowloon Bay, KLN',\n"," 'district': 'Kwun Tong District',\n"," 'latitude': 22.3062049,\n"," 'longitude': 114.21309471,\n"," 'contactNo': '+852 3465 6888, 09:30-18:00 Mon-Fri, except public holiday',\n","\n","# ... Skipping in the middle ... #\n","\n"," 'LGV': {'spaceUNL': 0, 'spaceEV': 0, 'spaceDIS': 0, 'space': 0},\n"," 'HGV': {'spaceUNL': 0, 'spaceEV': 0, 'spaceDIS': 0, 'space': 0},\n"," 'coach': {'spaceUNL': 0, 'spaceEV': 0, 'spaceDIS': 0, 'space': 0},\n"," 'motorCycle': {'spaceUNL': 0, 'spaceEV': 0, 'spaceDIS': 0, 'space': 0},\n"," 'creationDate': '2016-08-16 10:03:56',\n"," 'modifiedDate': '2024-01-03 13:28:25',\n"," 'publishedDate': '2018-12-12 12:22:47',\n"," 'lang': 'en_US'}\n","```\n","\n"],"metadata":{"id":"cSea-HnRSxKV"}},{"cell_type":"code","source":["# https://api.data.gov.hk/v1/carpark-info-vacancy?data=info&vehicleTypes=privateCar\n","import requests\n","from pprint import pprint\n","import json\n","\n","url = \"https://api.data.gov.hk/v1/carpark-info-vacancy\"\n","params = {\n"," \"data\": \"info\",\n"," \"vehicleTypes\": \"privateCar\"\n","}\n","\n","response = requests.get(url, params=params)\n","js_result_info = response.json()['results']\n","\n","url = \"https://api.data.gov.hk/v1/carpark-info-vacancy\"\n","\n","response = requests.get(url)\n","js_result_all = response.json()['results']\n","\n","output = []\n","for jrv in js_result_info:\n"," for jra in js_result_all:\n"," if jrv['park_Id'] == jra['park_Id']:\n"," output.append(jra)\n"," break\n","\n","js_row = output[0]\n","\n","pprint(js_row)\n","pprint(\"length of json: \"+str(len(output)))\n","\n","# Store js_result into js_result.json\n","with open('js_result_o.json', 'w') as outfile:\n"," json.dump(output, outfile, indent=2)\n","\n","task1_json = output\n"],"metadata":{"id":"eznQytp966Cz","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1729997896191,"user_tz":-480,"elapsed":3455,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"1e3b0014-b277-459f-b89d-040b6ddd92c7"},"execution_count":2,"outputs":[{"output_type":"stream","name":"stdout","text":["{'HGV': {'space': 0, 'spaceDIS': 0, 'spaceEV': 0, 'spaceUNL': 0},\n"," 'LGV': {'space': 0, 'spaceDIS': 0, 'spaceEV': 0, 'spaceUNL': 0},\n"," 'address': {'buildingName': 'Kai Tak Cruise Terminal',\n"," 'buildingNo': '33',\n"," 'dcDistrict': 'Kwun Tong District',\n"," 'floor': '1',\n"," 'region': 'KLN',\n"," 'streetName': 'Shing Fung Road',\n"," 'subDistrict': 'Kowloon Bay'},\n"," 'carpark_Type': 'multi-storey',\n"," 'coach': {'space': 0, 'spaceDIS': 0, 'spaceEV': 0, 'spaceUNL': 0},\n"," 'contactNo': '+852 3465 6888, 09:30-18:00 Mon-Fri, except public holiday',\n"," 'creationDate': '2016-08-16 10:03:56',\n"," 'displayAddress': '1st floor, Kai Tak Cruise Terminal, 33 Shing Fung Road, '\n"," 'Kowloon Bay, KLN',\n"," 'district': 'Kwun Tong District',\n"," 'facilities': ['disabilities', 'evCharger'],\n"," 'gracePeriods': [{'minutes': 10}],\n"," 'heightLimits': [{'height': 2}],\n"," 'lang': 'en_US',\n"," 'latitude': 22.3062049,\n"," 'longitude': 114.21309471,\n"," 'modifiedDate': '2024-01-03 13:28:25',\n"," 'motorCycle': {'space': 0, 'spaceDIS': 0, 'spaceEV': 0, 'spaceUNL': 0},\n"," 'name': 'Kai Tak Cruise Terminal Car Park 1',\n"," 'nature': 'commercial',\n"," 'openingHours': [{'excludePublicHoliday': False,\n"," 'periodEnd': '23:00',\n"," 'periodStart': '07:00',\n"," 'weekdays': ['MON',\n"," 'TUE',\n"," 'WED',\n"," 'THU',\n"," 'FRI',\n"," 'SAT',\n"," 'SUN',\n"," 'PH']}],\n"," 'opening_status': 'OPEN',\n"," 'park_Id': '10',\n"," 'paymentMethods': ['octopus', 'visa'],\n"," 'privateCar': {'hourlyCharges': [{'covered': 'covered',\n"," 'excludePublicHoliday': True,\n"," 'periodEnd': '23:00',\n"," 'periodStart': '07:00',\n"," 'price': 15,\n"," 'remark': '',\n"," 'type': 'hourly',\n"," 'usageMinimum': 1,\n"," 'weekdays': ['MON',\n"," 'TUE',\n"," 'WED',\n"," 'THU',\n"," 'FRI']},\n"," {'covered': 'covered',\n"," 'excludePublicHoliday': False,\n"," 'periodEnd': '23:00',\n"," 'periodStart': '07:00',\n"," 'price': 20,\n"," 'remark': '',\n"," 'type': 'hourly',\n"," 'usageMinimum': 1,\n"," 'weekdays': ['SAT', 'SUN', 'PH']}],\n"," 'space': 112,\n"," 'spaceDIS': 0,\n"," 'spaceEV': 0,\n"," 'spaceUNL': 0},\n"," 'publishedDate': '2018-12-12 12:22:47',\n"," 'renditionUrls': {'banner': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/banner.png',\n"," 'square': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/square.png',\n"," 'thumbnail': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/thumbnail.png'},\n"," 'website': 'http://www.kaitakcruiseterminal.com.hk/'}\n","'length of json: 472'\n"]}]},{"cell_type":"code","source":[],"metadata":{"id":"u3gqYUGH65OM"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Task 2. Load the json extracts into pandas data frame (40%)\n","\n","With the following instructions:\n","- The following columns are needed:\n"," - park_Id\n"," - name\n"," - displayAddress\n"," - district\n"," - latitude\n"," - longitude\n"," - opening_status\n"," - facilities\n"," - paymentMethods\n"," - modifiedDate\n","- 2A: Write a function for getting URL of carpark image.\n"," - and insert new column called \"url_image\"\n","- 2B: Write a python script for scrapping the dates of public holidays.\n","- 2C: Write a function for getting carpark today information, such as:\n"," - Carpark opening time of today: \"periodStart\"\n"," - Carpark closing time of today: \"periodEnd\"\n"," - Carpark Hourly Charge: \"price\"\n"," - Carpark Total Parking Space: \"space\" \n","- 2D: Combining all the stuffs, writing python script for outputting a pandas DataFrame.\n","\n","---\n","\n","Expected Output:\n","\n","![Example Image](https://drive.google.com/uc?id=1Jbjqx7ghZTYyyigml_tqJK2JxDTD5Etp)\n","\n","Note the dataset is with dimensions of 471 rows and 17 columns."],"metadata":{"id":"PHa-82SOXNsv"}},{"cell_type":"markdown","source":["## 2A. Write a function: Get URL of image of each carpark (10%)\n","We search anyone URL in the dictionaries 'renditionUrls', with the format:\n","\n","```\n","'renditionUrls': {'square': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/square.png',\n","'thumbnail': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/thumbnail.png',\n","'banner': 'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/banner.png'}\n","```\n","\n","**Write a function \"get_url_image\"** with\n","- input as \"js_row\", the json item for a single carpark\n","- return output as the first URL item in the dictionary\n","\n","Besides, in the function you need\n","- If dictionary 'renditionUrls' cannot be found, return None.\n","- If 'renditionUrls' is found but not the URL, return None as well.\n","\n","---\n","\n","Expected Output:\n","\n","\n","```\n","# get_url_image(js_row)\n","'https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/square.png'\n","```\n","\n"],"metadata":{"id":"elhOaT-NdLBS"}},{"cell_type":"code","source":["def get_url_image(js_row):\n"," return None if js_row.get('renditionUrls', {}).get(\"square\") == None else js_row['renditionUrls']['square']"],"metadata":{"id":"9q8VyUgFdXPu"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["print(get_url_image(js_row))"],"metadata":{"id":"iu7Isd3H7LeD","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1729996911229,"user_tz":-480,"elapsed":4,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"2c69bf4e-00d1-454f-8fab-15e627e958c1"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A693265207413252869411532657339312395903827562313.JPG/square.png\n"]}]},{"cell_type":"markdown","source":["## 2B. Write a script: Get Public Holidays in Hong Kong (10%)\n","\n","**Write a script for webscrapping with package BeautifulSoup** through the website: [General holidays for 2024](https://www.gov.hk/en/about/abouthk/holiday/2024.htm)\n","\n","save those dates in format 'YYYY-mm-dd' packed with a list in the variable 'ph_dates', as following:\n","\n","```\n","# ph_dates\n","\n","['2024-01-01',\n"," '2024-02-10',\n"," '2024-02-12',\n"," '2024-02-13',\n"," '2024-03-29',\n"," '2024-03-30',\n"," '2024-04-01',\n"," '2024-04-04',\n"," '2024-05-01',\n"," '2024-05-15',\n"," '2024-06-10',\n"," '2024-07-01',\n"," '2024-09-18',\n"," '2024-10-01',\n"," '2024-10-11',\n"," '2024-12-25',\n"," '2024-12-26']\n","```\n","\n"],"metadata":{"id":"FhQml8q_oqsq"}},{"cell_type":"code","source":["from pprint import pprint\n","import re\n","import requests\n","from bs4 import BeautifulSoup"],"metadata":{"id":"IH1C4Tse7Osh"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["\n","\n","txt_month =[\"January\", \"February\", \"March\", \"April\", \"May\", \"June\", \"July\", \"August\", \"September\", \"October\", \"November\", \"December\"]\n","\n","def formatNumber(number):\n"," return number.zfill(2)\n","\n","def txtMonthToIntMonth(txtMonth):\n"," return formatNumber(str((txt_month.index(txtMonth)+1)))\n","\n","def scrape_holiday():\n"," output = []\n"," url = 'https://www.gov.hk/en/about/abouthk/holiday/2024.htm'\n"," res = requests.get(url)\n"," soup = BeautifulSoup(res.text, 'html.parser')\n"," h1s = soup.select('tr>td.date')\n"," for h1 in h1s[1:]:\n"," temp = re.split(r'\\s', h1.text)\n"," temp1 = '2024-'+txtMonthToIntMonth(temp[1])+ '-'+ formatNumber(temp[0])\n"," output.append(temp1)\n"," return output\n","\n","ph_dates = scrape_holiday()"],"metadata":{"id":"yxGrHxqZ7Opt"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# ph_dates = ...your answer...\n","pprint(ph_dates)"],"metadata":{"id":"raUQhaQ47OkJ","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1729996912377,"user_tz":-480,"elapsed":9,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"17adaf35-dfb5-4790-fc23-ada129b4d172"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["['2024-01-01',\n"," '2024-02-10',\n"," '2024-02-12',\n"," '2024-02-13',\n"," '2024-03-29',\n"," '2024-03-30',\n"," '2024-04-01',\n"," '2024-04-04',\n"," '2024-05-01',\n"," '2024-05-15',\n"," '2024-06-10',\n"," '2024-07-01',\n"," '2024-09-18',\n"," '2024-10-01',\n"," '2024-10-11',\n"," '2024-12-25',\n"," '2024-12-26']\n"]}]},{"cell_type":"markdown","source":["## 2C. Write a function: Get information for each carpark, according to the date inputted. (10%)\n","\n","The hours opening/closing, hourly charge, and parking spaces can be found in dictionaries 'privateCar' and nested dictionary 'hourlyCharges', such as the following example:\n","\n","```\n"," 'privateCar': {'hourlyCharges': [{'weekdays': ['MON',\n"," 'TUE',\n"," 'WED',\n"," 'THU',\n"," 'FRI'],\n"," 'excludePublicHoliday': True,\n"," 'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 15,\n"," 'type': 'hourly',\n"," 'covered': 'covered',\n"," 'usageMinimum': 1,\n"," 'remark': ''},\n"," {'weekdays': ['SAT', 'SUN', 'PH'],\n"," 'excludePublicHoliday': False,\n"," 'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 20,\n"," 'usageMinimum': 1,\n"," 'type': 'hourly',\n"," 'covered': 'covered',\n"," 'remark': ''}],\n"," 'spaceUNL': 0,\n"," 'spaceEV': 0,\n"," 'spaceDIS': 0,\n"," 'space': 112},\n","```\n","\n","**Write a function \"get_todayinfo\"** with\n","- input as\n"," - \"js_row\", the json item for a single carpark\n"," - \"ph_dates\", the list of dates in string of public holidays\n"," - \"today_date_str\", the date in string with format \"YYYY-mm-dd\"\n","- return a dictionary with\n"," - \"periodStart\", time when carpark is opened\n"," - \"periodEnd\", time when carpark is closed\n"," - \"price\", hourly charge\n"," - \"space\", no. of parking spaces in the carpark.\n","\n","Besides, in the function you need\n","- Set to None if \"periodStart\", \"periodEnd\", or \"price\" have not been mentioned in data\n","- Set to 0 if \"space\" has not been mentioned in data.\n","- default value of parameter \"today_date_str\" is assigned to be None. Then today date is assigned in the function with variable \"today_date_str\"\n","\n","Be CAUTION: For some carparks, the keys of some dictionaries may be missed. You need to set default value once you cannot call some value in the dictionary.\n","\n","---\n","\n","Expected Outputs:\n","\n","```\n","# get_todayinfo(js_row, ph_dates)\n","\n","{'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 15,\n"," 'space': 112,\n"," 'today': '2024-10-18',\n"," 'today_weekday': 'FRI'}\n","\n","# get_todayinfo(js_row, ph_dates, '2024-12-25')\n","\n","{'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 20,\n"," 'space': 112,\n"," 'today': '2024-12-25',\n"," 'today_weekday': 'PH'}\n","```\n","\n","\n"],"metadata":{"id":"oABeEM7Iep8D"}},{"cell_type":"code","source":["import datetime\n","\n","def get_todayinfo(js_row, ph_dates, today_date_str = datetime.datetime.today().strftime('%Y-%m-%d')):\n"," given_date = datetime.datetime.strptime(today_date_str, '%Y-%m-%d')\n"," weekdays = ['MON', 'TUE', 'WED', 'THU', 'FRI', 'SAT', 'SUN']\n"," txt_weekday = \"PH\" if today_date_str in ph_dates else weekdays[given_date.weekday()]\n","\n"," output = {\n"," 'periodStart': '',\n"," 'periodEnd': '',\n"," 'price': 0,\n"," 'space': 0,\n"," 'today': today_date_str,\n"," 'today_weekday': txt_weekday\n"," }\n","\n"," if (js_row.get('privateCar') != None):\n"," jp = js_row['privateCar']\n"," if (jp.get('space') != None):\n"," output['space'] = js_row['privateCar']['space']\n"," if (jp.get('hourlyCharges') != None):\n"," for i in js_row['privateCar']['hourlyCharges']:\n"," if txt_weekday in i['weekdays']:\n"," output['periodStart'] = i['periodStart']\n"," output['periodEnd'] = i['periodEnd']\n"," output['price'] = i['price']\n"," break\n","\n"," output['today'] = today_date_str\n"," output['today_weekday'] = txt_weekday\n","\n"," return output"],"metadata":{"id":"64hacEppffjy"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["get_todayinfo(js_row, ph_dates)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4PKCPRUdYPU5","executionInfo":{"status":"ok","timestamp":1729996912377,"user_tz":-480,"elapsed":7,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"069201db-d7fa-421c-f357-4005ec45419a"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 20,\n"," 'space': 112,\n"," 'today': '2024-10-27',\n"," 'today_weekday': 'SUN'}"]},"metadata":{},"execution_count":65}]},{"cell_type":"code","source":["get_todayinfo(js_row, ph_dates, '2024-12-25')"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"wmCW-BnKYWRs","executionInfo":{"status":"ok","timestamp":1729996912377,"user_tz":-480,"elapsed":6,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"5b0df080-bbfc-4add-f30b-3554aafb1ccd"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["{'periodStart': '07:00',\n"," 'periodEnd': '23:00',\n"," 'price': 20,\n"," 'space': 112,\n"," 'today': '2024-12-25',\n"," 'today_weekday': 'PH'}"]},"metadata":{},"execution_count":66}]},{"cell_type":"markdown","source":["## 2D. Putting all the things together - Output the pandas data frame (10%)\n","\n","**Write a script to get the full dataset merged**.\n","\n","Expected Output:\n","\n","![Example Image](https://drive.google.com/uc?id=1Jbjqx7ghZTYyyigml_tqJK2JxDTD5Etp)\n","\n","Note the dataset is with dimensions of 471 rows and 17 columns.\n","\n","Save the dataset with name \"df_carpark\"."],"metadata":{"id":"lMPED8aQpbzh"}},{"cell_type":"code","source":["import pandas as pd"],"metadata":{"id":"FciJdfFS7oKz"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["hodgepodge = []\n","for jr in task1_json:\n"," hodgepodge.append({**jr, **get_todayinfo(jr, ph_dates), \"url_image\": get_url_image(jr)})\n","\n","test_df = pd.DataFrame(hodgepodge)"],"metadata":{"id":"Fqb-beuy7oIR"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# df_carpark = ...your answer...\n","\n","df_carpark = test_df[['park_Id','name',\n"," 'displayAddress', 'district','latitude','longitude', 'opening_status', 'facilities', 'paymentMethods',\n"," 'modifiedDate','url_image', 'periodStart', 'periodEnd', 'price','space', 'today', 'today_weekday']]\n","\n","df_carpark"],"metadata":{"id":"7vuMCITU7oF9","colab":{"base_uri":"https://localhost:8080/","height":894},"executionInfo":{"status":"ok","timestamp":1729996912377,"user_tz":-480,"elapsed":4,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"4c6c9fb1-8fe8-42ee-e494-739216a959c9"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" park_Id name \\\n","0 10 Kai Tak Cruise Terminal Car Park 1 \n","1 12 Amoy Plaza \n","2 27 Yau Lai Shopping Centre Carpark \n","3 30 Telford Plaza D Carpark \n","4 31 Telford Plaza I Carpark \n",".. ... ... \n","467 tdstt93 Hoi Yu Street Car Park No 1 \n","468 tdc150p1 Ying Tung Estate Carpark \n","469 tdstt97 On Sau Road \n","470 tdstt100 Hoi Wah Road (II) Car Park \n","471 tdc177p1 KTR350 Car Park \n","\n"," displayAddress district \\\n","0 1st floor, Kai Tak Cruise Terminal, 33 Shing F... Kwun Tong District \n","1 77 Ngau Tau Kok Raod, Kowloon Bay, KLN Kwun Tong District \n","2 LG1 & LG2, Yau Lai Shopping Centre, 9 Yau Tong... Kwun Tong District \n","3 33 Wai Yip Street, Kowloon Bay, KLN Kwun Tong District \n","4 33 Wai Yip Street, Kowloon Bay, KLN Kwun Tong District \n",".. ... ... \n","467 Hoi Yu Street Quarry Bay Eastern \n","468 12 YING TUNG ROAD Islands \n","469 On Sau Road, Kowloon Kwun Tong \n","470 Hoi Wah Road, Area 16, Tuen Mun, New Territories Tuen Mun \n","471 350 Kwun Tong Road, Kwun Tong, Kowloon Kwun Tong \n","\n"," latitude longitude opening_status \\\n","0 22.306205 114.213095 OPEN \n","1 22.324701 114.216753 OPEN \n","2 22.298512 114.237401 OPEN \n","3 22.322289 114.212043 OPEN \n","4 22.322686 114.213535 OPEN \n",".. ... ... ... \n","467 22.290065 114.214859 CLOSED \n","468 22.296614 113.953083 CLOSED \n","469 22.325902 114.231010 CLOSED \n","470 22.381421 113.973929 CLOSED \n","471 22.315039 114.218000 CLOSED \n","\n"," facilities paymentMethods \\\n","0 [disabilities, evCharger] [octopus, visa] \n","1 [evCharger, disabilities, unloading, washing] [cash, octopus] \n","2 [evCharger, disabilities] [octopus] \n","3 [evCharger] [octopus, visa, master] \n","4 [evCharger, disabilities] [octopus, visa, master] \n",".. ... ... \n","467 NaN NaN \n","468 NaN NaN \n","469 NaN NaN \n","470 NaN NaN \n","471 NaN NaN \n","\n"," modifiedDate url_image \\\n","0 2024-01-03 13:28:25 https://sps-opendata.pilotsmartke.gov.hk/rest/... \n","1 2024-07-12 21:55:17 None \n","2 2022-07-22 15:47:42 None \n","3 2018-01-31 17:20:14 None \n","4 2018-01-31 17:19:10 None \n",".. ... ... \n","467 NaN None \n","468 NaN None \n","469 NaN None \n","470 NaN None \n","471 NaN None \n","\n"," periodStart periodEnd price space today today_weekday \n","0 07:00 23:00 20 112 2024-10-27 SUN \n","1 00:00 24:00 25 389 2024-10-27 SUN \n","2 00:00 24:00 17 11 2024-10-27 SUN \n","3 00:00 24:00 22 70 2024-10-27 SUN \n","4 00:00 24:00 22 170 2024-10-27 SUN \n",".. ... ... ... ... ... ... \n","467 0 0 2024-10-27 SUN \n","468 0 0 2024-10-27 SUN \n","469 0 0 2024-10-27 SUN \n","470 0 0 2024-10-27 SUN \n","471 0 0 2024-10-27 SUN \n","\n","[472 rows x 17 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
park_IdnamedisplayAddressdistrictlatitudelongitudeopening_statusfacilitiespaymentMethodsmodifiedDateurl_imageperiodStartperiodEndpricespacetodaytoday_weekday
010Kai Tak Cruise Terminal Car Park 11st floor, Kai Tak Cruise Terminal, 33 Shing F...Kwun Tong District22.306205114.213095OPEN[disabilities, evCharger][octopus, visa]2024-01-03 13:28:25https://sps-opendata.pilotsmartke.gov.hk/rest/...07:0023:00201122024-10-27SUN
112Amoy Plaza77 Ngau Tau Kok Raod, Kowloon Bay, KLNKwun Tong District22.324701114.216753OPEN[evCharger, disabilities, unloading, washing][cash, octopus]2024-07-12 21:55:17None00:0024:00253892024-10-27SUN
227Yau Lai Shopping Centre CarparkLG1 & LG2, Yau Lai Shopping Centre, 9 Yau Tong...Kwun Tong District22.298512114.237401OPEN[evCharger, disabilities][octopus]2022-07-22 15:47:42None00:0024:0017112024-10-27SUN
330Telford Plaza D Carpark33 Wai Yip Street, Kowloon Bay, KLNKwun Tong District22.322289114.212043OPEN[evCharger][octopus, visa, master]2018-01-31 17:20:14None00:0024:0022702024-10-27SUN
431Telford Plaza I Carpark33 Wai Yip Street, Kowloon Bay, KLNKwun Tong District22.322686114.213535OPEN[evCharger, disabilities][octopus, visa, master]2018-01-31 17:19:10None00:0024:00221702024-10-27SUN
......................................................
467tdstt93Hoi Yu Street Car Park No 1Hoi Yu Street Quarry BayEastern22.290065114.214859CLOSEDNaNNaNNaNNone002024-10-27SUN
468tdc150p1Ying Tung Estate Carpark12 YING TUNG ROADIslands22.296614113.953083CLOSEDNaNNaNNaNNone002024-10-27SUN
469tdstt97On Sau RoadOn Sau Road, KowloonKwun Tong22.325902114.231010CLOSEDNaNNaNNaNNone002024-10-27SUN
470tdstt100Hoi Wah Road (II) Car ParkHoi Wah Road, Area 16, Tuen Mun, New TerritoriesTuen Mun22.381421113.973929CLOSEDNaNNaNNaNNone002024-10-27SUN
471tdc177p1KTR350 Car Park350 Kwun Tong Road, Kwun Tong, KowloonKwun Tong22.315039114.218000CLOSEDNaNNaNNaNNone002024-10-27SUN
\n","

472 rows × 17 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_carpark","summary":"{\n \"name\": \"df_carpark\",\n \"rows\": 472,\n \"fields\": [\n {\n \"column\": \"park_Id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 472,\n \"samples\": [\n \"tdc14p1004\",\n \"tdc14p2101\",\n \"tdcp2\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 466,\n \"samples\": [\n \"Choi Ming Car Park 1 \",\n \"Hing Tung Car Park \",\n \"Rumsey Street Car Park\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"displayAddress\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 437,\n \"samples\": [\n \"HUNG HOM ESTATE PHASE II CARPARK, 28 DAI WAN ROAD, HUNG HOM, KOWLOON\",\n \"Hin Keng Shopping Centre, 69 Che Kung Miu Road, Tai Wai, Sha Tin, New Territories\",\n \"6 Choi Ha Rd, Ngau Tau Kok\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"district\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 21,\n \"samples\": [\n \"Kwun Tong District\",\n \"Kwun Tong\",\n \"Sai Kung\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.06325756061661082,\n \"min\": 22.22033,\n \"max\": 22.55267812,\n \"num_unique_values\": 470,\n \"samples\": [\n 22.30666,\n 22.42166,\n 22.28740848\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.08822583620461727,\n \"min\": 113.93449828751,\n \"max\": 114.27423859,\n \"num_unique_values\": 469,\n \"samples\": [\n 114.252446,\n 114.226755,\n 114.15322694\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"opening_status\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"CLOSED\",\n \"OPEN\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"facilities\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"paymentMethods\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"modifiedDate\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 24,\n \"samples\": [\n \"2024-01-03 13:25:37\",\n \"2023-04-25 17:56:48\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"url_image\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 9,\n \"samples\": [\n \"https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A915718766070366444542505038806645269890059758729.png/square.png\",\n \"https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A859280094011967658179347734190846999967397591797.jpg/square.png\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"periodStart\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"07:00\",\n \"00:00\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"periodEnd\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"24:00\",\n \"\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 4,\n \"min\": 0,\n \"max\": 30,\n \"num_unique_values\": 11,\n \"samples\": [\n 30,\n 20\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"space\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 46,\n \"min\": 0,\n \"max\": 851,\n \"num_unique_values\": 24,\n \"samples\": [\n 40,\n 91\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"today\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2024-10-27\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"today_weekday\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"SUN\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":69}]},{"cell_type":"markdown","source":["# Task 3. Get Real-time Vacancy for each carpark (15%)\n","\n","Besides the carpark information, the API can also be called for the real-time vacancies.\n","\n","Please re-check the following:\n","- The API page is here. ([In Traditional Chinese](https://data.gov.hk/tc-data/dataset/hk-dpo-datagovhk1-carpark-info-vacancy/resource/f4c792c6-071c-4a64-888b-afeea33d5ad7), [In English](https://data.gov.hk/en-data/dataset/hk-dpo-datagovhk1-carpark-info-vacancy/resource/01752c62-a6b6-4ddc-bf2d-25efccadc143))\n","- and the data dictionary is [here](https://resource.data.one.gov.hk/opendata/carpark/Parking_Vacancy_Data_Specification.pdf).\n","\n","**Write a script for obtaining the full pandas data frame**, with the following instructions:\n","- In calling API, set the \"data\" parameter to be \"vacancy\"\n","- set the \"vehicleType\" parameter to be \"privateCar\" as well\n","- 3 columns are in dataset\n"," - \"park_Id\", for joining the carpark info dataset.\n"," - \"vacancy\", the no. of available carpark spaces\n"," - \"lastupdate\", Last Update time of the \"vacancy\" number.\n","- Only \"vacancy_type\" = \"A\", the vacancy no. will be taken into account.\n","- In default, set \"vacancy\" as 0, and \"lastupdate\" as None, if the carpark does not have this information.\n","- Name the dataset as \"df_vacancy\".\n","\n","---\n","\n","Expected JSON for a single carpark\n","\n","```\n","# js_result[0]\n","\n","{'park_Id': '10',\n"," 'privateCar': [{'vacancy_type': 'A',\n"," 'vacancy': 29,\n"," 'lastupdate': '2024-10-18 16:04:59'}]}\n","```\n","\n","Expected Output\n","\n","![Example Image](https://drive.google.com/uc?id=1I2NhcYdpQedgSw53k1nq5xKBeA6KKBXu)\n","\n","Note the vacancy dataset is still with 471 rows, same as the carpark info dataset.\n","\n","\n"],"metadata":{"id":"lF63KHwqznj4"}},{"cell_type":"code","source":["\n","url = \"https://api.data.gov.hk/v1/carpark-info-vacancy\"\n","params = {\n"," \"data\": \"vacancy\",\n"," \"vehicleTypes\": \"privateCar\"\n","}\n","\n","response = requests.get(url, params=params)\n","test_json = response.json()['results']"],"metadata":{"id":"cB9zDNPG7vXd"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["df = pd.json_normalize(test_json, record_path=['privateCar'], meta='park_Id')\n","\n","# Extract the 'vacancy' column and flatten it\n","df_vacancy = df[['park_Id', 'vacancy','lastupdate']]"],"metadata":{"id":"MhBS62lS7vVF"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# df_vacancy = ...your answer...\n","\n","df_vacancy"],"metadata":{"id":"IeuVM5-P7vPL","colab":{"base_uri":"https://localhost:8080/","height":423},"executionInfo":{"status":"ok","timestamp":1729997009156,"user_tz":-480,"elapsed":378,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"541b78b2-bef7-438d-872a-9715e7c55043"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" park_Id vacancy lastupdate\n","0 10 94 2024-10-27 10:31:48\n","1 12 189 2024-10-27 02:32:21\n","2 27 5 2024-10-25 09:11:28\n","3 30 20 2024-10-27 10:30:10\n","4 31 55 2024-10-27 07:04:11\n",".. ... ... ...\n","467 tdstt93 -1 2024-10-27 10:41:03\n","468 tdc150p1 2 2024-10-27 10:35:34\n","469 tdstt97 -1 2024-10-27 10:41:03\n","470 tdstt100 16 2024-10-27 10:40:09\n","471 tdc177p1 34 2024-10-27 10:40:26\n","\n","[472 rows x 3 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
park_Idvacancylastupdate
010942024-10-27 10:31:48
1121892024-10-27 02:32:21
22752024-10-25 09:11:28
330202024-10-27 10:30:10
431552024-10-27 07:04:11
............
467tdstt93-12024-10-27 10:41:03
468tdc150p122024-10-27 10:35:34
469tdstt97-12024-10-27 10:41:03
470tdstt100162024-10-27 10:40:09
471tdc177p1342024-10-27 10:40:26
\n","

472 rows × 3 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_vacancy","summary":"{\n \"name\": \"df_vacancy\",\n \"rows\": 472,\n \"fields\": [\n {\n \"column\": \"park_Id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 472,\n \"samples\": [\n \"tdc14p1004\",\n \"tdc14p2101\",\n \"tdcp2\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"vacancy\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 70,\n \"min\": -1,\n \"max\": 572,\n \"num_unique_values\": 79,\n \"samples\": [\n 1,\n 94,\n -1\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"lastupdate\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 79,\n \"samples\": [\n \"2024-10-27 10:40:03\",\n \"2024-10-27 10:31:48\",\n \"2024-10-27 10:41:03\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":76}]},{"cell_type":"markdown","source":["# Task 4. Join both datasets (10%)\n","\n","Finally, **write a script to join df_carpark and df_vacancy**, with the following instruction.\n","- Applying \"Left-Join\"\n","- on the common column \"park_Id\"\n","- saving as \"df_out\"\n","\n","---\n","\n","Expected Output\n","\n","![Example Image](https://drive.google.com/uc?id=1Nlc9PQ74rlqjQT_Oj200m0Vpg4om39lv)\n","\n","Note the dataset is with dimensions of 471 rows and 19 columns.\n","\n"],"metadata":{"id":"mdELHx582xC9"}},{"cell_type":"code","source":[" # df_out = ...your answer...\n","\n","df_vacancy.set_index('park_Id', inplace=True)\n","df_carpark.set_index('park_Id', inplace=True)\n","df_out=df_carpark.join(df_vacancy).reset_index(drop=False)\n","\n","df_out"],"metadata":{"id":"LYIZvKU37xxR","colab":{"base_uri":"https://localhost:8080/","height":894},"executionInfo":{"status":"ok","timestamp":1729996913324,"user_tz":-480,"elapsed":6,"user":{"displayName":"7goolook測試員","userId":"16118248937940283592"}},"outputId":"ab289c96-f1c9-4f8d-de5e-bd9cb383625f"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":[" park_Id name \\\n","0 10 Kai Tak Cruise Terminal Car Park 1 \n","1 12 Amoy Plaza \n","2 27 Yau Lai Shopping Centre Carpark \n","3 30 Telford Plaza D Carpark \n","4 31 Telford Plaza I Carpark \n",".. ... ... \n","467 tdstt93 Hoi Yu Street Car Park No 1 \n","468 tdc150p1 Ying Tung Estate Carpark \n","469 tdstt97 On Sau Road \n","470 tdstt100 Hoi Wah Road (II) Car Park \n","471 tdc177p1 KTR350 Car Park \n","\n"," displayAddress district \\\n","0 1st floor, Kai Tak Cruise Terminal, 33 Shing F... Kwun Tong District \n","1 77 Ngau Tau Kok Raod, Kowloon Bay, KLN Kwun Tong District \n","2 LG1 & LG2, Yau Lai Shopping Centre, 9 Yau Tong... Kwun Tong District \n","3 33 Wai Yip Street, Kowloon Bay, KLN Kwun Tong District \n","4 33 Wai Yip Street, Kowloon Bay, KLN Kwun Tong District \n",".. ... ... \n","467 Hoi Yu Street Quarry Bay Eastern \n","468 12 YING TUNG ROAD Islands \n","469 On Sau Road, Kowloon Kwun Tong \n","470 Hoi Wah Road, Area 16, Tuen Mun, New Territories Tuen Mun \n","471 350 Kwun Tong Road, Kwun Tong, Kowloon Kwun Tong \n","\n"," latitude longitude opening_status \\\n","0 22.306205 114.213095 OPEN \n","1 22.324701 114.216753 OPEN \n","2 22.298512 114.237401 OPEN \n","3 22.322289 114.212043 OPEN \n","4 22.322686 114.213535 OPEN \n",".. ... ... ... \n","467 22.290065 114.214859 CLOSED \n","468 22.296614 113.953083 CLOSED \n","469 22.325902 114.231010 CLOSED \n","470 22.381421 113.973929 CLOSED \n","471 22.315039 114.218000 CLOSED \n","\n"," facilities paymentMethods \\\n","0 [disabilities, evCharger] [octopus, visa] \n","1 [evCharger, disabilities, unloading, washing] [cash, octopus] \n","2 [evCharger, disabilities] [octopus] \n","3 [evCharger] [octopus, visa, master] \n","4 [evCharger, disabilities] [octopus, visa, master] \n",".. ... ... \n","467 NaN NaN \n","468 NaN NaN \n","469 NaN NaN \n","470 NaN NaN \n","471 NaN NaN \n","\n"," modifiedDate url_image \\\n","0 2024-01-03 13:28:25 https://sps-opendata.pilotsmartke.gov.hk/rest/... \n","1 2024-07-12 21:55:17 None \n","2 2022-07-22 15:47:42 None \n","3 2018-01-31 17:20:14 None \n","4 2018-01-31 17:19:10 None \n",".. ... ... \n","467 NaN None \n","468 NaN None \n","469 NaN None \n","470 NaN None \n","471 NaN None \n","\n"," periodStart periodEnd price space today today_weekday vacancy \\\n","0 07:00 23:00 20 112 2024-10-27 SUN 94 \n","1 00:00 24:00 25 389 2024-10-27 SUN 189 \n","2 00:00 24:00 17 11 2024-10-27 SUN 5 \n","3 00:00 24:00 22 70 2024-10-27 SUN 20 \n","4 00:00 24:00 22 170 2024-10-27 SUN 55 \n",".. ... ... ... ... ... ... ... \n","467 0 0 2024-10-27 SUN -1 \n","468 0 0 2024-10-27 SUN 2 \n","469 0 0 2024-10-27 SUN -1 \n","470 0 0 2024-10-27 SUN 16 \n","471 0 0 2024-10-27 SUN 34 \n","\n"," lastupdate \n","0 2024-10-27 10:31:48 \n","1 2024-10-27 02:32:21 \n","2 2024-10-25 09:11:28 \n","3 2024-10-27 10:30:10 \n","4 2024-10-27 07:04:11 \n",".. ... \n","467 2024-10-27 10:40:04 \n","468 2024-10-27 10:35:34 \n","469 2024-10-27 10:40:04 \n","470 2024-10-27 10:39:06 \n","471 2024-10-27 10:38:26 \n","\n","[472 rows x 19 columns]"],"text/html":["\n","
\n","
\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","
park_IdnamedisplayAddressdistrictlatitudelongitudeopening_statusfacilitiespaymentMethodsmodifiedDateurl_imageperiodStartperiodEndpricespacetodaytoday_weekdayvacancylastupdate
010Kai Tak Cruise Terminal Car Park 11st floor, Kai Tak Cruise Terminal, 33 Shing F...Kwun Tong District22.306205114.213095OPEN[disabilities, evCharger][octopus, visa]2024-01-03 13:28:25https://sps-opendata.pilotsmartke.gov.hk/rest/...07:0023:00201122024-10-27SUN942024-10-27 10:31:48
112Amoy Plaza77 Ngau Tau Kok Raod, Kowloon Bay, KLNKwun Tong District22.324701114.216753OPEN[evCharger, disabilities, unloading, washing][cash, octopus]2024-07-12 21:55:17None00:0024:00253892024-10-27SUN1892024-10-27 02:32:21
227Yau Lai Shopping Centre CarparkLG1 & LG2, Yau Lai Shopping Centre, 9 Yau Tong...Kwun Tong District22.298512114.237401OPEN[evCharger, disabilities][octopus]2022-07-22 15:47:42None00:0024:0017112024-10-27SUN52024-10-25 09:11:28
330Telford Plaza D Carpark33 Wai Yip Street, Kowloon Bay, KLNKwun Tong District22.322289114.212043OPEN[evCharger][octopus, visa, master]2018-01-31 17:20:14None00:0024:0022702024-10-27SUN202024-10-27 10:30:10
431Telford Plaza I Carpark33 Wai Yip Street, Kowloon Bay, KLNKwun Tong District22.322686114.213535OPEN[evCharger, disabilities][octopus, visa, master]2018-01-31 17:19:10None00:0024:00221702024-10-27SUN552024-10-27 07:04:11
............................................................
467tdstt93Hoi Yu Street Car Park No 1Hoi Yu Street Quarry BayEastern22.290065114.214859CLOSEDNaNNaNNaNNone002024-10-27SUN-12024-10-27 10:40:04
468tdc150p1Ying Tung Estate Carpark12 YING TUNG ROADIslands22.296614113.953083CLOSEDNaNNaNNaNNone002024-10-27SUN22024-10-27 10:35:34
469tdstt97On Sau RoadOn Sau Road, KowloonKwun Tong22.325902114.231010CLOSEDNaNNaNNaNNone002024-10-27SUN-12024-10-27 10:40:04
470tdstt100Hoi Wah Road (II) Car ParkHoi Wah Road, Area 16, Tuen Mun, New TerritoriesTuen Mun22.381421113.973929CLOSEDNaNNaNNaNNone002024-10-27SUN162024-10-27 10:39:06
471tdc177p1KTR350 Car Park350 Kwun Tong Road, Kwun Tong, KowloonKwun Tong22.315039114.218000CLOSEDNaNNaNNaNNone002024-10-27SUN342024-10-27 10:38:26
\n","

472 rows × 19 columns

\n","
\n","
\n","\n","
\n"," \n","\n"," \n","\n"," \n","
\n","\n","\n","
\n"," \n","\n","\n","\n"," \n","
\n","
\n","
\n"],"application/vnd.google.colaboratory.intrinsic+json":{"type":"dataframe","variable_name":"df_out","summary":"{\n \"name\": \"df_out\",\n \"rows\": 472,\n \"fields\": [\n {\n \"column\": \"park_Id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 472,\n \"samples\": [\n \"tdc14p1004\",\n \"tdc14p2101\",\n \"tdcp2\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 466,\n \"samples\": [\n \"Choi Ming Car Park 1 \",\n \"Hing Tung Car Park \",\n \"Rumsey Street Car Park\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"displayAddress\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 437,\n \"samples\": [\n \"HUNG HOM ESTATE PHASE II CARPARK, 28 DAI WAN ROAD, HUNG HOM, KOWLOON\",\n \"Hin Keng Shopping Centre, 69 Che Kung Miu Road, Tai Wai, Sha Tin, New Territories\",\n \"6 Choi Ha Rd, Ngau Tau Kok\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"district\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 21,\n \"samples\": [\n \"Kwun Tong District\",\n \"Kwun Tong\",\n \"Sai Kung\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.06325756061661082,\n \"min\": 22.22033,\n \"max\": 22.55267812,\n \"num_unique_values\": 470,\n \"samples\": [\n 22.30666,\n 22.42166,\n 22.28740848\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.08822583620461727,\n \"min\": 113.93449828751,\n \"max\": 114.27423859,\n \"num_unique_values\": 469,\n \"samples\": [\n 114.252446,\n 114.226755,\n 114.15322694\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"opening_status\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"CLOSED\",\n \"OPEN\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"facilities\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"paymentMethods\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"modifiedDate\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 24,\n \"samples\": [\n \"2024-01-03 13:25:37\",\n \"2023-04-25 17:56:48\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"url_image\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 9,\n \"samples\": [\n \"https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A915718766070366444542505038806645269890059758729.png/square.png\",\n \"https://sps-opendata.pilotsmartke.gov.hk/rest/getRendition/fs-1%3A859280094011967658179347734190846999967397591797.jpg/square.png\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"periodStart\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 6,\n \"samples\": [\n \"07:00\",\n \"00:00\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"periodEnd\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"24:00\",\n \"\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"price\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 4,\n \"min\": 0,\n \"max\": 30,\n \"num_unique_values\": 11,\n \"samples\": [\n 30,\n 20\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"space\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 46,\n \"min\": 0,\n \"max\": 851,\n \"num_unique_values\": 24,\n \"samples\": [\n 40,\n 91\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"today\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"2024-10-27\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"today_weekday\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"SUN\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"vacancy\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 70,\n \"min\": -1,\n \"max\": 574,\n \"num_unique_values\": 78,\n \"samples\": [\n 128\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"lastupdate\",\n \"properties\": {\n \"dtype\": \"object\",\n \"num_unique_values\": 79,\n \"samples\": [\n \"2024-10-27 10:39:08\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}"}},"metadata":{},"execution_count":73}]},{"cell_type":"markdown","source":["\n","\n","Finally we can make the following !\n","\n","![Example Image](https://drive.google.com/uc?id=1-I14TdnpkGwwObWs1ZNGt2Ufb2NMIBSH)\n","\n","The dataset can be used in Tableau dashboard which will be introduced in the lesson later."],"metadata":{"id":"fG6fLzPr3FiO"}},{"cell_type":"markdown","source":["# Congratuations ! That's all for the mid-term exam !
\n","# Please save this and upload this jupyter notebook to Blackboard.
\n","# Thanks !"],"metadata":{"id":"k1eYGlV6x8r4"}}]}