Agenda¶
- Structure
- Reading Files
- Inspect
- Descriptive Statistics
- Reshaping
- Selection
- Setting Value
- Insert and Drop
- Missing Value
- Combine
- Output File
- Reshaping II
- Grouping
- Binning
- Duplicate
Note¶
- 基於 Numpy
- 用於數據分析，提供容易操作的資料結構及分析工具
- 匯入 Pandas
In [1]:
import pandas as pd
也一併匯入 Numpy
In [2]:
import numpy as np
Structure¶
Series- 類似
numpy中的 1d-ndarray - 每個元素都有各自的索引值(標籤)
- 底層是
ndarray
- 類似
DataFrame- 類似
numpy中的 2d-ndarray - 由
pd.Series組成 - 可儲存不同類型的資料
- 有欄、列索引值(標籤)
- 類似
Panel- 三維數據結構
- 有資料集、欄、列索引值(標籤)
In [3]:
Taipei = np.array([17.0,8.3,20.1,22.6,27.0,27.8,30.1,30.3,29.4,27.1,24.0,17.5,])
type(Taipei)
Out[3]:
In [4]:
print(Taipei)
In [5]:
Taipei_series = pd.Series(Taipei)
type(Taipei_series)
Out[5]:
In [6]:
print(Taipei_series)
In [7]:
month = [ 'FEB', 'JAN', 'MAR', 'APR', 'MAY', 'JUN',
'JUL', 'AUG', 'SEP', 'OCT', 'NOV', 'DEC']
In [8]:
Taipei_series = pd.Series( Taipei ,
index = month )
In [9]:
print(Taipei_series)
In [10]:
Taipei = {'FEB': 17.0, 'JAN': 8.3, 'MAR': 20.1, 'APR': 22.6, 'MAY': 27.0, 'JUN': 27.8,
'JUL': 30.1, 'AUG': 30.3, 'SEP': 29.4, 'OCT': 27.1, 'NOV': 24.0, 'DEC': 17.5}
type(Taipei)
Out[10]:
In [11]:
print(Taipei)
In [12]:
Taipei_series = pd.Series(Taipei)
type(Taipei_series)
Out[12]:
In [13]:
print(Taipei_series)
Dictionary 的 Key 會變成 index
後面補充
In [14]:
Temperature = np.array([[17.0,8.3,20.1,22.6,27.0,27.8,30.1,30.3,29.4,27.1,24.0,17.5],
[21.1,21.7,22.9,25.7,28.4,28.4,29.4,30.2,29.7,28.9,26.9,21.6]])
type(Temperature)
Out[14]:
In [15]:
print(Temperature)
In [16]:
Temperature_dataframe = pd.DataFrame(Temperature)
type(Temperature_dataframe)
Out[16]:
In [17]:
print(Temperature_dataframe)
In [18]:
Temperature_dataframe = pd.DataFrame(Temperature,
index=['Taipei','Kaohsiung'],
columns = month)
type(Temperature_dataframe)
Out[18]:
In [19]:
print(Temperature_dataframe)
In [20]:
Temperature = {'FEB': [17.0,21.1], 'JAN': [8.3,21.7], 'MAR': [20.1,22.9], 'APR': [22.6,25.7],
'MAY': [27.0,28.4], 'JUN': [27.8,28.4],'JUL': [30.1,29.4], 'AUG': [30.3,30.2],
'SEP': [29.4,29.7], 'OCT': [27.1,28.9], 'NOV':[24.0,26.9], 'DEC': [17.5,21.6]}
type(Temperature)
Out[20]:
In [21]:
print(Temperature)
In [22]:
Temperature_dataframe = pd.DataFrame(Temperature)
type(Temperature_dataframe)
Out[22]:
In [23]:
print(Temperature_dataframe)
Dictionary 的 Key 會變成 column name
In [24]:
Temperature_dataframe = pd.DataFrame(Temperature,
columns = month)
type(Temperature_dataframe)
Out[24]:
In [25]:
print(Temperature_dataframe)
In [26]:
Temperature_dataframe = pd.DataFrame(Temperature,
columns = month,
index=['Taipei','Kaohsiung'])
type(Temperature_dataframe)
Out[26]:
In [27]:
print(Temperature_dataframe)
Note¶
Create Demo Data -- Temperature_dataframe¶
In [28]:
Temperature = {'FEB': [17.0,21.1,'Good'], 'JAN': [8.3,21.7,'Good'], 'MAR': [20.1,22.9,'Bad'],
'APR': [22.6,25.7,'Bad'], 'MAY': [27.0,28.4,'Bad'], 'JUN': [27.8,28.4,'Good'],
'JUL': [30.1,29.4,'Bad'], 'AUG': [30.3,30.2,'Bad'], 'SEP': [29.4,29.7,'Good'],
'OCT': [27.1,28.9,'Bad'], 'NOV':[24.0,26.9,'Good'], 'DEC': [17.5,21.6,'Good']}
In [29]:
Temperature_dataframe = pd.DataFrame(Temperature, columns = month)
print(Temperature_dataframe)
Note¶
In [30]:
Taipei = [17.0,8.3,20.1,22.6,27.0,27.8,30.1,30.3,29.4,27.1,24.0,17.5]
Kaohsiung = [21.1,21.7,22.9,25.7,28.4,28.4,29.4,30.2,29.7,28.9,26.9,21.6]
Quality = ['Good','Good','Bad','Bad','Bad','Good','Bad','Bad','Good','Bad','Good','Good']
Temperature2 = { 'Taipei' : Taipei ,
'Kaohsiung' : Kaohsiung ,
'Quality' : Quality }
In [31]:
Temperature_dataframe2 = pd.DataFrame(Temperature2, index = month)
print(Temperature_dataframe2)
Reading Files¶
Note¶
檢查目前所在目錄¶
In [32]:
import os
os.getcwd()
Out[32]:
Reading Files¶
Note¶
Reference : 政府開放資料 http://data.gov.tw/node/7325
In [33]:
Aboard = pd.read_csv('/Users/hsinyu/Desktop/Python資料分析/Data/Aboard.csv',encoding='big5')
In [34]:
Aboard.head()
Out[34]:
Reference : 內政部統計查詢網 http://statis.moi.gov.tw/micst/stmain.jsp?sys=100
In [35]:
Fire = pd.read_excel('/Users/hsinyu/Desktop/Python資料分析/Data/Fire.xlsx')
In [36]:
Fire.head()
Out[36]:
Reference : 台北市政府資料開放平台 http://data.taipei/opendata/datalist/datasetMeta?oid=4d29818c-a3ee-425d-b88a-22ac0c24c712
In [37]:
construction = pd.read_json('/Users/hsinyu/Desktop/Python資料分析/Data/construction.json')
In [38]:
construction.head(1)
Out[38]:
Note¶
Encoding Problem¶
In [39]:
import codecs
In [40]:
construction = pd.read_json(codecs.open('/Users/hsinyu/Desktop/Python資料分析/Data/construction.json', 'r', 'utf-8'))
In [41]:
Temperature_dataframe
Out[41]:
In [42]:
Temperature_dataframe2
Out[42]:
In [43]:
Temperature_dataframe.shape
Out[43]:
In [44]:
Temperature_dataframe2.shape
Out[44]:
In [45]:
Temperature_dataframe.dtypes
Out[45]:
In [46]:
Temperature_dataframe2.dtypes
Out[46]:
In [47]:
Temperature_dataframe2.columns
Out[47]:
In [48]:
Temperature_dataframe2.index
Out[48]:
In [49]:
Temperature_dataframe2.values
Out[49]:
In [50]:
Temperature_dataframe2.head(3)
Out[50]:
In [51]:
Temperature_dataframe2.head(5)
Out[51]:
In [52]:
Temperature_dataframe2.tail(3)
Out[52]:
In [53]:
Temperature_dataframe2.tail(5)
Out[53]:
In [54]:
Temperature_dataframe.info()
In [55]:
Temperature_dataframe2.info()
Descriptive Statistics¶
Note¶
Data science process flowchart¶
Reference : https://www.wikiwand.com/en/Data_analysis
In [56]:
Temperature_dataframe2.describe()
Out[56]:
Descriptive Statistics¶
Note¶
| Category | Continuous | |
| 次 數 | Central | Dispersion |
| 平均數 | 標準差 | |
| 中位數 | 全 距 | |
| 眾 數 | 四分位差 | |
Descriptive Statistics¶
Note¶
Descriptive Statistics ( Category )¶
治療癌症，共有兩個方法 ： Method A , Method B¶
以整體來看 Method A 比 Method B 好
Descriptive Statistics¶
Note¶
Descriptive Statistics ( Category )¶
Simpson's paradox¶
In [57]:
Temperature_dataframe2['Quality'].value_counts()
Out[57]:
Exercise¶
Q. 2002 - 2013 年國民出國目的地人數統計¶
- 資料中共有幾個觀測值
- 資料中共有幾個欄位及各欄位的名稱分別是什麼
- 資料中各個欄位的 Descriptive Statistics
Answers¶
In [58]:
Aboard_shape = Aboard.shape
print(Aboard_shape)
In [59]:
Aboard_shape[0]*Aboard_shape[1]
Out[59]:
In [60]:
Aboard.size
Out[60]:
Answers¶
In [61]:
len(Aboard.columns)
Out[61]:
Method2.¶
In [62]:
Aboard_shape[1]
Out[62]:
Answers¶
2. 資料中共有幾個欄位及各欄位的名稱分別是什麼¶
Aboard 的欄位名稱
In [63]:
Aboard.columns
Out[63]:
Answers¶
3. 資料中各個欄位的 Descriptive Statistics¶
Aboard 的敘述統計
In [64]:
Aboard.describe()
Out[64]:
What's wrong??
所有欄位都是全部都是文字(非數值)
Answers¶
Note 將 2002 - 2013 轉成數值型態¶
檢查 Aboard 欄位資料類型
In [65]:
Aboard.dtypes
Out[65]:
Answers¶
Note 將 2002 - 2013 轉成數值型態¶
檢查 Aboard 原始資料 ( csv file )
數字中含有 ,
Answers¶
Note 更新前的 Aboard¶
In [66]:
Aboard.head(1)
Out[66]:
Answers¶
Note 將 2002 - 2013 轉成數值型態¶
重新讀入 Aboard.csv ，加上 thousands=','
In [67]:
Aboard = pd.read_csv('/Users/hsinyu/Desktop/Python資料分析/Data/Aboard.csv',encoding='big5',sep=',', thousands=',')
In [68]:
Aboard.head(1)
Out[68]:
Answers¶
Note 將 2002 - 2013 轉成數值型態¶
重新檢查 Aboard 欄位資料類型
In [69]:
Aboard.dtypes
Out[69]:
2012,2013 從 object 轉成 int64
其他怎麼辦 ??
Answers¶
Note 將 2002 - 2011 轉成數值型態¶
再次檢查 Aboard 原始資料 ( csv file )
資料中 “-” 應該表示為 Missing Value (遺失值)
Note¶
.replace( A , B ) : 把資料中 A 取代成 B¶
將 Temperature_dataframe2 中 Bad 取代成 Not Good
In [70]:
Temperature_dataframe2.replace( 'Bad' , 'Not Good' )
Out[70]:
Answers¶
Note 將 2002 - 2011 轉成數值型態¶
利用 replace( ) ， 把“-”取代成 np.NaN
In [71]:
Aboard = Aboard.replace('-', np.NaN)
In [72]:
Aboard.head(2)
Out[72]:
Answers¶
Note 將 2002 - 2011 轉成數值型態¶
重新檢查 Aboard 欄位資料類型
In [73]:
Aboard.dtypes
Out[73]:
Note¶
pd.to_numeric( ) : 把資料轉換成數值¶
In [74]:
test_data = pd.Series(['1.0', '2', -3])
test_data.dtypes
Out[74]:
In [75]:
test_data = pd.to_numeric(test_data)
test_data.dtypes
Out[75]:
Answers¶
Note 將 2002 - 2011 轉成數值型態¶
Exercise q. 利用 for 迴圈，處理 2002-2011 的資料¶
Example.
Aboard['2003'] = pd.to_numeric(Aboard['2003'].str.replace(',', ''))
Answers¶
Note 將 2002 - 2011 轉成數值型態¶
Answers ¶
In [76]:
for i in Aboard.columns[2:12]:
Aboard[i] = pd.to_numeric(Aboard[i].str.replace(',', ''))
In [77]:
Aboard.dtypes
Out[77]:
Answers¶
3. 資料中各個欄位的 Descriptive Statistics¶
Aboard 連續型資料的敘述統計
In [78]:
Aboard.describe().astype(int)
Out[78]:
Answers¶
3. 資料中各個欄位的 Descriptive Statistics¶
Aboard 類別型資料的敘述統計
In [79]:
Aboard['Continent'].value_counts()
Out[79]:
Reshaping¶
Note¶
參數 axis=0 : 列、縱向(上下) vertical
參數 axis=1 : 行、橫向(左右) horizontal
In [80]:
Aboard_T = Aboard.T
In [81]:
Aboard_T.head(2)
Out[81]:
In [82]:
Aboard_T.shape
Out[82]:
In [83]:
Aboard.sort_index(axis=0,ascending=False)
Out[83]:
In [84]:
Aboard.sort_index(axis=1,ascending=True)
Out[84]:
In [85]:
Aboard.sort_values(by='2013',ascending=True)
Out[85]:
Selection¶
Slicing Subsets of Rows and Columns
- Using Column Name (Labels)
.loc：依照資料的索引值(標籤) ( column / index ) 選取.iloc( )：依照資料的位置選取- Boolean 選取
In [86]:
Aboard.Country
Out[86]:
In [87]:
Aboard['Country']
Out[87]:
In [88]:
Aboard[['Country','Continent']]
Out[88]:
Selection¶
Note¶
In [89]:
Aboard.head(2)
Out[89]:
In [90]:
Aboard.index
Out[90]:
Selection¶
Note¶
改變 Aboard 的 index
In [91]:
Aboard.index = Aboard['Country'].str.extract('^(?P<Country>[^\s]+)[\.\w\s]+$',expand=False)
del Aboard.index.name
In [92]:
Aboard.index
Out[92]:
In [93]:
Aboard.loc[:,'Country']
Out[93]:
In [94]:
Aboard.loc['香港']
Out[94]:
In [95]:
Aboard.loc['香港',['2002','2003']].astype(int)
Out[95]:
In [96]:
Aboard.iloc[:,1]
Out[96]:
In [97]:
Aboard.iloc[0,2:4]
Out[97]:
In [98]:
Aboard.loc[Aboard.loc[:,'2002']==0,'2002']
Out[98]:
Original
In [99]:
Aboard.loc['英國','2013']
Out[99]:
New
In [100]:
Aboard.loc['英國','2013'] = 34567
In [101]:
Aboard.loc['英國','2013']
Out[101]:
In [102]:
Aboard.loc['總計',:] = np.NaN
In [103]:
print(Aboard.loc['總計',:])
Setting Value¶
Note¶
更新 Aboard 中總計的資料
In [104]:
Aboard.loc['總計',:] = np.array([np.NaN,np.NaN,7319466,5923072,7780652,8208125,8671375,8963713,8465212,8145784,9416184,9584111,10241586,11087488])
In [105]:
Aboard.loc['總計',:]
Out[105]:
Exercise¶
Q. 現在有另一個檔案 Aboard_2.xlsx¶
- 讀入 Aboard_2.xlsx，並存一個名叫 Aboard_2 的 DataFrame
- 將 Aboard_2 轉置
- 改變 Aboard_2 的 column name ( 依序為
2013 2014 2015 2016)可以利用
Aboard_2.loc['Year',:].values的值
Answers¶
讀入 Aboard_2.xlsx
In [106]:
Aboard_2 = pd.read_excel('/Users/hsinyu/Desktop/Python資料分析/Data/Aboard_2.xlsx')
In [107]:
Aboard_2
Out[107]:
Answers¶
將 Aboard_2 轉置
In [108]:
Aboard_2 = Aboard_2.T
In [109]:
Aboard_2
Out[109]:
Answers¶
改變 Aboard_2 的 column name ( 依序為 2013 2014 2015 2016 )
可以利用
Aboard_2.loc['Year',:].values的值
In [110]:
Aboard_2.columns = Aboard_2.loc['Year',:].values
In [111]:
Aboard_2
Out[111]:
In [112]:
Aboard_2 = Aboard_2.drop(['Year'])
In [113]:
Aboard_2
Out[113]:
參數 axis=0 : 列、縱向(上下) vertical
參數 axis=1 : 行、橫向(左右) horizontal
In [114]:
Aboard_2 = Aboard_2.drop([2013],axis=1)
In [115]:
Aboard_2
Out[115]:
需新增的總計資料如下
In [116]:
total = pd.Series([23685810,26364480,29172200],index=[2014,2015,2016])
In [117]:
total.name = '總計'
In [118]:
total
Out[118]:
In [119]:
Aboard_2 = Aboard_2.append(total)
In [120]:
Aboard_2.tail(3)
Out[120]:
Missing Value¶
.isnull( ): 檢查哪裡是 missing value.notnull( ): 檢查哪裡不是 missing value.fillna( ): 填補(取代) missing value.dropna(): 刪除 missing value
In [121]:
Aboard.loc['柬埔寨','2002':'2013'].isnull()
Out[121]:
In [122]:
Aboard.loc['柬埔寨','2002':'2013'].notnull()
Out[122]:
In [123]:
Aboard.loc['大陸',:]
Out[123]:
In [124]:
Aboard.loc['大陸',:] = Aboard.loc['大陸',:].fillna(value=0)
In [125]:
Aboard.loc['大陸',:]
Out[125]:
In [126]:
Aboard
Out[126]:
參數 axis=0 : 列、縱向(上下) vertical
參數 axis=1 : 行、橫向(左右) horizontal
In [127]:
Aboard = Aboard.dropna(axis=0,how='any')
In [128]:
Aboard
Out[128]:
Note¶
Aboard = Aboard.dropna(axis=0,how='any')
how = 'any'：只要其中一筆有就刪除how = 'all'：全部都有才刪除
Combine¶
.concat( )：縱向、橫向合併，可以在axis = 參數指定，預設是axis = 0.merge( )：欲使用不同的合併方式，可以在how = 參數指定，預設是inner join
Combine¶
Note¶
檢查資料
In [129]:
Aboard.shape
Out[129]:
拆資料
In [130]:
Aboard_up = Aboard.iloc[:15, :]
Aboard_down = Aboard.iloc[15:, :]
In [131]:
Aboard_left = Aboard.iloc[:, :7]
Aboard_right = Aboard.iloc[:, 7:]
參數 axis=0 : 列、縱向(上下) vertical
參數 axis=1 : 行、橫向(左右) horizontal
In [132]:
Aboard_vertical = pd.concat([Aboard_up,Aboard_down])
In [133]:
Aboard_vertical.shape
Out[133]:
In [134]:
Aboard_vertical.head(2)
Out[134]:
參數 axis=0 : 列、縱向(上下) vertical
參數 axis=1 : 行、橫向(左右) horizontal
In [135]:
Aboard_horizontal = pd.concat([Aboard_left,Aboard_right],axis=1)
In [136]:
Aboard_horizontal.shape
Out[136]:
In [137]:
Aboard_horizontal.head(2)
Out[137]:
Combine¶
Note¶
SQL Join
Combine¶
Note¶
檢查資料 ( Aboard )
In [138]:
Aboard.shape
Out[138]:
In [139]:
Aboard.index
Out[139]:
Combine¶
Note¶
檢查資料 ( Aboard_2 )
In [140]:
Aboard_2.shape
Out[140]:
In [141]:
Aboard_2.index
Out[141]:
Combine¶
Note¶
建立一個名為 Key 的 column
In [142]:
Aboard = Aboard.iloc[:,0:14]
Aboard.loc[:,'Key'] = Aboard.index
In [143]:
Aboard_2.loc[:,'Key'] = Aboard_2.index
In [144]:
Aboard_inner = pd.merge(Aboard , Aboard_2)
In [145]:
Aboard_inner.shape
Out[145]:
In [146]:
Aboard_inner
Out[146]:
In [147]:
Aboard_outer = pd.merge(Aboard , Aboard_2 ,how='outer')
In [148]:
Aboard_outer.shape
Out[148]:
In [149]:
Aboard_outer
Out[149]:
In [150]:
Aboard_left = pd.merge(Aboard , Aboard_2 ,how='left')
In [151]:
Aboard_left.shape
Out[151]:
In [152]:
Aboard_left
Out[152]:
In [153]:
Aboard_right = pd.merge(Aboard , Aboard_2 ,how='right')
In [154]:
Aboard_right.shape
Out[154]:
In [155]:
Aboard_right
Out[155]:
In [156]:
Aboard_inner.to_csv("/Users/hsinyu/Desktop/Python資料分析/Data/Aboard_new.csv", index = False)
Note¶
Create Demo Data -- Temperature_Year¶
In [157]:
Year = ['2014','2014','2014','2015','2015','2015','2016','2016','2016']
City = ['Taipei','Taichung','Kaohsiung','Taipei','Taichung','Kaohsiung','Taipei','Taichung','Kaohsiung']
Average = [24.48,23.81,25.57,24.07,23.97,24.33,24.19,25.77,25.92]
Max = [30.5,30.1,30.3,30.0,29.7,30.6,30.3,29.4,30.2]
Min = [16.5,16.4,19.5,16.7,17.1,17.3,15.5,16.4,19.3]
Temperature_Year_table = {"Year" : Year,
"City" : City,
"Average" : Average,
"Max" : Max,
"Min" : Min}
Temperature_Year = pd.DataFrame(Temperature_Year_table)
print(Temperature_Year.shape)
Temperature_Year
Out[157]:
Note¶
Create Demo Data -- Temperature_Season¶
In [158]:
Year = ['2014','2014','2014','2014','2015','2015','2015','2015',
'2014','2014','2014','2014','2015','2015','2015','2015',
'2014','2014','2014','2014','2015','2015','2015','2015']
Season = ['Winter','Spring','Summer','Fall','Winter','Spring','Summer','Fall',
'Winter','Spring','Summer','Fall','Winter','Spring','Summer','Fall',
'Winter','Spring','Summer','Fall','Winter','Spring','Summer','Fall']
City = ['Taipei','Taipei','Taipei','Taipei','Taipei','Taipei','Taipei','Taipei',
'Taichung','Taichung','Taichung','Taichung','Taichung','Taichung','Taichung','Taichung',
'Kaohsiung','Kaohsiung','Kaohsiung','Kaohsiung','Kaohsiung','Kaohsiung','Kaohsiung','Kaohsiung']
Average=[16.6,22.2,29.57,25.57,18.8,22.57,29.53,25.37,
16.83,23.23,29.2,25.97,18.27,24.0,28.97,26.1,
20.0,25.43,29.53,27.3,19.87,26.03,29.63,27.57]
Temperature_Season_table = {"Year" : Year,
"City" : City,
"Season" : Season,
"Average" : Average }
Temperature_Season = pd.DataFrame(Temperature_Season_table)
print(Temperature_Season.shape)
Temperature_Season
Out[158]:
Note¶
Create Demo Data -- Temperature¶
In [159]:
Year = ['2014','2014','2014','2014','2014','2014','2014','2014','2014','2014','2014','2014',
'2015','2015','2015','2015','2015','2015','2015','2015','2015','2015','2015','2015',
'2016','2016','2016','2016','2016','2016','2016','2016','2016','2016','2016','2016']
Month = ['JAN','FEB','MAR','APR','MAY','JUN','JUL','AUG','SEP','OCT','NOV','DEC',
'JAN','FEB','MAR','APR','MAY','JUN','JUL','AUG','SEP','OCT','NOV','DEC',
'JAN','FEB','MAR','APR','MAY','JUN','JUL','AUG','SEP','OCT','NOV','DEC']
Season = ['Winter','Winter','Spring','Spring','Spring','Summer','Summer','Summer','Fall','Fall','Fall','Winter',
'Winter','Winter','Spring','Spring','Spring','Summer','Summer','Summer','Fall','Fall','Fall','Winter',
'Winter','Winter','Spring','Spring','Spring','Summer','Summer','Summer','Fall','Fall','Fall','Winter']
Taipei = [16.8,16.5,18.9,22.5,25.2,28.0,30.5,30.2,29.7,24.7,22.3,16.5,
16.7,20.8,18.9,22.7,26.1,30.0,30.0,28.6,27.4,25.2,23.5,18.9,
16.2,15.5,17.5,24.0,27.1,29.4,30.3,30.1,27.8,27.0,22.6,20.1]
Taichung = [16.4,17.0,19.8,24.1,25.8,28.6,30.1,28.9,29.2,25.6,23.1,17.1,
17.1,18.1,20.7,24.4,26.9,29.7,29.2,28.0,27.8,26.2,24.3,19.6,
16.8,16.4,18.3,24.9,27.6,28.8,29.4,28.9,27.9,27.5,23.4,20.4]
Kaohsiung = [19.5,20.3,22.6,25.9,27.8,29.2,30.3,29.1,29.5,27.2,25.2,20.2,
19.9,17.3,23.6,26.2,28.3,30.6,29.5,28.8,28.8,27.7,26.2,22.4,
19.3,19.6,21.6,26.9,28.9,29.7,30.2,29.4,28.4,28.4,25.7,22.9]
Temperature_table = { "Year" : Year,
"Month" : Month,
"Season" : Season,
"Taipei" : Taipei,
"Taichung" : Taichung,
"Kaohsiung" : Kaohsiung }
Temperature = pd.DataFrame(Temperature_table)
print(Temperature.shape)
Temperature.head()
Out[159]:
Reshaping II¶
Note¶
想知道 Temperature_Year 這組資料中，不同城市年均溫的變化狀況
In [160]:
Temperature_Year
Out[160]:
Temperature_Year.pivot_table(index='Year',columns='City',values='Average')
In [161]:
Temperature_Year.pivot_table(index='Year',columns='City')
Out[161]:
Exercise¶
Q. 利用 pivot_table 呈現出 Temperature_Year 這組資料中，各年度在不同城市最大溫度的變化¶
Answers¶
In [162]:
Temperature_Year.pivot_table(index='City',columns='Year',values='Max')
Out[162]:
Reshaping II¶
Note¶
想知道 Temperature_Season 這組資料中，不同城市季均溫的變化狀況
In [163]:
Temperature_Season
Out[163]:
Temperature_Season.pivot_table(index='Season',columns='City',values='Average',aggfunc=np.mean)
Reshaping II¶
Note¶
想知道 Temperature 這組資料中，不同城市在各年度下各個季節的季均溫狀況
In [164]:
Temperature
Out[164]:
Exercise¶
Q. 利用 pivot_table 呈現出 Temperature 這組資料 ， 台北市在各年度下各個季節的季均溫狀況¶
Answers¶
In [165]:
Temperature.pivot_table(index='Season',columns='Year',values='Taipei',aggfunc=np.mean)
Out[165]:
Grouping¶
Note¶
想知道 Temperature 這組資料中，不同城市在各年度下各個季節的季均溫狀況
In [166]:
Temperature
Out[166]:
In [167]:
Group_Year = Temperature.groupby(Temperature['Year'])
In [168]:
Group_Year
Out[168]:
In [169]:
Group_Year_mean = Group_Year.mean()
In [170]:
Group_Year_mean
Out[170]:
Grouping¶
Note¶
確認 Group_Year 的 Type
In [171]:
type(Group_Year_mean)
Out[171]:
In [172]:
Temperature['Taipei'].groupby(Temperature['Year']).mean()
Out[172]:
In [173]:
Temperature.groupby(['Year','Season']).mean()
Out[173]:
Note¶
Create Demo Data -- Temperature_Taipei¶
In [174]:
Month = ['JAN','FEB','MAR','APR','MAY','JUN','JUL','AUG','SEP','OCT','NOV','DEC','JAN']
Taipei = [16.8,16.5,18.9,22.5,25.2,28.0,30.5,30.2,29.7,24.7,22.3,16.5,16.8]
Temperature_Taipei_table = { "Month" : Month,
"Taipei" : Taipei }
Temperature_Taipei = pd.DataFrame(Temperature_Taipei_table)
print(Temperature_Taipei.shape)
Temperature_Taipei
Out[174]:
Binning¶
Note¶
想切分 Temperature_Taipei 這組資料
In [175]:
Temperature_Taipei
Out[175]:
In [176]:
cut_points = [0, 28, float("inf")]
group = ["冷", "熱"]
Temperature_Taipei['Feel'] = pd.cut(Temperature_Taipei["Taipei"], cut_points , labels = group)
In [177]:
Temperature_Taipei
Out[177]:
Duplicate¶
Note¶
想確認 Temperature_Taipei 這組資料的狀況
In [178]:
Temperature_Taipei
Out[178]:
In [179]:
Temperature_Taipei.duplicated()
Out[179]:
In [180]:
Temperature_Taipei = Temperature_Taipei.drop_duplicates()
In [181]:
Temperature_Taipei
Out[181]: