歐洲中期天氣預報中心再分析數據之下載

Table of contents

背景
- EAC4 數據庫基本資訊
- 作業流程
下載、轉檔、橫向整併
Reference

背景

歐洲中期天氣預報中心(ECMWF)之EAC4 (ECMWF Atmospheric Composition Reanalysis 4)長期蒐集、分析、重組全球的大氣成分數據，並以目前所理解的物化方程式予以同化，行成所謂「再分析」數據庫，並且對外提供。
其數據目前已建有與WRF-chem間的應用平台，以進行空氣品質的預報系統。也經常用在區域性的空氣品質解析(如Galmarini et al. 2021、Jeong and Hong 2021)。

EAC4 數據庫基本資訊

item	content
Data type	Gridded
Horizontal coverage	Global
Horizontal resolution	0.75°x0.75°
Vertical coverage	Surface, total column, model levels and pressure levels.
Vertical resolution	60 model levels. Pressure levels: 1000, 950, 925, 900, 850, 800, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, 20, 10, 7, 5, 3, 2, 1 hPa
Temporal coverage	2003 to 2021
Temporal resolution	3-hourly
File format	GRIB (optional conversion to netCDF)
Versions	Only one version
Update frequency	Twice a year with 4-6 month delay

作業流程

須先在ECMWF哥白尼倉庫註冊帳密(免費)。在User Profile處取得將API Key，將其寫進電腦的${HOME}/.cdsapirc檔案內，範例如下：

$ cat ~/.cdsapirc
url: https://ads.atmosphere.copernicus.eu/api/v2
key: 2556:e6b523da-4703-4dc7-bcfd-94cb2d8ed395 

下載cdsapi軟件：pip install cdsapi
到eac4網址勾選下載項目
在網址上選好選項後，點選「Show API request」複製retrieve{…}內容，貼在python程式內，以python程式進行下載。
下載完成後繼續轉變格式(ncl_convert2nc)、將nc檔案內容填入m3.nc檔案備用(進一步檢查)

下載、轉檔、橫向整併

空品數據之下載(get_d2.py)

這支程式與所用的模組基本上是哥白尼倉庫的提供的工具，但要下載全月的檔案，還是需要一些加工。
dt2jul是時間轉換常用的模組，地球科學上常用Juian date，這個模組力求簡單方便。

import cdsapi
import datetime
import os
c = cdsapi.Client()

def dt2jul(dt):
  yr=dt.year
  deltaT=dt-datetime.datetime(yr,1,1)
  deltaH=int((deltaT.total_seconds()-deltaT.days*24*3600)/3600.)
  return (yr*1000+deltaT.days+1,deltaH*10000)

SPECs及PARTs是再分析模式的空氣品質項目名稱，常用名稱如下
- 名稱的來源除了在eac4勾選網址之外，也可以在CAMS: Reanalysis data documentation內找到全部列表與說明。
- 這些名稱與CMAQ-cb6r3_ae7_aq之對照詳grb2D1m3RHO.py

SPECs =['carbon_monoxide', 'ethane', 'formaldehyde', 'isoprene', 'nitrogen_dioxide', 'nitrogen_monoxide', 'propane', 'sulphur_dioxide' ]
SPECs+=['ozone', 'ammonium', 'nitrate', 'olefins', 'organic_nitrates', 'paraffins']
PARTs =[
            'dust_aerosol_0.03-0.55um_mixing_ratio', 'dust_aerosol_0.55-0.9um_mixing_ratio', 'dust_aerosol_0.9-20um_mixing_ratio',
            'hydrophilic_black_carbon_aerosol_mixing_ratio', 'hydrophilic_organic_matter_aerosol_mixing_ratio', 'hydrophobic_black_carbon_aerosol_mixing_ratio',
            'hydrophobic_organic_matter_aerosol_mixing_ratio', 'nitric_acid', 'peroxyacetyl_nitrate',
            'sea_salt_aerosol_0.03-0.5um_mixing_ratio', 'sea_salt_aerosol_0.5-5um_mixing_ratio', 'sea_salt_aerosol_5-20um_mixing_ratio',
            'sulphate_aerosol_mixing_ratio',
        ]

按照空品項目、年代、月份「依序」下載
- 「不建議」同時、多線下載。系統可能會視為機器人攻擊。
- 以前月15日後的16日(run5)0時起算，到run12最末日21時結束
- 符合WRF批次定義的考量是因為CMAQ的BCON檔案會跟隨此一定義批次的起迄時間，而此處下載空品數據正是為形成或補充該BCON檔案的內容。

iyr=2019
for sp in SPECs+PARTs:
  for m in range(1,13):
    mo='{:02d}'.format(m)
    fname=sp+'_'+str(iyr)[2:]+mo+'.grib'
    if os.path.exists(fname):continue
    lastYr=(datetime.datetime(iyr,m,1)+datetime.timedelta(days=-1)).year
    lastMo=(datetime.datetime(iyr,m,1)+datetime.timedelta(days=-1)).month
    beg0=datetime.datetime(lastYr,lastMo,15)
    begd=beg0+datetime.timedelta(days=4*4)
    endd=beg0+datetime.timedelta(days=12*4)
    begd=str(begd.strftime("%Y-%m-%d"))
    endd=str(endd.strftime("%Y-%m-%d"))

以哥白尼倉庫提供的模組進行下載
- 層數model_level的定義詳見L60 model level definitions
- 時間：逐3小時，0時開始

    c.retrieve(
    'cams-global-reanalysis-eac4',
    {
    'model_level': ['21', '22', '23', '24', '25', '26', '27',
      '28', '29', '30', '31', '32', '33', '34', '35', '36', '37', '38',
      '39', '40', '42', '43', '44', '46', '47', '48', '49', '50', '51',
      '53', '54', '56', '57', '59'],
    'variable': sp,
    'date': begd+'/'+endd,
    'time': [
      '00:00', '03:00', '06:00',
      '09:00', '12:00', '15:00',
      '18:00', '21:00',
        ],

範圍：如果沒有界定範圍，檔案會很大。四圍經緯度邊界之順序為北、西、南、東，逆時針方向給定。
- d00範圍：北緯-10~50、東經60~180。'area': [50, 60, -10, 180,],
- d01範圍：北緯-2~47、東經80~160。'area': [47, 80, -2, 160,],
- d02範圍：北緯15~32、東經111~131。'area': [32, 111, 15, 131, ],

    'area': [
      32, 111, 15,
      131,
      ],
    'format': 'grib',
    },
     fname)  

轉檔

因為CMAQ、CAMx等空氣品質模式目前還沒有發展grib2格式的IO方式，如果要以VERDI檢視結果，還是要轉成nc格式比較方便。
雖然勾選網址有提供nc檔案格式的選項，但為嘗試性結果。此處還是以ncl_stable軟件包的ncl_convert2nc來進行轉檔。

for g in $(ls *grib);do 
  /opt/miniconda3/envs/ncl_stable/bin/ncl_convert2nc $g
done

warning
- NclGRIB2: codetable file “/opt/miniconda3/envs/ncl_stable/lib/ncarg/grib2_codetables/ecmwf/4/4.2.192.210.table” not a valid GRIB2 code table.
- Entry (192) not found in code table file /opt/miniconda3/envs/ncl_stable/lib/ncarg/grib2_codetables/ecmwf/0/0.0.table
- Entry (192) not found in code table file /opt/miniconda3/envs/ncl_stable/lib/ncarg/grib2_codetables/ecmwf/4/4.2.table
- Classic model NetCDF does not support string types, converting initial_time0 to a character array
- Dimension ‘ncl_strlen_0’ will be added

橫向合併

這項作業是將同一月份的27個分項檔案，按照相同的時間、空間軸整併成一個檔案。
做法：先用ncks -v取出空品濃度的矩陣，再使用ncks -A逐一附加即可。

y=$(n=$(( ${#PWD} - 1 ));echo $PWD|cut -c${n}-)
for m in {01..12};do
  fn=$y$m.nc
  if [ -e $fn ];then continue;fi
  cp ammonium_$y$m.nc $fn
  ncks -O --mk_rec_dmn initial_time0_hours $fn tmp.nc
  mv tmp.nc $fn
  for nc in $(ls *_$y$m.nc|grep -v ammonium);do
    var=$(ncdump -h $nc|grep float|grep P0_L|awkk 2|cut -d'(' -f1)
    ncks -O -v $var --mk_rec_dmn initial_time0_hours $nc tmp.nc
    ncks -A tmp.nc $fn
  done
done

污染項目的對照

eac4grib2檔案使用代碼而非污染項目之名稱，因此需要建立對照表。

for nc in $(ls *_1901.nc);do 
  var=$(ncdump -h $nc|grep float|grep P0_L|awkk 2|cut -d'(' -f1)
  spec=${nc/_1901.nc/}
  echo \"${var}\":\"${spec}\",
done

結果如下，將會用在grb2D1m3RHO.py

{
"VAR_192_217_21_P0_L105_GLL0":"ammonium_1902.nc",
"VAR_192_210_123_P0_L105_GLL0":"carbon_monoxide_1902.nc",
"VAR_192_210_4_P0_L105_GLL0":"dust_aerosol_0.03-0.55um_mixing_ratio_1902.nc",
"VAR_192_210_5_P0_L105_GLL0":"dust_aerosol_0.55-0.9um_mixing_ratio_1902.nc",
"VAR_192_210_6_P0_L105_GLL0":"dust_aerosol_0.9-20um_mixing_ratio_1902.nc",
"VAR_192_217_45_P0_L105_GLL0":"ethane_1902.nc",
"VAR_192_210_124_P0_L105_GLL0":"formaldehyde_1902.nc",
"VAR_192_210_9_P0_L105_GLL0":"hydrophilic_black_carbon_aerosol_mixing_ratio_1902.nc",
"VAR_192_210_7_P0_L105_GLL0":"hydrophilic_organic_matter_aerosol_mixing_ratio_1902.nc",
"VAR_192_210_10_P0_L105_GLL0":"hydrophobic_black_carbon_aerosol_mixing_ratio_1902.nc",
"VAR_192_210_8_P0_L105_GLL0":"hydrophobic_organic_matter_aerosol_mixing_ratio_1902.nc",
"VAR_192_217_16_P0_L105_GLL0":"isoprene_1902.nc",
"VAR_192_217_51_P0_L105_GLL0":"nitrate_1902.nc",
"VAR_192_217_6_P0_L105_GLL0":"nitric_acid_1902.nc",
"VAR_192_210_121_P0_L105_GLL0":"nitrogen_dioxide_1902.nc",
"VAR_192_217_27_P0_L105_GLL0":"nitrogen_monoxide_1902.nc",
"VAR_192_217_11_P0_L105_GLL0":"olefins_1902.nc",
"VAR_192_217_15_P0_L105_GLL0":"organic_nitrates_1902.nc",
"VAR_192_210_203_P0_L105_GLL0":"ozone_1902.nc",
"VAR_192_217_9_P0_L105_GLL0":"paraffins_1902.nc",
"VAR_192_217_13_P0_L105_GLL0":"peroxyacetyl_nitrate_1902.nc",
"VAR_192_217_47_P0_L105_GLL0":"propane_1902.nc",
"VAR_192_210_1_P0_L105_GLL0":"sea_salt_aerosol_0.03-0.5um_mixing_ratio_1902.nc",
"VAR_192_210_2_P0_L105_GLL0":"sea_salt_aerosol_0.5-5um_mixing_ratio_1902.nc",
"VAR_192_210_3_P0_L105_GLL0":"sea_salt_aerosol_5-20um_mixing_ratio_1902.nc",
"VAR_192_210_11_P0_L105_GLL0":"sulphate_aerosol_mixing_ratio_1902.nc",
"VAR_192_210_122_P0_L105_GLL0":"sulphur_dioxide_1902.nc",
}

Reference

ECMWF, EAC4 (ECMWF Atmospheric Composition Reanalysis 4), copernicus,record updated 2021-12-07 16:10:05 UTC
Helen Setchell, Carsten Maass, L60 model level definitions, confluence.ecmwf,修改于五月 22, 2019
Galmarini, S., Makar, P., Clifton, O., Hogrefe, C., Bash, J., Bianconi, R., Bellasio, R., Bieser, J., Butler, T., Ducker, J., Flemming, J., Hozdic, A., Holmes, C., Kioutsioukis, I., Kranenburg, R., Lupascu, A., Perez-Camanyo, J.L., Pleim, J., Ryu, Y.-H., San Jose, R., Schwede, D., Silva, S., Garcia Vivanco, M., and Wolke, R. (2021). Technical Note – AQMEII4 Activity 1: Evaluation of Wet and Dry Deposition Schemes as an Integral Part of Regional-Scale Air Quality Models (preprint). Gases/Atmospheric Modelling/Troposphere/Physics (physical properties and processes).
Jeong, U. and Hong, H. (2021). Assessment of Tropospheric Concentrations of NO2 from the TROPOMI/Sentinel-5 Precursor for the Estimation of Long-Term Exposure to Surface NO2 over South Korea. Remote Sensing 13 (10):1877. doi:10.3390/rs13101877.