A lot of my work involves dealing with spatial data: usually 2D, gridded datafiles in NetCDF format (or possibly GeoTIFF). The data can come from observations (e.g. EO data) or from model runs (e.g. JULES), and I have got used to using a wide variety of software to visualise these datasets (e.g. QGIS, ArcGIS, NCView, Panoply).
I'll go through some basics here quickly, then my visualisation tips are below. |
Some basics:
|
VISUALISE JULES OUTPUTS:
When run over a spatial grid, JULES can give output in either 1D ('land axis') or 2D formats, but unfortunately *** neither of these formats will load directly into a GIS package ***:
FORMATTING JULES 2D OUTPUT FOR GIS PACKAGES:
I'll assume here that the JULES output file is $FILE0 (generated from a run of the model using ancillary file $ANCILF), and the reformatted version will be called $FILE1. Run these lines on a Linux server (or similar):
export FILE0='$SCR/Almir/Almirante.Daily.2010.nc'
export FILE1='$SCR/Almir/Almirante.Daily_withdimvar.2010.nc'
export ANCILF='~/ancils/HSAncil_Almirante_1.0km_E2O_wh_dimrenamed.nc'
#Change the units of latitude and longitude (the 2d variables) from "degrees" to "degrees_north" and "degrees_east" and rename those 2d variables to "longitude_2d" and "latitude_2d".
ncatted -Oh -a units,latitude,o,c,"degrees_north" $FILE0 $FILE1
ncatted -Oh -a units,longitude,o,c,"degrees_east" $FILE1 $FILE1
#Rename the dimensions from x,y to lon,lat and add in corresponding dimension variables from the original ancillary file (which JULES doesn't output for them)
ncks -Oh -3 $FILE1 $FILE1
ncrename -Oh -v longitude,longitude_2d -v latitude,latitude_2d $FILE1
ncrename -Oh -d x,lon -d y,lat $FILE1
ncks -Ah -v lon $ANCILF $FILE1
ncks -Ah -v lat $ANCILF $FILE1
#The adding in of the proj4 attribute to lon and lat I think is an old standard that I think no longer applies, so I have commented those out
#ncatted -Oh -a proj4,lon,o,c,"+proj=longlat +datum=WGS84 +no_defs" $FILE1 $FILE1
#ncatted -Oh -a proj4,lat,o,c,"+proj=longlat +datum=WGS84 +no_defs" $FILE1 $FILE1
#I copy in the crs variable from $ANCILF here, though n.b. the extent attribute of crs will not be correct for non-global runs (it will still be for a global extent).
ncks -Ah -v crs $ANCILF $FILE1
#Even better would be to have the CRS information conform to the latest version of CF (1.5 at the moment I think, which is later than the format of the crs variable above), which I could get by running this command: gdal_translate -a_srs EPSG:4326 -of netCDF $FILE1 $FILE2
ncks -Oh -4 $FILE1 $FILE1
#I think it's better practice to fix the time dimension too (JULES leaves it as 'UNLIMITED' in extent, which means some software can't read the output file). Mysteriously, this seems to increase the file size by 80%, though (which is perhaps why JULES leaves it like it does).
ncks -Oh --fix_rec_dmn time $FILE1 $FILE1
OK: those commands should have successfully reformatted $FILE0 into $FILE1. Now you can load $FILE1 into a GIS package (see here).
When run over a spatial grid, JULES can give output in either 1D ('land axis') or 2D formats, but unfortunately *** neither of these formats will load directly into a GIS package ***:
- If your output is a 1D file, I would recommend immediately convert it to 2D following the steps here: in my opinion, the time you lose writing bespoke scripts for displaying a particular 1D file (although possible), will not be outweighed by the benefits. Just convert to 2D and go to the next point.
- If your output is a 2D file, then great, but JULES's default is not to include the longitude and latitude information in the output file (it has e.g. x running from 1 to 1440, but the corresponding longitude numbers are not in the file). This means that the file cannot be directly geolocated in a GIS package. Below are my steps for sorting this out:
FORMATTING JULES 2D OUTPUT FOR GIS PACKAGES:
I'll assume here that the JULES output file is $FILE0 (generated from a run of the model using ancillary file $ANCILF), and the reformatted version will be called $FILE1. Run these lines on a Linux server (or similar):
export FILE0='$SCR/Almir/Almirante.Daily.2010.nc'
export FILE1='$SCR/Almir/Almirante.Daily_withdimvar.2010.nc'
export ANCILF='~/ancils/HSAncil_Almirante_1.0km_E2O_wh_dimrenamed.nc'
#Change the units of latitude and longitude (the 2d variables) from "degrees" to "degrees_north" and "degrees_east" and rename those 2d variables to "longitude_2d" and "latitude_2d".
ncatted -Oh -a units,latitude,o,c,"degrees_north" $FILE0 $FILE1
ncatted -Oh -a units,longitude,o,c,"degrees_east" $FILE1 $FILE1
#Rename the dimensions from x,y to lon,lat and add in corresponding dimension variables from the original ancillary file (which JULES doesn't output for them)
ncks -Oh -3 $FILE1 $FILE1
ncrename -Oh -v longitude,longitude_2d -v latitude,latitude_2d $FILE1
ncrename -Oh -d x,lon -d y,lat $FILE1
ncks -Ah -v lon $ANCILF $FILE1
ncks -Ah -v lat $ANCILF $FILE1
#The adding in of the proj4 attribute to lon and lat I think is an old standard that I think no longer applies, so I have commented those out
#ncatted -Oh -a proj4,lon,o,c,"+proj=longlat +datum=WGS84 +no_defs" $FILE1 $FILE1
#ncatted -Oh -a proj4,lat,o,c,"+proj=longlat +datum=WGS84 +no_defs" $FILE1 $FILE1
#I copy in the crs variable from $ANCILF here, though n.b. the extent attribute of crs will not be correct for non-global runs (it will still be for a global extent).
ncks -Ah -v crs $ANCILF $FILE1
#Even better would be to have the CRS information conform to the latest version of CF (1.5 at the moment I think, which is later than the format of the crs variable above), which I could get by running this command: gdal_translate -a_srs EPSG:4326 -of netCDF $FILE1 $FILE2
ncks -Oh -4 $FILE1 $FILE1
#I think it's better practice to fix the time dimension too (JULES leaves it as 'UNLIMITED' in extent, which means some software can't read the output file). Mysteriously, this seems to increase the file size by 80%, though (which is perhaps why JULES leaves it like it does).
ncks -Oh --fix_rec_dmn time $FILE1 $FILE1
OK: those commands should have successfully reformatted $FILE0 into $FILE1. Now you can load $FILE1 into a GIS package (see here).