I've been using JULES since 2008 when I started on the work at Univ. Oxford that led to Marthews et al. (2012, GCB). During the first half of 2017 (now at CEH) I rewrote and reorganised the JULES website. As part of this, I created new pages there for all my JULES tutorials (under Training: especially see my tutorial JULES from Scratch) and a complete set of pages about how to get started using the model (under Getting Started).

Below are the remaining tools and tips that I regarded as a bit too 'developmental' to put on the official JULES website (at least for now).

Currently this page here is only about DATA VISUALISATION and GRID CONVERSION OPTIONS, but I've left the title as 'JULES Tools' so that I can make it more general in the future (I hope!). Also, I should stress that I'm not a Python user or an Iris user: I prefer using R as much as possible and this (obviously) influences what tools I find useful below.

Some tools available to CEH internal people only are [ask Toby for these].

For general information about Data Visualisation, please see Emma Robinson's talk about Interpreting JULES Output at the 2016 JULES Short Course here and her examples in Python and R here.

- USING 2D NETCDF FILES OUTPUT FROM JULES: Here are some links for dealing with 'normal' 2D NetCDF files (see also here).

   - On UNIX/Linux people seem to use GrADS (see also here and here), Xconv,  Iris (see also Cube Browser and Thea) or Ncview. Note: Xconv can show pp files, which ncview can’t, and also convert to netcdf if required (though the unix tool pptoncdf is generally better for that)

   - On Windows you can use Panoply (NASA's quick, basic viewer) or a GIS package like QGIS (see also here; has a lot of advantages if you want to compare JULES output to other raster or vector data, e.g. to calculate land surface fluxes over an irregular area like a river catchment).

   - You want to animate your NetCDF files, e.g. convert to a movie to include in a presentation? See here.

- USING 1D NETCDF FILES OUTPUT FROM JULES (i.e. where there is a 'land axis'): Gridded runs from JULES can output data in 2D or 1D NetCDF files. Most packages for displaying gridded data can handle 2D (see above), but very few can handle 1D, so what to do with that? There are several options:

   - On UNIX/Windows you can always of course write a tailor-made script to read the 1D Netcdf file into an array and then plot it. I've done this a few times now, but for me has two important disadvantages: (1) Firstly, for large files you will hit memory constraints very quickly on a script. Moving to Fortran or C removes the memory limitation, but you then have much more limited graphics for displaying the data. (2) Secondly, it's very difficult to code something that is 'general' in the sense that it can parse the input file and read in the data irrespective of how many variables you have (I usually have 50-60 in my JULES outputs), how many nonspatial dimensions it has (e.g. scpool, ntiles), what they are called (lat, latitude, latitude0, Latitude, lat_data, etc.) so you usually end up with something less than general and that means you need to recode it completely when you move on to the next application.

   - On UNIX/Linux you can use GrADS. You can display a 1D NetCDF file in GrADS if you can generate a .pdef file that correctly describes the grid of the datafile you have. See here for how to construct a pdef file for gridded output. I'm also told that you can get GrADS to output a 2D NetCDF file, which would mean that it can be used as a 1D -> 2D converter as well, but I don't know how to do that (Does anyone know more?).

For grid conversions 1D <-> 2D see some of the comments here, but basically I see only 3 real options:

1) You can write a bespoke Python or R script to read in the 1D .nc file and output a 2D .nc file. I've done this a couple of times and the biggest problems are (a) you run out of memory for even reasonably-sized NetCDF files and (b) it is very difficult to write a routine like this that is general enough that you can reuse it on a different project, so effectively to have to write a bespoke script each time.

2) You can load into ArcGIS and get ArcGIS to convert to 2D. Marcus Buechel kindly provided these tips in August 2019: "Converting the 1D output files produced by JULES when land_only=.true. into a more conventional 2D raster grid can be achieved using ArcMap. As far as I am aware the following tools have been available in ArcMap since version 10.3. As the output files are not produced on a regular latitude/longitude grid, you have to use the Make NetCDF Feature Layer in the Multidimension Toolbox, rather than Make NetCDF Raster Layer. The inputs are such:

1. Input the NetCDF file you wish to convert/visualise.
   
2. Select the variable(s) that you want to investigate.
   
3. Select ‘longitude’ as the X Variable and ‘latitude’ as the Y Variable.
   
4. Give your output file an appropriate name.
   
5. For the Row Dimensions box, select ‘x’. This is very important!
   
6. Finally select ‘time’ in the Dimension Values box.
   
7. Click OK to run the tool.
   

This should have produced a swath of feature points that represent all the land points in the JULES run. To convert these points into a raster, use the Feature to Raster tool in the Conversion Toolbox. Simply input the feature layer produced with the first tool, the field/variable you want to visualise, where you want the raster to be saved and the output cell size (which will depend on the spatial scale of the model). Hey presto, this will produce a raster on a 2D grid of the output data from JULES. Hope that helps. If you want it any more or less clearer, let me know!". Many thanks to Marcus for this one!


3) During August 216 I had the idea of doing grid conversions in a completely different way: I have used the NCO tools to 'atomise' the input file (e.g. with n spatial points) into n separate .nc files, each containing only the data for a single point. The NCO tools will preserve all nonspatial dimensions automatically if you extract each point separately using ncea. My R script keeps track of the spatial locations of all these files and then remerges them into a 2D grid using ncecat. Essentially, my script this time doesn't read in any data, but produces a .sh file with a sequence of (long) NCO commands and then I sh all those commands on UNIX to generate the 2D netcdf file. No memory limitations this time and it's 'general' because the NCO tools do the work. It's a bit slow (a file with 6604 land points took 3.5 hours to convert), but I can then use exactly the same script to convert another netcdf file of completely different nonspatial structure and I don't need to modify it, which is nice. Also, the conversion only needs to be done once and then I have a 2D netcdf file and I can view this in lots of different packages. THIS OPTION HAS ONLY BEEN PARTIALLY TESTED BUT YOU'RE WELCOME TO USE IT AND TELL ME HOW IT WORKS: Download here. Current limitations: only works with WFD or WFDEI data.

Just for information: in the future the Met Office (and therefore JULES) will be moving on to a more complex grid for at least some of its runs. For more information, see LFric links here and here and UM ticket here.