Where can I source the ancillary files I need for my JULES runs?
|
The advice I published on the JULES website about where to acquire ancillary data here is correct as far as it goes *, but I didn't give much detail on where JULES users routinely obtain their ancillary files.
Generally, JULES users use ancillary files obtained from one of three sources: (1) Generated using the ANTS ** system (available on Jasmin since March 2020). Note that JULES does not (yet) have a repository of pre-generated ancillary files available for download: you have to run ANTS yourself for each use case. (2) A bespoke script written to generate them, which you generally must write yourself *** (often, the current ANTS code can provide a starting point for this). (3) Asking around for help via personal contacts (i.e. emailing colleagues/collaborators and asking what files have been used on other projects). Of course, we do understand that this is pretty difficult if you do not yet have colleagues or collaborators (!). Additionally, given that available ancillary files for JULES have generally been created at the UK Met Office (although a few are now on the EIDC), your contacts generally need to be at the UK Met Office. |
|
My experience of these options:
(1) is tough: using ANTS is not straightforward and, although I have asked around quite a bit recently, I have not found anyone outside the UK Met Office who uses it regularly (perhaps people do use it but they know that if they admit this then I will ask them to help me out, however, so I can't be sure there is noone!).
(2) is more difficult than (1) (and, in my experience, takes at least several months) and
(3) is, I believe, the option of choice, which necessitates relentlessly emailing and chasing people who you think might be able to help you. This has got a bit easier with the advent of the Rose system because you can now access previously-used model configurations (Rose Suites) via Rosie Go, see what files were used there and email the suite owner. Some luck is required, however, and it is not unlikely that this search turns out to be fruitless for the particular needs of your project.
Finally, even if you do manage to acquire files obtained from (1), (2) or (3), they need to be checked thoroughly for consistency because JULES will abort with inconsistent ancillaries (e.g. if there are points that have both soil data and land_ice data: see line 972 of src/initialisation/standalone/initial_conditions/init_ic.inc where it states "Check that all land points have been identified as either soil or ice").
In previous projects I followed (3) (I started using JULES in ~2010 and option (1) above only became possible in 2020), but I was plagued by uncorrectable inconsistencies in the files I was supplied by colleagues. This meant I wrote a set of scripts to check, correct and harmonise those files, but when I started trying to move from 0.50 to 0.25 degree global resolution in 2020ish, I found that I could not source files of good enough quality. In 2021 I decided to change approach. Because I found ANTS too difficult to use (1), I decided to invest the time to follow (2), which led me to write the scripts I describe here.
(1) is tough: using ANTS is not straightforward and, although I have asked around quite a bit recently, I have not found anyone outside the UK Met Office who uses it regularly (perhaps people do use it but they know that if they admit this then I will ask them to help me out, however, so I can't be sure there is noone!).
(2) is more difficult than (1) (and, in my experience, takes at least several months) and
(3) is, I believe, the option of choice, which necessitates relentlessly emailing and chasing people who you think might be able to help you. This has got a bit easier with the advent of the Rose system because you can now access previously-used model configurations (Rose Suites) via Rosie Go, see what files were used there and email the suite owner. Some luck is required, however, and it is not unlikely that this search turns out to be fruitless for the particular needs of your project.
Finally, even if you do manage to acquire files obtained from (1), (2) or (3), they need to be checked thoroughly for consistency because JULES will abort with inconsistent ancillaries (e.g. if there are points that have both soil data and land_ice data: see line 972 of src/initialisation/standalone/initial_conditions/init_ic.inc where it states "Check that all land points have been identified as either soil or ice").
In previous projects I followed (3) (I started using JULES in ~2010 and option (1) above only became possible in 2020), but I was plagued by uncorrectable inconsistencies in the files I was supplied by colleagues. This meant I wrote a set of scripts to check, correct and harmonise those files, but when I started trying to move from 0.50 to 0.25 degree global resolution in 2020ish, I found that I could not source files of good enough quality. In 2021 I decided to change approach. Because I found ANTS too difficult to use (1), I decided to invest the time to follow (2), which led me to write the scripts I describe here.
* Although I believe now that I didn't stress enough there just how much of a roadblock this issue can be: if you don't have the right ancillaries, you will not be able to run JULES at all.
** ANTS is the UK Met Office's in-house, Python-based ancillary generation system (replacing the CAP, which retired in 2017). ANTS is an impressive system and a lot of work has gone into its development (see user guide here and code here), however as I put on the JULES website here, it is not yet a resource that can be used without expertise and training is not yet available.
*** Note that this requires (a) Familiarity with NetCDF files, standard grid formats and how to convert between them, (b) very good skills in scripting (NetCDF files can be generated using Fortran, Python, R, exported from ArcGIS or in many other ways) and (c) familiarity with online sources of appropriate input data (e.g. a high-resolution DEM). If you do not have this expertise, then you will need to follow option 1 and seek a collaborator at the UK Met Office.