Where can I get ancillary files for my JULES runs?
The advice I published on the JULES website about where to acquire ancillary data here is correct as far as it goes *, but I didn't give much detail on where JULES users routinely obtain their ancillary files.
Generally, JULES users use ancillary files obtained from one of three sources:
(1) Generated yourself using the ANTS ** system (available on Jasmin since March 2020).
(2) A bespoke script written to generate them, which you will have to write yourself *** from scratch.
(3) Asking around for help, i.e. emailing colleagues/collaborators and asking what files have been used on other projects. Of course, this is pretty difficult if you do not have colleagues or collaborators (!). Additionally, given that available ancillary files for JULES have generally been created at the UK Met Office (although a few are now on the EIDC), your contacts generally need to be at the UK Met Office.
Finally, n.b. you will have to check thoroughly any files obtained this way, because JULES will generally abort with inconsistent ancillaries (e.g. if there are points that have both soil data and land_ice data).
On the whole:
(1) is tough: using ANTS is not straightforward and I don't know anyone outside the UK Met Office who does so.
(2) is even tougher than (1) (very technically demanding and time-consuming) and
(3) I believe is the option of choice, which necessitates relentlessly emailing and chasing people who you think might be able to help you. Some luck is required, however, and you may well find that this search turns out to be fruitless for your project.
In previous projects I followed (3) but found that there were too many uncorrectable inconsistencies in the files I managed to obtain. Having written some scripts to check and correct those files, in 2021 I decided to stop losing so much time doing those checks and decided to write the scripts I describe here.
* Although I think now that I didn't stress enough there just how much of a cataclysmic roadblock this issue can be: if you don't have the right ancillaries, you will not be able to run JULES at all and this can be a hugely significant block to the progress of any JULES-based project.
** ANTS is the UK Met Office's in-house, Python-based ancillary generation system (replacing the CAP, which retired in 2017). ANTS is an impressive system and a lot of work has gone into its development (see user guide here and code here), however as I put on the JULES website here, it is not yet a resource that can be used without expertise and training is not yet available.
*** Note that this requires (a) Familiarity with NetCDF files, standard grid formats and how to convert between them, (b) very good skills in scripting (NetCDF files can be generated using Fortran, Python, R, exported from ArcGIS or in many other ways) and (c) familiarity with online sources of appropriate input data (e.g. a high-resolution DEM). If you do not have this expertise, then you will need to follow option 1 and seek a collaborator at the UK Met Office.