Select partitioning of multigrid meshes

[ukmo-juan-castillo]

Are there any linked Issues or Pull Requests?

MetOffice/lfric_apps#290

Brief description

When reading a multigrid mesh file, by default all meshes this file contains are partitioned. Partitioning all meshes in the multigrid mesh file imposes a series of conditions regarding the number of CPUs that can be used in a job. This can be a limitation in some applications that only need to process data on the 'master' mesh.

One simple solution would be just using mesh files containing one single mesh. For flexibility though, it would be desirable to give the option of partitioning all multigrid meshes or not. This functionality is already present in lfric_core via the any_maps parameter of the get_partition subroutine of the panel_decomposition_type object. On the other hand, this object subroutine is not called directly in most applications, so this functionality should be made more accessible.

Tagging users which may wish to comment.
Steve Mullerworth (@stevemullerworth) , Mike Hobson (@mike-hobson) , Matthew Hambley (@MatthewHambley), Lottie Turner (@mo-lottieturner), cjohnson-pi , mo-markweeks , Mike Bush (@michaelrbush)

If you want to stop getting spamming emails ...there is the unsubscribe button in the right hand column -------------------------->

Associated PRs

• Decide if partitioning all meshes in a multigrid mesh file #289
• Lfric2lfric no multi partition lfric_apps#291

Further details of the issue.

No response

[mo-rickywong]

Sorry, though I don't believe that the description is completely accurate.

• When reading a multigrid mesh file, by default all meshes this file contains are partitioned.
  Please explain by "...by default..."? lfric2lfric has it's own version of init_mesh, lfric2lfric is in control of which meshes it loads and which meshes it partitions, so I fail to see why this is an issue.
• Partitioning all meshes in the multigrid mesh file imposes a series of conditions regarding the number of CPUs that can be used in a job.
  It is not the partitioning of any/all meshes that imposes the restriction/defensive checks. It is because, the choice of file to read in contains inter-grid maps. This triggering of the defensive checks was added inappropriately by a previous developer without due oversight.
• it would be desirable to give the option of partitioning all multigrid meshes or not.
  That option is already available to lfric2lfric as lfric2lfric has its own version of init_mesh. It's the choice of input file and the previous defensive checks that are causing the issue.
• This functionality is already present in lfric_core via the any_maps parameter ...
  This is not a functionality, this is a result of the aforementioned developer adding in to incorrect parts of the code. It should not be made accessible

[ukmo-juan-castillo]

• "by default" here means that, if lfric finds a multigrid file with maps during partitioning, it will always partition all meshes contained in the multigrid file. Lfric2lfric has its own version of init_mesh, but it calls the partitioning code that is in lfric_core. Adding a version of the partitioning code in lfric2lfric would imply copying at least two module files from lfric_core to lfric2lfric and just changing a line of code.
• The defensive checks are triggered when global_mesh%get_nmaps() > 0. Agree that this is true when the multigrid mesh file has inter-grid maps.
• The option to partition all multigrid meshes is not available in lfric2lfric, as this option is not available in init_mesh but in the subroutine get_partition of the panel_decomposition object. The sequence of events is:

init_mesh calls create_local_mesh, which creates a partition_type object. During the creation of this object, the particular partitioner is called (e.g. partitioner_planar), which in turn calls the subroutine partitioner_rectangular_panels, which is the one that ultimately calls the get_partition subroutine that performs the checks. Only this last subroutine has a parameter deciding if the check takes place or not.

This shows that, to having a parameter in lfric2lfric deciding if the check is taking place or not, it is not sufficient to have a version of init_mesh, but it would also need to have a version of the partition_type object and the partitioners. Other option would be adding the parameter deciding if the check takes place or not in the init_mesh subroutine, but it will need to be transferred to the partition_type object constructor and the partitioners. If you know of another way to make the check parameter available to lfric2lfric I am open to suggestions.

• If this functionality should not be made accessible, then we can not do what it is requested in this issue. Another question is if the code needs to be rewritten, but that is out of the scope of this issue. We can either implement a solution with the code we currently have, or wait until the code is rewritten and then implement a better solution. In any case this issue is not fundamental to the workings of lfric2lfric because it just provides flexibility: we can always use mesh files containing just the meshes we will process, or without maps if we are not using the lfric2lfric map method.

[mo-rickywong]

If this functionality should not be made accessible, then we can not do what it is requested in this issue.

See

• Allow applications partitioning meshes to specify partition constraints (imposed by multigrid) #290

It allows the application to disable the MultiGrid constraints check without having to access to the any_maps variable (renamed here to something more helpful).

[ukmo-juan-castillo]

If this functionality should not be made accessible, then we can not do what it is requested in this issue.

See

* [Allow applications partitioning meshes to specify partition constraints (imposed by multigrid) #290](https://github.com/MetOffice/lfric_core/pull/290)

It allows the application to disable the MultiGrid constraints check without having to access to the any_maps variable (renamed here to something more helpful).

This one of the five alternatives I proposed, see MetOffice/lfric_apps#291. What I understood when I talked to you is that you did not like this solution either, but I am more than happy with it. Please let me know when this PR is merged to the trunk and I will update my part in lfric2lfric.

[mo-rickywong]

I believe there is a communication issue, your suggestions referred to accessing any_maps, duplication of code and placing methods relating to partitioning in the global_mesh object. I will prepare linked tickets for this and you can scitech review the apps PR.

[ukmo-juan-castillo]

Yes, something was lost in the communication process. I implemented the solution that I thought simpler because it implied changing less code, but I enumerated other approaches that could be followed, the first one the same you are implementing here. I am happy to review the PR if you consider I will be a good reviewer!