Error "Particle ID out of range" but particles should be in range

[jillbellovary]

I'm trying to create a "superzoom" IC from existing zoomed ICs. I'm using ICs that Michael Tremmel made of a zoomed MW+ galaxy, and I can use genetIC to reproduce his process. I then ran these ICs to z=10, and selected a cube that encompasses ~90% of the zoom region. I wish to further resample this region. I output the iords into a file and have verified that they are in the proper cube and they are all dark matter. Corentin has assisted me in creating the correct genetIC commands for my needs. But I keep getting: Error "Particle ID out of range"

I'll attach my param file here, all other files are quite large... but I'm glad to share more if it will help.
corentin_param.txt

I would appreciate any help!

[apontzen]

I think you are missing a mapper_relative_to. If I understand what you've done correctly, zoom_z10.iords comes from a run with that 1024^3 zoom grid, is that right? If so you need to add the right mapper_relative_to before append_IDFile.

E.g. suppose the paramfile with the 1024^3 zoom grid was called paramfile_with_1024_zoom.txt, then I'd add mapper_relative_to paramfile_with_1024_zoom.txt before the line append_IDfile ./zoom_z10.iords. Does that make sense/help?

BTW do you really want those velocity-zeroing commands? That is what we use in AMR runs, but shouldn't be necessary with SPH.

[jillbellovary]

ah OK I see... so. Rather than using the exact steps to create the Tremmel ICs (which I did use to make the cube and select iords), and THEN take a subsequent step to re-refine, I instead commented out the Tremmel-final refinement and used a NEW refinement do to it all in one go. This was Corentin's suggestion and made sense to me, but... now I can see how the iords from my list are out of range, they do not exist do they? And I can see that I need a mapper_relative_to but I am not sure what it would be; it would not be the Tremmel-made file.

So perhaps instead of doing one giant refinement (as Corentin had suggested) maybe there should be a step that directly follows from the Tremmel version (if so, I am not sure how to manage it, which is why I bothered Corentin initially).

As for the velocity-zeroing commands, they are from the Tremmel version and I do not know what they do, I am ignorant of their necessity.

[apontzen]

The mapper_relative_to should point to the precise paramfile that generated the ICs that were used for the run that generates the iords. It sounds to me like that was the tremmel-made file? But I may have misread

Yes you can also keep the tremmel refinement and do a fresh zoom on top of the existing zoom, though ironically you might still need a mapper_relative_to because earlier on you said mapper_relative_to ./decontaminated_paramfile_nearmint.txt so it would still be trying to interpret particles on that basis. So I am not sure there would be much advantage in your case to doing a zoom-inside-zoom rather than just a very high res zoom as Corentin suggested.

OK re the velocity commands. They are not going to be doing anything really bad, and so since you are mid-way through a project I wouldn't now remove them (they could slightly change the large scale structure).

[jillbellovary]

OK, it sat in the queue for a while... and still the same error. I added
mapper_relative_to ./this_is_michaels_param_file.txt
to the line prior to appending z10_zoomiords but I still got the same error as before

[apontzen]

OK... I don't think I can help much further without getting access to all the required files to run this for myself

[jillbellovary]

I'm using Stampede (in the US) to run this, which I assume you cannot access...

[apontzen]

No, but hopefully the paramfile and iord files are not prohibitively large to put somewhere...

[jillbellovary]

let me know if you got a google drive link emailed to you

[apontzen]

Yes, but unfortunately it's missing a number of files - volume_paramfile.txt and halo568_3rvir.txt are the ones that are definitely missing

[jillbellovary]

sorry I've added them!

[apontzen]

Thanks... so I created the ICs from this_is_michaels_param_file.txt... they have 223,207,607 particles in total, of which 200,843,959 are DM.

However the maximum iord in zoom_z10.iords is 235,691,295. So genetIC is correct there is something wrong here with the iords.

Most likely, the ICs used for the run which zoom_z10.iords comes from were not in fact generated with this_is_michaels_param_file.txt. In which case you need to find the paramfile for the actual ICs.

Other less likely possibilities include that somehow the iords are garbled, although I don't think I've ever seen that happen with changa.

[mtremmel]

Hey @jillbellovary did you already figure this out? I can't remember if we spoke about this offline but I was running into the same issue and remembered that the problem is that iords that changa assigns when SPLITGAS is turned on are different. A "buffer" equal to the initial Ngas is put in so that additional gas iords can be assigned later.

To make the DM iords match what genetIC expect you need to subtract Ngas from all of them (note that if you are extracting information from an adiabatic run, you can just use the Ngas at any timestep, otherwise use Ngas from the initial conditions)

Iord_DM_corrected = Iord_DM - Ngas_original

@apontzen has mentioned to me that it may be possible to put in a flag in genetic to do this conversion later, but for now this simple conversion will do the trick I think!

[jillbellovary]

This would explain why Andrew kept telling me that the iords in my file seemed like they were greater than the number of particles. I ended up not using an iord file at all and just mega-zooming the entire zoom region. But this info will be important to tell future people, including future me, if I ever do this again.