Merge pull request #46 from SMTG-Bham/dopey_fermi

Add `FermiSolver` class, with several other major updates

.gitignore

@@ -2,6 +2,7 @@
 
 # No PyCharm
 .idea/
+tests/__init__.py
 
 # Ignore Mac Finder metadata file
 .DS_Store

README.md

@@ -50,29 +50,31 @@ If you use `doped` in your research, please cite:
 - S. R. Kavanagh et al. [doped: Python toolkit for robust and repeatable charged defect supercell calculations](https://doi.org/10.21105/joss.06433). _Journal of Open Source Software_ 9 (96), 6433, **2024**
 
 ## `ShakeNBreak`
-As shown in the `doped` tutorials, it is highly recommended to use the [`ShakeNBreak`](https://shakenbreak.readthedocs.io/en/latest/) approach when calculating point defects in solids, to ensure you have identified the groundstate structures of your defects. As detailed in the [theory paper](https://doi.org/10.1038/s41524-023-00973-1), skipping this step can result in drastically incorrect formation energies, transition levels, carrier capture (basically any property associated with defects). This approach is followed in the [doped example notebook](https://github.com/SMTG-Bham/doped/blob/main/dope_workflow_example.ipynb), with a more in-depth explanation and tutorial given on the [ShakeNBreak](https://shakenbreak.readthedocs.io/en/latest/) website.
+As shown in the `doped` tutorials, it is highly recommended to use the [`ShakeNBreak`](https://shakenbreak.readthedocs.io/en/latest/) approach when calculating point defects in solids, to ensure you have identified the groundstate structures of your defects. As detailed in the [theory paper](https://doi.org/10.1038/s41524-023-00973-1), skipping this step can result in drastically incorrect formation energies, transition levels, carrier capture (basically any property associated with defects). This approach is followed in the [doped defect generation tutorial](https://doped.readthedocs.io/en/latest/generation_tutorial.html), with a more in-depth explanation and tutorial given on the [ShakeNBreak](https://shakenbreak.readthedocs.io/en/latest/) website.
 
 ![ShakeNBreak Summary](https://raw.githubusercontent.com/SMTG-Bham/ShakeNBreak/main/docs/SnB_Supercell_Schematic_PES_2sec_Compressed.gif)
 
 ## Studies using `doped`, so far
 
-- S. R. Kavanagh et al. **_Intrinsic point defect tolerance in selenium for indoor and tandem photovoltaics_** [_ChemRxiv_](https://doi.org/10.26434/chemrxiv-2024-91h02) 2024
-- X. Wang et al. **_Sulfur Vacancies Limit the Open-circuit Voltage of Sb<sub>2</sub>S<sub>3</sub> Solar Cells_** [_arXiv_](https://doi.org/10.48550/arXiv.2410.10560) 2024
+- Y. Fu & H. Lohan et al. **_Factors Enabling Delocalized Charge-Carriers in Pnictogen-Based
+Solar Absorbers: In-depth Investigation into CuSbSe<sub>2</sub>_** [_Nature Communications_](https://doi.org/10.1038/s41467-024-55254-2) 2025
+- S. R. Kavanagh **_Identifying Split Vacancies with Foundation Models and Electrostatics_** [_arXiv_](https://doi.org/10.48550/arXiv.2412.19330) 2025
+- S. R. Kavanagh et al. **_Intrinsic point defect tolerance in selenium for indoor and tandem photovoltaics_** [_ChemRxiv_](https://doi.org/10.26434/chemrxiv-2024-91h02) 2025
+- J. Hu et al. **_Enabling ionic transport in Li<sub>3</sub>AlP<sub>2</sub> the roles of defects and disorder_** [_Journal of Materials Chemistry A_](https://doi.org/10.1039/D4TA04347B) 2025
+- X. Wang et al. **_Sulfur Vacancies Limit the Open-circuit Voltage of Sb<sub>2</sub>S<sub>3</sub> Solar Cells_** [_ACS Energy Letters_](https://doi.org/10.1021/acsenergylett.4c02722) 2024
 - A. Zhang et al. **_Optimizing the n-type carrier concentration of an InVO<sub>4</sub> photocatalyst by codoing with donors and intrinsic defects_** [_Physical Review Applied_](https://doi.org/10.1103/PhysRevApplied.22.044047) 2024
 - M-L. Wang et al. **_Impact of sulfur doping on copper-substituted lead apatite_** [_Physical Review B_](https://doi.org/10.1103/PhysRevB.110.104109) 2024
 - S. Quadir et al. **_Low-Temperature Synthesis of Stable CaZn<sub>2</sub>P<sub>2</sub> Zintl Phosphide Thin Films as Candidate Top Absorbers_** [_Advanced Energy Materials_](https://doi.org/10.1002/aenm.202402640) 2024
 - M. Elgaml et al. **_Controlling the Superconductivity of Nb<sub>2</sub>Pd<sub>x</sub>S<sub>5</sub> via Reversible Li Intercalation_** [_Inorganic Chemistry_](https://pubs.acs.org/doi/full/10.1021/acs.inorgchem.3c03524) 2024
 - Z. Yuan & G. Hautier **_First-principles study of defects and doping limits in CaO_** [_Applied Physics Letters_](https://doi.org/10.1063/5.0211707) 2024
 - B. E. Murdock et al. **_Li-Site Defects Induce Formation of Li-Rich Impurity Phases: Implications for Charge Distribution and Performance of LiNi<sub>0.5-x</sub>M<sub>x</sub>Mn<sub>1.5</sub>O<sub>4</sub> Cathodes (M = Fe and Mg; x = 0.05–0.2)_** [_Advanced Materials_](https://doi.org/10.1002/adma.202400343) 2024
 - A. G. Squires et al. **_Oxygen dimerization as a defect-driven process in bulk LiNiO2<sub>2</sub>_** [_ACS Energy Letters_](https://pubs.acs.org/doi/10.1021/acsenergylett.4c01307) 2024
-- Y. Fu & H. Lohan et al. **_Factors Enabling Delocalized Charge-Carriers in Pnictogen-Based
-Solar Absorbers: In-depth Investigation into CuSbSe<sub>2</sub>_** [_arXiv_](https://doi.org/10.48550/arXiv.2401.02257) 2024
 - X. Wang et al. **_Upper efficiency limit of Sb<sub>2</sub>Se<sub>3</sub> solar cells_** [_Joule_](https://doi.org/10.1016/j.joule.2024.05.004) 2024
 - I. Mosquera-Lois et al. **_Machine-learning structural reconstructions for accelerated point defect calculations_** [_npj Computational Materials_](https://doi.org/10.1038/s41524-024-01303-9) 2024
 - W. Dou et al. **_Band Degeneracy and Anisotropy Enhances Thermoelectric Performance from Sb<sub>2</sub>Si<sub>2</sub>Te<sub>6</sub> to Sc<sub>2</sub>Si<sub>2</sub>Te<sub>6</sub>_** [_Journal of the American Chemical Society_](https://doi.org/10.1021/jacs.4c01838) 2024
 - K. Li et al. **_Computational Prediction of an Antimony-based n-type Transparent Conducting Oxide: F-doped Sb<sub>2</sub>O<sub>5</sub>_** [_Chemistry of Materials_](https://doi.org/10.1021/acs.chemmater.3c03257) 2024
 - S. Hachmioune et al. **_Exploring the Thermoelectric Potential of MgB<sub>4</sub>: Electronic Band Structure, Transport Properties, and Defect Chemistry_** [_Chemistry of Materials_](https://doi.org/10.1021/acs.chemmater.4c00584) 2024
-- J. Hu et al. **_Enabling ionic transport in Li<sub>3</sub>AlP<sub>2</sub> the roles of defects and disorder_** [_ChemRxiv_](https://doi.org/10.26434/chemrxiv-2024-3s0kh) 2024
+- Y. Zeng et al. **_Role of carbon in α-Al2O3:C crystals investigated with first-principles calculations and experiment_** [_Ceramics International_](https://doi.org/10.1016/j.ceramint.2024.12.512) 2024
 - X. Wang et al. **_Four-electron negative-U vacancy defects in antimony selenide_** [_Physical Review B_](https://journals.aps.org/prb/abstract/10.1103/PhysRevB.108.134102) 2023
 - Y. Kumagai et al. **_Alkali Mono-Pnictides: A New Class of Photovoltaic Materials by Element Mutation_** [_PRX Energy_](http://dx.doi.org/10.1103/PRXEnergy.2.043002) 2023
 - S. M. Liga & S. R. Kavanagh, A. Walsh, D. O. Scanlon, G. Konstantatos **_Mixed-Cation Vacancy-Ordered Perovskites (Cs<sub>2</sub>Ti<sub>1–x</sub>Sn<sub>x</sub>X<sub>6</sub>; X = I or Br): Low-Temperature Miscibility, Additivity, and Tunable Stability_** [_Journal of Physical Chemistry C_](https://doi.org/10.1021/acs.jpcc.3c05204) 2023

docs/Dev_ToDo.md

@@ -1,6 +1,6 @@
 # `doped` Development To-Do List
 ## Chemical potential
-- Need to recheck validity of approximations used for extrinsic competing phases (and code for this). Proper `vasp_std` setup (with `NKRED` folders like for defect calcs) and `vasp_ncl` generation.
+- Need to recheck validity of approximations used for extrinsic competing phases (and code for this)(see `full_sub_approach_understanding` folder). Proper `vasp_std` setup (with `NKRED` folders like for defect calcs) and `vasp_ncl` generation.
 - Efficient generation of competing phases for which there are many polymorphs? See SK notes from CdTe competing phases.
 
 ## Defect Complexes
@@ -12,7 +12,6 @@
   - `aide` labelling of defect species in formation energy plots? See `labellines` package for this (as used in `pymatgen-analysis-defects` chempots plotting)
   - Ordering of defects plotted (and thus in the legend) should be physically relevant (whether by energy, or defect type etc.)
   - Should have `ncols` as an optional parameter for the function, and auto-set this to 2 if the legend height exceeds that of the plot
-  - Don't show transition levels outside of the bandgap (or within a certain range of the band edge (good to have this as an option, as PHS analysis still isn't perfect... could do this criterion based on PHS analysis using P-ratio, which maybe is a bit better than default?), possibly using `pydefect` delocalisation analysis?), as these are shallow and not calculable with the standard supercell approach.
   - Option for degeneracy-weighted ('reduced') formation energy diagrams, similar to reduced energies in SOD. See Slack discussion and CdTe pyscfermi notebooks. Would be easy to implement if auto degeneracy handling implemented.
   - Could also add an optional right-hand-side y-axis for defect concentration (for a chosen anneal temp) to our TLD plotting (e.g. `concentration_T = None`) as done for thesis, noting in docstring that this obvs doesn't account for degeneracy!  
   - Separate `dist_tol` for interstitials vs (inequivalent) vacancies/substitutions? (See Xinwei chat) Any other options on this front?
@@ -85,18 +84,19 @@
 - Should flick through other defect codes (see
   https://shakenbreak.readthedocs.io/en/latest/Code_Compatibility.html, also `AiiDA-defects`) and see if
   there's any useful functionality we want to add!
-- Add short example notebook showing how to generate a defect PES for carrier capture or NEB calculations (don't use `IBRION = 2` for NEB).
 
 ## SK To-Do for next update:
-- `doped` repo/docs cleanup `TODO`s above
+- `doped` repo/docs cleanup `TODO`s above, and check through code TODOs
 - Update generation notebook to use structure file string input
 - Quick run through tutorial notebooks to check code all updated and running.
 - Clean up repo, removing old unnecessary git blobs
+- Should have a general refactor from `(bulk, defect)` to `(defect, bulk)` in inputs to functions (e.g. site-matching, symmetry functions etc), as this is most intuitive and then keep consistent throughout.
+- Configuration coordinate diagram generation tutorial, linked in other tutorials and codes (CarrierCapture.jl). For defect PESs for carrier capture or NEB calculations (don't use `IBRION = 2` for NEB).
+
 - Note in chempots tutorial that LaTeX table generator website can also be used with the `to_csv()` function to generate LaTeX tables for the competing phases.
-- Add note to chempots tutorial that if your bulk phase is lower energy than its version on the MP (e.g. distorted perovskite), then you should use this for your bulk competing phase calculation.
-- - Should have a general refactor from `(bulk, defect)` to `(defect, bulk)` in inputs to functions (e.g. site-matching, symmetry functions etc), as this is most intuitive and then keep consistent throughout.
+- Add note to chempots tutorial that if your bulk phase is lower energy than its version on the MP (e.g. distorted perovskite), then you should use this for your bulk competing phase calculation. 
+- Tests for configuration coordinate diagram generation code
 - Quick-start tutorial suggested by Alex G
 - Test chempot grid plotting tool.
-- `dist_tol` should also group defects for the concentration etc functions, currently doesn't (e.g. `CdTe_thermo.get_equilibrium_concentrations(limit="Te-rich", per_charge=False, fermi_level=0.5)` and `CdTe_thermo.dist_tol=10; CdTe_thermo.get_equilibrium_concentrations(limit="Te-rich", per_charge=False, fermi_level=0.5)`, same output)
 - Add example to chemical potentials / thermodynamics analysis tutorials of varying chemical potentials as a function of temperature/pressure (i.e. gas phases), using the `Spinney` functions detailed here (https://spinney.readthedocs.io/en/latest/tutorial/chemipots.html#including-temperature-and-pressure-effects-through-the-gas-phase-chemical-potentials) or possibly `DefAP` functions otherwise. Xinwei Sb2S3 stuff possibly a decent example for this, see our notebooks.
 - Deal with cases where "X-rich"/"X-poor" corresponds to more than one limit (pick one and warn user?)(e.g. Wenzhen Si2Sb2Te6). Can see `get_chempots` in `pmg-analysis-defects` for inspo on this.

docs/Tips.rst

@@ -382,6 +382,12 @@ PHS on the transition level diagram with a clear circle is shown on the right.
    Yu Kumagai, Naoki Tsunoda, Akira Takahashi, and Fumiyasu Oba
    Phys. Rev. Materials 5, 123803 (2021) -- 10.1103/PhysRevMaterials.5.123803
 
+In ``doped``, this eigenvalue analysis is performed automatically, and shallow/unstable defect charge
+states can be omitted from plotting and analysis using the ``unstable_entries`` argument and/or
+``DefectThermodynamics.prune_to_stable_entries()`` method. By default, parsed defect entries which are
+detected to be shallow ('perturbed host') states and unstable for Fermi levels in the band gap are omitted
+from plotting for clarity & accuracy.
+
 Spin Polarisation
 -----------------
 Proper accounting of spin polarisation and multiplicity is crucial for accurate defect calculations and
@@ -554,11 +560,6 @@ In the typical defect calculation workflow with ``doped`` (exemplified in the tu
   ``DefectsParser(json_filename="custom_name.json")``, but the default is
   ``{Host Chemical Formula}_defect_dict.json``.
 
-    - Additionally, a ``voronoi_nodes.json`` file is saved to the bulk supercell calculation directory if
-      any interstitial defects are parsed. This contains information about the Voronoi tessellation nodes
-      in the host structure, which are used for analysing interstitial positions but can be somewhat costly
-      to calculate – so are automatically saved to file once initially computed to reduce parsing times.
-
 - Additionally, if following the recommended structure-searching approach with ``ShakeNBreak`` as shown in
   the tutorials, ``distortion_metadata.json`` files will be written to the top directory (``output_path``,
   containing distortion information about all defects) and to each defect directory (containing just the
@@ -572,7 +573,7 @@ is not done automatically.
 ^^^^^^^^^^^^^^^^^^^^^
 Many analysis methods in ``doped`` return ``pandas`` ``DataFrame`` objects as the result, such as the
 ``get_symmetries_and_degeneracies()``, ``get_formation_energies()``, ``get_equilibrium_concentrations()``,
-``get_quenched_fermi_level_and_concentrations``, ``get_dopability_limits()``, ``get_doping_windows()`` and
+``get_fermi_level_and_concentrations``, ``get_dopability_limits()``, ``get_doping_windows()`` and
 ``get_transition_levels()`` methods for ``DefectThermodynamics`` objects, and the ``formation_energy_df``
 attribute and ``calculate_chempots()`` method for ``CompetingPhasesAnalyzer``. As mentioned in the
 tutorials, these ``DataFrame`` objects can be output to ``csv`` (or ``json``, ``xlsx`` etc., see the

docs/Troubleshooting.rst

@@ -44,8 +44,6 @@ Mis-matching Bulk and Defect Supercells
 When parsing defect calculations with ``doped``, if you provide bulk and defect supercells which do not
 match, you will see the following warning:
 
-.. code::
-
     Detected atoms far from the defect site (>X Å) with major displacements (>0.5 Å) in the defect
     supercell. This likely indicates a mismatch between the bulk and defect supercell definitions or an
     unconverged supercell size, both of which could cause errors in parsing. The mean displacement of the
@@ -138,7 +136,8 @@ A known issue with ``spglib`` is that it can give unnecessary errors or warnings
   spglib: get_bravais_exact_positions_and_lattice failed
   spglib: ref_get_exact_structure_and_symmetry failed.
 
-Typically this can be fixed by updating to ``spglib>=2.5`` with `pip install --upgrade spglib``.
+Typically this can be fixed by updating to ``spglib>=2.5`` with ``pip install --upgrade spglib``.
+
 .. see doped_spglib_warnings.ipynb
 
 ``ShakeNBreak``

docs/Tutorials.rst

@@ -15,5 +15,6 @@ tutorials:
    thermodynamics_tutorial
    chemical_potentials_tutorial
    advanced_analysis_tutorial
+   fermisolver_tutorial
    plotting_customisation_tutorial
    GGA_workflow_tutorial

docs/fermisolver_tutorial.ipynb

@@ -0,0 +1 @@
+../examples/fermisolver_tutorial.ipynb