Invalid k(E) values computed for path reaction.

[rwest]

Not sure what's going on here, but the methylformate example job dies with this:

Merging PDepNetwork #41809 and PDepNetwork #41810
Updating 2 modified unimolecular reaction networks...

========================================================================
Network Information (Network #41809)
-------------------
Isomers:
    H2CCCH(68)                                            347.693 kJ/mol
    CC#[C](459)                                           509.242 kJ/mol
    C1C=[C]1(4491)                                        513.113 kJ/mol
Reactant channels:
    H(17) + C3H2(69)                                      703.529 kJ/mol
    H(17) + [CH2]C#[C](2238)                              756.717 kJ/mol
    CH3(10) + C2(40)                                      968.493 kJ/mol
Product channels:
    H(17) + C1=C=C1(14986)                                706.706 kJ/mol
    H(17) + C1=[C][CH]1(14987)                            826.518 kJ/mol
    H(17) + C1[C]=[C]1(14988)                             988.462 kJ/mol
    [CH]1C=C1(14989)                                      377.133 kJ/mol
Path reactions:
    H2CCCH(68) <=> H(17) + C3H2(69)                       675.915 kJ/mol
    H(17) + [CH2]C#[C](2238) <=> H2CCCH(68)               756.717 kJ/mol
    H2CCCH(68) <=> C1C=[C]1(4491)                         573.629 kJ/mol
    CC#[C](459) <=> H2CCCH(68)                            654.052 kJ/mol
    CH3(10) + C2(40) <=> CC#[C](459)                      968.493 kJ/mol
    H(17) + [CH2]C#[C](2238) <=> CC#[C](459)              756.717 kJ/mol
    H(17) + C1=C=C1(14986) <=> C1C=[C]1(4491)             718.839 kJ/mol
    H(17) + C1=[C][CH]1(14987) <=> C1C=[C]1(4491)         826.518 kJ/mol
    H(17) + C1[C]=[C]1(14988) <=> C1C=[C]1(4491)          988.462 kJ/mol
    C1C=[C]1(4491) <=> [CH]1C=C1(14989)                   657.923 kJ/mol
========================================================================

Calculating densities of states for Network #41809...
Using 458 grains from 347.69 to 2042.67 kJ/mol in steps of 3.71 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #41809...
Using 200 grains from 347.69 to 1085.77 kJ/mol in steps of 3.71 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(68) <=> H(17) + C3H2(69):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 5.23653e-49
Error:     Expected Keq(292.578 K) = 2.85228e-57
Error:       Actual Keq(292.578 K) = 2.85228e-57
Traceback (most recent call last):
  File "rmg.py", line 134, in <module>
    cProfile.runctx(command, global_vars, local_vars, stats_file)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cProfile.py", line 49, in runctx
    prof = prof.runctx(statement, globals, locals)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cProfile.py", line 140, in runctx
    exec cmd in globals, locals
  File "<string>", line 1, in <module>
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/main.py", line 388, in execute
    self.reactionModel.enlarge(objectsToEnlarge)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/model.py", line 636, in enlarge
    self.updateUnimolecularReactionNetworks(database)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/model.py", line 1290, in updateUnimolecularReactionNetworks
    network.update(self, database, self.pressureDependence)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/pdep.py", line 443, in update
    K = self.calculateRateCoefficients(Tlist, Plist, method, grainSize=grainSize, grainCount=grainCount)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 852, in calculateRateCoefficients
    self.setConditions(T, P)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 761, in setConditions
    self.calculateMicrocanonicalRates()
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 613, in calculateMicrocanonicalRates
    raise NetworkError('Invalid k(E) values computed for path reaction "{0}" do not satisfy'.format(rxn))
rmgpy.measure.network.NetworkError: Invalid k(E) values computed for path reaction "H2CCCH(68) <=> H(17) + C3H2(69)" do not satisfy

Measure input file for network 41809 is at https://gist.github.com/2910382

[rwest]

I am still getting this after a trial merge of the 'molecule' branch.

Merging PDepNetwork #41809 and PDepNetwork #41810
Updating 2 modified unimolecular reaction networks...

========================================================================
Network Information (Network #41863)
-------------------
Isomers:
    CC=C=[CH](13976)                                      315.344 kJ/mol
    C(C)C#[C](5134)                                       486.735 kJ/mol
Reactant channels:
    CH3(10) + C3H2(69)                                    628.787 kJ/mol
    H(17) + C(=C)C#C(197)                                 498.607 kJ/mol
    CH3(10) + [CH2]C#[C](2305)                            681.976 kJ/mol
    C2(40) + C2H5(27)                                     943.022 kJ/mol
Product channels:
    H(17) + C(=C=[CH])[CH2](15712)                        664.527 kJ/mol
    H(17) + C[C]=C=[CH](13984)                            750.731 kJ/mol
    H(17) + C[CH]C#[C](13980)                             720.442 kJ/mol
    [C@@H]1(C)C=[C]1(31851)                               478.812 kJ/mol
    C([CH2])C#C(531)                                      354.849 kJ/mol
    C([CH2])C#[C](511) + H(17)                            903.775 kJ/mol
Path reactions:
    CH3(10) + C3H2(69) <=> CC=C=[CH](13976)               628.787 kJ/mol
    H(17) + C(=C=[CH])[CH2](15712) <=> CC=C=[CH](13976)      664.527 kJ/mol
    H(17) + C[C]=C=[CH](13984) <=> CC=C=[CH](13976)       750.731 kJ/mol
    H(17) + C(=C)C#C(197) <=> CC=C=[CH](13976)            502.791 kJ/mol
    H(17) + C[CH]C#[C](13980) <=> CC=C=[CH](13976)        720.442 kJ/mol
    CC=C=[CH](13976) <=> [C@@H]1(C)C=[C]1(31851)           541.28 kJ/mol
    C([CH2])C#C(531) <=> CC=C=[CH](13976)                 471.583 kJ/mol
    C(C)C#[C](5134) <=> CC=C=[CH](13976)                  631.545 kJ/mol
    CH3(10) + [CH2]C#[C](2305) <=> C(C)C#[C](5134)        681.976 kJ/mol
    C2(40) + C2H5(27) <=> C(C)C#[C](5134)                 943.022 kJ/mol
    H(17) + C[CH]C#[C](13980) <=> C(C)C#[C](5134)         720.442 kJ/mol
    C([CH2])C#[C](511) + H(17) <=> C(C)C#[C](5134)        903.775 kJ/mol
    C(C)C#[C](5134) <=> C([CH2])C#C(531)                  631.545 kJ/mol
========================================================================

Calculating densities of states for Network #41863...
Using 463 grains from 315.34 to 1998.47 kJ/mol in steps of 3.64 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #41863...
Using 200 grains from 315.34 to 1040.33 kJ/mol in steps of 3.64 kJ/mol to compute the k(T,P) values at 292.578 K
Using 200 grains from 315.34 to 1047.55 kJ/mol in steps of 3.68 kJ/mol to compute the k(T,P) values at 314.289 K
Using 200 grains from 315.34 to 1064.16 kJ/mol in steps of 3.76 kJ/mol to compute the k(T,P) values at 364.231 K
Using 200 grains from 315.34 to 1095.90 kJ/mol in steps of 3.92 kJ/mol to compute the k(T,P) values at 459.667 K
Using 203 grains from 315.34 to 1160.51 kJ/mol in steps of 4.18 kJ/mol to compute the k(T,P) values at 641.642 K
Using 232 grains from 315.34 to 1281.85 kJ/mol in steps of 4.18 kJ/mol to compute the k(T,P) values at 1011.65 K
Using 295 grains from 315.34 to 1545.44 kJ/mol in steps of 4.18 kJ/mol to compute the k(T,P) values at 1810.91 K
Using 403 grains from 315.34 to 1997.31 kJ/mol in steps of 4.18 kJ/mol to compute the k(T,P) values at 3163.57 K

========================================================================
Network Information (Network #41809)
-------------------
Isomers:
    H2CCCH(68)                                            347.693 kJ/mol
    CC#[C](459)                                           509.242 kJ/mol
    C1C=[C]1(4505)                                        513.113 kJ/mol
Reactant channels:
    C3H2(69) + H(17)                                      703.529 kJ/mol
    H(17) + [CH2]C#[C](2305)                              756.717 kJ/mol
    CH3(10) + C2(40)                                      968.493 kJ/mol
Product channels:
    H(17) + C1=C=C1(19322)                                706.706 kJ/mol
    H(17) + C1=[C][CH]1(19323)                            826.518 kJ/mol
    H(17) + C1[C]=[C]1(19324)                             988.462 kJ/mol
    [CH]1C=C1(19325)                                      377.133 kJ/mol
Path reactions:
    H2CCCH(68) <=> C3H2(69) + H(17)                       675.915 kJ/mol
    H(17) + [CH2]C#[C](2305) <=> H2CCCH(68)               756.717 kJ/mol
    H2CCCH(68) <=> C1C=[C]1(4505)                         573.629 kJ/mol
    CC#[C](459) <=> H2CCCH(68)                            654.052 kJ/mol
    CH3(10) + C2(40) <=> CC#[C](459)                      968.493 kJ/mol
    H(17) + [CH2]C#[C](2305) <=> CC#[C](459)              756.717 kJ/mol
    H(17) + C1=C=C1(19322) <=> C1C=[C]1(4505)             718.839 kJ/mol
    H(17) + C1=[C][CH]1(19323) <=> C1C=[C]1(4505)         826.518 kJ/mol
    H(17) + C1[C]=[C]1(19324) <=> C1C=[C]1(4505)          988.462 kJ/mol
    C1C=[C]1(4505) <=> [CH]1C=C1(19325)                   657.923 kJ/mol
========================================================================

Calculating densities of states for Network #41809...
Using 458 grains from 347.69 to 2042.67 kJ/mol in steps of 3.71 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #41809...
Using 200 grains from 347.69 to 1085.77 kJ/mol in steps of 3.71 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(68) <=> C3H2(69) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 7.44885e-49
Error:     Expected Keq(292.578 K) = 2.85228e-57
Error:       Actual Keq(292.578 K) = 2.85228e-57
Traceback (most recent call last):
  File "rmg.py", line 134, in <module>
    cProfile.runctx(command, global_vars, local_vars, stats_file)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cProfile.py", line 49, in runctx
    prof = prof.runctx(statement, globals, locals)
  File "/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cProfile.py", line 140, in runctx
    exec cmd in globals, locals
  File "<string>", line 1, in <module>
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/main.py", line 391, in execute
    self.reactionModel.enlarge(objectsToEnlarge)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/model.py", line 615, in enlarge
    self.updateUnimolecularReactionNetworks(database)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/model.py", line 1291, in updateUnimolecularReactionNetworks
    network.update(self, database, self.pressureDependence)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/rmg/pdep.py", line 443, in update
    K = self.calculateRateCoefficients(Tlist, Plist, method, grainSize=grainSize, grainCount=grainCount)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 852, in calculateRateCoefficients
    self.setConditions(T, P)
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 761, in setConditions
    self.calculateMicrocanonicalRates()
  File "/Users/rwest/Code/rmgpy/RMG-Py/rmgpy/measure/network.py", line 613, in calculateMicrocanonicalRates
    raise NetworkError('Invalid k(E) values computed for path reaction "{0}" do not satisfy'.format(rxn))
rmgpy.measure.network.NetworkError: Invalid k(E) values computed for path reaction "H2CCCH(68) <=> C3H2(69) + H(17)" do not satisfy

(But with the faster pdep it now only takes 6 hours to reach that point)

[rwest]

@faribas has now come across a job which runs into this problem within a couple of minutes of starting.
@jwallen, does your commit 406b3d8 suggest you've been having trouble here also, but in a slightly different context?

[rwest]

========================================================================
Network Information (Network #1)
-------------------
Isomers:
    Hyad(1)                                              -313.915 kJ/mol
Reactant channels:
    CH3OH(25) + CO(14)                                   -332.991 kJ/mol
    H2O(4) + CH2CO(37)                                   -310.981 kJ/mol
    CH2OH(26) + HCO(18)                                    4.6549 kJ/mol
    CH2CHO(34) + OH(10)                                   38.8102 kJ/mol
Product channels:
    [CH](C=O)O(73) + H(7)                                 47.6431 kJ/mol
    C(O)[C]=O(74) + H(7)                                  52.4103 kJ/mol
    C(C=O)[O](75) + H(7)                                  122.823 kJ/mol
Path reactions:
    CH3OH(25) + CO(14) <=> Hyad(1)                         24.741 kJ/mol
    H2O(4) + CH2CO(37) <=> Hyad(1)                       -124.255 kJ/mol
    CH2OH(26) + HCO(18) <=> Hyad(1)                        4.6549 kJ/mol
    CH2CHO(34) + OH(10) <=> Hyad(1)                       38.8102 kJ/mol
    [CH](C=O)O(73) + H(7) <=> Hyad(1)                     47.6431 kJ/mol
    C(O)[C]=O(74) + H(7) <=> Hyad(1)                      52.4103 kJ/mol
    C(C=O)[O](75) + H(7) <=> Hyad(1)                      122.823 kJ/mol
========================================================================

Calculating densities of states for Network #1...
Using 473 grains from -332.99 to 1066.32 kJ/mol in steps of 2.96 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #1...
Using 200 grains from -332.99 to 256.97 kJ/mol in steps of 2.96 kJ/mol to compute the k(T,P) values at 403.359 K
Error: For path reaction CH3OH(25) + CO(14) <=> Hyad(1):
Error:     Expected kf(403.359 K) = 5.0744e-41
Error:       Actual kf(403.359 K) = 5.57424e-40
Error:     Expected Keq(403.359 K) = 3.04059e-10
Error:       Actual Keq(403.359 K) = 3.04059e-10
-------------------------------------------------------------

So the kf values are an order of magnitude apart, but those are very small numbers (whatever the units may be). What is the machine precision? Are they really different?

[jwallen]

Yes, I still see this more often than I would like. Some of the error is probably just due to discretization, but I don't think that explains a full order of magnitude. This is especially true when using ILT to go from k(T) -> k(E), which by definition should give you the same k(T) at the end. (This assumes that Ea and the barrier height are the same, which is what we're usually assuming in RMG jobs.) Feel free to send me a few networks that are known to have the issue.

I've never really thought about machine precision being an issue. Now that you mention it, that could be problematic, since the formula is k(E) = A * rho(E - Ea) / rho(E), where rho(E) is the reactant density of states. For large Ea - as in this case - the ratio rho(E - Ea) / rho(E) is dividing a small number by a big number, which could be troublesome. I'll think about this some more; it may be that we don't care about this issue when the kf is negligible.

Looking at the first example in the thread, I see that the barrier height is negative. Not sure if RMG-Py handles this correctly, especially because I'm not really sure how it should be handled.

[rwest]

The first example in the thread is also very small kf: 1.31499e-46.
I think the last example in the thread is just the decomposition of

species(
    label='Hyad',
    reactive=True,
    structure=SMILES("C(C=O)O"),
)

but I'll send you an input file.

[rwest]

Just realized that although in all these cases the kf is very small (which still hints at numerical error accumulation?) a very small Keq means that the reverse rate could in fact be significant and important to get right?

Implementing an "ignore this problem if both kf and kr are less then 1e30" allows the Hyad example to get past the first problem, and run straight into another one:

Error: For path reaction CH3OH(22) + CO(11) <=> Hyad(1):
Error:     Expected kf(403.359 K) = 5.0744e-41
Error:       Actual kf(403.359 K) = 5.57493e-40
Error:     Expected Keq(403.359 K) = 3.04059e-10
Error:       Actual Keq(403.359 K) = 3.04059e-10
Error:     * Network error ignored because expected forward and reverse rate(s) < 1e-30
Error: For path reaction CH2CO(34) + H2O(8) <=> Hyad(1):
Error:     Expected kf(403.359 K) = 8.74996e-21
Error:       Actual kf(403.359 K) = 6.27463e-19
Error:     Expected Keq(403.359 K) = 7.84436e-07
Error:       Actual Keq(403.359 K) = 7.84436e-07

But what is "small" for a rate coefficient anyway? Are these even always in the same units (eg. all bimolecular?)

[faribas]

The 'sean-v8@26fbe21' still not working, I got the same error:

Warning: States node <Entry index=-1 label="C_R1"> and all its parents have data=None
Warning: For <Molecule "[CH2]O">, more characteristic frequencies were generated than vibrational modes allowed. Removed 1 groups (1 frequencies) to compensate.
Warning: For <Molecule "C[CH2]">, more characteristic frequencies were generated than vibrational modes allowed. Removed 1 groups (2 frequencies) to compensate.
Warning: States node <Entry index=-1 label="O_R1"> and all its parents have data=None
Warning: States node <Entry index=-1 label="Oxy"> and all its parents have data=None

========================================================================
Network Information (Network #323)
-------------------
Isomers:
    CH3CH2OH(44)                                         -248.823 kJ/mol
Reactant channels:
    H2O(4) + C2H4(32)                                    -207.671 kJ/mol
    CH3(19) + CH2OH(26)                                   109.466 kJ/mol
    C2H5(24) + OH(10)                                     142.111 kJ/mol
    CH3CHOH(45) + H(7)                                    143.826 kJ/mol
    CH2CH2OH(46) + H(7)                                   172.549 kJ/mol
    CH3CH2O(29) + H(7)                                    186.674 kJ/mol
Product channels:
Path reactions:
    H2O(4) + C2H4(32) <=> CH3CH2OH(44)                    14.4996 kJ/mol
    CH3(19) + CH2OH(26) <=> CH3CH2OH(44)                  109.466 kJ/mol
    C2H5(24) + OH(10) <=> CH3CH2OH(44)                    142.111 kJ/mol
    CH3CHOH(45) + H(7) <=> CH3CH2OH(44)                   143.826 kJ/mol
    CH2CH2OH(46) + H(7) <=> CH3CH2OH(44)                  172.549 kJ/mol
    CH3CH2O(29) + H(7) <=> CH3CH2OH(44)                   186.674 kJ/mol
========================================================================

Calculating densities of states for Network #323...
Using 482 grains from -248.82 to 1128.06 kJ/mol in steps of 2.86 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #323...
Using 200 grains from -248.82 to 320.82 kJ/mol in steps of 2.86 kJ/mol to compute the k(T,P) values at 403.359 K
Error: For path reaction H2O(4) + C2H4(32) <=> CH3CH2OH(44):
Error:     Expected kf(403.359 K) = 1.82749e-24
Error:       Actual kf(403.359 K) = 2.83046e-22
Error:     Expected Keq(403.359 K) = 0.00953824
Error:       Actual Keq(403.359 K) = 0.00953824

I changed cutoff to e-15, but again job crashed for the same reason:

========================================================================
Network Information (Network #326)
-------------------
Isomers:
    Aa(2)                                                -445.218 kJ/mol
Reactant channels:
    H2O(4) + CH2CO(37)                                   -310.981 kJ/mol
    CH3(19) + HOCO(16)                                   -58.3274 kJ/mol
    CH3CO(48) + OH(10)                                    10.5807 kJ/mol
    CH4(20) + CO2(15)                                    -487.247 kJ/mol
Product channels:
    C(=O)([CH2])O(208) + H(7)                            -35.9481 kJ/mol
    CC(=O)[O](209) + H(7)                                 12.0907 kJ/mol
Path reactions:
    H2O(4) + CH2CO(37) <=> Aa(2)                         -146.132 kJ/mol
    CH3(19) + HOCO(16) <=> Aa(2)                         -58.3274 kJ/mol
    C(=O)([CH2])O(208) + H(7) <=> Aa(2)                  -35.9481 kJ/mol
    CH3CO(48) + OH(10) <=> Aa(2)                          10.5807 kJ/mol
    CC(=O)[O](209) + H(7) <=> Aa(2)                       12.0907 kJ/mol
    CH4(20) + CO2(15) <=> Aa(2)                          -155.874 kJ/mol
========================================================================

Calculating densities of states for Network #326...
Using 454 grains from -487.25 to 954.81 kJ/mol in steps of 3.18 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for Network #326...
Using 200 grains from -487.25 to 146.24 kJ/mol in steps of 3.18 kJ/mol to compute the k(T,P) values at 403.359 K
Error: For path reaction H2O(4) + CH2CO(37) <=> Aa(2):
Error:     Expected kf(403.359 K) = 1.17697e-17
Error:       Actual kf(403.359 K) = 3.13281e-14
Error:     Expected Keq(403.359 K) = 2.27301e+09
Error:       Actual Keq(403.359 K) = 2.27301e+09
Error:     * Network error ignored because expected forward and/or reverse rate(s) < 1e-15
Error: For path reaction CH4(20) + CO2(15) <=> Aa(2):
Error:     Expected kf(403.359 K) = 1.05109e-37
Error:       Actual kf(403.359 K) = 1.329e-36
Error:     Expected Keq(403.359 K) = 7.22931e-13
Error:       Actual Keq(403.359 K) = 7.22931e-13
Error:     * Network error ignored because expected forward and/or reverse rate(s) < 1e-15
Using 200 grains from -487.25 to 155.60 kJ/mol in steps of 3.23 kJ/mol to compute the k(T,P) values at 431.513 K
Error: For path reaction H2O(4) + CH2CO(37) <=> Aa(2):
Error:     Expected kf(431.513 K) = 3.57026e-16
Error:       Actual kf(431.513 K) = 7.49454e-13
Error:     Expected Keq(431.513 K) = 1.47752e+08
Error:       Actual Keq(431.513 K) = 1.47752e+08
Error:     * Network error ignored because expected forward and/or reverse rate(s) < 1e-15
Error: For path reaction CH4(20) + CO2(15) <=> Aa(2):
Error:     Expected kf(431.513 K) = 8.02439e-35
Error:       Actual kf(431.513 K) = 1.03782e-33
Error:     Expected Keq(431.513 K) = 1.55559e-12
Error:       Actual Keq(431.513 K) = 1.55559e-12
Error:     * Network error ignored because expected forward and/or reverse rate(s) < 1e-15
Using 200 grains from -487.25 to 176.85 kJ/mol in steps of 3.34 kJ/mol to compute the k(T,P) values at 495.409 K
Error: For path reaction H2O(4) + CH2CO(37) <=> Aa(2):
Error:     Expected kf(495.409 K) = 2.03546e-13
Error:       Actual kf(495.409 K) = 3.47943e-10
Error:     Expected Keq(495.409 K) = 943896
Error:       Actual Keq(495.409 K) = 943896

[rwest]

Yes, even allowing relatively large discrepancies only seems to delay the inevitable
See the commit comments on @stroiano's branch: sean-v8/RMG-Py@master...kE-bypass
I'm still not sure how large is OK though, or if something worse is at play.

[rwest]

In commit 22f4dd3 I made it so that if it gets an invalid k(E) error, it tries again with twice as many grains. This seemed to help a lot, but running the methylformate example, it runs for three hours, then this happens:

Calculating densities of states for PDepNetwork #15975...
Using 486 grains from 338.60 to 2155.75 kJ/mol in steps of 3.75 kJ/mol to compute densities of states
Calculating phenomenological rate coefficients for PDepNetwork #15975...
Using 200 grains from 338.60 to 1085.05 kJ/mol in steps of 3.75 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.26235e-55
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.5266e-66
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 400 grains from 338.60 to 1085.05 kJ/mol in steps of 1.87 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 5.79451e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 2.78384e-66
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 800 grains from 338.60 to 1085.05 kJ/mol in steps of 0.93 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 2.21623e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 1.39569e-66
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 1600 grains from 338.60 to 1085.05 kJ/mol in steps of 0.47 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.78488e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 9.61517e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 3200 grains from 338.60 to 1085.05 kJ/mol in steps of 0.23 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.47857e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 7.93517e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 6400 grains from 338.60 to 1085.05 kJ/mol in steps of 0.12 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.29574e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 7.20241e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 12800 grains from 338.60 to 1085.05 kJ/mol in steps of 0.06 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.23712e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.8711e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 25600 grains from 338.60 to 1085.05 kJ/mol in steps of 0.03 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.22069e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.7152e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 51200 grains from 338.60 to 1085.05 kJ/mol in steps of 0.01 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.20094e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.63572e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 102400 grains from 338.60 to 1085.05 kJ/mol in steps of 0.01 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.19407e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.59716e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 204800 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.19066e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.57798e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 409600 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.18895e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56841e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 819200 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.18775e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56354e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 1638400 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.18818e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56593e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 3276800 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.1884e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56712e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 6553600 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.18847e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56771e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 13107200 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K
Error: For path reaction H2CCCH(65) <=> C3H2(60) + H(17):
Error:     Expected kf(292.578 K) = 1.31499e-46
Error:       Actual kf(292.578 K) = 1.18855e-56
Error:     Expected Keq(292.578 K) = 5.89833e-66
Error:       Actual Keq(292.578 K) = 6.56801e-67
Warning: Increasing number of grains, decreasing grain size and trying again.
Using 26214400 grains from 338.60 to 1085.05 kJ/mol in steps of 0.00 kJ/mol to compute the k(T,P) values at 292.578 K

Those attempts take about another 3 hours, then it crashes.
Profiling data suggests the job spent 60% of its time in {method 'mapDensityOfStates' of 'rmgpy.pdep.configuration.Configuration' objects}

We need a strategy for when adding more grains doesn't help.

[jwallen]

Clearly increasing the number of grains is not helping in this system, as the actual kf barely changes at all. This suggests a real (not a numerical) error. If you can send me the network I'll see if I can identify what's going on.

[jwallen]

The problem with this particular network is that the reaction H2CCCH(65)<=>C3H2(60)+H(17), coming from Glarborg/C2, has an activation energy that is ~20 kcal/mol below the barrier height (as estimated from the enthalpy of reaction). Using the ILT method to get k(E) requires that the activation energy and the barrier height be the same. Since the k(T) estimate came from a library instead of an estimate, its Ea was not adjusted to be at least dHrxn.
Incidentally, the thermo for all of these species is coming from DFT_QCI_thermo, so it's not a group additivity estimate.

I'm not quite sure what the appropriate course of action is.