Undefined index/Uninitialized string offset running polyfill-intl-idn against Unicode test suite

[TRowbotham]

As promised in #159 (comment), here is the stack trace when running symfony/polyfill-intl-idn against the Unicode test suite. I haven't actually looked at the code to see what may be happening.

STR:

1. Make sure you disable ext-intl or use the Symfony\Polyfill\Intl\Idn\Idn class directly.
2. Use idn_to_utf8();
3. Set the $domain parameter to the first string in the test data set.
4. Set the $options parameter to 0. (doesn't matter what the value is)
5. Set the $variant parameter to INTL_IDNA_VARIANT_UTS46 (doesn't matter; same problem using INTL_IDNA_VARIANT_2003)

Minimal example test case:

$source = 'Ü.xn--Tda';
idn_to_utf8($source, 0, INTL_IDNA_VARIANT_UTS46, $info);

There were 7 errors:

1. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set Add support of JSON_THROW_ON_ERROR global constant #168 ('Ü.xn--Tda', 'ü.ü', '', 'xn--tda.xn--tda', '', '', '')
   Undefined index: T

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

2. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set Add Normalizer::getRawDecomposition() and normalizer_get_raw_decomposition() #169 ('Ü.xn--Tda', 'ü.ü', '', 'xn--tda.xn--tda', '', '', '')
   Undefined index: T

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

3. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set [Mbstring] weird behaviour with mb_substr on PHP5 versions #184 ('xn--0.pt', '', '[P4]', '', '', '', '')
   Uninitialized string offset: 1

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

4. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set Incorrect implementation of hrtime #186 ('xn--a-Ä.pt', 'xn--a-ä.pt', '[P4]', 'xn--xn--a--gua.pt', '', '', '')
   Undefined index: �

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

5. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set Ensure hrtime is monotonic #187 ('xn--a-Ä.pt', 'xn--a-ä.pt', '[P4]', 'xn--xn--a--gua.pt', '', '', '')
   Undefined index: A

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

6. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set [grapheme] fix handling negative offsets #188 ('xn--a-ä.pt', 'xn--a-ä.pt', '[P4]', 'xn--xn--a--gua.pt', '', '', '')
   Undefined index: �

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

7. Rowbot\Idna\Test\SymfonyPolyfill::testToUtf8 with data set [Mbstring] fix mb_strrpos with negative offset #189 ('xn--a-ä.pt', '', '[P4]', 'xn--xn--a--gua.pt', '', '', '')
   Undefined index: �

/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:266
/home/trevor/GitHub/IDNA/vendor/symfony/polyfill-intl-idn/Idn.php:114
/home/trevor/GitHub/IDNA/tests/SymfonyPolyfill.php:63

[TRowbotham]

After taking a quick look, the problem seems to be that $decodeTable only accounts for characters [a-z0-9], but needs to also check for [A-Z] as idn_to_utf8() isn't lowercasing the input. Additionally, $decodeTable needs to account for all bytes in the range 00..7F and check if a byte exists on line 266 and fail if it doesn't exist/isn't [a-zA-Z0-9].

[TRowbotham]

Closing now that #267 is merged.