Annotate metrics

[prasunanand]

Changes made:

• Add mypy to setup.cfg
• Add ArrayLike type
• Modify ci scripts
• Add annotations to dask_ml.metrics

Partially fixes #607

[TomAugspurger]

Thanks! Gave a quick pass through. Will look more later.

[TomAugspurger]

I don't know typing well enough, but I think there's a way to say that the Type[X] will exactly match the type of the first element of the result tuple? So you know that if you put in a dask.Array you'll get back a dask.Array rather than an numpy.ndarray?

[prasunanand]

I am not sure about this :/ .

[TomAugspurger]

OK to punt on it for now.

[TomAugspurger]

Would be good to check with scikit-learn what the actual supported types are. Can this be an array, which just isn't implemented yet?

[prasunanand]

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_squared_error.html

multi_output is a string

[TomAugspurger]

If you merge master then the unrelated CI issues should be fixed.

[prasunanand]

@TomAugspurger , Request your review.

[TomAugspurger]

Looking close, thanks!

[TomAugspurger]

Can you add mypy to the linting check in azure-pipelines/posix.yaml?

And there are some linting issues there which aren't failing the build. https://dev.azure.com/dask-dev/dask/_build/results?buildId=818&view=logs&j=34469798-9dec-5482-c96c-ad1eda9ec58d&t=3856b5af-eaa9-56ed-b4cc-e7b6fad51231&l=12 Can you fix those? I'll figure out why CI isn't failing.

[prasunanand]

Thanks @TomAugspurger . I am working on the changes.

[TomAugspurger]

Can you merge master? There will be a merge conflict with posix.yaml. You'll need to move your mypy check to ci/code_checks.sh.

[prasunanand]

I think it should have worked.

dask_ml/metrics/pairwise.py:218: error: Invalid index type "Union[str, Callable[[Any, Any], float]]" for "Dict[str, Any]"; expected type "str"

https://dev.azure.com/dask-dev/dask/_build/results?buildId=834&view=logs&j=9dcbdf5b-840d-5ddd-a5ac-58624932ec86&t=96578b6e-0c8a-5a4c-e6c4-a33b762a5e3a&l=25

[TomAugspurger]

I think it should have worked.

I think PAIRWISE_KERNEL_FUNCTIONS is inferred as Dict[str, Any]. You could perhaps add an annotation to it.

Or... perhaps add an assert isinstance(metric, str) to ~L216. That will convince mypy that the only way we reach that code is when metric is a str, which I think is correct.

---

Just FYI @rth @thomasjpfan. IIRC you're investigating type annotations for scikit-learn. I'm not sure if there's a way for us to share types, since we have different definitions of "ArrayLike", but we can coordinate in places where we overlap.

[prasunanand]

Thanks :)
assert isinstance(metric, str) worked

[prasunanand]

@TomAugspurger , It's all good now :)

[TomAugspurger]

Thanks, this looks good!

[rth]

I'm not sure if there's a way for us to share types, since we have different definitions of "ArrayLike", but we can coordinate in places where we overlap.

@TomAugspurger As far as I know, in scikit-learn we won't type array like objects for now precisely because it's unclear what the type for "ArrayLike" input should be. So at least there shouldn't be an issue of types being over-constrained on the scikit-learn side.

[mrocklin]

So at least there shouldn't be an issue of types being over-constrained on the scikit-learn side.

This is true of most things I think :)