module google.cloud has no attribute firestore - Incorrect

[adamcunnington-mlg]

Bug Report

module google.cloud has no attribute firestore
The bug is that this statement is not true.

There have been a few related stackoverflow issues (e.g. 1, 2) relating to incorrect "module x as no attribute y" specifically relating to google.cloud.
Some issues were sorted by recreating the virtual env.
Other issues on this repo were historically traced back to using incremental mode but solution seemed incomplete (e.g. 1, 2).
We disabled mypy cache (e.g. incremental mode) which dealt with a chunk of the problem

To Reproduce
Hard to know exactly how to reproduce because the problem with this issue is it seems random and indeterministic - even after disabling incremental mode.

All I can tell you is in our code we do:

from google.cloud import bigquery, firestore

Both google-cloud-bigquery and google-cloud-firestore are in setup.cfg/requirements.txt and installed in the virtual environment which mypy is being activated from.

We have no issues (currently) with the bigquery import but we do with firestore. They are completely identical for all intents and purposes here.

Expected Behaviour

No complaint about missing attribute.

Actual Behaviour

Incorrect complaint about missing attribute.

Your Environment

mypy 0.950 (compiled: yes)
python 3.9.x

Please can we get to the bottom of this once and for all?

[hauntsaninja]

Are you using namespace_packages=True? If not, please try with that. This will be made the default at some point soon.

0.950 should have fixes for all known incremental mode bugs, in particular, #12250 . If you do get a repro, that would be helpful.

[adamcunnington-mlg]

Will try and see if it fixes

[adamcunnington-mlg]

It has no effect unfortunately - which is unsurprising anyway because then we'd have to understand why bigquery wasn't causing an issue here but firestore was.

For now, we will just type: ignore.
I don't know how we can reproduce it.

[shevron]

I'm seeing a similar behavior - with:

from google.cloud import bigquery, storage

I get:

$ mypy --no-incremental mynamespace/
mynamespace/package/module.py:19: error: Module "google.cloud" has no attribute "storage"

This is with mypy 0.961;
My code is using namespace packages (i.e. mynamespace in the example above is a namespace package), but so far I haven't had any issues.

I was able to work around it for now by doing:

from google.cloud import bigquery
from google.cloud.storage import Client as StorageClient

which of course may not work for everyone, but for me it's good enough. I am not sure why it would work, but it does.

EDIT: this is on Python 3.10.2 + Poetry

[hauntsaninja]

Okay, had some free time, so looked into this. There are a couple different things alluded to in this issue:

The first issue is that google-cloud-firestore (and google-cloud-storage) are not py.typed packages. This means that mypy does not know anything about them and things are expected to fail. Note that google-cloud-bigquery does have a py.typed, which is why it doesn't cause any issues.

λ mypy -c 'from google.cloud import firestore'
<string>:1: error: Skipping analyzing "google.cloud": module is installed, but missing library stubs or py.typed marker
<string>:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

The second issue is that mypy gives an unhelpful diagnostic / gives a diagnostic with an inconsistent error code in cases involving multiple imports from the same namespace package ("no attribute" instead of the clearer message mentioning py.typed above). See #10360 for more details. I believe this issue is mainly just a duplicate of that one, so I'll be closing this.

The third issue (alluded to here, but without repro) is with incremental mode. Several issues with these have been fixed, but there is one recently reported issue that is not yet fixed, see #13085. I'll have a fix for this soon (which should hopefully be more robust than patches to similar prior issues).