Update Facets Subset Logic

[herrmann-e]

Overview

For each facet group (each set of keys joined by an "or" or "and"), we should decide whether to use it based on whether the keys are a subset of the facets that the provider defines. This is now on a facet group level, not on the level of the entire list of facet groups.

If we have the following:

facets = [
    {
        'type': 'or',
        'facets': [
            {'key': 'user_id', 'value': 2},
            {'key': 'client_group_id', 'value': 4},
        ]
    },
    {
        'type': 'or',
        'facets': [
            {'key': 'feature_needed', 'value': 'evestmentData'},
        ]
    },
]

then:

• for custom security and model portfolio providers, we should use the first facet group, but not the second.
• for separate account providers, we should use the second facet group, but not the first.
• for providers without any of these facets, we should just return a normal search

How to test

• check out this branch
• in /sites/ycharts, check out this branch AND RUN yc_build. you need to build due to the changes to the docker process that were merged on develop.
• yc_ssh & python manage.py autocompleter --store --remove --clear_cache --name='separate_account'
• in /sites/ycharts, perform the following modifications:
  • edit .docker/docker-compose.yml to update the web step (can copy/paste)

  web:
      image: ycharts_app
      env_file:
          - .docker-env.secret.env
      depends_on:
          valkey:
              condition: service_healthy
      volumes:
          - /sites/ycharts:/sites/ycharts
          - /sites/django-autocompleter:/sites/django-autocompleter
      ports:
          - '8000:8000'
      working_dir: /sites/ycharts
      stdin_open: true
      tty: true
      command: >
          /bin/bash -c "/sites/ycharts/.docker/docker_start.sh no-install-node-modules &&
                        source /sites/ycharts/confs/airflow/dev_airflow_env_settings.sh &&
                        pip install -e /sites/django-autocompleter &&
                        python /sites/ycharts/manage.py runserver [::]:8000"

  • edit requirements.txt to remove the lines:

  django-redis-autocompleter==1.1.2
  # via -r requirements.in
  

• start yc_django
• verify that you have both evestment and separate account data features locally
• in the search bar, type 'tortoise strategic'. confirm that you see both the evestment version and the morningstar version.
• confirm that the results for the other security types match the ones that you get when this branch is not checked out
• find an e2e test user in admin & designate them as 'morningstar data only'. create a subscription feature usage for them of separate account data and confirm they have no evestment data feature
• impersonate them, and confirm that you can only see morningstar separate accounts when typing 'tortoise strategic' into the search bar
• find an e2e test user in admin & designate them as 'evestment data only'. create a subscription feature usage for them of evestment data and confirm they have no morningstar data feature
• impersonate them, and confirm that you can only see evestment separate accounts when typing 'tortoise strategic' into the search bar
• find an e2e test user in admin and verify that they have neither of the relevant features.
• impersonate them and confirm that you can see both evestment and morningstar smas when typing 'tortoise strategic' into the search bar.

[coveralls]

Pull Request Test Coverage Report for Build 20437227736

Details

• 28 of 31 (90.32%) changed or added relevant lines in 2 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage decreased (-0.1%) to 93.074%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
autocompleter/base.py	27	30	90.0%

Files with Coverage Reduction	New Missed Lines	%
autocompleter/base.py	1	93.35%

Totals
Change from base Build 15002648044:	-0.1%
Covered Lines:	981
Relevant Lines:	1054

---

💛 - Coveralls

[herrmann-e]

one weird thing i'm noticing on my local is that when you type 'tortoise' into the search bar slowly, the results flip from including evestment to not with each letter that's typed.... i'm not sure why that's happening. seems likely to be part of my changes somehow because it isn't happening on staging or production.
still investigating ☝️
the behavior in the shell (with local autocompleter changes installed) seems normal though...... shows same results as on staging/prod :/
i take it back.... it stopped happening.

[anthonyp97]

Looks great! Do you know the procedure for creating a new version of this in pypi: https://pypi.org/project/django-redis-autocompleter/

[anthonyp97]

Confirming my understanding: so we are keeping the subset logic check but instead of checking all facets passed in is a subset of the provider's facets, we now check each facet group passed in is a subset of the provider facets. So we need to make sure we pass in new facet groups if we want them to work with providers that have unique facets

[herrmann-e]

Yeah exactly, each 'and' or 'or' facet group will be treated independently of each other; for each facet group we'll check to see if it's a subset of provider facets to see whether we can evaluate and use that particular facet group.

[herrmann-e]

@anthonyp97 bumped up the version in autocompleter/__init__.py but i don't know the procedure for creating a new version beyond that 👀

[anthonyp97]

@herrmann-e so you first merge this to master and then we follow these steps: https://github.com/ycharts/django-autocompleter/blob/master/DEVELOPMENT.md.

Feel free to merge and I can help with pushing to pypi

[herrmann-e]

@anthonyp97 merged but may need help with the pypi stuff yeah 🙏

[anthonyp97]

@herrmann-e cool I just uploaded the new version to pypi using our token:

(ycharts_systems)  anthonypanat@Anthonys-MacBook-Pro  /sites/django-autocompleter   master  twine upload dist/*          
Uploading distributions to https://upload.pypi.org/legacy/
Uploading django_redis_autocompleter-1.1.3-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.6/28.6 kB • 00:00 • 108.4 MB/s
Uploading django_redis_autocompleter-1.1.3.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27.5/27.5 kB • 00:00 • 89.9 MB/s

View at:
https://pypi.org/project/django-redis-autocompleter/1.1.3/

Check it out here: https://pypi.org/project/django-redis-autocompleter/
You can update to this newer version for hotfix