Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENP Docker overhaul #125

Closed
8 of 10 tasks
cris-oddball opened this issue Jan 3, 2025 · 8 comments
Closed
8 of 10 tasks

ENP Docker overhaul #125

cris-oddball opened this issue Jan 3, 2025 · 8 comments
Assignees
@kalbfled
Labels
Dev Reviewed Reviewed by Tech Lead Notify Board trigger PM Reviewed Reviewed by Product Manager QA Reviewed Reviewed by Quality Assurance QA Issue requires QA collaboration

Comments

@cris-oddball
Copy link
Contributor

cris-oddball commented Jan 3, 2025 •
edited
Loading

User Story - Business Need

Whilst performing other work, there appear to be problems with the way the container is Dockerized. This ticket aims to overhaul that container and documentation.

  • Sync with Kyle when ticket is picked up.
  • Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story(ies)

As a VA ENP dev
I want a consistent, repeatable Docker container
So that development is simplified

Additional Info and Resources

Issue 1: the Poetry version is pinned down to 1.8.3 when we upgraded to 1.8.4 within the last three months.
Issue 2: Kyle had issues running the existing container (see him for details).
Issue 3: bash is not in the Docker build but the script runs bash
Image

Acceptance Criteria

  • The task needs to be running and the load balancer needs to point to it. (done this sprint)
  • the ENP docker container is overhauled for consistency
  • Docker locks it to Poetry 1.8.4
  • if a bash script is to be run, then bash must be in the $PATH, or clear documentation is included for how the script runs
  • Issues discovered from the GHA Deploy ticket are addressed (such as cert/port/etc.)
  • Documentation is updated
  • This work is added to the sprint review slide deck (key win bullet point and demo slide)

QA Considerations

  • Validate all work performed.

Potential Dependencies
@cris-oddball cris-oddball added Notify Board trigger QA Issue requires QA collaboration QA Reviewed Reviewed by Quality Assurance labels Jan 3, 2025
@cris-oddball cris-oddball mentioned this issue Dec 9, 2024
Closed
6 tasks
@kbelikova-oddball kbelikova-oddball added the PM Reviewed Reviewed by Product Manager label Jan 6, 2025
@k-macmillan k-macmillan added the Dev Reviewed Reviewed by Tech Lead label Jan 6, 2025
@k-macmillan
Copy link
Member

k-macmillan commented Jan 22, 2025 •
edited
Loading

Need to re-evaluate if necessary.
Discussed with the dev that recently overhauled the container. We want to keep this until after the container has been deployed because we expect some wrinkles that will need to be ironed out.

@kalbfled kalbfled self-assigned this Feb 26, 2025
@cris-oddball
Copy link
Contributor Author

QA Conversation

  • May need to talk to other people about any issues they experienced with the Docker container
  • Needs a standardized way to run, @EvanParish may have that info
  • Ensure Poetry is in the docker container as pinned to 1.8.4
  • Ensure bash can run - evan may have resolved this
  • Check documentation?

@kalbfled
Copy link
Member

kalbfled commented Feb 26, 2025 •
edited
Loading

At the start of this ticket:

  • Poetry is already pinned to 1.8.4
  • Bash is installed in /bin, which is in the path
  • The image builds, but the container won't run using the Docker Compose file

@EvanParish
Copy link
Contributor

  • The image builds, but the container won't run using the Docker Compose file

Do you have the api database running when running docker compose up? Because that's necessary now.

@kalbfled
Copy link
Member

I did not have the notification-api database running, but running it first doesn't clear the error.

enp-api  | 2025-02-27 15:58:31.996 | ERROR    | logging:callHandlers:1762 - Traceback (most recent call last):
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
enp-api  |     self._dbapi_connection = engine.raw_connection()
enp-api  |                              ^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3298, in raw_connection
enp-api  |     return self.pool.connect()
enp-api  |            ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 449, in connect
enp-api  |     return _ConnectionFairy._checkout(self)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
enp-api  |     fairy = _ConnectionRecord.checkout(pool)
enp-api  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
enp-api  |     rec = pool._do_get()
enp-api  |           ^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
enp-api  |     with util.safe_reraise():
enp-api  |          ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
enp-api  |     raise exc_value.with_traceback(exc_tb)
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
enp-api  |     return self._create_connection()
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
enp-api  |     return _ConnectionRecord(self)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
enp-api  |     self.__connect()
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
enp-api  |     with util.safe_reraise():
enp-api  |          ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
enp-api  |     raise exc_value.with_traceback(exc_tb)
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
enp-api  |     self.dbapi_connection = connection = pool._invoke_creator(self)
enp-api  |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 646, in connect
enp-api  |     return dialect.connect(*cargs, **cparams)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 622, in connect
enp-api  |     return self.loaded_dbapi.connect(*cargs, **cparams)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/psycopg.py", line 733, in connect
enp-api  |     await_only(creator_fn(*arg, **kw))
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
enp-api  |     return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
enp-api  |     value = await result
enp-api  |             ^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/psycopg/connection_async.py", line 114, in connect
enp-api  |     attempts = await conninfo_attempts_async(params)
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/psycopg/_conninfo_attempts_async.py", line 49, in conninfo_attempts_async
enp-api  |     raise e.OperationalError(str(last_exc))
enp-api  | psycopg.OperationalError: [Errno -2] Name does not resolve
enp-api  | 
enp-api  | The above exception was the direct cause of the following exception:
enp-api  | 
enp-api  | Traceback (most recent call last):
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/starlette/routing.py", line 693, in lifespan
enp-api  |     async with self.lifespan_context(app) as maybe_state:
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
enp-api  |     return await anext(self.gen)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
enp-api  |     async with original_context(app) as maybe_original_state:
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
enp-api  |     return await anext(self.gen)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
enp-api  |     async with original_context(app) as maybe_original_state:
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
enp-api  |     return await anext(self.gen)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
enp-api  |     async with original_context(app) as maybe_original_state:
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
enp-api  |     return await anext(self.gen)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/app/main.py", line 64, in lifespan
enp-api  |     await init_db()
enp-api  |   File "/app/app/db/db_init.py", line 41, in init_db
enp-api  |     await init_napi_metadata()
enp-api  |   File "/app/app/db/db_init.py", line 83, in init_napi_metadata
enp-api  |     async with _engine_napi_read.connect() as conn:
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/base.py", line 121, in __aenter__
enp-api  |     return await self.start(is_ctxmanager=True)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/engine.py", line 274, in start
enp-api  |     await greenlet_spawn(self.sync_engine.connect)
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
enp-api  |     result = context.throw(*sys.exc_info())
enp-api  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3274, in connect
enp-api  |     return self._connection_cls(self)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 148, in __init__
enp-api  |     Connection._handle_dbapi_exception_noconnection(
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2439, in _handle_dbapi_exception_noconnection
enp-api  |     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 146, in __init__
enp-api  |     self._dbapi_connection = engine.raw_connection()
enp-api  |                              ^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 3298, in raw_connection
enp-api  |     return self.pool.connect()
enp-api  |            ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 449, in connect
enp-api  |     return _ConnectionFairy._checkout(self)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
enp-api  |     fairy = _ConnectionRecord.checkout(pool)
enp-api  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
enp-api  |     rec = pool._do_get()
enp-api  |           ^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 179, in _do_get
enp-api  |     with util.safe_reraise():
enp-api  |          ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
enp-api  |     raise exc_value.with_traceback(exc_tb)
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
enp-api  |     return self._create_connection()
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
enp-api  |     return _ConnectionRecord(self)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
enp-api  |     self.__connect()
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
enp-api  |     with util.safe_reraise():
enp-api  |          ^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
enp-api  |     raise exc_value.with_traceback(exc_tb)
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
enp-api  |     self.dbapi_connection = connection = pool._invoke_creator(self)
enp-api  |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/create.py", line 646, in connect
enp-api  |     return dialect.connect(*cargs, **cparams)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 622, in connect
enp-api  |     return self.loaded_dbapi.connect(*cargs, **cparams)
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/psycopg.py", line 733, in connect
enp-api  |     await_only(creator_fn(*arg, **kw))
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
enp-api  |     return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
enp-api  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
enp-api  |     value = await result
enp-api  |             ^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/psycopg/connection_async.py", line 114, in connect
enp-api  |     attempts = await conninfo_attempts_async(params)
enp-api  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
enp-api  |   File "/app/.venv/lib/python3.12/site-packages/psycopg/_conninfo_attempts_async.py", line 49, in conninfo_attempts_async
enp-api  |     raise e.OperationalError(str(last_exc))
enp-api  | sqlalchemy.exc.OperationalError: (psycopg.OperationalError) [Errno -2] Name does not resolve
enp-api  | (Background on this error at: https://sqlalche.me/e/20/e3q8)
enp-api  | 
enp-api  | 2025-02-27 15:58:31.997 | ERROR    | logging:callHandlers:1762 - Application startup failed. Exiting.

@kalbfled
Copy link
Member

I found what seems to be an easy solution to the above traceback. In .env.docker, I changed "host.docker.internal", which is not valid on Linux (probably because Docker Desktop isn't intalled), in the API database connection string to "ci-db-1" , which resolves because all the running containers are in the "ci_default" network.

kalbfled added a commit that referenced this issue Feb 28, 2025
@kalbfled
#125 Docker configuration changes
fd93367
kalbfled added a commit that referenced this issue Mar 3, 2025
@kalbfled
#125 Docker configuration changes
aa637d8
@kalbfled kalbfled mentioned this issue Mar 3, 2025
Merged
14 tasks
@kalbfled
Copy link
Member

kalbfled commented Mar 3, 2025

The branch is deployed, and routes seem to work with Postman. QA tells me she is happy with it. I didn't see anything in the GHA ticket to lead me to believe there are any additional problems I need to address.

github-merge-queue bot pushed a commit that referenced this issue Mar 3, 2025
@kalbfled
#125 Docker configuration changes (#183)
0940e90
@cris-oddball
Copy link
Contributor Author

cris-oddball commented Mar 3, 2025 •
edited
Loading

PR Approved and Merged.

Tested locally by pulling down the code, and walking through the wiki to...

  • Run the notification-api DB, ENP DB and ENP in containers. Tested that routes work in Swagger.
  • Run the notification-api DB and ENP DB in containers and ENP outside of a container. Tested that routes work in Swagger.

Tested in the Cloud by deploying to Perf

  • Tested that the 4 hosted endpoints return data, as expected, using Postman

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kalbfled kalbfled

Labels
Dev Reviewed Reviewed by Tech Lead Notify Board trigger PM Reviewed Reviewed by Product Manager QA Reviewed Reviewed by Quality Assurance QA Issue requires QA collaboration
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@kalbfled @MackHalliday @k-macmillan @EvanParish @cris-oddball @npmartin-oddball @kbelikova-oddball