Add Robust Unit and Integration Tests for Product APITest complete suite #31

VedanshiAwasthi · 2025-04-07T11:40:28Z

This PR introduces a set of critical backend improvements that enhance the database configuration, API response reliability, and testing infrastructure. These updates ensure a more maintainable, predictable, and production-ready system.

What’s Included

Model Enhancements
Added db_alias in meta for all MongoEngine models to support multiple database environments (prod, test).
Ensures smoother integration and test isolation.
Database Configuration
Updated mongoengine.connect() to explicitly define uuidRepresentation and read_preference.
Boosts compatibility with new MongoDB drivers and improves replica set behavior.
View Refactor
Improved response format consistency across all API endpoints.
Unified error handling structure for better frontend-debugging and client parsing.
URL Updates
Cleaned and restructured urls.py for clarity and scalability.
Testing Improvements
->Fixed test failures caused by outdated validation logic.
->Added a full testing environment setup:
->pytest.ini for centralized test config
->Seed scripts to initialize DB with sample products and categories.
->Developed integration + unit tests for:
->Category Creation / Update / Fetch
->Product CRUD workflows
->Asserted response structure, HTTP status codes, and DB mutations.

~~ Why It Matters~~
Production Resilience: Cleaner error handling prevents silent failures and speeds up debugging.
Test Reliability: Clear separation of test DB via db_alias and seeded data ensures consistent results.
Scalability: Refactored code and URLs prepare the app for future module expansion.
Developer Velocity: New tests catch regressions early and act as documentation for expected behavior.

…rror, adjusted Docker Compose setup, and verified API response in the browser.

…dpoints

… a product category and to fetch a list of products belonging to a particular category.

…r DB routing

…gister category endpoints

… category/product views

…support test coverage

…r local testing

… category/product views

…support test coverage

ashutoshkeshri · 2025-04-07T12:12:11Z

backend/django_app/settings.py

 DEBUG = True

-ALLOWED_HOSTS = []
+ALLOWED_HOSTS = ["*"]


We should avoid '*'. This allows requests from any domain

I have removed '*' from ALLOWED_HOSTS and restricted it to localhost and 127.0.0.1 for development. Also ensured CORS_ALLOWED_ORIGINS only allow localhost addresses. Please let me know if any further changes are needed!

ashutoshkeshri · 2025-04-07T12:16:27Z

backend/django_app/urls.py

+# ]
+
+
+def home(request):


Let's remove these from urls.py.
https://nalexn.github.io/separation-of-concerns/

Thanks for the suggestion! I’ve removed the unnecessary logic from urls.py as it’s no longer needed.

ashutoshkeshri · 2025-04-07T12:19:46Z

backend/docker-compose.yaml

    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: example
+    command: ["mongod", "--auth"]


Thanks! Glad the setup is on point!

backend/django_app/settings.py

ashutoshkeshri · 2025-04-07T13:06:24Z

backend/products/admin.py

+from django.contrib import admin
+from .models import Product
+
+# admin.site.register(Product)


Remove comments which no longer is need. Applicable everywhere.

I’ve removed the unnecessary comments to keep the code clean and more readable. Let me know if there's anything else I can improve!

ashutoshkeshri · 2025-04-07T13:08:58Z

backend/products/errorHandler.py

@@ -0,0 +1,18 @@
+from rest_framework.response import Response


Checkout File naming convention - https://realpython.com/python-pep8/#how-to-choose-names

Thanks for the suggestion! I’ll make sure to rename the file according to PEP 8 conventions for better readability and consistency.

ashutoshkeshri · 2025-04-07T13:12:54Z

backend/products/models/CategoryModel.py

+import mongoengine
+import datetime
+
+class ProductCategory(mongoengine.Document):


You can import all the necessary Classes from monogengine directly.

from mongoengine import Document, StringField, DateTimeField

I’ve updated the imports to directly include only the necessary classes from mongoengine in both CategoryModel and ProductModel

ashutoshkeshri · 2025-04-07T13:16:08Z

backend/products/models/CategoryModel.py

+    created_at = mongoengine.DateTimeField(default=lambda: datetime.datetime.now(datetime.UTC))
+    updated_at = mongoengine.DateTimeField(default=lambda: datetime.datetime.now(datetime.UTC))


Since all of the models would need created_at and updated_at. We can create a base class for setting them up and inherit them wherever you need.

class TimestampedDocument(Document): created_at = DateTimeField(default=datetime.datetime.utcnow) updated_at = DateTimeField(default=datetime.datetime.utcnow) meta = {'abstract': True}

I’ve created a TimestampedDocument base class to handle the created_at and updated_at fields, and now Product and ProductCategory inherits from it. I’ve updated the save() method to use datetime.utcnow() for consistency with the TimestampedDocument base class.

ashutoshkeshri · 2025-04-07T13:19:15Z

backend/products/models/CategoryModel.py

+    def save(self, *args, **kwargs):
+        self.updated_at = datetime.datetime.now(datetime.UTC)
+        return super(ProductCategory, self).save(*args, **kwargs)
+


Great that you have overriden the method to update updated at field.
Explore signals to see how we can do this without override. https://www.tutorialspoint.com/mongoengine/mongoengine_signals.htm#:~:text=Signals%20are%20events%20dispatched%20by,receive%20signals%20from%20many%20senders.

Thanks for the suggestion! I've explored MongoEngine signals and implemented the pre_save signal to automatically update the updated_at field.

ashutoshkeshri · 2025-04-07T13:22:26Z

backend/products/models/ProductModel.py

+
+
+


And linter to fix these issues

Thanks for the suggestion. I will use linter (pylint) to identify and fix any issues, including removing unnecessary imports, formatting corrections, and adding docstrings where necessary.

ashutoshkeshri · 2025-04-07T13:35:13Z

backend/products/models/ProductModel.py

+from .CategoryModel import ProductCategory
+
+class Product(mongoengine.Document):
+    name = mongoengine.StringField(max_length=255, required=True)


Should title be unique?

I believe the title field should be unique, as each product category needs to have a distinct identifier. Ensuring uniqueness will help prevent any potential issues with having multiple categories with the same name, which could lead to confusion or errors in the system.

ashutoshkeshri · 2025-04-07T13:42:38Z

backend/products/repositories/Category.py

+class CategoryRepository:
+    @staticmethod
+    def create(title, description=None):
+        if ProductCategory.objects.filter(title=title).first():


Use Get() instead of filter here.

I’ve updated both the create and update functions as per your suggestions. I replaced filter() with get() in the create function for better performance and applied proper exception handling in the update function.

ashutoshkeshri · 2025-04-07T13:49:15Z

backend/products/repositories/Category.py

+    def create(title, description=None):
+        if ProductCategory.objects.filter(title=title).first():
+            raise ValueError("Category with this title already exists.")
+        category = ProductCategory.objects.create(title=title, description=description)


Can you check ProductCategory(title=title, description=description) creates document.
Also do check this out - https://www.dabapps.com/insights/django-models-and-encapsulation/

I have checked and confirmed that ProductCategory(title=title, description=description) is successfully creating the document as expected.

ashutoshkeshri · 2025-04-07T13:56:23Z

backend/products/repositories/Category.py

+
+    @staticmethod
+    def get_category_by_title(title):
+        return ProductCategory.objects.filter(title__iexact=title).first()


Lets store only upper/lowecase for category

Thank you for your suggestion. At present, the case-insensitive approach is being used and several existing functions depend on it, and they have been thoroughly tested. Transitioning to a case-sensitive implementation would require substantial changes and additional testing. For now, I would prefer to maintain the current approach, but I am open to revisiting this if necessary in the future.

ashutoshkeshri · 2025-04-16T13:02:17Z

backend/products/repositories/ProductRepository.py

+class ProductRepository:
+
+    @staticmethod
+    def createProd(prod_details):


Please add types in function definition.
This is for better readability.

I've updated the functions to include type hints as per your recommendation. This will improve the readability of the code and help with understanding the expected inputs and outputs for each function.

ashutoshkeshri · 2025-04-25T03:47:40Z

backend/products/tests/seed_data.py

+def seed_database():
+
+    # Clear existing data
+    ProductCategory.objects.delete()


Can you add error handling for db errors.

I've implemented error handling for MongoDB connections and seeding. The db fixture handles connection issues and switches to mongomock if MongoDB is unavailable. The seed_database function also logs and handles exceptions for better debugging.

ashutoshkeshri · 2025-04-25T03:50:24Z

backend/products/tests/seed_data.py

+    ProductCategory.objects.delete()
+    Product.objects.delete()
+
+    # Create categories


Lets create multiple items for testing.
Also This function currently does many things. Can you create helper function to clear, generate data.

Refactored seed_database function by creating helper functions for clearing data, generating categories, and creating products. Each function has been equipped with error handling and docstrings for better clarity and maintainability.

ashutoshkeshri · 2025-04-25T03:54:47Z

backend/products/tests/conftest.py

+    if mongodb_use_local:
+
+        mongoengine.connect(
+            db='test_products_db',


We should avoid using magic strings.
https://softwareengineering.stackexchange.com/questions/365339/what-is-wrong-with-magic-strings.

Let create constants and use it like below

MONGO_HOST = os.getenv('MONGO_HOST', 'localhost') MONGO_PORT = int(os.getenv('MONGO_PORT', 27017)) MONGO_USER = os.getenv('MONGO_USER', '') MONGO_PASS = os.getenv('MONGO_PASS', '') MONGO_AUTH_DB = os.getenv('MONGO_AUTH_DB', 'admin') TEST_DB_NAME = 'test_products_db'

Rrefactored the MongoDB connection to use constants for configuration values, retrieved from environment variables. This avoids magic strings, improving code maintainability and flexibility.

ashutoshkeshri · 2025-04-25T03:56:20Z

backend/products/tests/conftest.py

+    return is_mongodb_available()
+
+@pytest.fixture(scope="function")
+def db(mongodb_use_local):


Can you add proper loggers and error handling

Added logging and improved error handling for MongoDB connections and database seeding. Replaced print statements with structured logs for better traceability and debugging.

ashutoshkeshri · 2025-04-25T04:02:02Z

backend/products/serializers.py

+
+        categories = ProductCategory.objects(title__in=value)
+        if len(categories) != len(value):
+            raise serializers.ValidationError("Some categories are invalid.")


this doesn't gives us which item is invalid.
Can you also add what item is invalid. This will help in debugging data issues properly.

VedanshiAwasthi and others added 27 commits March 1, 2025 09:59

Updated URL patterns, tested API on Postman, resolved Docker engine e…

b8f15b7

…rror, adjusted Docker Compose setup, and verified API response in the browser.

Handle URL-encoded characters in hello_name endpoint

9e56b7d

Updated README

f4dac96

Updated README

6499ec1

Created Product model

a8ed77a

Implemented CRUD operations for Product API with serialization and en…

f15a23b

…dpoints

Added basic validations to all APIs

45fdf38

added pagination

57cf729

added error messages and validation failures

30915f0

Implemented CSR for crud operations

ceca2a2

Error Handling

4211960

setup mongondb with mongoengine, docker compose

1cb7dc1

Built relationship between product and category model. Added apis for…

9d7f838

… a product category and to fetch a list of products belonging to a particular category.

Implemented functionality to add or remove categories from a product

3d02250

Added unit test for API

ee3a543

chore(models): specify 'db_alias' in MongoEngine model meta for prope…

08fb0b6

…r DB routing

chore(config): update mongo connection with uuidRepresentation and re…

31da1df

…gister category endpoints

refactor(views): improve API response structure and error handling in…

3778c39

… category/product views

updated urls

7880e14

test-fix(validation): improve error handling and validation logic to …

1ad91b3

…support test coverage

chore(tests): add test environment setup and seed scripts for smoothe…

e2dbed7

…r local testing

chore: add pytest configuration file

cb9f7bc

test: add integration and unit tests for category and product workflows

b92bb43

refactor(views): improve API response structure and error handling in…

97e3a5b

… category/product views

test-fix(validation): improve error handling and validation logic to …

53a90af

…support test coverage

test: add integration and unit tests for category and product workflows

df6a639

Merge branch 'main' into test--complete--suite

ce852b1

ashutoshkeshri reviewed Apr 7, 2025

View reviewed changes

backend/django_app/settings.py Show resolved Hide resolved

ashutoshkeshri reviewed Apr 7, 2025

View reviewed changes

ashutoshkeshri reviewed Apr 16, 2025

View reviewed changes

ashutoshkeshri reviewed Apr 25, 2025

View reviewed changes

		@@ -0,0 +1,18 @@
		from rest_framework.response import Response

		created_at = mongoengine.DateTimeField(default=lambda: datetime.datetime.now(datetime.UTC))
		updated_at = mongoengine.DateTimeField(default=lambda: datetime.datetime.now(datetime.UTC))

Add Robust Unit and Integration Tests for Product APITest complete suite #31

Are you sure you want to change the base?

Add Robust Unit and Integration Tests for Product APITest complete suite #31

Uh oh!

Conversation

VedanshiAwasthi commented Apr 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashutoshkeshri Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashutoshkeshri Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ashutoshkeshri Apr 16, 2025 •

edited

Loading

ashutoshkeshri Apr 25, 2025 •

edited

Loading