Add aggregation functions for URLs statistics to reduce load times

[Zingzy]

This PR addresses Issue #42

Summary by Sourcery

Add aggregation functions for URL statistics to improve performance and reduce load times by using aggregation pipelines. Refactor data handling to streamline code and enhance efficiency, and sanitize bot names in URL redirection logic to ensure valid database keys.

New Features:

• Introduce aggregation functions for URL statistics to optimize data retrieval and reduce load times.

Enhancements:

• Refactor URL data handling to use aggregation pipelines, improving performance and reducing code complexity.
• Sanitize bot names in URL redirection logic to prevent invalid characters in database keys.

[sourcery-ai]

Reviewer's Guide by Sourcery

This PR optimizes URL statistics retrieval by introducing MongoDB aggregation functions and refactoring the data processing logic. The changes move data transformation from application-level to database-level processing, reducing memory usage and improving response times. The implementation includes new aggregation pipeline utilities and modifications to existing routes to leverage these optimizations.

Sequence diagram for URL statistics retrieval

sequenceDiagram
    participant Client
    participant Server
    participant MongoDB

    Client->>Server: Request /stats/<short_code>
    Server->>MongoDB: Call aggregate_url(pipeline)
    MongoDB-->>Server: Return aggregated URL data
    Server-->>Client: Render stats.html with URL data

Loading

Class diagram for URL data aggregation

classDiagram
    class MongoUtils {
        +load_url(id, projection)
        +aggregate_url(pipeline)
        +load_emoji_url(alias, projection)
        +aggregate_emoji_url(pipeline)
    }

    class PipelineUtils {
        +get_stats_pipeline(short_code)
        +_create_field_transform(field_name)
    }

    MongoUtils --> PipelineUtils: uses

    note for MongoUtils "New aggregation functions added"
    note for PipelineUtils "New utility for creating aggregation pipelines"

Loading

File-Level Changes

Change	Details	Files
Introduce MongoDB aggregation functions to replace manual data processing	Add new aggregate_url and aggregate_emoji_url functions to handle database aggregations Create utility functions to generate MongoDB aggregation pipelines Implement field transformation logic for browser, OS, country, and referrer statistics	utils/mongo_utils.py utils/pipeline_utils.py
Optimize URL data retrieval and processing in routes	Replace manual data processing with aggregation pipeline calls Add projection parameter to limit initial data retrieval Simplify null checks and default value handling Remove redundant data processing loops and transformations	blueprints/stats.py
Improve bot detection and handling	Add sanitization for bot names to prevent MongoDB operator injection Fix bot statistics storage format	blueprints/url_shortener.py

Possibly linked issues

• Optimize Stats retrieval and calculation using Aggregation Pipelines #42: The PR implements MongoDB aggregation pipelines to optimize stats retrieval, directly addressing the issue.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @Zingzy - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[Zingzy]

@sourcery-ai review

[sourcery-ai]

Hey @Zingzy - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider using more specific exception handling instead of catching generic Exceptions in aggregate_url() and aggregate_emoji_url()

Here's what I looked at during the review

• 🟢 General issues: all looks good
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟡 Complexity: 2 issues found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}