Update backend, frontend, sample data, readme, opensearch schema

Author: tqtg

README.md

@@ -27,8 +27,8 @@ Cornac is endorsed by ACM RecSys for evaluating and reproducing recommendation a
 The architecture consists of the following components:
 
 1. **Cornac-AB Backend Server** ([Backend source code](backend)): Handles API endpoints and business logic.
-2. **Cornac-AB Frontend** ([Frontend source code](frontend)): Offers a user interface for interacting with the A/B tests.
-3. **Books-AB User Interaction Frontend** ([User Interaction Frontend source code](democlient)): Provides a frontend for user interactions.
+2. **Books-AB User Interaction Frontend** ([User Interaction Frontend source code](democlient)): Provides a frontend for user interactions.
+3. **Cornac-AB Frontend** ([Frontend source code](frontend)): Offers a user interface to setup, track and evaluate models for the A/B tests.
 4. **OpenSearch & OpenSearch Dashboards** ([Official site](https://opensearch.org)): Data indexing, search, and visualization.
 5. **GoodReads 10k Dataset** ([Goodbooks-10k repository](https://github.com/zygmuntz/goodbooks-10k)): Preloaded data for demonstration purposes.
 
@@ -40,15 +40,92 @@ docker compose up
 ```
 This command will start all the required components and load the GoodReads dataset into OpenSearch for A/B testing and visualization.
 
-## Accessing the Solution
+## Usage Guide
+
+### Accessing the Solution
 
 Once the containers are running, you can access the various parts of the solution via the following URLs:
-- Cornac-AB Backend Server `localhost:8080`
-- Cornac-AB Frontend `localhost:8081`
-- Books-AB User Interaction Frontend `localhost:8082`
+- [Books-AB User Interaction Frontend](#2) `localhost:8082`
+- [Cornac-AB Frontend]() `localhost:8081`
 - OpenSearch API `localhost:9200`
 - OpenSearch Dashboards `localhost:5601`
 
+### 1. Cornac-AB Backend
+
+This backend server is built on [Spring](https://spring.io/). Spring is a production grade scalable framework for building web applications.
+
+This solution connects to a local h2 database (which could be easily replaceable with most SQL databases supported by Spring).
+
+[Cornac instances](https://cornac.readthedocs.io/en/stable/user/iamadeveloper.html#running-an-api-service) (Based on Flask) are run on this container, and restarts automatically should it be found to be down.
+
+The [Spring Data OpenSearch](https://github.com/opensearch-project/spring-data-opensearch) library has been used to connect the backend to the OpenSearch service.
+
+### 2. Books-AB User Interaction Frontend
+
+#### Accessing the frontend
+http://localhost:8082/
+
+Included in this solution is a sample frontend that showcases how the solution receives user interactions.
+
+#### A. Selecting a User ID
+
+<img src="screenshots/demo-1.png" alt="demo" width="500"/> 
+
+Users will first enter their User ID. To decrease the size of the sample data, we only included past book history data of user IDs 100-199.
+
+#### B. Viewing Books Recommendations
+<img src="screenshots/demo-2.png" alt="demo" width="500"/> 
+
+Click **Explore Books**. A particular model is allocated by the backend, which provides recommendations as on this explore page. Recommendation records are stored in OpenSearch as well. 
+
+#### C. Providing Book Feedback
+Users could select to view more details by clicking on them.
+
+<img src="screenshots/demo-3.png" alt="demo" width="500"/> 
+
+When a user clicks on a book, we register that as a feedback. This feedback will be attributed with the **click** action on OpenSearch.
+
+<img src="screenshots/demo-4.png" alt="demo" width="500"/>  
+
+Further in this view, users will be able to rate the book by click the stars icon, which will be attributed with a **rate** action as a feedback on OpenSearch.
+
+#### D. Viewing User Interactions
+
+
+User interaction in this frontend will be recorded in OpenSearch as **recommendations** and **feedbacks**. These can be viewed in the Cornac-AB frontend dashboards in real-time, as shown in the next section.
+
+#### 3. Cornac-AB Frontend
+
+#### Accessing the frontend
+http://localhost:8081/
+
+<img src="screenshots/frontend-1.png" alt="demo" width="500"/> 
+
+You'll be welcomed with this screen.
+
+#### A. Viewing the Dashboards
+
+<img src="screenshots/frontend-2.png" alt="demo" width="500"/>
+
+Going to the dashboard screen will show you multiple dashboards, including the Users, Recommendations and Feedback dashboards. Sample data based on the Goodbooks 10k dataset has already been generated and inserted for you.
+
+#### B. Evaluating your Models
+<img src="screenshots/frontend-3.png" alt="demo" width="500"/>
+
+Under the Feedback Dashboard section, you will be able to filter data, and further compare your models using the Cornac evaluation features by selecting the **Run Cornac Evaluation** button.
+
+<img src="screenshots/frontend-4.png" alt="demo" width="500"/>
+
+A summary of the data that will be put through Cornac's evaluation services will be shown. You could add more metrics by selecting the **Add Metric** button will allow you to add more metrics as shown below.
+
+<img src="screenshots/frontend-5.png" alt="demo" width="300"/>
+
+#### C. Viewing Evaluation Results
+
+<img src="screenshots/frontend-6.png" alt="demo" width="500"/>
+
+You will then be shown with the metric results, along with the p-values of individual models to evaluate the performance of your models.
+
 ## Further Usage
 
 Cornac-AB is a solution which showcases how A/B Testing could be done and visualized as a forward testing experiment. Feel free to further contribute, or fork the repository and extend it to your own application needs.
@@ -61,6 +138,20 @@ This project welcomes contributions and suggestions. Before contributing, please
 
 If you use Cornac in a scientific publication, we would appreciate citations to the following papers:
 
+<details>
+  <summary><a href="https://ieeexplore.ieee.org/abstract/document/9354572">Cornac-AB: An Open-Source Recommendation Framework with Native A/B Testing Integration</a>, Ong <i>et al.</i>, In Proceedings of the ACM Web Conference 2024.</summary>
+
+  ```
+  @inproceedings{ong2024cornacab,
+    title={Cornac-AB: An Open-Source Recommendation Framework with Native A/B Testing Integration},
+    author={Ong, Darryl and Truong, Quoc-Tuan and Lauw, Hady W},
+    journal={Proceedings of the ACM Web Conference 2024},
+    pages={xx--yy},
+    year={2024}
+  }
+  ```
+</details>
+
 <details>
   <summary><a href="http://jmlr.org/papers/v21/19-805.html">Cornac: A Comparative Framework for Multimodal Recommender Systems</a>, Salah <i>et al.</i>, Journal of Machine Learning Research, 21(95):1–5, 2020.</summary>
 
@@ -77,34 +168,6 @@ If you use Cornac in a scientific publication, we would appreciate citations to
   ```
 </details>
 
-<details>
-  <summary><a href="https://ieeexplore.ieee.org/abstract/document/9354572">Exploring Cross-Modality Utilization in Recommender Systems</a>, Truong <i>et al.</i>, IEEE Internet Computing, 25(4):50–57, 2021.</summary>
-
-  ```
-  @article{truong2021exploring,
-    title={Exploring Cross-Modality Utilization in Recommender Systems},
-    author={Truong, Quoc-Tuan and Salah, Aghiles and Tran, Thanh-Binh and Guo, Jingyao and Lauw, Hady W},
-    journal={IEEE Internet Computing},
-    year={2021},
-    publisher={IEEE}
-  }
-  ```
-</details>
-
-<details>
-  <summary><a href="http://jmlr.org/papers/v21/19-805.html">Multi-Modal Recommender Systems: Hands-On Exploration</a>, Truong <i>et al.</i>, ACM Conference on Recommender Systems, 2021.</summary>
-
-  ```
-  @inproceedings{truong2021multi,
-    title={Multi-modal recommender systems: Hands-on exploration},
-    author={Truong, Quoc-Tuan and Salah, Aghiles and Lauw, Hady},
-    booktitle={Fifteenth ACM Conference on Recommender Systems},
-    pages={834--837},
-    year={2021}
-  }
-  ```
-</details>
-
 ## License
 
 [Apache License 2.0](LICENSE)

backend/src/main/java/ai/preferred/cornac/entity/Feedback.java

@@ -26,6 +26,8 @@ public class Feedback {
     private String userId;
     @Field(name="item_id", type = FieldType.Keyword)
     private String itemId;
+    @Field(name="recommendation_id", type = FieldType.Keyword)
+    private String recommendationId;
     private String model;
     private Integer rating;
     private String action; // 'click', 'rating'

backend/src/main/java/ai/preferred/cornac/entity/RecommendLog.java

@@ -23,12 +23,13 @@ public class RecommendLog {
     private String id;
     @Field(type = FieldType.Keyword)
     private Integer experimentId;
-    @Field(type = FieldType.Keyword)
+    @Field(name="user_id", type = FieldType.Keyword)
     private String userId;
     @Field(type = FieldType.Date, pattern = "uuuu-MM-dd'T'HH:mm:ss.SSSSSS||uuuu-MM-dd'T'HH:mm:ss.SSS||uuuu-MM-dd'T'HH:mm:ss||uuuu-MM-dd", format = {})
     private LocalDateTime timestamp;
     @Field(type = FieldType.Keyword)
     private List<String> recommendations;
+    private String model;
 
     private boolean isFallback;
     private String fallbackReason;

backend/src/main/java/ai/preferred/cornac/service/CornacService.java

@@ -371,6 +371,9 @@ public CornacEvaluationResponse postCornacInstanceEvaluation(CornacInstance corn
                 .body(BodyInserters.fromValue(evaluationRequest))
                 .retrieve()
                 .bodyToMono(CornacEvaluationResponse.class)
+                .doOnError(error -> {
+                    error.printStackTrace();
+                })
                 .block();
     }
 

backend/src/main/java/ai/preferred/cornac/service/FeedbackService.java

@@ -36,22 +36,15 @@ public Feedback addFeedback(String recommendId, String itemId, Integer rating, S
         UserAbAllocation userAbAllocation = userAbAllocationRepository.findByExperimentIdAndUserId(
                 recommendLog.getExperimentId(), recommendLog.getUserId()
         );
-        List<CornacInstance> cornacInstances = cornacInstanceRepository.findCornacInstanceByExperimentId(recommendLog.getExperimentId());
-
-        String cornacInstanceName;
-        if (userAbAllocation == null) {
-            cornacInstanceName = "unallocated";
-        } else {
-            cornacInstanceName = cornacInstances.get(userAbAllocation.getAbGroup()).getServiceName();
-        }
 
         Feedback feedback = new Feedback();
         feedback.setItemId(itemId);
         feedback.setRating(rating);
         feedback.setUserId(recommendLog.getUserId());
         feedback.setTimestamp(LocalDateTime.now());
         feedback.setExperimentId(recommendLog.getExperimentId());
-        feedback.setModel(cornacInstanceName);
+        feedback.setModel(recommendLog.getModel());
+        feedback.setRecommendationId(recommendId);
         feedback.setAction(action);
 
         return feedbackRepository.save(feedback);

backend/src/main/java/ai/preferred/cornac/service/RecommendService.java

@@ -129,6 +129,7 @@ public RecommendLogDto getRecommendations(String userId, String k){
         log.setUserId(userId);
         log.setTimestamp(LocalDateTime.now());
         log.setRecommendations(recommendation.getRecommendations());
+        log.setModel(cornacInstance == null ? "unallocated" : cornacInstance.getServiceName());
 
         RecommendLog recommendLog = recommendLogRepository.save(log);
         return convertToRecommendLogDto(recommendLog);