Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: db setting error and duplicated keyword submission #113

Merged
merged 51 commits into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
112019d
feat: merge dev (#36)
binaryYuki Jan 14, 2025
c53ac21
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 14, 2025
cf02614
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 14, 2025
8354a76
feat: file upload handler
JAAAACCCCCCKKKK Jan 15, 2025
2079e31
feat: file upload handler
JAAAACCCCCCKKKK Jan 15, 2025
609fdf8
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 15, 2025
ad8962a
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 15, 2025
f2f3e49
ready for editing test
JAAAACCCCCCKKKK Jan 15, 2025
f338da5
Merge branch 'master' of https://github.com/JAAAACCCCCCKKKK/Zephyr
JAAAACCCCCCKKKK Jan 15, 2025
ff0d1f8
test: routes.
JAAAACCCCCCKKKK Jan 15, 2025
6abec62
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 15, 2025
ac020d8
test message
JAAAACCCCCCKKKK Jan 15, 2025
9091cc7
Delete src/main/java/Zephyr/草稿纸
binaryYuki Jan 15, 2025
d2aa241
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 15, 2025
f75703a
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 17, 2025
46f3612
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 17, 2025
19a1c6d
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 17, 2025
08d7f77
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 17, 2025
254ad48
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 17, 2025
ddb36e0
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 17, 2025
2e4e4e8
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 18, 2025
469173b
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 18, 2025
2cac4e1
Merge branch 'master' of https://github.com/iiiyuki/Zephyr
actions-user Jan 18, 2025
0579497
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 19, 2025
478bb08
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 19, 2025
46d49f0
feat: a naive approach of a text based online fraud detector.
JAAAACCCCCCKKKK Jan 19, 2025
d78e924
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 19, 2025
d1c42d9
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 19, 2025
566449a
chore: update gitignore config
JAAAACCCCCCKKKK Jan 20, 2025
5d15c74
chore: update gitignore config
JAAAACCCCCCKKKK Jan 20, 2025
6b035e5
feat: cache manager
binaryYuki Jan 20, 2025
825361d
feat: caches
binaryYuki Jan 20, 2025
c0e8a6a
Merge pull request #1 from binaryYuki/JAAAACCCCCCKKKK/master
JAAAACCCCCCKKKK Jan 20, 2025
67928e5
chore: correct db error
JAAAACCCCCCKKKK Jan 20, 2025
69b6ea2
feat: suspicious substring uploader.
JAAAACCCCCCKKKK Jan 20, 2025
4e420c8
Merge branch 'master' into master
JAAAACCCCCCKKKK Jan 20, 2025
b992b26
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 21, 2025
dcdc5ee
chore: debugging handleKeywordSubmit.
JAAAACCCCCCKKKK Jan 21, 2025
3f84531
feat: keyword uploader
JAAAACCCCCCKKKK Jan 22, 2025
4ee3d86
feat: keyword uploader
JAAAACCCCCCKKKK Jan 23, 2025
2316a75
Merge remote-tracking branch 'origin/master'
JAAAACCCCCCKKKK Jan 23, 2025
c0f1897
chore:fix bugs on keyword submission
JAAAACCCCCCKKKK Jan 23, 2025
9b90f8a
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 24, 2025
148b207
Merge branch 'iiiyuki:master' into master
JAAAACCCCCCKKKK Jan 24, 2025
9b598e4
feat: universal entity manager in main verticle and subroutes
JAAAACCCCCCKKKK Jan 27, 2025
4815d12
Merge branch 'master' into master
JAAAACCCCCCKKKK Jan 28, 2025
d78e06a
fix: db setting error and duplicated keyword submission
JAAAACCCCCCKKKK Jan 31, 2025
4fea363
fix: db setting error and duplicated keyword submission
JAAAACCCCCCKKKK Feb 3, 2025
749c9b3
fix: jdbc grammar violations
binaryYuki Feb 3, 2025
4edfcf1
chore: remove random comments
binaryYuki Feb 3, 2025
231d75d
fix: misusing of status codes
binaryYuki Feb 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 71 additions & 109 deletions src/main/java/Zephyr/JackRoutes.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,15 @@
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.*;

import static Zephyr.MainVerticle.dbHelperInstance;


/**
* @author Jingyu Wang
*/
Expand Down Expand Up @@ -53,7 +60,7 @@ public Router getSubRouter() {
);

router.post("/analyze/text/uploads").handler(ctx -> {
ctx.fail(402);
ctx.fail(501);
// List<FileUpload> uploads = ctx.fileUploads();
// for(FileUpload u:uploads){
// String fileName = u.fileName();
Expand Down Expand Up @@ -83,25 +90,6 @@ public Router getSubRouter() {
</form>""");
});

router.post("/analyze/text/uploads").handler(ctx -> {
List<FileUpload> uploads = ctx.fileUploads();
for(FileUpload u:uploads){
String fileName = u.fileName();
String tail = fileName.substring(fileName.lastIndexOf("."));
if ("text/plain".equals(u.contentType())
&&".txt".equals(tail)
&&"UTF-8".equals(u.charSet())
&&u.size()<=50000)
//校验通过,开始处置
{
handleFileUpload(ctx, u);
} else{
//校验不通过,强制删除
ctx.fail(400);

}
}
});

/**
* 添加一个关键词
Expand Down Expand Up @@ -137,82 +125,65 @@ private void handleInfo(RoutingContext ctx) {
.end(response.encode());
}

private void handleKeywordSubmit(RoutingContext ctx){
//获取请求体
RequestBody body = ctx.body();
//转化为JSON对象
JsonObject object = body.asJsonObject();
String keyword = object.getString("input");
EntityManager entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
//提取所有已知的可疑关键词
List<AcceptedSequences> acceptedSequences =
entityManager
.createQuery("SELECT a FROM AcceptedSequences a", AcceptedSequences.class)
.getResultList();
//如关键词已存在,则增加它的权重1,并返回
for(int i=0;i<acceptedSequences.size();i++){
if(acceptedSequences.get(i).getList().getFirst().equals(keyword)){
acceptedSequences.get(i).setRate(acceptedSequences.get(i).getRate()+1);
entityManager.getTransaction().begin();
entityManager.persist(acceptedSequences.get(i));
entityManager.getTransaction().commit();
entityManager.close();
JsonObject response = new JsonObject()
.put("status", "uploaded")
.put("content", keyword)
.put("timestamp", System.currentTimeMillis());

ctx.response()
.putHeader("Content-Type", "application/json")
.end(response.encode());
return;
}
}
entityManager.close();
entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
//遍历完成仍不存在,存入新关键词
try {
// Begin a transaction
entityManager.getTransaction().begin();

// 创建一个新AcceptedSequences对象
private void handleKeywordSubmit(RoutingContext ctx) {
// 获取请求体
JsonObject object = ctx.body().asJsonObject();
String keyword = object.getString("input").toLowerCase();
JsonObject updateBias = new JsonObject();

try (Connection connection = dbHelper.getDataSource().getConnection()) {
// 首先检查是否已经有匹配的记录
String selectQuery = "SELECT rate FROM accepted_sequences WHERE content = ?";
try (PreparedStatement selectStmt = connection.prepareStatement(selectQuery)) {
selectStmt.setString(1, keyword);
ResultSet rs = selectStmt.executeQuery();

if (rs.next()) {
// 如果记录已存在,更新 rate
int newRate = rs.getInt("rate") + 1;
String updateQuery = "UPDATE accepted_sequences SET rate = ?, last_updated_at = ? WHERE content = ?";
try (PreparedStatement updateStmt = connection.prepareStatement(updateQuery)) {
updateStmt.setInt(1, newRate);
updateStmt.setLong(2, System.currentTimeMillis());
updateStmt.setString(3, keyword);
updateStmt.executeUpdate();
}

AcceptedSequences newSequence = new AcceptedSequences();
newSequence.setContent(keyword);
newSequence.setTimeStampString(""+System.currentTimeMillis());
newSequence.setRate(1);
newSequence.setCreatedAt(""+System.currentTimeMillis()
);
// Persist the new sequence
entityManager.persist(newSequence);
updateBias.put("success", true)
.put("content", keyword)
.put("rate", newRate)
.put("timestamp", System.currentTimeMillis());
} else {
// 如果记录不存在,插入新记录
String insertQuery = "INSERT INTO accepted_sequences (content, rate, created_at, last_updated_at) VALUES (?, ?, ?, ?)";
try (PreparedStatement insertStmt = connection.prepareStatement(insertQuery)) {
insertStmt.setString(1, keyword);
insertStmt.setInt(2, 1);
insertStmt.setLong(3, System.currentTimeMillis());
insertStmt.setLong(4, System.currentTimeMillis());
insertStmt.executeUpdate();
}

// Commit the transaction
entityManager.getTransaction().commit();
} catch (Exception e) {
// Rollback the transaction in case of errors
if (entityManager.getTransaction().isActive()) {
entityManager.getTransaction().rollback();
updateBias.put("success", true)
.put("content", keyword)
.put("rate", 1)
.put("timestamp", System.currentTimeMillis());
}
}
ctx.response().setStatusCode(500).end("Error: " + e.getMessage());
} finally {
// Close the entity manager
entityManager.close();
//return response
JsonObject response = new JsonObject()
.put("status", "uploaded")
.put("content", keyword)
.put("timestamp", System.currentTimeMillis());

ctx.response()
.putHeader("Content-Type", "application/json")
.end(response.encode());
} catch (SQLException e) {
updateBias.put("success", false).put("message", "Database error: " + e.getMessage());
}

// 将结果返回给客户端
ctx.response()
.putHeader("Content-Type", "application/json")
.end(updateBias.encode());
}

//关键词检测器 A naive approach of a text-based fraud detector.
private void handleFileUpload(RoutingContext ctx, FileUpload u){
Path path = Paths.get("Zephyr", "uploads", u.uploadedFileName());
EntityManager entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
EntityManager entityManager = dbHelperInstance.getEntityManager();
try {
// Begin a transaction
entityManager.getTransaction().begin();
Expand Down Expand Up @@ -246,7 +217,7 @@ private void handleFileUpload(RoutingContext ctx, FileUpload u){
// Close the entity manager
entityManager.close();
}
entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
entityManager = dbHelperInstance.getEntityManager();
//提取指定文件
List<Uploads> uploads = entityManager.createQuery
("SELECT filePath FROM Uploads WHERE processed = FALSE", Uploads.class).getResultList();
Expand Down Expand Up @@ -276,24 +247,30 @@ private void handleFileUpload(RoutingContext ctx, FileUpload u){
.putHeader("Content-Type", "application/json")
.end(response.encode());
}
upload.setProcessed(true);
//处理结束, 文件在DB改为可覆盖状态,被BodyHandler从文件夹移除(Line:48)
upload.setProcessed(true);
entityManager.close();
}
}
//遍历列表后仍未发现指定文件,处理失败
ctx.fail(404);
entityManager.close();
}

private int[] processFile(String path) {
/*施工中*/
int res = 0;
int lines = 0;
List<AcceptedSequences> acceptedSequences;
try (EntityManager entityManager = dbHelper.getEntityManagerFactory().createEntityManager()) {
try {
EntityManager entityManager = dbHelperInstance.getEntityManager();
//提取所有已知的可疑关键词
acceptedSequences = entityManager
.createQuery("SELECT a FROM AcceptedSequences a", AcceptedSequences.class)
.getResultList();
entityManager.close();
}
catch (Exception e){
throw new RuntimeException(e);
}
String line;
try{
Expand All @@ -314,27 +291,13 @@ private int[] processFile(String path) {
return new int[]{res, lines};
}

private void submitKeyWord(String keyword){
EntityManager entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
//提取所有已知的可疑关键词
List<AcceptedSequences> acceptedSequences =
entityManager
.createQuery("SELECT a From AcceptedSequences a", AcceptedSequences.class)
.getResultList();
for (AcceptedSequences acceptedSequences1: acceptedSequences){
if(acceptedSequences1.getList().getFirst().equals(keyword)){

}
}
}

// orm test
private void testOrm(RoutingContext ctx) {
// 假设要查找或更新 ID 为 1 的实体
Long id = 1L;

// Get the entity manager
EntityManager entityManager = dbHelper.getEntityManagerFactory().createEntityManager();
EntityManager entityManager = dbHelperInstance.getEntityManager();

try {
// Begin a transaction
Expand Down Expand Up @@ -373,19 +336,18 @@ private void testOrm(RoutingContext ctx) {
}

// 新EntityManager
entityManager = dbHelper.getEntityManagerFactory().createEntityManager();

entityManager = dbHelperInstance.getEntityManager();
// 从数据库中查询所有服务
List<Service> services = entityManager.createQuery("SELECT s FROM Service s", Service.class).getResultList();
JsonObject response = new JsonObject()
.put("status", "ok")
.put("message", "Test ORM Success")
.put("timestamp", System.currentTimeMillis())
.put("services", services);
entityManager.close();
ctx.response()
.putHeader("Content-Type", "application/json")
.end(response.encode());
entityManager.close();
}
}

Expand Down
39 changes: 19 additions & 20 deletions src/main/java/Zephyr/MainVerticle.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
import io.vertx.ext.web.Router;
import io.vertx.ext.web.RoutingContext;
import io.vertx.ext.web.handler.BodyHandler;
import jakarta.persistence.EntityManagerFactory;

import java.nio.file.Paths;
import java.sql.Connection;
Expand All @@ -24,7 +23,7 @@
*/
public class MainVerticle extends AbstractVerticle {
private static final Logger logger = LoggerFactory.getLogger(MainVerticle.class);

public static dbHelper dbHelperInstance;
@Override
public void start(Promise<Void> startPromise) {
logger.info("Starting MainVerticle initialization...");
Expand Down Expand Up @@ -53,7 +52,7 @@ public void start(Promise<Void> startPromise) {
private Future<Void> initializeServices() {
Promise<Void> promise = Promise.promise();

dbHelper db = new dbHelper(vertx);
dbHelperInstance = new dbHelper(vertx);
ValKeyManager valKeyManager = ValKeyManager.getInstance();

// 测试 ValKey 连接
Expand All @@ -67,7 +66,7 @@ private Future<Void> initializeServices() {
}

// 初始化数据库
db.init(ar -> {
dbHelperInstance.init(ar -> {
if (ar.succeeded()) {
logger.info("Database initialized successfully.");
promise.complete();
Expand All @@ -94,26 +93,26 @@ private Router setupHttpServer() {

// 添加 BodyHandler
router.route().handler(BodyHandler.create()
.setBodyLimit(50_000)
.setDeleteUploadedFilesOnEnd(true)
.setHandleFileUploads(true)
.setUploadsDirectory(Paths.get("Zephyr", "uploads").toString())
.setMergeFormAttributes(true)
);
.setBodyLimit(50_000)
.setDeleteUploadedFilesOnEnd(true)
.setHandleFileUploads(true)
.setUploadsDirectory(Paths.get("Zephyr", "uploads").toString())
.setMergeFormAttributes(true)
);

// 配置子路由
router.route("/api/jack/*").subRouter(new JackRoutes(vertx).getSubRouter());
router.route("/api/austin/*").subRouter(new AustinRoutes(vertx).getSubRouter());
router.route("/api/v1/*").subRouter(new YukiRoutes(vertx).getSubRouter());
// 配置子路由
router.route("/api/jack/*").subRouter(new JackRoutes(vertx).getSubRouter());
router.route("/api/austin/*").subRouter(new AustinRoutes(vertx).getSubRouter());
router.route("/api/v1/*").subRouter(new YukiRoutes(vertx).getSubRouter());

// 添加健康检查路由
router.get("/healthz").handler(this::handleHealthCheck);
// 添加健康检查路由
router.get("/healthz").handler(this::handleHealthCheck);

// 错误处理
setupErrorHandlers(router);
// 错误处理
setupErrorHandlers(router);

return router;
}
return router;
}

private void handleHealthCheck(RoutingContext ctx) {
JsonObject responseObject = new JsonObject();
Expand Down
Loading