Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes of Broken CI #27

Merged
merged 23 commits into from
Mar 22, 2023
Merged

Fixes of Broken CI #27

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
49de5c4
Include new wrapper files in test CmakeLists file
MohitMaliDeveloper Mar 3, 2023
e14a123
Removed wrapper.app from testing
MohitMaliDeveloper Mar 3, 2023
f7e4a5b
Fix path of compiled so file
MohitMaliDeveloper Mar 3, 2023
9b7067d
Fix path according to new folder structure
MohitMaliDeveloper Mar 3, 2023
37618f2
Reverting minSdk,targetSdk version
MohitMaliDeveloper Mar 6, 2023
f427d53
Fix cpp error while compiling
MohitMaliDeveloper Mar 6, 2023
38043b0
Refactored test cases
MohitMaliDeveloper Mar 10, 2023
306916f
Fixed jni directory in test cases
MohitMaliDeveloper Mar 10, 2023
9d962a3
Fixed generated jar path
MohitMaliDeveloper Mar 13, 2023
45f3e86
Fixed jacoco report path
MohitMaliDeveloper Mar 13, 2023
7d07392
Re-enable failed test cases
MohitMaliDeveloper Mar 13, 2023
09539e2
Added test cases for libkiwix package
MohitMaliDeveloper Mar 14, 2023
9a2573e
Removed extra spaces from main.html, we have tested small.zim and the…
MohitMaliDeveloper Mar 14, 2023
97f2b4d
Added searcher test cases
MohitMaliDeveloper Mar 14, 2023
441a154
Add book to library for bookmark test
MohitMaliDeveloper Mar 14, 2023
c5bf83a
Removed absolute path of JNI_INCLUDE_DIR
MohitMaliDeveloper Mar 16, 2023
3073712
Fix typo in `Library.getBookmarks`
mgautierfr Mar 20, 2023
285817f
Fix creation of Bookmark.
mgautierfr Mar 20, 2023
ee9a47d
main.html has extra spaces, which small.zim file not containing so in…
Mar 21, 2023
57dd239
Remove bookmark test cases added
Mar 21, 2023
4ee1e7f
Added new test cases for bookmark
Mar 21, 2023
33dd97d
Removed commented code
Mar 22, 2023
160428d
Changes after review
Mar 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/pull_request.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ jobs:

- name: Compile Project
run: |
./gradlew generateHeaderFilesFromJavaWrapper
./gradlew build
./gradlew generateHeaderFilesFromJavaWrapper build

- name: create unit coverage
run: |
Expand All @@ -42,4 +41,4 @@ jobs:
with:
name: Tests Coverage Report
path: |
android-libkiwixbuild/build/coverage-report/
lib/build/coverage-report/
4 changes: 2 additions & 2 deletions install_deps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ printf "${Green}Downloading libzim ${NC}\n"
printf "\n${Green}Done! ${NC}\n"

printf "${Green}Coping libzim header and so files ${NC}\n"
./gradlew checkCurrentLibzimDate copyLibzimHeaderFiles copyLibzimAndroidArm copyLibzimAndroidArm64 copyLibzimAndroidx86 copyLibzimAndroidx86_64 copyLibzimLinux_x86_64 renameLibzimSoFile
./gradlew checkCurrentLibzimDate checkCurrentLinuxLibzimDate copyLibzimHeaderFiles copyLibzimAndroidArm copyLibzimAndroidArm64 copyLibzimAndroidx86 copyLibzimAndroidx86_64 copyLibzimLinux_x86_64 renameLibzimSoFile
printf "\n${Green}Down! ${NC}\n"

printf "${Green}Downloading libkiwix ${NC}\n"
./gradlew downloadLibkiwixSoAndHeaderFiles unzipLibkiwix
printf "\n${Green}Done! ${NC}\n"

printf "${Green}Coping libkiwix header and so files ${NC}\n"
./gradlew checkCurrentLibkiwixDate copyLibkiwixHeaderFiles copyLibkiwixAndroidArm copyLibkiwixAndroidArm64 copyLibkiwixAndroidx86 copyLibkiwixAndroidx86_64 copyLibkiwixLinux_x86_64 renameLibkiwixSoFile
./gradlew checkCurrentLibkiwixDate checkCurrentLinuxLibkiwixDate copyLibkiwixHeaderFiles copyLibkiwixAndroidArm copyLibkiwixAndroidArm64 copyLibkiwixAndroidx86 copyLibkiwixAndroidx86_64 copyLibkiwixLinux_x86_64 renameLibkiwixSoFile
printf "\n${Green}Done! ${NC}\n"
26 changes: 18 additions & 8 deletions lib/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,9 @@ ext.libkiwix_base_url = 'https://download.kiwix.org/nightly'
ext.libzim_base_url = 'https://download.openzim.org/nightly'
// change this date to get latest libzim .so and header files
ext.nightly_date_for_libkiwix = project.properties["nightly_date_for_libkiwix"] ?: ""
ext.nightly_date_for_libkiwix_linux = project.properties["nightly_date_for_libkiwix_linux"] ?: ""
ext.nightly_date_for_libzim = project.properties["nightly_date_for_libzim"] ?: ""
ext.nightly_date_for_libzim_linux = project.properties["nightly_date_for_libzim_linux"] ?: ""

ext.libkiwix_version = project.properties["libkiwix_version"] ?: ""
ext.libzim_version = project.properties["libzim_version"] ?: ""
Expand All @@ -100,6 +102,10 @@ task checkCurrentLibzimDate() {
project.ext.set("nightly_date_for_libzim", getDateFromPath(buildDir.path, "libzim_android-arm64-"))
}

task checkCurrentLinuxLibzimDate() {
project.ext.set("nightly_date_for_libzim_linux", getDateFromPath(buildDir.path, "libzim_linux-x86_64-"))
}

task unzipLibzim(type: Copy) {
// unzip android arm
from tarTree(buildDir.path + "/libzim_android-arm.tar.gz")
Expand Down Expand Up @@ -150,8 +156,8 @@ task copyLibzimAndroidx86_64(type: Copy) {

task copyLibzimLinux_x86_64(type: Copy) {
// copying linux_x86_64 so file
project.ext.set("libzim_version", getFileFromFolder(buildDir.path + "/libzim_linux-x86_64-" + nightly_date_for_libzim + "/lib/x86_64-linux-gnu/"))
from buildDir.path + "/libzim_linux-x86_64-" + nightly_date_for_libzim + "/lib/x86_64-linux-gnu/" + libzim_version
project.ext.set("libzim_version", getFileFromFolder(buildDir.path + "/libzim_linux-x86_64-" + nightly_date_for_libzim_linux + "/lib/x86_64-linux-gnu/"))
from buildDir.path + "/libzim_linux-x86_64-" + nightly_date_for_libzim_linux + "/lib/x86_64-linux-gnu/" + libzim_version
into buildDir.path
}

Expand Down Expand Up @@ -180,6 +186,10 @@ task checkCurrentLibkiwixDate() {
project.ext.set("nightly_date_for_libkiwix", getDateFromPath(buildDir.path, "libkiwix_android-arm64-"))
}

task checkCurrentLinuxLibkiwixDate() {
project.ext.set("nightly_date_for_libkiwix_linux", getDateFromPath(buildDir.path, "libkiwix_linux-x86_64-"))
}

static String getDateFromPath(String path, String matchesString) {
File folder = new File(path)
if (folder.exists()) {
Expand Down Expand Up @@ -241,8 +251,8 @@ task copyLibkiwixAndroidx86_64(type: Copy) {

task copyLibkiwixLinux_x86_64(type: Copy) {
// copying linux_x86_64 so file
project.ext.set("libkiwix_version", getFileFromFolder(buildDir.path + "/libkiwix_linux-x86_64-" + nightly_date_for_libkiwix + "/lib/x86_64-linux-gnu/"))
from buildDir.path + "/libkiwix_linux-x86_64-" + nightly_date_for_libkiwix + "/lib/x86_64-linux-gnu/" + libkiwix_version
project.ext.set("libkiwix_version", getFileFromFolder(buildDir.path + "/libkiwix_linux-x86_64-" + nightly_date_for_libkiwix_linux + "/lib/x86_64-linux-gnu/"))
from buildDir.path + "/libkiwix_linux-x86_64-" + nightly_date_for_libkiwix_linux + "/lib/x86_64-linux-gnu/" + libkiwix_version
into buildDir.path
}

Expand All @@ -267,13 +277,13 @@ task renameLibkiwixSoFile(type: Copy) {

task copyBuildKiwixSoFile(type: Copy) {
// copying linux_x86_64 so file
from projectDir.path + "/src/androidTests/java/org/kiwix/kiwixlib/libbuildkiwix.so"
from projectDir.path + "/src/test/libbuildkiwix.so"
into buildDir.path
}

task createCodeCoverageReport(type: Exec) {
workingDir "${projectDir}/src/androidTests/java/org/kiwix/kiwixlib/"
commandLine 'sh', '-c', "bash 'compile_and_run_test.sh' ${buildDir}/libs/*app*.jar $buildDir"
workingDir "${projectDir}/src/test/"
commandLine 'sh', '-c', "bash 'compile_and_run_test.sh' ${buildDir}/libs/*lib*.jar $buildDir"
}

task checkCurrentJavaVersion() {
Expand All @@ -285,7 +295,7 @@ task checkCurrentJavaVersion() {

task generateHeaderFilesFromJavaWrapper(type: Exec) {
workingDir "${projectDir}/src/main/java/org/kiwix/"
commandLine 'bash', '-c', "javac -h ${buildDir}/include/javah_generated/ -d ${buildDir}/libzim/ ${getLibzimFiles()} ${getLibkiwixFiles()}"
commandLine 'bash', '-c', "javac -h ${buildDir}/include/javah_generated/ -d ${projectDir}/src/test/ ${getLibzimFiles()} ${getLibkiwixFiles()}"
}

String getLibkiwixFiles() {
Expand Down
8 changes: 7 additions & 1 deletion lib/src/main/cpp/libkiwix/library.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,14 @@ METHOD(jobjectArray, getBookmarks, jboolean onlyValidBookmarks) {
auto bookmarks = THIS->getBookmarks(TO_C(onlyValidBookmarks));
jobjectArray retArray = createArray(env, bookmarks.size(), "org/kiwix/libkiwix/Bookmark");
size_t index = 0;
jclass wrapperClass = env->FindClass("org/kiwix/libkiwix/Bookmark");
jmethodID initMethod = env->GetMethodID(wrapperClass, "<init>", "(J)V");

for (auto bookmark: bookmarks) {
auto wrapper = BUILD_WRAPPER("org/kiwix/libkiwx/Bookmark", bookmark);
// This double new is necessary as we need to allocate the bookmark itself (as a shared_ptr) on the heap but
// we also want the shared_ptr to be stored in the head as we want to have a ptr (cast as long) to it.
shared_ptr<kiwix::Bookmark>* handle = new shared_ptr<kiwix::Bookmark>(new kiwix::Bookmark(std::move(bookmark)));
jobject wrapper = env->NewObject(wrapperClass, initMethod, reinterpret_cast<jlong>(handle));
env->SetObjectArrayElement(retArray, index++, wrapper);
}
return retArray;
Expand Down
2 changes: 2 additions & 0 deletions lib/src/main/cpp/utils.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
#include <jni.h>

#include <mutex>
#include <memory>
#include <cassert>
#include <string>
#include <vector>
#include <set>
Expand Down
4 changes: 4 additions & 0 deletions lib/src/main/java/org/kiwix/libkiwix/Bookmark.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ public Bookmark() {
setNativeBookmark();
}

private Bookmark(long handle) {
nativeHandle = handle;
}

public native void setBookId(String bookId);
public native void setBookTitle(String bookTitle);
public native void setUrl(String url);
Expand Down
57 changes: 31 additions & 26 deletions lib/src/test/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,43 @@ cmake_minimum_required (VERSION 3.16)
project (buildkiwix)

add_library(buildkiwix
SHARED
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/base64.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/book.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/filter.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/kiwixicu.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/kiwixreader.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/kiwixsearcher.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/kiwixserver.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/library.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/manager.cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/utils.h
SHARED
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/book.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/bookmark.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/filter.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/illustration.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/kiwixicu.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/kiwixserver.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/library.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libkiwix/manager.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/archive.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/blob.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/entry.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/entry_iterator.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/item.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/query.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/search.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/search_iterator.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/searcher.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/suggestion_item.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/suggestion_iterator.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/suggestion_search.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/libzim/suggestion_searcher.cpp
${PROJECT_SOURCE_DIR}/../main/cpp/utils.h
Comment on lines +5 to +27
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests should not recompile the wrapper.
It should use the already compiled wrapper (with build target) as kiwix-android will use it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not running test cases on android architecture its linux variant upon which test needs to run .

)

find_package(JNI)

if (JNI_FOUND)
message (STATUS "JNI_INCLUDE_DIRS=${JNI_INCLUDE_DIRS}")
message (STATUS "JNI_LIBRARIES=${JNI_LIBRARIES}")
endif()

include_directories(
/opt/hostedtoolcache/jdk/11.0.17/x64/include
/opt/hostedtoolcache/jdk/11.0.17/x64/include/linux
${CMAKE_SOURCE_DIR}/../../../../../main/cpp
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/include/libkiwix
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/include/libzim
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/include/javah_generated
${CMAKE_SOURCE_DIR}/../../../../../main/cpp/include/utils
${JNI_INCLUDE_DIRS}
${PROJECT_SOURCE_DIR}/../main/cpp
${PROJECT_SOURCE_DIR}/../../build/include/libkiwix
${PROJECT_SOURCE_DIR}/../../build/include/libzim
${PROJECT_SOURCE_DIR}/../../build/include/javah_generated
)

target_link_libraries(buildkiwix
LINK_PUBLIC
${CMAKE_SOURCE_DIR}/../../../../../../build/libkiwix.so
${CMAKE_SOURCE_DIR}/../../../../../../build/libzim.so
LINK_PUBLIC
${PROJECT_SOURCE_DIR}/../../build/libkiwix.so
${PROJECT_SOURCE_DIR}/../../build/libzim.so
)
9 changes: 5 additions & 4 deletions lib/src/test/compile_and_run_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ cmake .
make

# Copy generated .so file to build directory to run test cases
cd ../../../../../../../
cd ../../../
./gradlew copyBuildKiwixSoFile
cd app/src/androidTests/java/org/kiwix/kiwixlib/
cd lib/src/test

KIWIX_LIB_JAR=$1
if [ -z $KIWIX_LIB_JAR ]
Expand Down Expand Up @@ -45,5 +45,6 @@ java -Djava.library.path="$KIWIX_LIB_DIR" \
|| die "Unit test failed"

java -jar jacoco-0.8.7/lib/jacococli.jar report jacoco.exec \
--classfiles org/kiwix/kiwixlib/ \
--html ../../../../../../../build/coverage-report --xml coverage.xml
--classfiles org/kiwix/libkiwix/ \
--classfiles org/kiwix/libzim/ \
--html ../../build/coverage-report --xml coverage.xml
12 changes: 6 additions & 6 deletions lib/src/test/small_zimfile_data/main.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<html>
<head>
<head>
<meta charset="UTF-8">
<title>Test ZIM file</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
</head>

<body>
Test ZIM file
</body>
</html>
<body>
Test ZIM file
</body>
</html>
Loading