Add Android sample (JNI binding + Java wrapper class)#101
Open
shubham0204 wants to merge 2 commits intomonatis:mainfrom
Open
Add Android sample (JNI binding + Java wrapper class)#101shubham0204 wants to merge 2 commits intomonatis:mainfrom
shubham0204 wants to merge 2 commits intomonatis:mainfrom
Conversation
…size, update tests to remove magnitude comparison
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds
examples/clip.android, an Android sample that interfaces with the shared libraries ofclip.cppthrough JNI. The structure of the project is highly inspired fromllama.cpp/examples/llama.android.Summary of changes
examples/clip.androidis an Android project with two modulesappandclip.appmodule contains the sources for a demo application which allows the user to select an image and enter a text description to compare them (uses Kotlin/Compose)clipmodule containsclip/src/main/cpp/clip-androidthat contains the JNI interfaces which allow communication with the underlying functions fromclip.h.clipmodule containsclip/src/main/java/android/clip/cpp/CLIPAndroid.javawhich is wrapper class containingnativemethods.clipmodule also houses an instrumented testclip/src/androidTest/java/android/example/clip/CLIPInstrumentedTest.ktwritten in KotlinREADME.mdis included in theexamples/clip.androiddirectory which describes how to run the sample app and the instrumented testclip.cppis modified to address issue (Possible) Incorrect assignment of image mean/std values inclip_model_load()#99CMakeLists.txtis modified to add-mcpu=nativewhen${CMAKE_SYSTEM_PROCESSOR} MATCHES "aarch64"only if-DCLIP_NATIVEis enabledclip_image_preprocess_no_resizeinclip.handclip.cppwhich does works exactly likeclip_image_preprocessbut does not perform linear interpolation (no image resizing). Android'sBitmapclass and Python's PIL can handle bilinear interpolation easily, so it would be great to have a method that skips resizing.Note
Though the
clipmodule here in built as an Android library module, it can be built into a 'Java library' module (packaged as a JAR) and used in other Java-only projects as a port toclip.cppIt was fun working on this sample, and I would like @monatis to review it and share feedback.