-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose Llava as a shared library for downstream projects #3613
Merged
Merged
Changes from 17 commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
0209d39
wip llava python bindings compatibility
damian0815 3c10d9f
add external llava API
damian0815 770dc9d
add base64 in-prompt image support
damian0815 8224ca5
wip refactor image loading
damian0815 c693208
refactor image load out of llava init
damian0815 0889117
cleanup
damian0815 f83c060
further cleanup; move llava-cli into its own file and rename
damian0815 e2cd07c
move base64.hpp into common/
damian0815 f8eddcf
collapse clip and llava libraries
damian0815 b9f533b
move llava into its own subdir
damian0815 f21af51
wip
damian0815 708928c
fix bug where base64 string was not removed from the prompt
damian0815 09edb7e
get libllava to output in the right place
damian0815 2847ecf
expose llava methods in libllama.dylib
damian0815 e3261ff
cleanup memory usage around clip_image_*
damian0815 d64891b
cleanup and refactor *again*
damian0815 5a91551
update headerdoc
damian0815 e84003b
Move llava back to examples
monatis 8037034
build with cmake, not tested (WIP)
monatis 52143f7
Editorconfig
monatis c6b8844
Merge branch 'master' into llava-lib
monatis 32bf7bf
Editorconfig
monatis 53dca51
Build with make
monatis b927772
Build with make
monatis 01f06e2
Fix cyclical depts on Windows
monatis ad97e0e
attempt to fix build on Windows
monatis 71ea278
Merge branch 'master' into llava-lib
monatis 1f8c866
attempt to fix build on Windows
monatis d6be69f
Upd TODOs
monatis 5b8b9ef
attempt to fix build on Windows+CUDA
monatis b9bacc7
Revert changes in cmake
monatis 9f03ac7
Fix according to review comments
monatis 22f43fc
Support building as a shared library
monatis 3548029
address review comments
cebtenzzre File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -44,7 +44,7 @@ models-mnt | |
/infill | ||
/libllama.so | ||
/llama-bench | ||
/llava | ||
/llava-cli | ||
/main | ||
/metal | ||
/perplexity | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -698,6 +698,7 @@ target_include_directories(llama PUBLIC .) | |
target_compile_features(llama PUBLIC cxx_std_11) # don't bump | ||
target_link_libraries(llama PRIVATE | ||
ggml | ||
llava | ||
${LLAMA_EXTRA_LIBS} | ||
) | ||
|
||
|
@@ -707,6 +708,22 @@ if (BUILD_SHARED_LIBS) | |
if (LLAMA_METAL) | ||
set_target_properties(llama PROPERTIES RESOURCE "${CMAKE_CURRENT_SOURCE_DIR}/ggml-metal.metal") | ||
endif() | ||
|
||
# By default, symbols provided by the sublibs that are not used by mainlib (which is all of them in this case) | ||
# are not used. This changes that. | ||
if (WIN32) | ||
set_target_properties(llama PROPERTIES | ||
LINK_FLAGS "/WHOLEARCHIVE" | ||
cebtenzzre marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) | ||
elseif (APPLE) | ||
set_target_properties(llama PROPERTIES | ||
LINK_FLAGS "-Wl,-all_load" | ||
) | ||
else () | ||
set_target_properties(llama PROPERTIES | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I removed this clausule as the code I added on line 701 is more restrictive |
||
LINK_FLAGS "-Wl,--whole-archive" | ||
) | ||
endif () | ||
endif() | ||
|
||
|
||
|
@@ -793,6 +810,7 @@ endif() | |
# | ||
|
||
add_subdirectory(common) | ||
add_subdirectory(llava) | ||
|
||
if (LLAMA_BUILD_TESTS AND NOT CMAKE_JS_VERSION) | ||
include(CTest) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,6 +3,7 @@ | |
set(TARGET common) | ||
|
||
add_library(${TARGET} OBJECT | ||
base64.hpp | ||
common.h | ||
common.cpp | ||
sampling.h | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.