Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose Llava as a shared library for downstream projects #3613

Merged
merged 34 commits into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0209d39
wip llava python bindings compatibility
damian0815 Oct 13, 2023
3c10d9f
add external llava API
damian0815 Oct 13, 2023
770dc9d
add base64 in-prompt image support
damian0815 Oct 13, 2023
8224ca5
wip refactor image loading
damian0815 Oct 14, 2023
c693208
refactor image load out of llava init
damian0815 Oct 14, 2023
0889117
cleanup
damian0815 Oct 14, 2023
f83c060
further cleanup; move llava-cli into its own file and rename
damian0815 Oct 14, 2023
e2cd07c
move base64.hpp into common/
damian0815 Oct 14, 2023
f8eddcf
collapse clip and llava libraries
damian0815 Oct 14, 2023
b9f533b
move llava into its own subdir
damian0815 Oct 14, 2023
f21af51
wip
damian0815 Oct 14, 2023
708928c
fix bug where base64 string was not removed from the prompt
damian0815 Oct 14, 2023
09edb7e
get libllava to output in the right place
damian0815 Oct 14, 2023
2847ecf
expose llava methods in libllama.dylib
damian0815 Oct 14, 2023
e3261ff
cleanup memory usage around clip_image_*
damian0815 Oct 14, 2023
d64891b
cleanup and refactor *again*
damian0815 Oct 15, 2023
5a91551
update headerdoc
damian0815 Oct 15, 2023
e84003b
Move llava back to examples
monatis Nov 2, 2023
8037034
build with cmake, not tested (WIP)
monatis Nov 2, 2023
52143f7
Editorconfig
monatis Nov 5, 2023
c6b8844
Merge branch 'master' into llava-lib
monatis Nov 5, 2023
32bf7bf
Editorconfig
monatis Nov 5, 2023
53dca51
Build with make
monatis Nov 5, 2023
b927772
Build with make
monatis Nov 5, 2023
01f06e2
Fix cyclical depts on Windows
monatis Nov 5, 2023
ad97e0e
attempt to fix build on Windows
monatis Nov 5, 2023
71ea278
Merge branch 'master' into llava-lib
monatis Nov 5, 2023
1f8c866
attempt to fix build on Windows
monatis Nov 6, 2023
d6be69f
Upd TODOs
monatis Nov 6, 2023
5b8b9ef
attempt to fix build on Windows+CUDA
monatis Nov 6, 2023
b9bacc7
Revert changes in cmake
monatis Nov 6, 2023
9f03ac7
Fix according to review comments
monatis Nov 6, 2023
22f43fc
Support building as a shared library
monatis Nov 6, 2023
3548029
address review comments
cebtenzzre Nov 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ models-mnt
/infill
/libllama.so
/llama-bench
/llava
/llava-cli
/main
/metal
/perplexity
Expand Down
18 changes: 18 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -698,6 +698,7 @@ target_include_directories(llama PUBLIC .)
target_compile_features(llama PUBLIC cxx_std_11) # don't bump
target_link_libraries(llama PRIVATE
ggml
llava
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
llava
-Wl,--whole-archive llava
-Wl,--no-whole-archive

${LLAMA_EXTRA_LIBS}
)

Expand All @@ -707,6 +708,22 @@ if (BUILD_SHARED_LIBS)
if (LLAMA_METAL)
set_target_properties(llama PROPERTIES RESOURCE "${CMAKE_CURRENT_SOURCE_DIR}/ggml-metal.metal")
endif()

# By default, symbols provided by the sublibs that are not used by mainlib (which is all of them in this case)
# are not used. This changes that.
if (WIN32)
set_target_properties(llama PROPERTIES
LINK_FLAGS "/WHOLEARCHIVE"
cebtenzzre marked this conversation as resolved.
Show resolved Hide resolved
)
elseif (APPLE)
set_target_properties(llama PROPERTIES
LINK_FLAGS "-Wl,-all_load"
)
else ()
set_target_properties(llama PROPERTIES
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this clausule as the code I added on line 701 is more restrictive

LINK_FLAGS "-Wl,--whole-archive"
)
endif ()
endif()


Expand Down Expand Up @@ -793,6 +810,7 @@ endif()
#

add_subdirectory(common)
add_subdirectory(llava)

if (LLAMA_BUILD_TESTS AND NOT CMAKE_JS_VERSION)
include(CTest)
Expand Down
1 change: 1 addition & 0 deletions common/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
set(TARGET common)

add_library(${TARGET} OBJECT
base64.hpp
common.h
common.cpp
sampling.h
Expand Down
Loading