feat(server): near-duplicate detection#8228

Merged

zackpollard merged 49 commits intomainfrom feat/duplicate-detection

May 16, 2024

+1,254-61

Commits on May 16, 2024

duplicate detection job, entity, config

mertalev
authored and
Zack Pollard
committed
queueing

mertalev
authored and
Zack Pollard
committed
job panel, update api

mertalev
authored and
Zack Pollard
committed
use embedding in db instead of fetching

mertalev
authored and
Zack Pollard
committed
disable concurrency

mertalev
authored and
Zack Pollard
committed
only queue visible assets

mertalev
authored and
Zack Pollard
committed
handle multiple duplicateIds

mertalev
authored and
Zack Pollard
committed
update concurrent queue check

mertalev
authored and
Zack Pollard
committed
add provider

mertalev
authored and
Zack Pollard
committed
add web placeholder, server endpoint, migration, various fixes

mertalev
authored and
Zack Pollard
committed
update sql

mertalev
authored and
Zack Pollard
committed
select embedding by default

mertalev
authored and
Zack Pollard
committed
rename variable

mertalev
authored and
Zack Pollard
committed
simplify

mertalev
authored and
Zack Pollard
committed
remove separate entity, handle re-running with different threshold, set default back to 0.02

mertalev
authored and
Zack Pollard
committed
fix tests

mertalev
authored and
Zack Pollard
committed
add tests

mertalev
authored and
Zack Pollard
committed
add index to entity

mertalev
authored and
Zack Pollard
committed
formatting

mertalev
authored and
Zack Pollard
committed
update asset mock

mertalev
authored and
Zack Pollard
committed
fix upsertJobStatus signature

mertalev
authored and
Zack Pollard
committed
update sql

mertalev
authored and
Zack Pollard
committed
formatting

mertalev
authored and
Zack Pollard
committed
default to 0.03

mertalev
authored and
Zack Pollard
committed
optimize clustering

mertalev
authored and
Zack Pollard
committed
use asset's duplicateId if present

mertalev
authored and
Zack Pollard
committed
update sql

mertalev
authored and
Zack Pollard
committed
update tests

mertalev
authored and
Zack Pollard
committed
expose admin setting

mertalev
authored and
Zack Pollard
committed
refactor

mertalev
authored and
Zack Pollard
committed
formatting

mertalev
authored and
Zack Pollard
committed
skip if ml is disabled

mertalev
authored and
Zack Pollard
committed
debug trash e2e

mertalev
authored and
Zack Pollard
committed
remove from web

mertalev
authored and
Zack Pollard
committed
remove from sidebar

mertalev
authored and
Zack Pollard
committed
test if ml is disabled

mertalev
authored and
Zack Pollard
committed
update sql

mertalev
authored and
Zack Pollard
committed
separate duplicate detection from clip in config, disable by default for now

mertalev
authored and
Zack Pollard
committed
fix doc

mertalev
authored and
Zack Pollard
committed
lower minimum maxDistance

mertalev
authored and
Zack Pollard
committed
update api

mertalev
authored and
Zack Pollard
committed
Add and Use Duplicate Detection Feature Flag (#9364 )

authored and
Zack Pollard
committed
chore: fixes and additions after rebase

zackpollard
authored and
Zack Pollard
committed
chore: update api (remove new Role enum)

zackpollard
authored and
Zack Pollard
committed
fix: left join smart search so getAll works without machine learning

zackpollard
authored and
Zack Pollard
committed
test: trash e2e go back to checking length of assets is zero

zackpollard
authored and
Zack Pollard
committed
chore: regen api after rebase
zackpollard
committed
test: fix tests after rebase
zackpollard
committed
redundant join
mertalev
committed