Commit Graph

29 Commits

Author SHA1 Message Date
Phlogi 8c6adf7157 feat(server): resolve duplicates (#25316)
* feat(web): Synchronize information from deduplicated images

* Added new settings menu to the the deduplication tab.
* The toggable options in the settings are synchronization of: albums, favorites, ratings, description, visibility and location.
* When synchronizing the albums, the resolved images will be added to all albums of the duplicates.
* When synchronizing the favorite status, the resolved images will be marked as favorite, if at least one selectable image is marked as favorite.
* When synchronizing the ratings, the highest rating from the selectable images will be applied to the resolved image.
* When synchronizing the description, all descriptions from the selectable images will be merged into one description for the resolved image.
* When synchronizing the visibility, the most restrictive visibility setting from the selectable images will be applied to the resolved image.
* When synchronizing the location, if exactly one unique location exists among the selectable images, this location will be applied to the resolved image.
* There is no additional UI for these settings to keep the visual clutter minimal. The settings are applied automatically based on the user's preferences.

* Replace addAssetToAlbums with copyAsset

* fix linter

* feat(web): add duplicate sync fields and fix typo

* feat(web): add tag sync and enhance duplicate resolution

This update introduces tag synchronization for duplicate resolution,
ensuring all unique tag IDs from duplicates are applied to kept assets.
The visibility sync logic is updated to use a simplified ordering, as the hidden status items will never show up in a duplicate set.
Album synchronization now merges albums directly via addAssetsToAlbums; as the approach with copyAsset API endpoint was ineffiecient.
Description, rating, and location sync logic is improved for correctness.
and deduplication. i18n strings were added / updated.

* feat(server): move duplicate resolution to backend with sync and stacking

Moves duplicate metadata synchronization from frontend to backend, enabling robust
batch operations and proper validation. This is an improved refactor of PR #13851.

New endpoints:
- POST /duplicates/resolve - batch resolve with configurable metadata sync
- POST /duplicates/stack - create stacks from duplicate groups
- GET /duplicates - now includes suggestedKeepAssetIds based on file size and EXIF

Key changes:
- Move sync logic (albums, tags, favorites, ratings, descriptions, location, visibility) to server
- Add server-side metadata merge policies with proper conflict resolution
- Replace client-side resolution logic with new backend endpoints
- Add comprehensive E2E tests (70+ test cases) and unit tests
- Update OpenAPI specs and TypeScript SDK

No breaking changes - only additions to existing API.

* feat(preferences): enable all duplicate sync settings by default

* chore: clean up

* chore: clean up

* refactor: rename & clean up

* fix: preference upgrade

* chore: linting

* refactor(e2e): use updateAssets API for setAssetDuplicateId

* fix: visibility sync logic in duplicate resolution

* fix(duplicate): write description to exifUpdate

Previously the duplicate resolution populated assetUpdate.description even
though description belongs to exif info.

* fix(duplicate): remove redundant updateLockedColumns wrapper

updateAllExif already computes lockedProperties via distinctLocked
using Object.keys(options). The wrapper added a lockedProperties key
to the options object, causing the spurious string 'lockedProperties'
to be stored in the lockedProperties array.

* fix(duplicate): write merged tags to asset_exif to survive metadata re-extraction

During duplicate resolution, replaceAssetTags correctly wrote merged tag
IDs to the tag_asset table, but never updated asset_exif.tags or locked
the tags property. The subsequent SidecarWrite → AssetExtractMetadata
chain calls applyTagList, which destructively replaces tag_asset rows
with whatever is in asset_exif.tags — still the original per-asset tags,
not the merged set.

Write merged tag values to asset_exif.tags via updateAllExif (which also
locks the property via distinctLocked), and queue SidecarWrite when tags
change so they persist to the sidecar file.

* docs(duplicates): clarify location and tag sync behavior

* refactor(duplicate): remove sync settings, always sync all metadata on resolve

Remove DuplicateSyncSettingsDto and the per-field sync toggles
(albums, favorites, rating, description, visibility, location, tags).
Duplicate resolution now unconditionally syncs all metadata from
trashed assets to kept assets.

- Remove DuplicateSyncSettingsDto and settings field from DuplicateResolveDto
- Update DuplicateService to always run all sync logic without conditionals
- Delete DuplicateSettingsModal.svelte and settings gear button from UI
- Remove DuplicateSettings type and duplicateSettings persisted store
- Update unit and e2e tests to remove settings from resolve requests

* docs: update duplicates utility to reflect automatic metadata sync

* docs(web): replace duplicates info modal with link to documentation

* chore: clean up

* fix: add missing type cast to jsonAgg in duplicate repository getAll

* fix: skip persisting rating=0 in duplicate merge to avoid unnecessary sidecar write

---------

Co-authored-by: Toni <51962051+EinToni@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jason@rasm.me>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2026-03-26 18:33:55 +00:00
Daniel Dietzler 34ce68095d chore: upgrade to kysely 0.28.11 (#26744) 2026-03-11 16:17:31 +01:00
Daniel Dietzler 7cb355279e chore: remove asset stubs (#26187) 2026-02-13 11:00:31 -05:00
Daniel Dietzler 7e0356e227 refactor: more small tests (#26159) 2026-02-12 08:34:32 -05:00
Daniel Dietzler e54678e0d6 refactor: small tests (#26141) 2026-02-11 11:49:00 -05:00
Jason Rasmussen bcb968e3d1 refactor: job names (#19949) 2025-07-15 18:39:00 -04:00
Jason Rasmussen e73abe0762 refactor: enum casing (#19946) 2025-07-15 14:50:13 -04:00
Jason Rasmussen 910661e75c chore: remove unused mocks (#19299) 2025-06-19 10:35:09 -05:00
Jason Rasmussen 5cd186d3d4 refactor: duplicate queries (#19136) 2025-06-12 14:23:02 -04:00
Mert a6a4dfcfd3 fix(server): queueing for duplicate detection (#18380)
* fix queueing

* update tests
2025-05-20 08:44:39 -05:00
Alex d33ce13561 feat(server): visibility column (#17939)
* feat: private view

* pr feedback

* sql generation

* feat: visibility column

* fix: set visibility value as the same as the still part after unlinked live photos

* fix: test

* pr feedback
2025-05-06 12:12:48 -05:00
Jason Rasmussen 8c5116bc1d refactor: stream search duplicates (#17991) 2025-04-30 10:40:32 -04:00
Jason Rasmussen 17e720440d refactor: new asset-job repository (#17622)
* refactor: new asset-job repository

* fix: broken medium tests on main
2025-04-15 10:24:51 -04:00
Jason Rasmussen fdbe6d649f refactor: remove smart search entity (#17447)
refactor: smart search entity
2025-04-08 09:56:45 -04:00
Mert a708649504 fix(server): skip stacked assets in duplicate detection (#16380)
* skip stacked assets in duplicate detection

* update sql

* handle stacking after duplicate detection runs
2025-02-27 10:16:13 -06:00
Jason Rasmussen fa5aeaf539 refactor: last repository (#16042) 2025-02-11 22:15:56 +00:00
Jason Rasmussen 9d85272c2b refactor: repositories (#16036) 2025-02-11 14:08:13 -05:00
Jason Rasmussen 735f8d661e refactor: test mocks (#16008) 2025-02-11 00:47:42 +01:00
Jason Rasmussen 03948a69e2 refactor: system metadata repository (#15954) 2025-02-07 17:26:49 -05:00
Jason Rasmussen d3446f3092 refactor: logging repository (#15540) 2025-01-23 08:31:30 -05:00
Mert 2e12c46980 refactor(server): use kysely (#12857) 2025-01-09 11:15:41 -05:00
Pranay Pandey 380fc06979 fix: remove duplicateIds on unique assets (#13752) 2024-11-04 10:03:03 -05:00
Daniel Dietzler 9d0f03808c chore: finishing unit tests for a couple of services (#13292) 2024-10-08 17:08:49 -04:00
Jason Rasmussen 4ea281f854 refactor: service dependencies (#13108)
refactor(server): simplify service dependency management
2024-10-02 10:54:35 -04:00
Jason Rasmussen 63437529e1 refactor(server): config file env (#13100) 2024-10-01 16:03:55 -04:00
bo0tzz e54c18367b chore: Lower default duplicate detection distance (#10703) 2024-06-30 11:36:02 -04:00
Alex 6879bcb7a4 chore(server): duplication default settings (#9781) 2024-05-26 20:51:41 -04:00
Mert e7c8501930 fix(server): search duplicates of the same asset type (#9747)
* search by type

* make sql

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-26 22:04:23 +00:00
Mert ff52300624 refactor(server): duplicate controller and service (#9542)
* duplicate controller and service

* change endpoint name

* fix search tests

* remove unused import

* add to index
2024-05-16 19:39:33 -04:00