Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TrackDAO/GlobalTrackCache: Handle file aliasing #3026

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion src/library/dao/trackdao.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1227,7 +1227,20 @@ TrackPointer TrackDAO::getTrackFromDB(TrackId trackId) const {
// Just to be safe, but this should never happen!!
return pTrack;
}
DEBUG_ASSERT(pTrack->getId() == trackId);
if (pTrack->getId() != trackId) {
// This happens if two different tracks are referencing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// The cacheResolver() failed to resolve the track by it's trackId, but has found a different track listed in the database, referencing ...

It took me a bit to remind me how everything is connected. So I think I think this comment helps here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that the using code keeps has the old track ID already on stack and keeps using it.
So we either need to inform the using code about the changed track ID or fail here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not able to solve all downstream issues and also haven't spotted any serious issues at first sight.

// the same (physical) file on the file system!! Due to
// symbolic links different locations may resolve to the
// same canonical location. We can only load each file
// once at a time.
kLogger.warning()
<< "Returning already loaded track"
<< pTrack->getId()
<< "instead of"
<< trackId
<< "with the same canonical file location"
<< pTrack->getFileInfo().canonicalFilePath();
}
if (cacheResolver.getLookupResult() == GlobalTrackCacheLookupResult::HIT) {
// Due to race conditions the track might have been reloaded
// from the database in the meantime. In this case we abort
Expand Down
79 changes: 52 additions & 27 deletions src/track/globaltrackcache.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -391,8 +391,34 @@ bool GlobalTrackCache::isEmpty() const {
return m_tracksById.empty() && m_tracksByCanonicalLocation.empty();
}

TrackPointer GlobalTrackCache::lookupByRef(
const TrackRef& trackRef) {
TrackPointer trackPtr;
if (trackRef.hasId()) {
trackPtr = lookupById(trackRef.getId());
if (trackPtr) {
return trackPtr;
}
}
if (trackRef.hasCanonicalLocation()) {
trackPtr = lookupByCanonicalLocation(trackRef.getCanonicalLocation());
if (trackPtr) {
if (trackRef.hasId() &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this ever be true because of the check in line 426?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If another track with a different id but the same canonical location has already been stored in the cache we will get here. That's part of the problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we might check that the found track also already has a valid id. Otherwise, this is a rare, intermediate edge case.

trackRef.getId() != trackPtr->getId()) {
kLogger.warning()
<< "Found a different track with the same canonical location:"
<< "expected =" << trackRef
<< "actual =" << createTrackRef(*trackPtr);
}
return trackPtr;
}
}
return trackPtr;
}

TrackPointer GlobalTrackCache::lookupById(
const TrackId& trackId) {
TrackPointer trackPtr;
const auto trackById(m_tracksById.find(trackId));
if (m_tracksById.end() != trackById) {
// Cache hit
Expand All @@ -402,45 +428,43 @@ TrackPointer GlobalTrackCache::lookupById(
<< trackId
<< trackById->second->getPlainPtr();
}
return revive(trackById->second);
trackPtr = revive(trackById->second);
DEBUG_ASSERT(trackPtr);
} else {
// Cache miss
if (traceLogEnabled()) {
kLogger.trace()
<< "Cache miss for"
<< trackId;
}
return TrackPointer();
}
return trackPtr;
}

TrackPointer GlobalTrackCache::lookupByRef(
const TrackRef& trackRef) {
if (trackRef.hasId()) {
return lookupById(trackRef.getId());
TrackPointer GlobalTrackCache::lookupByCanonicalLocation(
const QString& canonicalLocation) {
TrackPointer trackPtr;
const auto trackByCanonicalLocation(
m_tracksByCanonicalLocation.find(canonicalLocation));
if (m_tracksByCanonicalLocation.end() != trackByCanonicalLocation) {
// Cache hit
if (traceLogEnabled()) {
kLogger.trace()
<< "Cache hit for"
<< canonicalLocation
<< trackByCanonicalLocation->second->getPlainPtr();
}
trackPtr = revive(trackByCanonicalLocation->second);
DEBUG_ASSERT(trackPtr);
} else {
const auto canonicalLocation = trackRef.getCanonicalLocation();
const auto trackByCanonicalLocation(
m_tracksByCanonicalLocation.find(canonicalLocation));
if (m_tracksByCanonicalLocation.end() != trackByCanonicalLocation) {
// Cache hit
if (traceLogEnabled()) {
kLogger.trace()
<< "Cache hit for"
<< canonicalLocation
<< trackByCanonicalLocation->second->getPlainPtr();
}
return revive(trackByCanonicalLocation->second);
} else {
// Cache miss
if (traceLogEnabled()) {
kLogger.trace()
<< "Cache miss for"
<< canonicalLocation;
}
return TrackPointer();
// Cache miss
if (traceLogEnabled()) {
kLogger.trace()
<< "Cache miss for"
<< canonicalLocation;
}
}
return trackPtr;
}

TrackPointer GlobalTrackCache::revive(
Expand Down Expand Up @@ -515,7 +539,8 @@ void GlobalTrackCache::resolve(
<< "Resolving track by canonical location"
<< trackRef.getCanonicalLocation();
}
auto strongPtr = lookupByRef(trackRef);
auto strongPtr = lookupByCanonicalLocation(
trackRef.getCanonicalLocation());
if (strongPtr) {
// Cache hit
if (debugLogEnabled()) {
Expand Down
6 changes: 4 additions & 2 deletions src/track/globaltrackcache.h
Original file line number Diff line number Diff line change
Expand Up @@ -226,10 +226,12 @@ private slots:
void relocateTracks(
GlobalTrackCacheRelocator* /*nullable*/ pRelocator);

TrackPointer lookupById(
const TrackId& trackId);
TrackPointer lookupByRef(
const TrackRef& trackRef);
TrackPointer lookupById(
const TrackId& trackId);
TrackPointer lookupByCanonicalLocation(
const QString& canonicalLocation);

TrackPointer revive(GlobalTrackCacheEntryPointer entryPtr);

Expand Down
31 changes: 20 additions & 11 deletions src/track/trackref.cpp
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#include "track/trackref.h"

#include <QDebugStateSaver>

bool TrackRef::verifyConsistency() const {
// Class invariant: The location can only be set together with
Expand All @@ -16,17 +17,25 @@ bool TrackRef::verifyConsistency() const {
}

std::ostream& operator<<(std::ostream& os, const TrackRef& trackRef) {
return os << '[' << trackRef.getLocation().toStdString()
<< " | " << trackRef.getCanonicalLocation().toStdString()
<< " | " << trackRef.getId()
<< ']';

return os
<< "TrackRef{"
<< trackRef.getLocation().toStdString()
<< ','
<< trackRef.getCanonicalLocation().toStdString()
<< ','
<< trackRef.getId()
<< '}';
}

QDebug operator<<(QDebug debug, const TrackRef& trackRef) {
debug.nospace() << '[' << trackRef.getLocation()
<< " | " << trackRef.getCanonicalLocation()
<< " | " << trackRef.getId()
<< ']';
return debug.space();
QDebug operator<<(QDebug dbg, const TrackRef& trackRef) {
const QDebugStateSaver saver(dbg);
dbg = dbg.maybeSpace() << "TrackRef";
return dbg.nospace()
<< '{'
<< trackRef.getLocation()
<< ','
<< trackRef.getCanonicalLocation()
<< ','
<< trackRef.getId()
<< '}';
}