Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

621: Generic Read Metadata #1019

Merged
merged 26 commits into from
Apr 16, 2024
Merged

621: Generic Read Metadata #1019

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6d651cc
Do some cool stuff with collectors
halprin Apr 3, 2024
cf5ac75
Merge branch 'main' into story-621-read_metadata_generic
halprin Apr 12, 2024
8d4a5a1
Get the metadata storage working again with the latest changes
halprin Apr 12, 2024
9ceabf8
Remove unused method
halprin Apr 12, 2024
8ee28a7
Fix DatabasePartnerMetadataStorageTest unit tests
halprin Apr 12, 2024
9a52b40
Move a test over
halprin Apr 12, 2024
6a67e1d
Moved more tests over
halprin Apr 12, 2024
820636f
Remove some noise
halprin Apr 12, 2024
08897e4
Merge branch 'main' into story-621-read_metadata_generic
halprin Apr 15, 2024
6a0ab81
Fix most PostgresDoa unit tests
halprin Apr 15, 2024
aadc45b
Fix last test of PostgresDoaTest
halprin Apr 15, 2024
2d438de
JavaDocs
halprin Apr 15, 2024
0a15c0e
Iterator throws NoSuchElementException
halprin Apr 15, 2024
6905389
const for Received Message Id
halprin Apr 15, 2024
7d623bc
ResultSetIteratorTest
halprin Apr 15, 2024
b3d6631
Test exception case for fetchManyData too
halprin Apr 16, 2024
13c8842
Merge branch 'main' into story-621-read_metadata_generic
halprin Apr 16, 2024
a9537cd
Move over the logic to get linked IDs given the submissionId out of t…
halprin Apr 16, 2024
341dd78
Move tests around
halprin Apr 16, 2024
abe1f86
Make the DB message link storage use the updated DbDao
halprin Apr 16, 2024
47aa2e9
Test getting a message link
halprin Apr 16, 2024
deeb4be
Test partialMessageLinkFromResultSet
halprin Apr 16, 2024
ef72e8e
Test changes to the readPrivateKey works test
halprin Apr 16, 2024
faae0fa
Clean-up JjwtEngineTest test
halprin Apr 16, 2024
8c86655
Use a Map.Entry instead
halprin Apr 16, 2024
a7f1860
Make MessageLink immutable and test equals and hashCode
halprin Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 119 additions & 15 deletions ...ava/gov/hhs/cdc/trustedintermediary/external/database/DatabasePartnerMetadataStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,38 @@

import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadata;
import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadataException;
import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadataMessageType;
import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadataStatus;
import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadataStorage;
import gov.hhs.cdc.trustedintermediary.wrappers.Logger;
import gov.hhs.cdc.trustedintermediary.wrappers.formatter.Formatter;
import gov.hhs.cdc.trustedintermediary.wrappers.formatter.FormatterProcessingException;
import gov.hhs.cdc.trustedintermediary.wrappers.formatter.TypeReference;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.sql.Types;
import java.time.Instant;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
import javax.inject.Inject;

/** Implements the {@link PartnerMetadataStorage} using a database. */
public class DatabasePartnerMetadataStorage implements PartnerMetadataStorage {

private static final DatabasePartnerMetadataStorage INSTANCE =
new DatabasePartnerMetadataStorage();

private static final String METADATA_TABLE_RECEIVED_MESSAGE_ID = "received_message_id";

@Inject DbDao dao;

@Inject Logger logger;

@Inject Formatter formatter;
private static final DatabasePartnerMetadataStorage INSTANCE =
new DatabasePartnerMetadataStorage();

private DatabasePartnerMetadataStorage() {}

Expand All @@ -35,12 +45,25 @@ public static DatabasePartnerMetadataStorage getInstance() {
public Optional<PartnerMetadata> readMetadata(final String uniqueId)
throws PartnerMetadataException {
try {
PartnerMetadata data = (PartnerMetadata) dao.fetchMetadata(uniqueId);
return Optional.ofNullable(data);
PartnerMetadata metadata =
dao.fetchFirstData(
connection -> {
try {
PreparedStatement statement =
connection.prepareStatement(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice. For some time I've been thinking of having a wrapper class that will handle the prepared statements. Thoughts on having the wrapper class so that both storage classes can use it with the dao?

"SELECT * FROM metadata where received_message_id = ? OR sent_message_id = ?");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point I think we should separate SQL code from Java code. In the meantime I've been placing the sql code in a variable at the start of the method where it's used, but we should probably start putting them in a more centralized place soon. It could be external files or java constants

statement.setString(1, uniqueId);
statement.setString(2, uniqueId);
return statement;
} catch (SQLException e) {
throw new RuntimeException(e);
}
},
this::partnerMetadataFromResultSet);

return Optional.ofNullable(metadata);
} catch (SQLException e) {
throw new PartnerMetadataException("Error retrieving metadata", e);
} catch (FormatterProcessingException e) {
throw new PartnerMetadataException("Error formatting metadata", e);
}
}

Expand All @@ -50,7 +73,7 @@ public void saveMetadata(final PartnerMetadata metadata) throws PartnerMetadataE

try {
List<DbColumn> columns = createDbColumnsFromMetadata(metadata);
dao.upsertData("metadata", columns, "(received_message_id)");
dao.upsertData("metadata", columns, "(" + METADATA_TABLE_RECEIVED_MESSAGE_ID + ")");
} catch (SQLException e) {
throw new PartnerMetadataException("Error saving metadata", e);
} catch (FormatterProcessingException e) {
Expand All @@ -63,32 +86,113 @@ public Set<PartnerMetadata> readMetadataForSender(String sender)
throws PartnerMetadataException {
Set<PartnerMetadata> consolidatedMetadata;
try {
consolidatedMetadata = dao.fetchMetadataForSender(sender);
consolidatedMetadata =
dao.fetchManyData(
connection -> {
try {
PreparedStatement statement =
connection.prepareStatement(
"SELECT * FROM metadata WHERE sender = ?");
statement.setString(1, sender);
return statement;
} catch (SQLException e) {
throw new RuntimeException(e);
}
},
this::partnerMetadataFromResultSet,
Collectors.toSet());

return consolidatedMetadata;

} catch (SQLException e) {
throw new PartnerMetadataException("Error retrieving consolidated metadata", e);
} catch (FormatterProcessingException e) {
throw new PartnerMetadataException("Error formatting consolidated metadata", e);
}
return consolidatedMetadata;
}

@Override
public Set<PartnerMetadata> readMetadataForMessageLinking(String submissionId)
throws PartnerMetadataException {

Set<PartnerMetadata> metadataSet;
try {
metadataSet = dao.fetchMetadataForMessageLinking(submissionId);
} catch (SQLException | FormatterProcessingException e) {
metadataSet =
dao.fetchManyData(
connection -> {
try {
PreparedStatement statement =
connection.prepareStatement(
"""
SELECT m2.*
FROM metadata m1
JOIN metadata m2
ON m1.placer_order_number = m2.placer_order_number
AND m1.sending_application_id = m2.sending_application_id
AND m1.sending_facility_id = m2.sending_facility_id
WHERE m1.sent_message_id = ?;
""");
statement.setString(1, submissionId);
return statement;
} catch (SQLException e) {
throw new RuntimeException(e);
}
},
this::partnerMetadataFromResultSet,
Collectors.toSet());

return metadataSet;
} catch (SQLException e) {
throw new PartnerMetadataException("Error retrieving metadata", e);
}
return metadataSet;
}

PartnerMetadata partnerMetadataFromResultSet(ResultSet resultSet) {
try {
Instant timeReceived = null;
Instant timeDelivered = null;
Timestamp timestampReceived = resultSet.getTimestamp("time_received");
Timestamp timestampDelivered = resultSet.getTimestamp("time_delivered");
if (timestampReceived != null) {
timeReceived = timestampReceived.toInstant();
}

if (timestampDelivered != null) {
timeDelivered = timestampDelivered.toInstant();
}

return new PartnerMetadata(
resultSet.getString(METADATA_TABLE_RECEIVED_MESSAGE_ID),
resultSet.getString("sent_message_id"),
resultSet.getString("sender"),
resultSet.getString("receiver"),
timeReceived,
timeDelivered,
resultSet.getString("hash_of_message"),
PartnerMetadataStatus.valueOf(resultSet.getString("delivery_status")),
resultSet.getString("failure_reason"),
PartnerMetadataMessageType.valueOf(resultSet.getString("message_type")),
formatter.convertJsonToObject(
resultSet.getString("sending_application_details"),
new TypeReference<>() {}),
formatter.convertJsonToObject(
resultSet.getString("sending_facility_details"),
new TypeReference<>() {}),
formatter.convertJsonToObject(
resultSet.getString("receiving_application_details"),
new TypeReference<>() {}),
formatter.convertJsonToObject(
resultSet.getString("receiving_facility_details"),
new TypeReference<>() {}),
resultSet.getString("placer_order_number"));
} catch (SQLException | FormatterProcessingException e) {
throw new RuntimeException(e);
}
}

private List<DbColumn> createDbColumnsFromMetadata(PartnerMetadata metadata)
throws FormatterProcessingException {
return List.of(
new DbColumn(
"received_message_id",
METADATA_TABLE_RECEIVED_MESSAGE_ID,
metadata.receivedSubmissionId(),
false,
Types.VARCHAR),
Expand Down
22 changes: 13 additions & 9 deletions etor/src/main/java/gov/hhs/cdc/trustedintermediary/external/database/DbDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
package gov.hhs.cdc.trustedintermediary.external.database;

import gov.hhs.cdc.trustedintermediary.etor.metadata.partner.PartnerMetadata;
import gov.hhs.cdc.trustedintermediary.wrappers.formatter.FormatterProcessingException;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.List;
import java.util.Set;
import java.util.function.Function;
import java.util.stream.Collector;

/** Interface for accessing the database for metadata */
public interface DbDao {
void upsertData(String tableName, List<DbColumn> values, String conflictTarget)
throws SQLException;

Object fetchMetadata(String uniqueId) throws SQLException, FormatterProcessingException;

Set<PartnerMetadata> fetchMetadataForSender(String sender)
throws SQLException, FormatterProcessingException;
<T> T fetchFirstData(
Function<Connection, PreparedStatement> sqlGenerator, Function<ResultSet, T> converter)
throws SQLException;

Set<PartnerMetadata> fetchMetadataForMessageLinking(String submissionId)
throws SQLException, FormatterProcessingException;
<T, S> S fetchManyData(
Function<Connection, PreparedStatement> sqlGenerator,
Function<ResultSet, T> converter,
Collector<? super T, ?, S> collector)
throws SQLException;
}
Loading
Loading