Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error logging for Unsuccessful L10n Fetch ( plus code clean up) #544

Merged
merged 99 commits into from
May 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
f4c1214
Loading configuration from the JSON configuration file
jessiejuachon Mar 11, 2020
1a4d95e
Removing code that is not yet ready to be used from the sample applic…
jessiejuachon Mar 11, 2020
5e72b73
fixing failing code scan
jessiejuachon Mar 11, 2020
73d85ca
Adding a cache properties map to hold cache details such as etag and …
jessiejuachon Mar 12, 2020
2a38107
Removing white space
jessiejuachon Mar 12, 2020
c7f5d4f
Fixing failing unit test
jessiejuachon Mar 12, 2020
056891a
Parse headers, response code, response message from an HTTP response …
jessiejuachon Mar 12, 2020
a2ba98b
Storing response headers in Map of cache properties
jessiejuachon Mar 17, 2020
17978bd
Fixing failing unit test due to NullPointerException when doing Map.p…
jessiejuachon Mar 17, 2020
6c69170
Changes after code review.
jessiejuachon Mar 17, 2020
41a8aa1
Removing exception handling for now
jessiejuachon Mar 18, 2020
dd2455d
Deprecating VIPCfg.initialize; cleaning up code
jessiejuachon Mar 19, 2020
e26d372
Storing the http response code in the cache
jessiejuachon Mar 20, 2020
01ab69e
Using cached etag as if-none-match request header value in the reques…
jessiejuachon Mar 25, 2020
750c9d2
Handling 404 response from Singleton service
jessiejuachon Mar 26, 2020
529e5cc
Using HTTP repsonse's Cache-Control max-age and timestamp to dertermi…
jessiejuachon Mar 26, 2020
b40fcf9
Moving constants to URLUtils
jessiejuachon Mar 26, 2020
fa123f9
Fixing failing test
jessiejuachon Mar 27, 2020
30b2054
Fixing failing code scan
jessiejuachon Mar 27, 2020
0c69787
Merge branch 'g11n-java-client' into g11n-java-client
Xiaochao8 Mar 27, 2020
ffc0a74
Cleaning up import in ComponentService,java
jessiejuachon Mar 30, 2020
d8450e4
Cleaning up code - caching/expiry
jessiejuachon Mar 30, 2020
d7f7641
Merge branch 'g11n-java-client' of https://github.com/jessiejuachon/s…
jessiejuachon Mar 30, 2020
0dcd827
Adding tests for new caching workflow; enabling old caching workflow …
jessiejuachon Mar 31, 2020
d06adb3
Fixing a failing test
jessiejuachon Mar 31, 2020
48c8558
Adding mock server response
jessiejuachon Mar 31, 2020
4e45bed
Removing unused file sampleconfig.json
jessiejuachon Mar 31, 2020
6a40dc8
Changes from code review
jessiejuachon Mar 31, 2020
4c3c12d
Value of VIPCfg.cacheExpiredTime is -1 when value is not set in confi…
jessiejuachon Mar 31, 2020
7e0d939
Changing a LinkedHashMap to a HashMap because insertion order does no…
jessiejuachon Apr 2, 2020
baef0d0
Adding CacheItem object to contain map of data and a another map for …
jessiejuachon Apr 3, 2020
02a96e9
Fixing failing unit test
jessiejuachon Apr 3, 2020
78d88f7
Fixing code scan issues
jessiejuachon Apr 3, 2020
655c223
Code cleanup
jessiejuachon Apr 3, 2020
8746915
Lazily instantiating cacheProps; code clean up and comments
jessiejuachon Apr 5, 2020
897a03c
Changes after code review
jessiejuachon Apr 7, 2020
3a4e183
Adding header to CacheItem.java
jessiejuachon Apr 7, 2020
b08feb4
Fixing code san issue. Synchronize on a final field
jessiejuachon Apr 7, 2020
8f57e24
Fixing code scan bug
jessiejuachon Apr 7, 2020
72c8799
Changes after code review
jessiejuachon Apr 7, 2020
92edfc8
Updating CasheService.isContainComponent to return false if cache key…
jessiejuachon Apr 7, 2020
2042554
HttpRequester if-none-match header and other customized headers are a…
jessiejuachon Apr 8, 2020
ae707ff
Code clean up; keep old caching expiration logic
jessiejuachon Apr 8, 2020
82818d5
Fixing code scan bug
jessiejuachon Apr 8, 2020
31d4ff6
Merge branch 'g11n-java-client' into g11n-java-client
jessiejuachon Apr 8, 2020
c9f83d7
code clean up
jessiejuachon Apr 9, 2020
78dcc1c
Removing response code and response msg from cache; Passing cacheItem…
jessiejuachon Apr 9, 2020
9281c64
Changes after code review
jessiejuachon Apr 10, 2020
2657762
Adding header to CacheItem.java file
jessiejuachon Apr 10, 2020
9e80f09
Fixing failing unit test
jessiejuachon Apr 10, 2020
68467f3
Changes after code review
jessiejuachon Apr 10, 2020
d67c72e
Fixing code scan bugs
jessiejuachon Apr 10, 2020
8591973
Not storing anything in cache if response is neither 200 nor 304
jessiejuachon Apr 13, 2020
22a5b29
Loading messages from specified offline resource bundles
jessiejuachon Apr 15, 2020
ce2488b
Merge branch 'g11n-java-client' of https://github.com/vmware/singleto…
jessiejuachon Apr 15, 2020
1092963
Adding header
jessiejuachon Apr 15, 2020
35d1d97
Fixing failing unie test
jessiejuachon Apr 15, 2020
ada17f1
Code changes after review; code clean up
jessiejuachon Apr 16, 2020
fc60eef
Using java.nio.file.Paths
jessiejuachon Apr 17, 2020
3e70449
Offline mode as fallback when service fetch fails
jessiejuachon Apr 17, 2020
609d8e4
Resetting VIPCfg and I18nFactory instances for each test
jessiejuachon Apr 17, 2020
dd19b1c
Resetting VIPCfg and I18nFactory after test
jessiejuachon Apr 17, 2020
7389f8b
Fixing tests
jessiejuachon Apr 17, 2020
5b92e44
Limit to wiremock test logging
jessiejuachon Apr 20, 2020
dfb41b5
Decreasing unit test log
jessiejuachon Apr 20, 2020
21a7105
this test is timing out and trying to connect 70+ times
jessiejuachon Apr 21, 2020
84343d5
Adding comments to code
jessiejuachon Apr 21, 2020
be8e443
Merge branch 'g11n-java-client' of https://github.com/vmware/singleto…
jessiejuachon Apr 21, 2020
b805d36
Adding timestamp to cache fromn local bundle
jessiejuachon Apr 21, 2020
17bc16c
Deprecating methods in TranslationMessage; adding new method getMessage
jessiejuachon Apr 23, 2020
15da010
Added tests for TranslationMessage.getMessage
jessiejuachon Apr 24, 2020
631de86
Fixing test
jessiejuachon Apr 24, 2020
ffd9adb
Fixing test
jessiejuachon Apr 24, 2020
c00f7ae
Fixing test
jessiejuachon Apr 24, 2020
d0504c4
Removing comment field because source collection is not supported any…
jessiejuachon Apr 24, 2020
79b18d0
Code clean up
jessiejuachon Apr 24, 2020
ec9f5db
Changes from code review
jessiejuachon Apr 24, 2020
2e8bd36
Code clean up
jessiejuachon Apr 24, 2020
4e44a8c
Optional SourceOpt in initialization
jessiejuachon Apr 26, 2020
de5c4ee
Fixing test
jessiejuachon Apr 27, 2020
406cb2a
Adding comments to code
jessiejuachon Apr 28, 2020
c80696a
Adding comment for javadoc
jessiejuachon Apr 28, 2020
79d111e
Cleaning up code
jessiejuachon Apr 28, 2020
24d8d5d
Code clean up
jessiejuachon Apr 28, 2020
23e62b6
Fixing test
jessiejuachon Apr 28, 2020
369643f
Removing source message fallback from this PR
jessiejuachon Apr 29, 2020
4e67e25
Merge branch 'g11n-java-client' into g11n-java-client
jessiejuachon Apr 29, 2020
636b5b4
Default locale in config file instead of hardcoded; get supported loc…
jessiejuachon May 5, 2020
71a87e5
Merge branch 'g11n-java-client' into g11n-java-client
jessiejuachon May 5, 2020
b9a90e3
Fixing test
jessiejuachon May 5, 2020
060666d
Fixing tests
jessiejuachon May 5, 2020
52e3695
Default locale to Locale.ENGLISH if not set in config file
jessiejuachon May 5, 2020
2858ebc
Changed after code review: code clean up, bug fix
jessiejuachon May 6, 2020
bbe3047
Update src/main/java/com/vmware/vipclient/i18n/messages/api/opt/local…
jessiejuachon May 8, 2020
fa56388
Update src/main/java/com/vmware/vipclient/i18n/messages/api/opt/local…
jessiejuachon May 8, 2020
08cd96d
code clean up, error logging, bug fix
jessiejuachon May 8, 2020
e89ff56
Merge branch 'g11n-java-client' of https://github.com/vmware/singleto…
jessiejuachon May 8, 2020
119aebd
code clean up
jessiejuachon May 8, 2020
e8a937d
code clean up
jessiejuachon May 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,6 @@ public void addCacheData(String key, String value) {
this.cachedData.put(key, value);
}

public boolean isCachedDataEmpty() {
return this.cachedData.isEmpty();
}

public synchronized void addCachedData(Map<String, String> cachedData) {
if (cachedData != null)
this.cachedData.putAll(cachedData);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,6 @@ public class ConstantsMsg {
public static final String SERVER_RETURN_ERROR = "Server returned error! Status: %d. Message: %s";
public static final String SERVER_CONTENT_ERROR = "The content from server is wrong!";
public static final String UNKNOWN_ERROR = "Unknown error.";
public static final String GET_MESSAGES_FAILED = "Failed to get messages for component {0}, locale: {1}.";

}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

import java.util.ListIterator;
import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.FutureTask;

import org.json.simple.JSONValue;
import org.json.simple.parser.ParseException;
Expand All @@ -15,12 +17,13 @@
import com.vmware.vipclient.i18n.VIPCfg;
import com.vmware.vipclient.i18n.base.DataSourceEnum;
import com.vmware.vipclient.i18n.base.cache.MessageCacheItem;
import com.vmware.vipclient.i18n.base.cache.CacheMode;
import com.vmware.vipclient.i18n.base.cache.persist.DiskCacheLoader;
import com.vmware.vipclient.i18n.base.cache.persist.Loader;
import com.vmware.vipclient.i18n.common.ConstantsMsg;
import com.vmware.vipclient.i18n.messages.api.opt.server.ComponentBasedOpt;
import com.vmware.vipclient.i18n.messages.dto.MessagesDTO;
import com.vmware.vipclient.i18n.util.ConstantsKeys;
import com.vmware.vipclient.i18n.util.FormatUtils;
import com.vmware.vipclient.i18n.util.JSONUtils;
import com.vmware.vipclient.i18n.util.LocaleUtility;

public class ComponentService {
private MessagesDTO dto = null;
Expand All @@ -40,39 +43,74 @@ public ComponentService(MessagesDTO dto) {
public void getMessages(final MessageCacheItem cacheItem, ListIterator<DataSourceEnum> msgSourceQueueIter) {
if (!msgSourceQueueIter.hasNext())
return;

long timestampOld = cacheItem.getTimestamp();
DataSourceEnum dataSource = (DataSourceEnum) msgSourceQueueIter.next();
dataSource.createMessageOpt(dto).getComponentMessages(cacheItem);
long timestampNew = cacheItem.getTimestamp();

// If failed to get messages from the dataSource, try the next dataSource in the queue
if (cacheItem.getCachedData().isEmpty()) {
getMessages(cacheItem, msgSourceQueueIter);
// If failed to get messages from the data source
if (timestampNew == timestampOld) {
// Try the next dataSource in the queue
if (msgSourceQueueIter.hasNext()) {
getMessages(cacheItem, msgSourceQueueIter);
// If no more data source in queue, log the error. This means that neither online nor offline fetch succeeded.
} else {
logger.error(FormatUtils.format(ConstantsMsg.GET_MESSAGES_FAILED, LocaleUtility.getDefaultLocale(),
dto.getComponent(), dto.getLocale()));
}
}
}

public Map<String, String> getComponentTranslation() {
CacheService cs = new CacheService(dto);

if (cs.isContainComponent()) {
return cs.getCacheOfComponent().getCachedData();
} else {
// Messages are not cached in memory, so try to look in disk cache
if (VIPCfg.getInstance().getCacheMode() == CacheMode.DISK) {
Loader loader = VIPCfg.getInstance().getCacheManager()
.getLoaderInstance(DiskCacheLoader.class);
Map<String, String> cachedMessages = loader.load(dto.getCompositStrAsCacheKey());
if (cachedMessages != null) // Messages are in disk cache
return cachedMessages;
}

// Prepare a new CacheItem to store cache properties
MessageCacheItem cacheItem = new MessageCacheItem();
// Pass this cacheItem to getMessages so that it will be populated from the http request
this.getMessages(cacheItem, VIPCfg.getInstance().getMsgOriginsQueue().listIterator());
// Store the messages and properties in cache using a single CacheItem object
cs.addCacheOfComponent(cacheItem);
return cacheItem.getCachedData();
}
return fetchMessages().getCachedData();
}

public MessageCacheItem fetchMessages() {
CacheService cacheService = new CacheService(dto);
Map<String, String> cacheOfComponent = null;
MessageCacheItem cacheItem = null;
if (cacheService.isContainComponent()) { // Item is in cache
cacheItem = cacheService.getCacheOfComponent();
cacheOfComponent = cacheItem.getCachedData();
if (cacheItem.isExpired()) { // cacheItem has expired
// Update the cache in a separate thread
populateCacheTask(cacheService, dto, cacheItem);
}
} else { // Item is not in cache
// Create a new cacheItem object to be stored in cache
cacheItem = new MessageCacheItem();
getMessages(cacheItem, VIPCfg.getInstance().getMsgOriginsQueue().listIterator());
cacheOfComponent = cacheItem.getCachedData();

if (cacheOfComponent != null && !cacheOfComponent.isEmpty()) {
cacheService.addCacheOfComponent(cacheItem);
}
}
return cacheItem;
}

private void populateCacheTask(final CacheService cacheService, MessagesDTO dto, MessageCacheItem cacheItem) {
Callable<MessageCacheItem> callable = () -> {
try {

// Pass cacheItem to getMessages so that:
// 1. A previously stored etag, if any, can be used for the next HTTP request.
// 2. CacheItem properties such as etag, timestamp and maxAgeMillis can be refreshed
// with new properties from the next HTTP response.
getMessages(cacheItem, VIPCfg.getInstance().getMsgOriginsQueue().listIterator());

return cacheItem;
} catch (Exception e) {
// To make sure that the thread will close
// even when an exception is thrown
return null;
}
};
FutureTask<MessageCacheItem> task = new FutureTask<MessageCacheItem>(callable);
Thread thread = new Thread(task);
thread.start();
}

public boolean isComponentAvailable() {
boolean r = false;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.Callable;
import java.util.concurrent.FutureTask;

import org.json.simple.JSONObject;
import org.json.simple.JSONValue;
Expand All @@ -33,7 +31,7 @@ public class StringService {
public String getString(MessagesDTO dto) {
String key = dto.getKey();

MessageCacheItem cacheItem = fetchMessages(dto);
MessageCacheItem cacheItem = new ComponentService(dto).fetchMessages();

// If failed to get MessageCacheItem of a non-default locale,
// use MessageCacheItem of the default locale instead.
Expand All @@ -42,11 +40,11 @@ public String getString(MessagesDTO dto) {
MessagesDTO defaultLocaleDTO = new MessagesDTO(dto.getComponent(),
dto.getKey(), dto.getSource(), LocaleUtility.getDefaultLocale().toLanguageTag(), null);
// MessageCacheItem of the default locale
cacheItem = fetchMessages(defaultLocaleDTO);
cacheItem = new ComponentService(defaultLocaleDTO).fetchMessages();

// The MessageCacheItem for the requested locale will be a reference
// to the MessageCacheItem of the default locale
if (!cacheItem.isCachedDataEmpty()) {
if (!cacheItem.getCachedData().isEmpty()) {
CacheService cacheService = new CacheService(dto);
cacheService.addCacheOfComponent(cacheItem);
cacheOfComponent = cacheItem.getCachedData();
Expand All @@ -56,55 +54,6 @@ public String getString(MessagesDTO dto) {

}

public MessageCacheItem fetchMessages(MessagesDTO dto) {
CacheService cacheService = new CacheService(dto);
Map<String, String> cacheOfComponent = null;
MessageCacheItem cacheItem = null;
if (cacheService.isContainComponent()) { // Item is in cache
cacheItem = cacheService.getCacheOfComponent();
cacheOfComponent = cacheItem.getCachedData();
if (cacheItem.isExpired()) { // cacheItem has expired
// Update the cache in a separate thread
populateCacheTask(cacheService, dto, cacheItem);
}
} else { // Item is not in cache
// Create a new cacheItem object to be stored in cache
cacheItem = new MessageCacheItem();
cacheOfComponent = populateCache(cacheService, dto, cacheItem).getCachedData();

if (cacheOfComponent != null && !cacheOfComponent.isEmpty()) {
cacheService.addCacheOfComponent(cacheItem);
}
}
return cacheItem;
}

private void populateCacheTask(final CacheService cacheService, MessagesDTO dto, MessageCacheItem cacheItem) {
Callable<MessageCacheItem> callable = () -> {
try {
// Use the cacheProps that is already in the cache.
MessageCacheItem updatedCacheItem = populateCache(cacheService, dto, cacheItem);
return updatedCacheItem;
} catch (Exception e) {
// To make sure that the thread will close
// even when an exception is thrown
return null;
}
};
FutureTask<MessageCacheItem> task = new FutureTask<MessageCacheItem>(callable);
Thread thread = new Thread(task);
thread.start();
}

private MessageCacheItem populateCache(CacheService cacheService, MessagesDTO dto, MessageCacheItem cacheItem) {
// Pass cacheitem to getMessages so that:
// 1. A previously stored etag, if any, can be used for the next HTTP request.
// 2. CacheItem properties such as etag, timestamp and maxAgeMillis can be refreshed
// with new properties from the next HTTP response.
new ComponentService(dto).getMessages(cacheItem, VIPCfg.getInstance().getMsgOriginsQueue().listIterator());

return cacheItem;
}

public String postString(MessagesDTO dto) {
String r = "";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,5 +67,4 @@ public class ConstantsKeys {
public static final String DISPLAY_NAME = "displayName";
public static final String JSON_KEYSET = "jsonkeyset";
public static final String TRANSLATION_STATUS = "translationStatus";

}