Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading offline resource bundles #511

Merged
merged 87 commits into from
Apr 30, 2020
Merged
Show file tree
Hide file tree
Changes from 82 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
f4c1214
Loading configuration from the JSON configuration file
jessiejuachon Mar 11, 2020
1a4d95e
Removing code that is not yet ready to be used from the sample applic…
jessiejuachon Mar 11, 2020
5e72b73
fixing failing code scan
jessiejuachon Mar 11, 2020
73d85ca
Adding a cache properties map to hold cache details such as etag and …
jessiejuachon Mar 12, 2020
2a38107
Removing white space
jessiejuachon Mar 12, 2020
c7f5d4f
Fixing failing unit test
jessiejuachon Mar 12, 2020
056891a
Parse headers, response code, response message from an HTTP response …
jessiejuachon Mar 12, 2020
a2ba98b
Storing response headers in Map of cache properties
jessiejuachon Mar 17, 2020
17978bd
Fixing failing unit test due to NullPointerException when doing Map.p…
jessiejuachon Mar 17, 2020
6c69170
Changes after code review.
jessiejuachon Mar 17, 2020
41a8aa1
Removing exception handling for now
jessiejuachon Mar 18, 2020
dd2455d
Deprecating VIPCfg.initialize; cleaning up code
jessiejuachon Mar 19, 2020
e26d372
Storing the http response code in the cache
jessiejuachon Mar 20, 2020
01ab69e
Using cached etag as if-none-match request header value in the reques…
jessiejuachon Mar 25, 2020
750c9d2
Handling 404 response from Singleton service
jessiejuachon Mar 26, 2020
529e5cc
Using HTTP repsonse's Cache-Control max-age and timestamp to dertermi…
jessiejuachon Mar 26, 2020
b40fcf9
Moving constants to URLUtils
jessiejuachon Mar 26, 2020
fa123f9
Fixing failing test
jessiejuachon Mar 27, 2020
30b2054
Fixing failing code scan
jessiejuachon Mar 27, 2020
0c69787
Merge branch 'g11n-java-client' into g11n-java-client
Xiaochao8 Mar 27, 2020
ffc0a74
Cleaning up import in ComponentService,java
jessiejuachon Mar 30, 2020
d8450e4
Cleaning up code - caching/expiry
jessiejuachon Mar 30, 2020
d7f7641
Merge branch 'g11n-java-client' of https://github.com/jessiejuachon/s…
jessiejuachon Mar 30, 2020
0dcd827
Adding tests for new caching workflow; enabling old caching workflow …
jessiejuachon Mar 31, 2020
d06adb3
Fixing a failing test
jessiejuachon Mar 31, 2020
48c8558
Adding mock server response
jessiejuachon Mar 31, 2020
4e45bed
Removing unused file sampleconfig.json
jessiejuachon Mar 31, 2020
6a40dc8
Changes from code review
jessiejuachon Mar 31, 2020
4c3c12d
Value of VIPCfg.cacheExpiredTime is -1 when value is not set in confi…
jessiejuachon Mar 31, 2020
7e0d939
Changing a LinkedHashMap to a HashMap because insertion order does no…
jessiejuachon Apr 2, 2020
baef0d0
Adding CacheItem object to contain map of data and a another map for …
jessiejuachon Apr 3, 2020
02a96e9
Fixing failing unit test
jessiejuachon Apr 3, 2020
78d88f7
Fixing code scan issues
jessiejuachon Apr 3, 2020
655c223
Code cleanup
jessiejuachon Apr 3, 2020
8746915
Lazily instantiating cacheProps; code clean up and comments
jessiejuachon Apr 5, 2020
897a03c
Changes after code review
jessiejuachon Apr 7, 2020
3a4e183
Adding header to CacheItem.java
jessiejuachon Apr 7, 2020
b08feb4
Fixing code san issue. Synchronize on a final field
jessiejuachon Apr 7, 2020
8f57e24
Fixing code scan bug
jessiejuachon Apr 7, 2020
72c8799
Changes after code review
jessiejuachon Apr 7, 2020
92edfc8
Updating CasheService.isContainComponent to return false if cache key…
jessiejuachon Apr 7, 2020
2042554
HttpRequester if-none-match header and other customized headers are a…
jessiejuachon Apr 8, 2020
ae707ff
Code clean up; keep old caching expiration logic
jessiejuachon Apr 8, 2020
82818d5
Fixing code scan bug
jessiejuachon Apr 8, 2020
31d4ff6
Merge branch 'g11n-java-client' into g11n-java-client
jessiejuachon Apr 8, 2020
c9f83d7
code clean up
jessiejuachon Apr 9, 2020
78dcc1c
Removing response code and response msg from cache; Passing cacheItem…
jessiejuachon Apr 9, 2020
9281c64
Changes after code review
jessiejuachon Apr 10, 2020
2657762
Adding header to CacheItem.java file
jessiejuachon Apr 10, 2020
9e80f09
Fixing failing unit test
jessiejuachon Apr 10, 2020
68467f3
Changes after code review
jessiejuachon Apr 10, 2020
d67c72e
Fixing code scan bugs
jessiejuachon Apr 10, 2020
8591973
Not storing anything in cache if response is neither 200 nor 304
jessiejuachon Apr 13, 2020
22a5b29
Loading messages from specified offline resource bundles
jessiejuachon Apr 15, 2020
ce2488b
Merge branch 'g11n-java-client' of https://github.com/vmware/singleto…
jessiejuachon Apr 15, 2020
1092963
Adding header
jessiejuachon Apr 15, 2020
35d1d97
Fixing failing unie test
jessiejuachon Apr 15, 2020
ada17f1
Code changes after review; code clean up
jessiejuachon Apr 16, 2020
fc60eef
Using java.nio.file.Paths
jessiejuachon Apr 17, 2020
3e70449
Offline mode as fallback when service fetch fails
jessiejuachon Apr 17, 2020
609d8e4
Resetting VIPCfg and I18nFactory instances for each test
jessiejuachon Apr 17, 2020
dd19b1c
Resetting VIPCfg and I18nFactory after test
jessiejuachon Apr 17, 2020
7389f8b
Fixing tests
jessiejuachon Apr 17, 2020
5b92e44
Limit to wiremock test logging
jessiejuachon Apr 20, 2020
dfb41b5
Decreasing unit test log
jessiejuachon Apr 20, 2020
21a7105
this test is timing out and trying to connect 70+ times
jessiejuachon Apr 21, 2020
84343d5
Adding comments to code
jessiejuachon Apr 21, 2020
be8e443
Merge branch 'g11n-java-client' of https://github.com/vmware/singleto…
jessiejuachon Apr 21, 2020
b805d36
Adding timestamp to cache fromn local bundle
jessiejuachon Apr 21, 2020
17bc16c
Deprecating methods in TranslationMessage; adding new method getMessage
jessiejuachon Apr 23, 2020
15da010
Added tests for TranslationMessage.getMessage
jessiejuachon Apr 24, 2020
631de86
Fixing test
jessiejuachon Apr 24, 2020
ffd9adb
Fixing test
jessiejuachon Apr 24, 2020
c00f7ae
Fixing test
jessiejuachon Apr 24, 2020
d0504c4
Removing comment field because source collection is not supported any…
jessiejuachon Apr 24, 2020
79b18d0
Code clean up
jessiejuachon Apr 24, 2020
ec9f5db
Changes from code review
jessiejuachon Apr 24, 2020
2e8bd36
Code clean up
jessiejuachon Apr 24, 2020
4e44a8c
Optional SourceOpt in initialization
jessiejuachon Apr 26, 2020
de5c4ee
Fixing test
jessiejuachon Apr 27, 2020
406cb2a
Adding comments to code
jessiejuachon Apr 28, 2020
c80696a
Adding comment for javadoc
jessiejuachon Apr 28, 2020
79d111e
Cleaning up code
jessiejuachon Apr 28, 2020
24d8d5d
Code clean up
jessiejuachon Apr 28, 2020
23e62b6
Fixing test
jessiejuachon Apr 28, 2020
369643f
Removing source message fallback from this PR
jessiejuachon Apr 29, 2020
4e67e25
Merge branch 'g11n-java-client' into g11n-java-client
jessiejuachon Apr 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

import com.vmware.vipclient.i18n.I18nFactory;
import com.vmware.vipclient.i18n.VIPCfg;
import com.vmware.vipclient.i18n.base.DataSourceEnum;
import com.vmware.vipclient.i18n.base.cache.FormattingCache;
import com.vmware.vipclient.i18n.base.cache.MessageCache;
import com.vmware.vipclient.i18n.util.LocaleUtility;
Expand All @@ -33,11 +34,24 @@ public static void main(String[] args) {
} catch (VIPClientInitException e) {
System.out.println(e.getMessage());
}

cfg.initializeVIPService();
cfg.createTranslationCache(MessageCache.class);
cfg.createFormattingCache(FormattingCache.class);
I18nFactory.getInstance(cfg);

// Demonstrate French locale (online mode - from remote service)
demonstrate(thislocale);

// Demonstrate Filipino locale (offline mode - from local bundle)
cfg.setMessageOrigin(DataSourceEnum.Bundle);
cfg.setOfflineResourcesBaseUrl("offlineBundles/");
LocaleUtility.setLocale(new Locale("fil"));
demonstrate(new Locale("fil"));

}

private static void demonstrate(Locale locale) {
// Get translation
String key = "global_text_username";
String source = "User name";
Expand All @@ -49,9 +63,9 @@ public static void main(String[] args) {
System.out.println(">>>>>> Get translation by key: \"" + key + "\"");
String trans2 = Translation.getTranslation2(key);
System.out.println(trans2);

System.out.println(">>>>>> Check translation status of key: \"" + key + "\"");
boolean bReady = Translation.isTranslationReady("default", key, thislocale);
boolean bReady = Translation.isTranslationReady("default", key, locale);
System.out.println(bReady);

// Number format
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"component": "JAVA",
"messages": {
"LeadTest": "[{0}] Test alert",
"global_text_username": "User name",
"table.host": "Host",
"sample.plural.key1": "{0, plural, one{There is one file on disk \"{1}\".} other{There are # files on disk \"{1}\".}}"
},
"locale": "en-US"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"component": "JAVA",
"messages": {
"LeadTest": "[{0}] Alerto sa pagsusuri",
"global_text_username": "Pangalan ng gumagamit",
"table.host": "Punong-abala"
},
"locale": "fil"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"component": "JAVA",
"messages": {
"LeadTest": "[{0}] Alerte de test",
"global_text_username": "Nom d'utilisateur",
"table.host": "Hôte"
},
"locale": "fr"
}
10 changes: 7 additions & 3 deletions src/main/java/com/vmware/vipclient/i18n/I18nFactory.java
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ public static synchronized I18nFactory getInstance(VIPCfg cfg) {
if (factory == null) {
factory = new I18nFactory(cfg);
}
factory.setCfg(cfg);
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to remove this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next PR

return factory;
}

Expand All @@ -76,10 +77,13 @@ public Message getMessageInstance(Class c, VIPCfg cfg) {
Message i = null;
if (c == null) {
return i;
} else if (this.getCfg().getVipServer() == null
|| this.getCfg().getProductName() == null
} else if (this.getCfg().getProductName() == null
|| this.getCfg().getVersion() == null) {
logger.error("VipServer|ProductName|Version is null!");
logger.error("ProductName|Version is null!");
return i;
} else if (this.getCfg().getVipServer() == null
&& this.getCfg().getOfflineResourcesBaseUrl() == null) {
logger.error("One of offlineResourcesBaseUrl and vipServer must not be null!");
Xiaochao8 marked this conversation as resolved.
Show resolved Hide resolved
return i;
}
String key;
Expand Down
80 changes: 73 additions & 7 deletions src/main/java/com/vmware/vipclient/i18n/VIPCfg.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

import java.net.MalformedURLException;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.ResourceBundle;

Expand All @@ -19,6 +21,7 @@
import com.vmware.vipclient.i18n.base.cache.CacheMode;
import com.vmware.vipclient.i18n.base.cache.TranslationCacheManager;
import com.vmware.vipclient.i18n.exceptions.VIPClientInitException;
import com.vmware.vipclient.i18n.messages.api.opt.SourceOpt;
import com.vmware.vipclient.i18n.messages.dto.MessagesDTO;
import com.vmware.vipclient.i18n.messages.service.ProductService;

Expand All @@ -36,15 +39,18 @@ public class VIPCfg {
private TranslationCacheManager translationCacheManager;

// data origin
@Deprecated
private DataSourceEnum messageOrigin = DataSourceEnum.VIP;

private List<DataSourceEnum> msgOriginsQueue = new LinkedList<DataSourceEnum>();

// cache mode
private CacheMode cacheMode = CacheMode.MEMORY;

private String cachePath;

// define the global parameters
private boolean pseudo;
@Deprecated
private boolean collectSource;
private boolean cleanCache;
private long cacheExpiredTime;
Expand All @@ -56,7 +62,18 @@ public class VIPCfg {
private String version;
private String vipServer;
private String i18nScope = "numbers,dates,currencies,plurals,measurements";

private String offlineResourcesBaseUrl;

/**
* The optional SourceOpt object to access source messages. If SourceOpt is defined, then you can use a source message:
* a. as fallback if neither localized message nor default locale message was not retrieved successfully
* b. for pseudo-translation
* c. if the message hasn't been collected for localization
* e.g. ResourceBundleSrcOpt is the SourceOpt implementation that retrieves source messages from a .properties file.
* If source messages need to come from another location such as a DB, then have another implementation like DBSourceOpt.
*/
private SourceOpt srcOpt;
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved

// define key for cache management
public static final String CACHE_L3 = "CACHE_L3";
public static final String CACHE_L2 = "CACHE_L2";
Expand Down Expand Up @@ -86,6 +103,10 @@ public static synchronized VIPCfg getInstance() {
}
return gcInstance;
}

public static synchronized void resetInstance() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when this function will be called?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See @before in CacheServiceTest.java and OfflineModeTest.java
VipCfg is different for each test so have to reset it.
I see that for other tests, you are manually putting configurations back to original as a workaround.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change this.

gcInstance = null;
}

/**
* create a default instance of VIPCfg
Expand Down Expand Up @@ -120,9 +141,19 @@ public void initialize(String vipServer, String productName, String version) {
}

/**
* initialize the instance by a properties file
* Initialize VIPCfg instance using a configuration file
*
* @param cfg The configuration file
* @param srcOpt The optional SourceOpt object which gives access to source messages
*/
public void initialize(String cfg, SourceOpt srcOpt) throws VIPClientInitException {
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved
initialize(cfg);
this.setSrcOpt(srcOpt);
}
/**
* Initialize VIPCfg instance using a configuration file
*
* @param cfg
* @param cfg The configuration file
*/
public void initialize(String cfg) throws VIPClientInitException {
ResourceBundle prop = ResourceBundle.getBundle(cfg);
Expand All @@ -138,8 +169,17 @@ public void initialize(String cfg) throws VIPClientInitException {
}
if (prop.containsKey("version"))
this.version = prop.getString("version");
if (prop.containsKey("vipServer"))

// Remote VIP resources takes priority over offline resources
// so add DataSourceEnum.VIP first to msgOriginsQueue
if (prop.containsKey("vipServer")) {
this.vipServer = prop.getString("vipServer");
this.msgOriginsQueue.add(DataSourceEnum.VIP);
}
if (prop.containsKey("offlineResourcesBaseUrl")) {
this.offlineResourcesBaseUrl = prop.getString("offlineResourcesBaseUrl");
this.msgOriginsQueue.add(DataSourceEnum.Bundle);
}
if (prop.containsKey("pseudo"))
this.pseudo = Boolean.parseBoolean(prop.getString("pseudo"));
if (prop.containsKey("collectSource"))
Expand All @@ -159,7 +199,6 @@ public void initialize(String cfg) throws VIPClientInitException {
if (prop.containsKey("cacheExpiredTime"))
this.cacheExpiredTime = Long.parseLong(prop
.getString("cacheExpiredTime"));

}

/**
Expand Down Expand Up @@ -303,10 +342,12 @@ public void setPseudo(boolean pseudo) {
this.pseudo = pseudo;
}

@Deprecated
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved
public boolean isCollectSource() {
return collectSource;
}

@Deprecated
public void setCollectSource(boolean collectSource) {
this.collectSource = collectSource;
}
Expand Down Expand Up @@ -350,11 +391,13 @@ public boolean isMachineTranslation() {
public void setMachineTranslation(boolean machineTranslation) {
this.machineTranslation = machineTranslation;
}


@Deprecated
public DataSourceEnum getMessageOrigin() {
return messageOrigin;
}

@Deprecated
public void setMessageOrigin(DataSourceEnum messageOrigin) {
this.messageOrigin = messageOrigin;
}
Expand Down Expand Up @@ -393,4 +436,27 @@ public void setCachePath(String cachePath) {
this.cachePath = cachePath;
}

public String getOfflineResourcesBaseUrl() {
return offlineResourcesBaseUrl;
}

public void setOfflineResourcesBaseUrl(String offlineResourcesBaseUrl) {
this.offlineResourcesBaseUrl = offlineResourcesBaseUrl;
}

public List<DataSourceEnum> getMsgOriginsQueue() {
return msgOriginsQueue;
}

public void setMsgOriginsQueue(List<DataSourceEnum> msgOriginsQueue) {
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved
this.msgOriginsQueue = msgOriginsQueue;
}

public SourceOpt getSrcOpt() {
return srcOpt;
}

public void setSrcOpt(SourceOpt srcOpt) {
this.srcOpt = srcOpt;
}
jessiejuachon marked this conversation as resolved.
Show resolved Hide resolved
}
19 changes: 18 additions & 1 deletion src/main/java/com/vmware/vipclient/i18n/base/DataSourceEnum.java
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,23 @@
*/
package com.vmware.vipclient.i18n.base;

import com.vmware.vipclient.i18n.messages.api.opt.MessageOpt;
import com.vmware.vipclient.i18n.messages.api.opt.local.LocalMessagesOpt;
import com.vmware.vipclient.i18n.messages.api.opt.server.ComponentBasedOpt;
import com.vmware.vipclient.i18n.messages.dto.MessagesDTO;

public enum DataSourceEnum {
Bundle, VIP
Bundle {
@Override
public MessageOpt createMessageOpt(MessagesDTO dto) {
return new LocalMessagesOpt(dto);
}
},
VIP {
@Override
public MessageOpt createMessageOpt(MessagesDTO dto) {
return new ComponentBasedOpt(dto);
}
};
public abstract MessageOpt createMessageOpt(MessagesDTO dto);
}
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ public synchronized boolean put(String cacheKey, CacheItem itemToCache) {
if (cacheItem == null) {
cachedComponentsMap.put(cacheKey, (MessageCacheItem) itemToCache);
} else {
cacheItem.addCacheItem((MessageCacheItem) itemToCache);
cacheItem.setCacheItem((MessageCacheItem) itemToCache);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/*

* Copyright 2019 VMware, Inc.
* SPDX-License-Identifier: EPL-2.0
*/
Expand Down Expand Up @@ -31,16 +32,28 @@ public MessageCacheItem (Map<String, String> dataMap) {

private String etag;
private long timestamp;
private Long maxAgeMillis = 864000000l;
private Long maxAgeMillis = 86400000l;

private final Map<String, String> cachedData = new HashMap<String, String>();

public final Map<String, String> cachedData = new HashMap<String, String>();
public void addCacheData(String key, String value) {
this.cachedData.put(key, value);
}

public boolean isCachedDataEmpty() {
return this.cachedData.isEmpty();
}

public synchronized void addCachedData(Map<String, String> cachedData) {
if (cachedData != null)
this.cachedData.putAll(cachedData);
Xiaochao8 marked this conversation as resolved.
Show resolved Hide resolved
}

public synchronized void addCacheItem (MessageCacheItem cacheItem) {
public synchronized void setCacheItem (MessageCacheItem cacheItem) {
// Do not update cacheItem if timestamp is earlier than current.
// An older timestamp comes from an old thread that was blocked.
if (cacheItem.getTimestamp() < this.timestamp)
return;
Comment on lines +52 to +56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this logic, we should prevent querying fro server parallelly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave it if much effort.

this.addCachedData(cacheItem.getCachedData());
this.etag = cacheItem.etag;
this.timestamp = cacheItem.timestamp;
Expand Down Expand Up @@ -76,6 +89,10 @@ public synchronized void setMaxAgeMillis(Long maxAgeMillis) {
}

public boolean isExpired() {
// If offline mode only, cache never expires.
if (VIPCfg.getInstance().getVipServer() == null) {
return false;
}
// If maxAgeFromConfig is present, it means it is using the old way
// of caching expiration, so do not expire individual CacheItem object
if (VIPCfg.getInstance().getCacheExpiredTime() != 0) {
Expand All @@ -94,5 +111,5 @@ public boolean isExpired() {

return System.currentTimeMillis() - responseTimeStamp > maxAgeMillis;
}

}
Loading