[Refactor] Dubbo Cloud Native (#4844)

* Polish #4542 : [Enhancement] Adapt the Java standard Event/Listener mechanism * Polish #4541 : [Feature] Add local File System DynamicConfigurationFactory‘s extension * Polish #4541 : Bugfix * Polish #4541 : Optimization * Polish #4541 : Add the compatibility for PollingWatchService on the some platforms * Polish #4541 : Add delay publish without ThreadPoolExecutor * Polish #4541 : Refactor the extension name * Polish #4541 : Add remove ops * Polish #4541 : Add testable constructor * Polish #4541 : Add getConfigGroups method * Polish #4610 : [Refactor] Refactor the bootstrap module * Polish #4541 : Fix the nulling URL issue * Polish #4622 : [Refactor] Refactor ConfigManager * Polish #4622 : [Refactor] Refactor ConfigManager * Polish #4622 : Support multiple configcenters * Polish #4671 : ServiceNameMapping will not map the group, version and protocol * update referenceCount log (#4683) Add comments to support multiple shared connections * Polish /#4687 : Remove the duplicated test code in dubbo-config-spring (#4688) * #4685 修改代码if判断false问题 if (hasException == false)修改成if (!hasException) (#4695) * Fixed Service annotation method parameters are not in effect (#4598) * keep demo simple, and switch to use zookeeper as registry center (#4705) * keep demo simple, and switch to use zookeeper as registry center * remove comment * @reference auto-wires the instance of generic interface #4594 (#4677) * try to shorten maven output to make travis build pass (#4710) * use CountDownLatch to check zk registry if establish connection (#4589) * Minor change * Rename the extension name of WritableMetadataService * Polish #4759 : [Refactor] Change the signature of methods of MetadataService #4759 * Merge remote-tracking branch 'upstream/master' into dubbo-cloud-native # Conflicts: # dubbo-all/pom.xml # dubbo-bom/pom.xml # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/AbstractInterfaceConfig.java # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/ApplicationConfig.java # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/ConfigCenterConfig.java # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/ReferenceConfig.java # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/RegistryConfig.java # dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/ServiceConfig.java # dubbo-config/dubbo-config-api/src/test/java/org/apache/dubbo/config/ReferenceConfigTest.java # dubbo-configcenter/dubbo-configcenter-api/src/main/java/org/apache/dubbo/configcenter/DynamicConfiguration.java # dubbo-configcenter/dubbo-configcenter-api/src/test/java/org/apache/dubbo/configcenter/mock/MockDynamicConfiguration.java # dubbo-configcenter/dubbo-configcenter-consul/src/main/java/org/apache/dubbo/configcenter/consul/ConsulDynamicConfiguration.java # dubbo-configcenter/dubbo-configcenter-etcd/src/test/java/org/apache/dubbo/configcenter/support/etcd/EtcdDynamicConfigurationTest.java # dubbo-configcenter/dubbo-configcenter-nacos/src/main/java/org/apache/dubbo/configcenter/support/nacos/NacosDynamicConfiguration.java # dubbo-configcenter/dubbo-configcenter-nacos/src/test/java/org/apache/dubbo/configcenter/support/nacos/NacosDynamicConfigurationTest.java # dubbo-configcenter/dubbo-configcenter-zookeeper/src/main/java/org/apache/dubbo/configcenter/support/zookeeper/ZookeeperDynamicConfiguration.java # dubbo-metadata/dubbo-metadata-api/src/main/java/org/apache/dubbo/metadata/definition/model/MethodDefinition.java # dubbo-metadata/dubbo-metadata-api/src/main/java/org/apache/dubbo/metadata/report/identifier/MetadataIdentifier.java # dubbo-metadata/dubbo-metadata-api/src/main/java/org/apache/dubbo/metadata/report/support/AbstractMetadataReport.java # dubbo-metadata/dubbo-metadata-api/src/test/java/org/apache/dubbo/metadata/report/identifier/MetadataIdentifierTest.java # dubbo-metadata/dubbo-metadata-definition-protobuf/src/main/java/org/apache/dubbo/metadata/definition/protobuf/ProtobufTypeBuilder.java # dubbo-metadata/dubbo-metadata-definition-protobuf/src/test/java/org/apache/dubbo/metadata/definition/protobuf/ProtobufTypeBuilderTest.java # dubbo-metadata/pom.xml # dubbo-registry/dubbo-registry-api/src/main/java/org/apache/dubbo/registry/integration/AbstractConfiguratorListener.java # dubbo-registry/dubbo-registry-nacos/src/main/java/org/apache/dubbo/registry/nacos/NacosRegistry.java # dubbo-registry/dubbo-registry-nacos/src/main/java/org/apache/dubbo/registry/nacos/NacosRegistryFactory.java # dubbo-rpc/dubbo-rpc-xml/src/main/java/org/apache/dubbo/xml/rpc/protocol/xmlrpc/XmlRpcProtocol.java * Polish #3984 : Add the implementation of Page<ServiceInstance> getInstances(String serviceName, int offset, int pageSize, boolean healthyOnly) * Code merge * Fix the cases * Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud-native # Conflicts: # dubbo-bootstrap/src/test/java/org/apache/dubbo/bootstrap/DubboServiceProviderBootstrap.java # dubbo-metadata/dubbo-metadata-definition-protobuf/pom.xml # dubbo-registry/dubbo-registry-api/src/test/java/org/apache/dubbo/registry/support/ServiceOrientedRegistryTest.java # dubbo-registry/dubbo-registry-consul/src/main/java/org/apache/dubbo/registry/consul/ConsulServiceDiscoveryFactory.java # dubbo-registry/dubbo-registry-etcd3/src/main/java/org/apache/dubbo/registry/etcd/EtcdServiceDiscovery.java * Refactor ConfigManager * Refactor ConfigManager * Resolve the issues on ConfigManager * Refactor and add test-cases for ConfigManager * Polish #4774 : [Feature] Dubbo Cloud Native - To Support in Spring * Polish #4808 : [Feature] Add the registered/unregistered event mechanism ShutdownHook * Polish #4807 : [Feature] Add the callback mechanism ShutdownHook #4807 * Polish #4813 : [Feature] add Prioritized implementation for ServiceInstanceCustomizer * Polish #4815 : [Feature] Add the ServiceLoader for Dubbo's services or components * Polish #4815 : [Feature] Add the ServiceLoader for Dubbo's services or components * Polish #4813 : [Feature] add Prioritized implementation for ServiceInstanceCustomizer * Polish #4807 : Add sort implementation * Refactor * Refactor
apache · Aug 16, 2019 · d6d0bec · d6d0bec
1 parent dd360f4
commit d6d0bec
Show file tree

Hide file tree

Showing 66 changed files with 756 additions and 623 deletions.
diff --git a/dubbo-bootstrap/src/main/java/org/apache/dubbo/bootstrap/DubboBootstrap.java b/dubbo-bootstrap/src/main/java/org/apache/dubbo/bootstrap/DubboBootstrap.java
@@ -21,7 +21,6 @@
 import org.apache.dubbo.common.config.configcenter.DynamicConfiguration;
 import org.apache.dubbo.common.config.configcenter.wrapper.CompositeDynamicConfiguration;
 import org.apache.dubbo.common.context.Lifecycle;
-import org.apache.dubbo.common.extension.ExtensionLoader;
 import org.apache.dubbo.common.logger.Logger;
 import org.apache.dubbo.common.logger.LoggerFactory;
 import org.apache.dubbo.common.utils.CollectionUtils;
@@ -50,20 +49,22 @@
 import org.apache.dubbo.config.utils.ReferenceConfigCache;
 import org.apache.dubbo.event.EventDispatcher;
 import org.apache.dubbo.event.EventListener;
+import org.apache.dubbo.metadata.MetadataService;
+import org.apache.dubbo.metadata.MetadataServiceExporter;
+import org.apache.dubbo.metadata.WritableMetadataService;
 import org.apache.dubbo.metadata.report.MetadataReportInstance;
 import org.apache.dubbo.registry.client.AbstractServiceDiscoveryFactory;
 import org.apache.dubbo.registry.client.DefaultServiceInstance;
 import org.apache.dubbo.registry.client.ServiceDiscovery;
 import org.apache.dubbo.registry.client.ServiceInstance;
-import org.apache.dubbo.rpc.Protocol;
-import org.apache.dubbo.rpc.ProtocolServer;
 
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Set;
+import java.util.SortedSet;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.locks.Condition;
@@ -75,13 +76,11 @@
 import static java.util.concurrent.Executors.newSingleThreadExecutor;
 import static org.apache.dubbo.common.config.ConfigurationUtils.parseProperties;
 import static org.apache.dubbo.common.config.configcenter.DynamicConfiguration.getDynamicConfiguration;
-import static org.apache.dubbo.common.constants.CommonConstants.GROUP_CHAR_SEPERATOR;
-import static org.apache.dubbo.common.constants.CommonConstants.METADATA_DEFAULT;
-import static org.apache.dubbo.common.constants.CommonConstants.METADATA_KEY;
 import static org.apache.dubbo.common.constants.CommonConstants.METADATA_REMOTE;
 import static org.apache.dubbo.common.utils.StringUtils.isNotEmpty;
 import static org.apache.dubbo.config.context.ConfigManager.getInstance;
-import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.MEATADATA_STORED_TYPE_KEY;
+import static org.apache.dubbo.metadata.WritableMetadataService.getExtension;
+import static org.apache.dubbo.registry.client.metadata.ServiceInstanceMetadataUtils.setMetadataStorageType;
 import static org.apache.dubbo.remoting.Constants.CLIENT_KEY;
 
 /**
@@ -130,7 +129,13 @@ public class DubboBootstrap implements Lifecycle {
      */
     private volatile boolean onlyRegisterProvider = false;
 
-    private ServiceInstance serviceInstance;
+    private volatile boolean defaultMetadataStorageType = true;
+
+    private volatile ServiceInstance serviceInstance;
+
+    private volatile MetadataService metadataService;
+
+    private volatile MetadataServiceExporter metadataServiceExporter;
 
     public DubboBootstrap() {
         DubboShutdownHook.getDubboShutdownHook().register();
@@ -147,6 +152,19 @@ public DubboBootstrap onlyRegisterProvider(boolean onlyRegisterProvider) {
         return this;
     }
 
+    public boolean isOnlyRegisterProvider() {
+        return onlyRegisterProvider;
+    }
+
+    public boolean isDefaultMetadataStorageType() {
+        return defaultMetadataStorageType;
+    }
+
+    public DubboBootstrap defaultMetadataStorageType(boolean defaultMetadataStorageType) {
+        this.defaultMetadataStorageType = defaultMetadataStorageType;
+        return this;
+    }
+
     public DubboBootstrap metadataReport(MetadataReportConfig metadataReportConfig) {
         configManager.addMetadataReport(metadataReportConfig);
         return this;
@@ -161,7 +179,6 @@ public DubboBootstrap metadataReports(List<MetadataReportConfig> metadataReportC
         return this;
     }
 
-
     // {@link ApplicationConfig} correlative methods
 
     /**
@@ -419,6 +436,10 @@ public DubboBootstrap initialize() {
 
             useRegistryAsConfigCenterIfNecessary();
 
+            initMetadataService();
+
+            initMetadataServiceExporter();
+
             initialized = true;
 
             if (logger.isInfoEnabled()) {
@@ -430,6 +451,24 @@ public DubboBootstrap initialize() {
         return this;
     }
 
+    /**
+     * Initialize {@link MetadataService} from {@link WritableMetadataService}'s extension
+     */
+    private void initMetadataService() {
+        this.metadataService = getExtension(isDefaultMetadataStorageType());
+    }
+
+    /**
+     * Initialize {@link MetadataServiceExporter}
+     */
+    private void initMetadataServiceExporter() {
+        this.metadataServiceExporter = new ConfigurableMetadataServiceExporter()
+                .setApplicationConfig(getApplication())
+                .setRegistries(configManager.getRegistries())
+                .setProtocols(configManager.getProtocols())
+                .metadataService(metadataService);
+    }
+
     private void loadRemoteConfigs() {
         // registry ids to registry configs
         List<RegistryConfig> tmpRegistries = new ArrayList<>();
@@ -512,26 +551,16 @@ public DubboBootstrap start() {
         }
         if (!isStarted()) {
 
+            // 1. export Dubbo Services
             exportServices();
 
-            // Not only provider register and some services are exported
-            if (!onlyRegisterProvider && !configManager.getServices().isEmpty()) {
-                /**
-                 * export {@link MetadataService}
-                 */
-                // TODO, only export to default registry?
-                ApplicationConfig applicationConfig = configManager.getApplication().orElseThrow(() -> new IllegalStateException("ApplicationConfig cannot be null"));
-                if (!METADATA_REMOTE.equals(applicationConfig.getMetadata())) {
-                    exportMetadataService(
-                            applicationConfig,
-                            configManager.getRegistries(),
-                            configManager.getProtocols()
-                    );
-                }
-                /**
-                 * Register the local {@link ServiceInstance}
-                 */
-                registerServiceInstance(applicationConfig);
+            // 2. export MetadataService
+            exportMetadataService();
+
+            // Not only provider register
+            if (!isOnlyRegisterProvider() || hasExportedServices()) {
+                //3. Register the local ServiceInstance if required
+                registerServiceInstance();
             }
 
             referServices();
@@ -545,6 +574,14 @@ public DubboBootstrap start() {
         return this;
     }
 
+    private boolean hasExportedServices() {
+        return !metadataService.getExportedURLs().isEmpty();
+    }
+
+    private ApplicationConfig getApplication() {
+        return configManager.getApplication().orElseThrow(() -> new IllegalStateException("ApplicationConfig cannot be null"));
+    }
+
     /**
      * Block current thread to be await.
      *
@@ -578,6 +615,7 @@ public DubboBootstrap await() {
     public DubboBootstrap stop() {
         if (isInitialized() && isStarted()) {
             unregisterServiceInstance();
+            unexportMetadataService();
             unexportServices();
             started = false;
         }
@@ -699,14 +737,17 @@ public DubboBootstrap addEventListener(EventListener<?> listener) {
         return this;
     }
 
-    private List<URL> exportMetadataService(ApplicationConfig applicationConfig,
-                                            Collection<RegistryConfig> globalRegistryConfigs,
-                                            Collection<ProtocolConfig> globalProtocolConfigs) {
-        ConfigurableMetadataServiceExporter exporter = new ConfigurableMetadataServiceExporter();
-        exporter.setApplicationConfig(applicationConfig);
-        exporter.setRegistries(globalRegistryConfigs);
-        exporter.setProtocols(globalProtocolConfigs);
-        return exporter.export();
+    /**
+     * export {@link MetadataService} and get the exported {@link URL URLs}
+     *
+     * @return {@link MetadataServiceExporter#getExportedURLs()}
+     */
+    private List<URL> exportMetadataService() {
+        return metadataServiceExporter.export().getExportedURLs();
+    }
+
+    private void unexportMetadataService() {
+        metadataServiceExporter.unexport();
     }
 
     private void exportServices() {
@@ -724,38 +765,42 @@ private void referServices() {
         configManager.getReferences().forEach(cache::get);
     }
 
-    public boolean isOnlyRegisterProvider() {
-        return onlyRegisterProvider;
-    }
+    private void registerServiceInstance() {
 
-    private void registerServiceInstance(ApplicationConfig applicationConfig) {
-        ExtensionLoader<Protocol> loader = ExtensionLoader.getExtensionLoader(Protocol.class);
-        Set<String> protocols = loader.getLoadedExtensions();
-        if (CollectionUtils.isEmpty(protocols)) {
-            throw new IllegalStateException("There should has at least one Protocol specified.");
-        }
+        ApplicationConfig application = getApplication();
 
-        String protocol = findOneProtocolForServiceInstance(protocols);
+        String serviceName = application.getName();
 
-        Protocol protocolInstance = loader.getExtension(protocol);
+        URL exportedURL = selectMetadataServiceExportedURL();
 
-        String serviceName = applicationConfig.getName();
-        // TODO, only support exporting one server
-        ProtocolServer server = protocolInstance.getServers().get(0);
-        String[] address = server.getAddress().split(GROUP_CHAR_SEPERATOR);
-        String host = address[0];
-        int port = Integer.parseInt(address[1]);
+        String host = exportedURL.getHost();
 
-        ServiceInstance serviceInstance = initServiceInstance(
-                serviceName,
-                host,
-                port,
-                applicationConfig.getMetadata() == null ? METADATA_DEFAULT : applicationConfig.getMetadata()
-        );
+        int port = exportedURL.getPort();
+
+        ServiceInstance serviceInstance = initServiceInstance(serviceName, host, port);
 
         getServiceDiscoveries().forEach(serviceDiscovery -> serviceDiscovery.register(serviceInstance));
     }
 
+    private URL selectMetadataServiceExportedURL() {
+
+        URL selectedURL = null;
+
+        SortedSet<String> urlValues = metadataService.getExportedURLs();
+
+        for (String urlValue : urlValues) {
+            URL url = URL.valueOf(urlValue);
+            if ("rest".equals(url.getProtocol())) { // REST first
+                selectedURL = url;
+                break;
+            } else {
+                selectedURL = url; // If not found, take any one
+            }
+        }
+
+        return selectedURL;
+    }
+
     /**
      * Use rest protocol if there's one, otherwise, choose the first one available.
      *
@@ -791,9 +836,9 @@ private void unregisterServiceInstance() {
 
     }
 
-    private ServiceInstance initServiceInstance(String serviceName, String host, int port, String metadataType) {
+    private ServiceInstance initServiceInstance(String serviceName, String host, int port) {
         this.serviceInstance = new DefaultServiceInstance(serviceName, host, port);
-        this.serviceInstance.getMetadata().put(MEATADATA_STORED_TYPE_KEY, metadataType);
+        setMetadataStorageType(serviceInstance, isDefaultMetadataStorageType());
         return this.serviceInstance;
     }
 

diff --git a/dubbo-bootstrap/src/test/java/org/apache/dubbo/bootstrap/DubboServiceConsumerBootstrap.java b/dubbo-bootstrap/src/test/java/org/apache/dubbo/bootstrap/DubboServiceConsumerBootstrap.java
@@ -17,16 +17,9 @@
 package org.apache.dubbo.bootstrap;
 
 import org.apache.dubbo.bootstrap.rest.UserService;
-import org.apache.dubbo.config.ApplicationConfig;
-import org.apache.dubbo.config.MetadataReportConfig;
 import org.apache.dubbo.config.ReferenceConfig;
 import org.apache.dubbo.config.context.ConfigManager;
 
-import java.util.HashMap;
-import java.util.Map;
-
-import static org.apache.dubbo.common.constants.CommonConstants.METADATA_KEY;
-
 /**
  * Dubbo Provider Bootstrap
  *
@@ -40,7 +33,7 @@ public static void main(String[] args) throws Exception {
                 .application("dubbo-consumer-demo")
                 // Zookeeper
                 .registry("zookeeper", builder -> builder.address("zookeeper://127.0.0.1:2181?registry.type=service&subscribed.services=dubbo-provider-demo"))
-                .metadataReport(new MetadataReportConfig("zookeeper://127.0.0.1:2181"))
+//                .metadataReport(new MetadataReportConfig("zookeeper://127.0.0.1:2181"))
                 // Nacos
 //                .registry("consul", builder -> builder.address("consul://127.0.0.1:8500?registry.type=service&subscribed.services=dubbo-provider-demo").group("namespace1"))
                 .reference("echo", builder -> builder.interfaceClass(EchoService.class).protocol("dubbo"))

diff --git a/dubbo-bootstrap/src/test/java/org/apache/dubbo/bootstrap/DubboServiceProvider2Bootstrap.java b/dubbo-bootstrap/src/test/java/org/apache/dubbo/bootstrap/DubboServiceProvider2Bootstrap.java
@@ -16,6 +16,9 @@
  */
 package org.apache.dubbo.bootstrap;
 
+import org.apache.dubbo.bootstrap.rest.UserService;
+import org.apache.dubbo.bootstrap.rest.UserServiceImpl;
+
 /**
  * Dubbo Provider Bootstrap
  *
@@ -31,8 +34,10 @@ public static void main(String[] args) {
                 // Nacos
 //                .registry("nacos", builder -> builder.address("nacos://127.0.0.1:8848?registry.type=service"))
 //                .registry(RegistryBuilder.newBuilder().address("etcd3://127.0.0.1:2379?registry.type=service").build())
-                .protocol(builder -> builder.port(20885).name("dubbo"))
-                .service(builder -> builder.id("test").interfaceClass(EchoService.class).ref(new EchoServiceImpl()))
+                .protocol("dubbo", builder -> builder.port(20885).name("dubbo"))
+                .protocol("rest", builder -> builder.port(9090).name("rest"))
+                .service(builder -> builder.id("echo").interfaceClass(EchoService.class).ref(new EchoServiceImpl()).protocolIds("dubbo"))
+                .service(builder -> builder.id("user").interfaceClass(UserService.class).ref(new UserServiceImpl()).protocolIds("rest"))
                 .start()
                 .await();
     }

diff --git a/...java/org/apache/dubbo/common/config/configcenter/file/FileSystemDynamicConfiguration.java b/...java/org/apache/dubbo/common/config/configcenter/file/FileSystemDynamicConfiguration.java
@@ -169,6 +169,10 @@ public class FileSystemDynamicConfiguration extends AbstractDynamicConfiguration
 
     private final Map<File, List<ConfigurationListener>> listenersRepository;
 
+    public FileSystemDynamicConfiguration() {
+        this(URL.valueOf("file:///default"));
+    }
+
     public FileSystemDynamicConfiguration(URL url) {
         this(initDirectory(url), getEncoding(url), getThreadPoolPrefixName(url), getThreadPoolSize(url),
                 getThreadPoolKeepAliveTime(url));

diff --git a/dubbo-common/src/main/java/org/apache/dubbo/common/constants/RegistryConstants.java b/dubbo-common/src/main/java/org/apache/dubbo/common/constants/RegistryConstants.java
@@ -67,7 +67,7 @@ public interface RegistryConstants {
      */
     String SERVICE_REGISTRY_TYPE = "service";
 
-    String SERVICE_REGISTRY_PROTOCOL = "service-oriented-registry";
+    String SERVICE_REGISTRY_PROTOCOL = "service-discovery-registry";
 
     /**
      * The parameter key of the subscribed service names for Service-Oriented Registry

diff --git a/dubbo-common/src/main/java/org/apache/dubbo/common/context/Lifecycle.java b/dubbo-common/src/main/java/org/apache/dubbo/common/context/Lifecycle.java
@@ -21,7 +21,7 @@
  *
  * @since 2.7.4
  */
-public interface Lifecycle {
+public interface Lifecycle<V> {
 
     /**
      * Initialize the component before {@link #start() start}

diff --git a/dubbo-common/src/main/java/org/apache/dubbo/common/extension/ExtensionLoader.java b/dubbo-common/src/main/java/org/apache/dubbo/common/extension/ExtensionLoader.java
@@ -353,6 +353,16 @@ public T getExtension(String name) {
         return (T) instance;
     }
 
+    /**
+     * Get the extension by specified name if found, or {@link #getDefaultExtension() returns the default one}
+     *
+     * @param name the name of extension
+     * @return non-null
+     */
+    public T getOrDefaultExtension(String name) {
+        return containsExtension(name)  ? getExtension(name) : getDefaultExtension();
+    }
+
     /**
      * Return default extension, return <code>null</code> if it's not configured.
      */
@@ -543,6 +553,10 @@ private T createExtension(String name) {
         }
     }
 
+    private boolean containsExtension(String name) {
+        return getExtensionClasses().containsKey(name);
+    }
+
     private T injectExtension(T instance) {
         try {
             if (objectFactory != null) {