Merge branch 'develop' of https://github.com/Angel-ML/angel into develop

Angel-ML · Aug 20, 2019 · 0db7fd8 · 0db7fd8
2 parents 114574c + 6e385a5
commit 0db7fd8
Show file tree

Hide file tree

Showing 154 changed files with 3,240 additions and 2,230 deletions.
diff --git a/Community.md b/Community.md
@@ -1,4 +1,4 @@
-Welcome to the Angel community! This COMMUNITY.md explains where to get help, and different ways to be part of the community. 
+Welcome to the Angel community! This document explains where to get help, and different ways to be part of the community. 
 
 The Angel project community comprises a group of core committer and a sub-group that forms the Technical Steering Committee (the “TSC”) which governs the project.
 
@@ -25,7 +25,7 @@ The TSC may add additional members to the TSC by a 2/3 majority vote.
 - [fitzwang](https://github.com/wangcaihua) - **Fitz Wang** <[fitzwang.ml@gmail.com](fitzwang.ml@gmail.com) >
 - [leleyu](https://github.com/leleyu)- **Lele Yu** <[yulele214@gmail.com](yulele214@gmail.com) >
 - [lbluesjjw](https://github.com/bluesjjw)- **Jeremy Jiang** <[blue.jwjiang@gmail.com](blue.jwjiang@gmail.com) >
- 
+
 ### TSC Meeting
 
 - The TSC meets regularly in a voice meeting call
@@ -53,10 +53,47 @@ Committers should be familiar with the guidelines for new contributors in [CONT
 
 
 
-
-
-
-
-
-
-
+## Contributor
+
+We would like to sincerely thank the following community members for their contributions to Angel.
+
+- [paynie](https://github.com/paynie) - **Paynie Xiao**
+- [andyyehoo](https://github.com/andyyehoo) - **Andy Huang**
+- [leleyu](https://github.com/leleyu) - **Lele Yu**
+- [shunanzhang](https://github.com/shunanzhang) - **Shunan Zhang**
+- [lbluesjjw](https://github.com/bluesjjw) - **Jeremy Jiang**
+- [TAAAN](https://github.com/TAAAN) - **Yunkun Tan**
+- [weiwee](https://github.com/weiwee) - **Wenbin Wei**
+- [rachelsunrh](https://github.com/rachelsunrh) - **Rachel Sun**
+- [ZuwenYou](https://github.com/ZuwenYou) - **Zuwen You**
+- [fitzwang](https://github.com/wangcaihua) - **Fitz Wang**
+- [luxin0123](https://github.com/luxin0123) - **Ally Yu**
+- [xujie32](https://github.com/xujie32) - **Joy Xu**
+- [FLYINGQQ](https://github.com/FLYINGQQ)
+- [yundai424](https://github.com/yundai424) - **Ariel Dai**
+- [hbghhy](https://github.com/hbghhy)
+- [duoyu119](https://github.com/duoyu119) - **Duo Yu**
+- [facaiy](https://github.com/facaiy) - **Facai Yan**
+- [f7753](https://github.com/f7753) - **Lewin Ma**
+- [ouyangwen-it](https://github.com/ouyangwen-it) - **Gdp Ouyang**
+- [ericzhang-cn](https://github.com/ericzhang-cn) - **Eric Zhang**
+- [ljch2018](https://github.com/ljch2018)
+- [liuzhanhao](https://github.com/liuzhanhao) - **Zhanhao (Jasper) Liu**
+- [AugustoYao](https://github.com/AugustoYao) - **Augusto Yao**
+- [20100507](https://github.com/20100507)
+- [yd19tc](https://github.com/yd19tc)
+- [witgo](https://github.com/witgo) - **Guoqiang Li**
+- [Morrissss](https://github.com/Morrissss)
+- [sunchenglong](https://github.com/sunchenglong) - **Chenglong Sun**
+- [quhailong](https://github.com/quhailong) - **Hailong Qu**
+- [patrikwang](https://github.com/patrikwang)
+- [mtunique](https://github.com/mtunique) - **Tao Meng**
+- [ibingoogle](https://github.com/ibingoogle)
+- [ghandzhipeng](https://github.com/ghandzhipeng) - **Zhipeng Zhang**
+- [Suanec](https://github.com/Suanec)
+- [sustcsonglin](https://github.com/sustcsonglin)
+- [endymecy](https://github.com/endymecy)
+- [StrikeW](https://github.com/StrikeW)
+- [wqh17101](https://github.com/wqh17101)
+- [Oneal65](<https://github.com/Oneal65>) - **Oneal Liu**
+- [cheney2015](<https://github.com/cheney2015>) - **Cristal Chen**
diff --git a/README.md b/README.md
@@ -4,6 +4,7 @@
 [![license](http://img.shields.io/badge/license-Apache2.0-brightgreen.svg?style=flat)](https://github.com/tencent/angel/blob/master/LICENSE)
 [![Release Version](https://img.shields.io/badge/release-2.2.0-red.svg)](https://github.com/tencent/angel/releases)
 [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/tencent/angel/pulls)
+[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3058/badge)](https://bestpractices.coreinfrastructure.org/projects/3058)
 
 This project **Angel** is a high-performance distributed machine learning platform based on the philosophy of Parameter Server. It is tuned for performance with big data from Tencent and has a wide range of applicability and stability, demonstrating increasing advantage in handling higher dimension model. Angel is jointly developed by Tencent and Peking University, taking account of both high availability  in industry and innovation in academia. **Angel** is developed with **Java** and **Scala**.  It supports running on **Yarn** and **Kubernetes**. With the **PS Service** abstraction, it provides two modules, namely **Spark on Angel** and **Pytorch on Angel** separately, which enable integrate the power of Spark/PyTorch and Parameter Server for distributed training.  Graph Computing and deep learning frameworks support is under development and will be released in the future.
 
@@ -63,3 +64,4 @@ We welcome everyone interested in machine learning to contribute code, create is
 
 
 
+
diff --git a/angel-ps/bin/angel-local-submit b/angel-ps/bin/angel-local-submit
@@ -24,7 +24,7 @@ bin=`cd "$bin"; pwd`
 lib="$bin"/../lib
 
 export ANGEL_HOME="$bin/../"
-export ANGEL_VERSION=3.0.0-SNAPSHOT
+export ANGEL_VERSION=3.0.0
 
 
 for f in "$lib"/*.jar; do

diff --git a/angel-ps/bin/angel-submit b/angel-ps/bin/angel-submit
@@ -13,7 +13,7 @@
 # or implied. See the License for the specific language governing permissions and limitations under
 # the License.
 
-export ANGEL_VERSION=3.0.0-SNAPSHOT
+export ANGEL_VERSION=3.0.0
 export PATH=${JAVA_HOME}/bin:$PATH
 
 if [ "${HADOOP_HOME}" != "" ]; then

diff --git a/angel-ps/bin/angel-submit-with-jarvis b/angel-ps/bin/angel-submit-with-jarvis
@@ -17,7 +17,7 @@
 
 # export HADOOP_HOME=""
 # export JAVA_HOME=""
-export ANGEL_VERSION=3.0.0-SNAPSHOT
+export ANGEL_VERSION=3.0.0
 export PATH=$JAVA_HOME/bin:$PATH
 
 if [ "$HADOOP_HOME" = "" ]; then

diff --git a/angel-ps/conf/angel-site.xml b/angel-ps/conf/angel-site.xml
@@ -18,7 +18,7 @@
 <configuration>
   <property>
     <name>angel.job.libjars</name>
-    <value>${ANGEL_HOME}/lib/jniloader-1.1.jar,${ANGEL_HOME}/lib/native_system-java-1.1.jar,${ANGEL_HOME}/lib/arpack_combined_all-0.1.jar,${ANGEL_HOME}/lib/all-1.1.2.pom,${ANGEL_HOME}/lib/core-1.1.2.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/jackson-annotations-2.6.5.jar,${ANGEL_HOME}/lib/jackson-core-2.7.7.jar,${ANGEL_HOME}/lib/jackson-core-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-databind-2.7.7.jar,${ANGEL_HOME}/lib/jackson-jaxrs-1.9.13.jar,${ANGEL_HOME}/lib/jackson-mapper-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-module-paranamer-2.6.5.jar,${ANGEL_HOME}/lib/jackson-module-scala_2.11-2.6.5.jar,${ANGEL_HOME}/lib/jackson-xc-1.9.13.jar,${ANGEL_HOME}/lib/json4s-ast_2.11-3.2.11.jar,${ANGEL_HOME}/lib/json4s-core_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-scalap_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-jackson_2.11-3.5.3.jar,${ANGEL_HOME}/lib/netty-all-4.1.17.Final.jar,${ANGEL_HOME}/lib/angel-ps-mllib-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-tools-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/scala-reflect-2.11.8.jar,${ANGEL_HOME}/lib/memory-0.8.1.jar,${ANGEL_HOME}/lib/sketches-core-0.8.1.jar,${ANGEL_HOME}/lib/commons-pool-1.6.jar,${ANGEL_HOME}/lib/kryo-shaded-4.0.0.jar,${ANGEL_HOME}/lib/kryo-serializers-0.42.jar,${ANGEL_HOME}/lib/scala-library-2.11.8.jar,${ANGEL_HOME}/lib/angel-ps-core-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-psf-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/fastutil-7.1.0.jar,${ANGEL_HOME}/lib/sizeof-0.3.0.jar,${ANGEL_HOME}/lib/minlog-1.3.0.jar,${ANGEL_HOME}/lib/breeze_2.11-0.13.jar,${ANGEL_HOME}/lib/angel-format-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/angel-math-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/angel-mlcore-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/libopenblas.so,${ANGEL_HOME}/lib/commons-math-2.2.jar</value>
+    <value>${ANGEL_HOME}/lib/jniloader-1.1.jar,${ANGEL_HOME}/lib/native_system-java-1.1.jar,${ANGEL_HOME}/lib/arpack_combined_all-0.1.jar,${ANGEL_HOME}/lib/all-1.1.2.pom,${ANGEL_HOME}/lib/core-1.1.2.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/jackson-annotations-2.6.5.jar,${ANGEL_HOME}/lib/jackson-core-2.7.7.jar,${ANGEL_HOME}/lib/jackson-core-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-databind-2.7.7.jar,${ANGEL_HOME}/lib/jackson-jaxrs-1.9.13.jar,${ANGEL_HOME}/lib/jackson-mapper-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-module-paranamer-2.6.5.jar,${ANGEL_HOME}/lib/jackson-module-scala_2.11-2.6.5.jar,${ANGEL_HOME}/lib/jackson-xc-1.9.13.jar,${ANGEL_HOME}/lib/json4s-ast_2.11-3.2.11.jar,${ANGEL_HOME}/lib/json4s-core_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-scalap_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-jackson_2.11-3.5.3.jar,${ANGEL_HOME}/lib/netty-all-4.1.17.Final.jar,${ANGEL_HOME}/lib/angel-ps-mllib-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-tools-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/scala-reflect-2.11.8.jar,${ANGEL_HOME}/lib/memory-0.8.1.jar,${ANGEL_HOME}/lib/sketches-core-0.8.1.jar,${ANGEL_HOME}/lib/commons-pool-1.6.jar,${ANGEL_HOME}/lib/kryo-shaded-4.0.0.jar,${ANGEL_HOME}/lib/kryo-serializers-0.42.jar,${ANGEL_HOME}/lib/scala-library-2.11.8.jar,${ANGEL_HOME}/lib/angel-ps-core-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-psf-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/fastutil-7.1.0.jar,${ANGEL_HOME}/lib/sizeof-0.3.0.jar,${ANGEL_HOME}/lib/minlog-1.3.0.jar,${ANGEL_HOME}/lib/breeze_2.11-0.13.jar,${ANGEL_HOME}/lib/angel-format-0.1.1.jar,${ANGEL_HOME}/lib/angel-math-0.1.1.jar,${ANGEL_HOME}/lib/angel-mlcore-0.1.1.jar,${ANGEL_HOME}/lib/libopenblas.so,${ANGEL_HOME}/lib/commons-math-2.2.jar</value>
   </property>
 
   <property>

diff --git a/angel-ps/core/pom.xml b/angel-ps/core/pom.xml
@@ -22,7 +22,7 @@
     <parent>
         <groupId>com.tencent.angel</groupId>
         <artifactId>angel-ps</artifactId>
-        <version>3.0.0-SNAPSHOT</version>
+        <version>3.0.0</version>
     </parent>
     <artifactId>angel-ps-core</artifactId>
     <name>angel-ps-core</name>
@@ -64,22 +64,6 @@
             <artifactId>kubernetes-client</artifactId>
             <version>3.0.0</version>
         </dependency>
-        <dependency>
-            <groupId>com.tencent.angel</groupId>
-            <artifactId>angel-math</artifactId>
-            <!--version>3.0.0</version-->
-            <version>0.1.0-SNAPSHOT</version>
-        </dependency>
-        <dependency>
-            <groupId>com.tencent.angel</groupId>
-            <artifactId>angel-format</artifactId>
-            <version>0.1.0-SNAPSHOT</version>
-        </dependency>
-        <dependency>
-            <groupId>com.tencent.angel</groupId>
-            <artifactId>angel-format</artifactId>
-            <version>0.1.0-SNAPSHOT</version>
-        </dependency>
     </dependencies>
 
     <build>

diff --git a/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java b/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.common;
 
 import java.io.DataInputStream;

diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.config
 
 import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, Pod, PodBuilder}

diff --git a/...l-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala b/...l-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.config
 
 private[angel] object Constants {

diff --git a/...core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala b/...core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.config
 
 import com.tencent.angel.conf.AngelConf

diff --git a/...rc/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala b/...rc/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.config
 
 import io.fabric8.kubernetes.api.model.HasMetadata

diff --git a/...rc/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala b/...rc/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.config
 
 private[angel] sealed trait KubernetesVolumeSpecificConf

diff --git a/...a/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala b/...a/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.features
 
 import com.tencent.angel.conf.AngelConf

diff --git a/...ava/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala b/...ava/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.features
 
 import com.tencent.angel.conf.AngelConf

diff --git a/...main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala b/...main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.features
 
 import com.tencent.angel.kubernetesmanager.deploy.config.{AngelPod, KubernetesConf, KubernetesRoleSpecificConf}

diff --git a/...main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala b/...main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala
@@ -1,3 +1,19 @@
+/*
+ * Tencent is pleased to support the open source community by making Angel available.
+ *
+ * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in
+ * compliance with the License. You may obtain a copy of the License at
+ *
+ * https://opensource.org/licenses/Apache-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ *
+ */
 package com.tencent.angel.kubernetesmanager.deploy.features
 
 import com.tencent.angel.kubernetesmanager.deploy.config.{AngelPod, KubernetesConf, KubernetesMasterSpecificConf}