From f262267143363a4ec5240e14e0c871a0afc741ba Mon Sep 17 00:00:00 2001 From: joyjxu Date: Thu, 15 Aug 2019 20:14:13 +0800 Subject: [PATCH 01/20] fixed "PNG" error --- docs/algo/afm_on_angel_en.md | 4 ++-- docs/algo/dcn_on_angel_en.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/algo/afm_on_angel_en.md b/docs/algo/afm_on_angel_en.md index 9925df7e6..00554dd8f 100644 --- a/docs/algo/afm_on_angel_en.md +++ b/docs/algo/afm_on_angel_en.md @@ -3,7 +3,7 @@ ## 1. Introduction AFM(Attention Neural Factorization Machines) is a improved FM method, which which learns the importance of each feature interaction from data via a neural attention network. -![AFM](../img/AFM.PNG) +![AFM](../img/AFM.png) ### 1.1 BiInteractionCrossTiled layer BiInteractionCrossTiled is a layer that calculates each second-order feature interaction and concatenates them as a vector. @@ -12,7 +12,7 @@ BiInteractionCrossTiled is a layer that calculates each second-order feature int ParamSharedFC layer is a fully connected layer with shared parameters, as explained in the following formulas, where w is the shared parameters for every feature interaction. h is another shared parameter vector for the next layer. The attention weights are calculated by applying a softmax operation on the layer's output. -![ParamSharedFC](../img/ParamSharedFC.PNG) +![ParamSharedFC](../img/ParamSharedFC.png) ## 2. Execution & Performance ### 2.1 Network Construction by Json diff --git a/docs/algo/dcn_on_angel_en.md b/docs/algo/dcn_on_angel_en.md index c2a4a7bcb..703de05cf 100644 --- a/docs/algo/dcn_on_angel_en.md +++ b/docs/algo/dcn_on_angel_en.md @@ -3,13 +3,13 @@ ## 1. Introduction DCN(Deep & Cross Network) explicitly applies feature crossing at each layer, requires no manual feature engineering, and adds negligible extra complexity to the DNN model. -![DCN](../img/DCN.PNG) +![DCN](../img/DCN.png) ### 1.1 Cross Layer DCN is combined of two parts: cross network and deep network. The deep network is a simple dnn, stacked by several fully connected layers. The Cross network, in other hands, is stacked by several cross layers. Each cross layer has two inputs: the original embedding layer and the output of the last cross layer. The calculating method is as follows. -![CrossLyer](../img/CrossLyer.PNG) +![CrossLyer](../img/CrossLyer.png) ### 1.3 Combination From 56bcbec95e9b1f5e4bc12146578de0eeecf59462 Mon Sep 17 00:00:00 2001 From: paynie Date: Fri, 16 Aug 2019 10:40:37 +0800 Subject: [PATCH 02/20] update line doc --- docs/algo/sona/line_sona_en.md | 150 +++++++++++++++++++++------------ 1 file changed, 98 insertions(+), 52 deletions(-) diff --git a/docs/algo/sona/line_sona_en.md b/docs/algo/sona/line_sona_en.md index 4d0159777..a47aad0ec 100644 --- a/docs/algo/sona/line_sona_en.md +++ b/docs/algo/sona/line_sona_en.md @@ -1,52 +1,98 @@ -# LINE - -> LINE (Large-scale Information Network Embedding) algorithm is one of the well-known algorithms in the field of Network Embedding. It embeds graph data into vector space as to use vertor-based machine learning algorithm to handle graph datas. - -## Algorithm Introduction - -The LINE algorithm is a network representation learning algorithm(also be considered as a preprocessing algorithm for graph data). The algorithm recieve a network as input and, produces the vector representation for each node. The LINE algorithm mainly focuses on optimizing two objective functions: - -![line_loss](../../img/line_loss.png) - -where, p_1 characterizes the first-order similarity between nodes (direct edge), and p_2 depicts the second-order similarity between nodes (similar neighbors). in other words, - -- If there are joints between two nodes, then the two nodes are also close in the embedded vector space -- If the neighbors of two nodes are similar, then in the embedded vector space, the two nodes are also close - -For more details, please refer to the paper [[1]](https://arxiv.org/abs/1503.03578) - -## Distributed Implemention - -The implementation of the LINE algorithm refers to Yahoo's paper [[2]](https://arxiv.org/abs/1606.08495), which - -1. splitting the Embedding vector into multiple PSs by dimension - -2. processing the dot product between nodes partially in each PS, and merge in spark executors. - -3. calculates the gradient for each node in executors - -4. pushes the gradient to all PS, and then update vectors - -![line_structure](../../img/line_structure.png) - -## Running - -### Algorithm IO parameters - -- input: hdfs path, undirected graph, nodes need to be consecutively numbered starting from 0, separated by white space or comma, for example: - 0 2 - 2 1 - 3 1 - 3 2 - 4 1 -- modelPath: hdfs path, the final model save path is modelPath/epoch_checkpoint_x, where x represents the xth round epoch -- modelCPInterval: save the model every few rounds of epoch - -### Algorithm parameters - -- vectorDim: The vector space dimension of the embedding vector and the vector dimension of the context (meaning that the model space occupied by the second-order optimization is twice the first-order optimization under the same parameters) -- negSample: The algorithm samples negative sampling optimization, indicating the number of negative sampling nodes used by each pair -- learningRate: The learning rate affects the results of the algorithm -- BatchSize: the size of each mini batch -- maxEpoch: the number of rounds used by the sample, the sample will be shuffled after each round -- Order: Optimize the order, 1 or 2 \ No newline at end of file +# LINE + +> LINE (Large-scale Information Network Embedding) algorithm is one of the well-known algorithms in the field of Network Embedding. It embeds graph data into vector space as to use vertor-based machine learning algorithm to handle graph datas. + +## Algorithm Introduction + +The LINE algorithm is a network representation learning algorithm(also be considered as a preprocessing algorithm for graph data). The algorithm recieve a network as input and, produces the vector representation for each node. The LINE algorithm mainly focuses on optimizing two objective functions: + +![line_loss](../../img/line_loss.png) + +where, p_1 characterizes the first-order similarity between nodes (direct edge), and p_2 depicts the second-order similarity between nodes (similar neighbors). in other words, + +- If there are joints between two nodes, then the two nodes are also close in the embedded vector space +- If the neighbors of two nodes are similar, then in the embedded vector space, the two nodes are also close + +For more details, please refer to the paper [[1]](https://arxiv.org/abs/1503.03578) + +## Distributed Implemention +There are currently two completely different implementation versions in Spark On Angel: **LINE V1** and **LINE V2**. Each of these two methods has advantages and disadvantages and application scenarios. + +In LINE V1, we push the training data to the PS and then training the model in PS. It can avoid a large amount of network IO caused by pulling the model, but it is not suitable for scenes with smaller node embedding dimensions, because it has some limitations: the number of model partitions must be smaller than the node embedding dimension, and the node embedding dimension must be the integer number of partitions. If the node dimension is small, the number of model partitions must be very small, PS is easy to become a bottleneck. + +in LINE V2, we pull model from PS to Worker, then training the model in Worker, so it is suitable for the case where the node embedding dimension is not very high. In addition, LINE V2 is much more stable. **In general, we strongly recommend LINE V2**. + +### LINE V1 + +The implementation of the LINE algorithm refers to Yahoo's paper [[2]](https://arxiv.org/abs/1606.08495), which + +The model is divided by column, it means that each partition contains all nodes. + +1. split the Embedding vector into multiple PSs by dimension + +2. process the dot product between nodes partially in each PS, and merge in spark executors. + +3. calculate the gradient for each node in executors + +4. push the gradient to all PS, and then update vectors + + +### LINE V2 + +The model is divided by node id range, it means that each partition contains part of node + +1. negative sample and pull the nodes embedding from PSs + +2. process the dot product between nodes in executors + +3. calculate the gradient for each node in executors + +4. push the gradient to all PS, and then update nodes embedding + +## Running + +### Algorithm IO parameters + +- input: hdfs path, undirected graph, nodes need to be consecutively numbered starting from 0, separated by white space or comma, for example: + 0 2 + 2 1 + 3 1 + 3 2 + 4 1 +- output: hdfs path, the final model save path is output/epoch_checkpoint_x, where x represents the xth round epoch +- saveModelInterval: save the model every few rounds of epoch +- checkpointInterval: write the model checkpoint every few rounds of epoch + +### Algorithm parameters + +- embedding: The vector space dimension of the embedding vector and the vector dimension of the context (meaning that the model space occupied by the second-order optimization is twice the first-order optimization under the same parameters) +- negative: The algorithm samples negative sampling optimization, indicating the number of negative sampling nodes used by each pair +- stepSize: The learning rate affects the results of the algorithm +- batchSize: the size of each mini batch +- epoch: the number of rounds used by the sample, the sample will be shuffled after each round +- order: Optimize the order, 1 or 2 +- subSample sub sample or not, true or false +- remapping: remapping the node id or not, true or false + +### Submitting scripts +``` +input=hdfs://my-hdfs/data +output=hdfs://my-hdfs/model + +source ./bin/spark-on-angel-env.sh +$SPARK_HOME/bin/spark-submit \ + --master yarn-cluster\ + --conf spark.ps.instances=1 \ + --conf spark.ps.cores=1 \ + --conf spark.ps.jars=$SONA_ANGEL_JARS \ + --conf spark.ps.memory=10g \ + --name "kcore angel" \ + --jars $SONA_SPARK_JARS \ + --driver-memory 5g \ + --num-executors 1 \ + --executor-cores 4 \ + --executor-memory 10g \ + --class org.apache.spark.angel.examples.graph.LINEExample2 \ + ./lib/angelml-0.1.0-SNAPSHOT.jar + input:$input output:$output embedding:128 negative:5 epoch:100 stepSize:0.01 batchSize:1000 numParts:2 subSample:false remapping:false order:2 interval:5 +``` \ No newline at end of file From a796009339a69e94ac18c31089cac5461ae0a726 Mon Sep 17 00:00:00 2001 From: paynie Date: Fri, 16 Aug 2019 10:44:47 +0800 Subject: [PATCH 03/20] add CII Best Practices --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 9ddeeea48..f4fd13f41 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ [![license](http://img.shields.io/badge/license-Apache2.0-brightgreen.svg?style=flat)](https://github.com/tencent/angel/blob/master/LICENSE) [![Release Version](https://img.shields.io/badge/release-2.2.0-red.svg)](https://github.com/tencent/angel/releases) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/tencent/angel/pulls) +[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3058/badge)](https://bestpractices.coreinfrastructure.org/projects/3058) This project **Angel** is a high-performance distributed machine learning platform based on the philosophy of Parameter Server. It is tuned for performance with big data from Tencent and has a wide range of applicability and stability, demonstrating increasing advantage in handling higher dimension model. Angel is jointly developed by Tencent and Peking University, taking account of both high availability in industry and innovation in academia. **Angel** is developed with **Java** and **Scala**. It supports running on **Yarn** and **Kubernetes**. With the **PS Service** abstraction, it provides two modules, namely **Spark on Angel** and **Pytorch on Angel** separately, which enable integrate the power of Spark/PyTorch and Parameter Server for distributed training. Graph Computing and deep learning frameworks support is under development and will be released in the future. @@ -61,3 +62,4 @@ We welcome everyone interested in machine learning to contribute code, create is + From 85c9a044a030a0a8e23576742c70eb179ad63e4e Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 15:42:09 +0800 Subject: [PATCH 04/20] add doc for angel run on kubernetes. --- docs/deploy/run_on_kubernetes_en.md | 167 ++++++++++++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 docs/deploy/run_on_kubernetes_en.md diff --git a/docs/deploy/run_on_kubernetes_en.md b/docs/deploy/run_on_kubernetes_en.md new file mode 100644 index 000000000..7c5d179b7 --- /dev/null +++ b/docs/deploy/run_on_kubernetes_en.md @@ -0,0 +1,167 @@ +# Running Angel On Kubernetes + +Angel can also run on clusters managed by [Kubernetes](https://kubernetes.io/). This feature makes use of native Kubernetes scheduler that has been added to Angel. + +### 1. **Preparing for the run environment** + +The environment requirements for running Angel on Kubernetes include: + +* A runnable distribution of Angel 3.0.0 or above. + +* A running Kubernetes cluster at version >= 1.6 with access configured to it using kubectl. If you do not already have a working Kubernetes cluster, you may set up a test cluster on your local machine using [minikube](https://kubernetes.io/docs/setup/learning-environment/minikube/). + * If use minikube, we recommend using the latest release of minikube with the DNS addon enabled. + * Be aware that the default minikube configuration is not enough for running Angel applications. We recommend 1 CPUs and 2g of memory for per angel ps/worker to be able to start a example. + +* You must have appropriate permissions to list, create, edit and delete pods in your cluster. You can verify that you can list these resources by running kubectl auth can-i pods. + * The service account credentials used by the Angel Master pods must be allowed to create pods, services and configmaps. + +* You must have Kubernetes DNS configured in your cluster. + +### 2. **Angel on Kubernetes Job Example** + +* Docker images + Kubernetes requires users to supply images that can be deployed into containers within pods. The images are built to be run in a container runtime environment that Kubernetes supports. Docker is a container runtime + environment that is frequently used with Kubernetes. We provide a basic image for the user, you can pull it down and run or customize your own image through it. Follow the pull command: + `docker pull 202102/angel:v3.0.0` + +* Distributed File System + we both support cephfs and hdfs distributed file system, to ensure that the kubernetes cluster's pods are accessible + +* Namespace && [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) + * Create a namespace for angel, namespaces are ways to divide cluster resources between multiple users (via resource quota). Angel on Kubernetes can use namespaces to launch Angel applications. This can be made use of + through the `angel.kubernetes.namespace` configuration. Follow the command to create a namespace: + `kubectl create namespace angel-group` + * Create a service account, the Angel master pod uses a Kubernetes service account to access the Kubernetes API server to create and watch parameter server and worker pods. the service account must be granted a Role or + ClusterRole that allows angel master pod to create pods and services. Follow the command to create service account: + `kubectl create serviceaccount angel` + * To create a RoleBinding or ClusterRoleBinding, grant a service account a Role or ClusterRole. The following command creates an edit ClusterRole in the angel-group namespace and grants it to the angel service account + created above: + `kubectl create clusterrolebinding angel-role --clusterrole=edit --serviceaccount=angel-group:angel --namespace=angel-group` + +* Submitting the Job + * data preparation + Put your data to cephfs or hdfs, angel example data in `$ANGEL_HOME/data`. This example uses `$ANGEL_HOME/data/a9a_123d_train.libsvm` as data, assuming the data has been uploaded to cephfs. + + * Use `angel-submit` under the `bin` directory in the distribution package to submit Angel jobs to the Kubernetes cluster + > **please make sure the cluster has enough resources; for the following example, at least 6GB memory and 3 vcores are needed to start the job** + + ```$xslt + export ANGEL_HOME=/path/to/angel + $ANGEL_HOME/bin/angel-submit \ + -Dangel.deploy.mode=KUBERNETES \ + -Dangel.kubernetes.master= \ + -Dangel.app.submit.class=com.tencent.angel.ml.core.graphsubmit.GraphRunner \ + -Dangel.action.type=train \ + -Dangel.kubernetes.container.image=202102/angel:v3.0.0 \ + -Dangel.kubernetes.master.volumes.hostPath.test.mount.path=/angel/mountpath \ + -Dangel.kubernetes.master.volumes.hostPath.test.options.path=/cephfs/basepath/to/angel \ + -Dangel.kubernetes.executor.volumes.hostPath.test.mount.path=/angel/mountpath \ + -Dangel.kubernetes.executor.volumes.hostPath.test.options.path=/cephfs/basepath/to/angel \ + -Dangel.kubernetes.namespace=angel-group \ + -Dangel.kubernetes.serviceaccount=angel \ + -Dangel.ps.number=2 \ + -Dangel.ps.memory.gb=2 \ + -Dangel.am.memory.gb=1 \ + -Dangel.workergroup.number=2\ + -Dangel.worker.memory.gb=2 \ + -Dangel.train.data.path=file:///angel/mountpath/to/data \ + -Dangel.save.model.path=file:///angel/mountpath/to/model \ + -Dangel.tmp.output.path.prefix=/angel/mountpath/to/tmp \ + -Dangel.output.path.deleteonexist=true \ + -Dangel.log.path=file:///angel/mountpath/to/angel_log \ + -Dangel.job.name=angel_lr_test \ + -Dml.epoch.num=5 \ + -Dml.model.class.name=com.tencent.angel.ml.classification.LogisticRegression \ + -Dml.feature.index.range=123 \ + -Dml.model.size=123 \ + -Dml.data.validate.ratio=0.1 \ + -Dml.data.type=libsvm \ + -Dml.learn.rate=0.01 \ + -Dml.reg.l2=0.03 \ + -Dml.inputlayer.optimizer=ftrl + ``` + while run on kubernetes, `angel.deploy.mode` must set `KUBERNETES` and `angel.kubernetes.master` must set kubernetes + cluster api server url, If you have a Kubernetes cluster setup, one way to discover the apiserver URL is by executing kubectl cluster-info. + ```$xslt + kubectl cluster-info + Kubernetes master is running at http://127.0.0.1:8080 + ``` + When using the cephfs, you need to mount path to ensure angel master/worker can access data in cephfs. + +### 3. **Other Features** + +* Dependency Management + + Application dependencies can be pre-mounted into custom-built Docker images. Those dependencies can be added to the classpath + by referencing them with / use `angel.kubernetes.master.extraClassPath` and `angel.kubernetes.executor.extraClassPath` in angel-submit script + or setting the ANGEL_EXTRA_CLASSPATH environment variable in your Dockerfiles. For example, if you want use hdfs, then your images need to contain hadoop + client and hadoop dependency. + +* Using Kubernetes Volumes + + Users can mount the following types of Kubernetes [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) into the Angel master and ps/worker pods: + * hostPath: mounts a file or directory from the host node’s filesystem into a pod. + * emptyDir: an initially empty volume created when a pod is assigned to a node. + * persistentVolumeClaim: used to mount a PersistentVolume into a pod. + + To mount a volume of any of the types above into the Angel master pod, use the following configuration property: + ```$xslt + --conf angel.kubernetes.master.volumes.[VolumeType].[VolumeName].mount.path= + --conf angel.kubernetes.master.volumes.[VolumeType].[VolumeName].mount.readOnly= + ``` + Specifically, VolumeType can be one of the following values: hostPath, emptyDir, and persistentVolumeClaim. VolumeName is the name you want to use for the + volume under the volumes field in the pod specification. For example, the path of a hostPath with volume name test can be specified using the following property: + + ```$xslt + angel.kubernetes.master.volumes.hostPath.test.options.path=/cephfs/basepath/to/angel + ``` + The configuration properties for mounting volumes into the Angel ps/worker pods use prefix `angel.kubernetes.executor.` instead of `angel.kubernetes.master.`. + +* Logs & Debug + * Accessing Logs + Logs can be accessed using the Kubernetes API and the kubectl CLI. When a Angel application is running, it’s possible to stream logs from the application using: + ```$xslt + kubectl -n logs -f + ``` + The same logs can also be accessed through the `Kubernetes dashboard` if installed on the cluster. + + * Debug + To get some basic information about the scheduling decisions made around the Angel master pod, you can run: + ```$xslt + kubectl -n describe pod + kubectl -n get pod -o yaml --export + ``` + If the pod has encountered a runtime error, the status can be probed further using: + ```$xslt + kubectl -n logs + ``` + +### 4. **Configuration** + +The following configurations are specific to Angel on Kubernetes. + +| Property Name | Default | Meaning | +| --- | --- | --- | +| angel.kubernetes.master | none | kubernetes cluster api server url | +| angel.kubernetes.namespace | default | The namespace that will be used for running the angel master and angel ps/worker pods. | +| angel.kubernetes.container.image | none | Container image to use for the Angel application. This is usually of the form example.com/repo/angel:v3.0.0. This configuration is required and must be provided by the user, we provide a base image `docker pull 202102/angel:v3.0.0` . | +| angel.kubernetes.container.image.pullPolicy | IfNotPresent | Container image pull policy used when pulling images within Kubernetes, valid values are `Always`, `Never`, and `IfNotPresent`. | +| angel.kubernetes.serviceaccount | default | Service account that is used when running the angel master pod. The angel master pod uses this service account when requesting angel ps/worker pods from the API server | +| angel.kubernetes.master.label.[LabelName] | none | Add the label specified by LabelName to the angel master pod. | +| angel.kubernetes.master.annotation.[AnnotationName] | none | Add the annotation specified by AnnotationName to the angel master pod. | +| angel.kubernetes.executor.label.[LabelName] | none | Add the label specified by LabelName to the angel ps/worker pod. | +| angel.kubernetes.executor.annotation.[AnnotationName] | none | Add the annotation specified by AnnotationName to the angel ps/worker pod. | +| angel.kubernetes.masterEnv.[EnvironmentVariableName] | none | Add the environment variable specified by EnvironmentVariableName to the Angel master process. The user can specify multiple of these to set multiple environment variables. | +| angel.kubernetes.executorEnv.[EnvironmentVariableName] | none | Add the environment variable specified by EnvironmentVariableName to the Angel ps/worker process. The user can specify multiple of these to set multiple environment variables. | +| angel.kubernetes.master.extraClassPath | none | Add the extra classpath to the angel master process | +| angel.kubernetes.executor.extraClassPath | none | Add the extra classpath to the angel ps/worker process | +| angel.kubernetes.master.volumes.[VolumeType].[VolumeName].mount.path | none | Add the Kubernetes Volume named VolumeName of the VolumeType type to the angel master pod on the path specified in the value. | +| angel.kubernetes.master.volumes.[VolumeType].[VolumeName].mount.readOnly | none | Specify if the mounted volume is read only or not. | +| angel.kubernetes.master.volumes.[VolumeType].[VolumeName].options.[OptionName] | none | Configure Kubernetes Volume options passed to the Kubernetes with OptionName as key having specified value, must conform with Kubernetes option format. | +| angel.kubernetes.executor.volumes.[VolumeType].[VolumeName].mount.path | none | Add the Kubernetes Volume named VolumeName of the VolumeType type to the angel ps/worker pod on the path specified in the value. | +| angel.kubernetes.executor.volumes.[VolumeType].[VolumeName].mount.readOnly | none | Specify if the mounted volume is read only or not. | +| angel.kubernetes.executor.volumes.[VolumeType].[VolumeName].options.[OptionName] | none | Configure Kubernetes Volume options passed to the Kubernetes with OptionName as key having specified value, must conform with Kubernetes option format. | +| angel.kubernetes.master.secrets.[SecretName] | none | Add the Kubernetes Secret named SecretName to the angel master pod on the path specified in the value. | +| angel.kubernetes.executor.secrets.[SecretName] | none | Add the Kubernetes Secret named SecretName to the angel ps/worker pod on the path specified in the value. | + +All angel on kubernetes configuration information can be referred to `AngelConf` class. \ No newline at end of file From 86249924f6be0471c65d154c090daae8f74e3587 Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 15:48:06 +0800 Subject: [PATCH 05/20] update run_on_kubernetes_en.md --- docs/deploy/run_on_kubernetes_en.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/deploy/run_on_kubernetes_en.md b/docs/deploy/run_on_kubernetes_en.md index 7c5d179b7..f73d54e92 100644 --- a/docs/deploy/run_on_kubernetes_en.md +++ b/docs/deploy/run_on_kubernetes_en.md @@ -38,12 +38,12 @@ The environment requirements for running Angel on Kubernetes include: created above: `kubectl create clusterrolebinding angel-role --clusterrole=edit --serviceaccount=angel-group:angel --namespace=angel-group` -* Submitting the Job +* **Submitting the Job** * data preparation Put your data to cephfs or hdfs, angel example data in `$ANGEL_HOME/data`. This example uses `$ANGEL_HOME/data/a9a_123d_train.libsvm` as data, assuming the data has been uploaded to cephfs. * Use `angel-submit` under the `bin` directory in the distribution package to submit Angel jobs to the Kubernetes cluster - > **please make sure the cluster has enough resources; for the following example, at least 6GB memory and 3 vcores are needed to start the job** + > **please make sure the cluster has enough resources; for the following example, at least 9GB memory and 5 vcores are needed to start the job** ```$xslt export ANGEL_HOME=/path/to/angel From 9b09ded11843251a0fb3f81a46ec2a0839ff9a85 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?cristalchen=28=E9=99=88=E4=BB=81=E6=B4=81=29?= Date: Fri, 16 Aug 2019 16:56:02 +0800 Subject: [PATCH 06/20] add english caption --- docs/basic/lossfunc_on_angel_en.md | 89 ++++++++++++++ docs/basic/optimizer_on_angel_en.md | 167 +++++++++++++++++++++++++++ docs/basic/transfunc_on_angel _en.md | 35 ++++++ 3 files changed, 291 insertions(+) create mode 100644 docs/basic/lossfunc_on_angel_en.md create mode 100644 docs/basic/optimizer_on_angel_en.md create mode 100644 docs/basic/transfunc_on_angel _en.md diff --git a/docs/basic/lossfunc_on_angel_en.md b/docs/basic/lossfunc_on_angel_en.md new file mode 100644 index 000000000..996ce5f68 --- /dev/null +++ b/docs/basic/lossfunc_on_angel_en.md @@ -0,0 +1,89 @@ +# Loss function in Angel + +## 1. Loss function in Angel +There are a lot of loss functons in Angel, which can be classified into two categories +- Classification loss function +- Regression loss function + +Classification loss function can be shown in the following table: + +name|expression|description +---|---|--- +CrossEntoryLoss| ![](http://latex.codecogs.com/png.latex?-\sum_iy_i\log{p_i(x)}) | is used for classification, logistic regression is a special case and can also be used for multi-class Softmax, which requires input to be a probability.![](http://latex.codecogs.com/png.latex?p_i(x)), rather than ![](http://latex.codecogs.com/png.latex?f(x)) +LogLoss| ![](http://latex.codecogs.com/png.latex?\log{\(1+e^{-yf(x)}\)}) |is used fFor classification, it is the loss function of logistic regression, which can be regarded as a special column of CrossEntoryLoss function, which is embodied by Sigmoid![](http://latex.codecogs.com/png.latex?p_i(x)) +SoftmaxLoss| ![](http://latex.codecogs.com/png.latex?-\sum_iI(y=i)\log\frac{x^{x_i}}{\sum_je^{x_j}}) | It is a special form of CrossEntoryLoss, which is embodied in the way of Softmax![](http://latex.codecogs.com/png.latex?p_i(x)) +HingeLoss| ![](http://latex.codecogs.com/png.latex?\max{\(0,1-yf(x)\)}) | loss function of SVM + +Graphically represented as follows: + +![classifcationloss](../img/classifcationloss.png) + +The regression loss function is shown in the following table + +name|expression|description +---|---|--- +L2Loss | ![](http://latex.codecogs.com/png.latex?\|y-f(x)\|_2^2) | is used for regression, and is the loss function of least squares regression +HuberLoss | ![](http://latex.codecogs.com/png.latex?\left\\{\begin{array}{ll}\delta\cdot\(abs(x)-\frac{\delta}{2}\),&abs(x)>\delta\\\\\frac{1}{2}x^2,&abs(x)\le\delta\end{array}\right.) | is used for regression, and it uses a quadratic function near 0, and uses a function once in other places to solve the problem that the absolute value function is non-differentiable around 0. The model obtained by Huber loss is more robust. + +Graphically represented as follows: + +![regressionloss](../img/regressionloss.png) + +## 2. implementation of Loss in Angel +loss in angel all implement LossFunc Trait, as follows: +```scala +trait LossFunc extends Serializable { + def calLoss(modelOut: Matrix, graph: AngelGraph): Double + + def loss(pred: Double, label: Double): Double + + def calGrad(modelOut: Matrix, graph: AngelGraph): Matrix + + def predict(modelOut: Matrix): Matrix +} +``` +It can be seen that the loss in angel not only has the function of calculating loss, but also the functions of `calculating gradient` and `predicting. It is precisely because of the gradient calculation implemented in Loss that the reverse conduction has a starting point. In LossLayer Calculating the gradient is to call the calGrad of lossFunc directly, as follows: +```scala + override def calGradOutput(): Matrix = { + val start = System.currentTimeMillis() + status match { + gradOutput = lossFunc.calGrad(output, graph) + status = STATUS.Backward + case _ => + } + val end = System.currentTimeMillis() + gradOutput + } +``` +Another function is `prediction`. Calculating the predicted value in LossLayer is directly calling the predict of lossFunc, as follows: +```scala + override def predict(): Matrix = { + status match { + case STATUS.Null => + calOutput() + case _ => + } + + lossFunc.predict(output) + } +``` + +## 3. Json of Loss Function +### 3.1 Loss function without parameters +In addition to huberloss, the other lossfunc is a lossless function with no parameters. There are two expressions, as follows +```json +"lossfunc": "logloss" + +"lossfunc": { + "type": "logloss" +} +``` + +### 3.2 Loss function of parameter +Only huberloss, as follows: +```json +"lossfunc": { + "type": "huberloss", + "delta": 0.1 +} +``` diff --git a/docs/basic/optimizer_on_angel_en.md b/docs/basic/optimizer_on_angel_en.md new file mode 100644 index 000000000..f0d98b17b --- /dev/null +++ b/docs/basic/optimizer_on_angel_en.md @@ -0,0 +1,167 @@ +# Optimizer in Angel + +There are many optimization methods for machine learning, but in the big data scenario, the most used method is a series of methods based on SGD. In Angel, only a small number of optimization methods are implemented, as follows: +- Methods based on stochastic gradient descent + - SDG: here is mini-batch SGD (mini-batch Stochastic gradient descent) + - Momentum: SGD with momentum + - AdaGrad: SGD with Hessian diagonal approximation + - AdaDelta: + - Adam: SGD with momentum and diagonal Hessian approximation +- Online learning method + - FTRL: Follow The Regularized Leader, an online learining method + + + + +## 1. SGD +The update equation for SGD is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}\bold{x}_{t+1}=\bold{x}_t-\eta\Delta\bold{x}_t) + +where, ![](http://latex.codecogs.com/png.latex?\eta)is learning rate. Using SGD can add ![](http://latex.codecogs.com/png.latex?L_1,L_2) Regularization, the actual optimization is based on PGD(proximal gradient descent). + +There are two kinds of json expressions, as follows: +```json +"optimizer": "sgd", + +"optimizer": { + "type": "sgd", + "reg1": 0.01, + "reg2": 0.02 +} +``` + +## 2. Momentum +Momentum's update formula is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}\bold{v}_t=\gamma\bold{v}_{t-1}+\Delta\bold{x}_t,\bold{x}_{t+1}=\bold{x}_t-\eta\bold{v}_t) + +where, ![](http://latex.codecogs.com/png.latex?\gamma)is momentum factor, ![](http://latex.codecogs.com/png.latex?\eta)is learning rate. Besides, Momentum can be added ![](http://latex.codecogs.com/png.latex?L_2) regularization. The default optimization method in Angel is Momentum. + +There are two types of json expressions, as follows: +```json +"optimizer": "momentum", + +"optimizer": { + "type": "momentum", + "momentum": 0.9, + "reg2": 0.01 +} +``` + +## 3. AdaGrad +The update formula for AdaGrad (here is the exponential smoothing version, ie RMSprop) is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}n_t=\beta%20n_{t-1}+(1-\beta)g_{t}^2) + +![model](http://latex.codecogs.com/png.latex?\dpi{150}\Delta%20\theta=-\frac{\eta}{\sqrt{n_t+\epsilon}}\cdot%20g_t) + +where, ![](http://latex.codecogs.com/png.latex?\beta)is smoothing factor, ![](http://latex.codecogs.com/png.latex?\eta)is learning rate. Besides, AdaGrad can be added ![](http://latex.codecogs.com/png.latex?L_1,L2)regularization. + +There are two types of json expressions, as follows: +```json +"optimizer": "adagrad", + +"optimizer": { + "type": "adagrad", + "beta": 0.9, + "reg1": 0.01, + "reg2": 0.01 +} +``` + +## 4. AdaDelta +The update formula for AdaDelta (here is the exponential smoothing version, ie RMSprop) is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}n_t=\alpha%20n_{t-1}+(1-\alpha)g_{t}^2) + +![model](http://latex.codecogs.com/png.latex?\dpi{150}\Delta%20\theta=-\frac{\sqrt{v_{t-1}+\epsilon}}{\sqrt{n_t+\epsilon}}\cdot%20g_t) + +![model](http://latex.codecogs.com/png.latex?\dpi{150}v_t=\beta%20v_{t-1}+(1-\beta)\Delta%20\theta_{t}^2) + + +where, ![](http://latex.codecogs.com/png.latex?\alpha,%20\beta)is smoothing factor. AdaDelta can be added ![](http://latex.codecogs.com/png.latex?L_1,L2) regularization. + +There are two types of json expressions, as follows: +```json +"optimizer": "adadelta", + +"optimizer": { + "type": "adadelta", + "alpha": 0.9, + "beta": 0.9, + "reg1": 0.01, + "reg2": 0.01 +} +``` + +## 5. Adam +Adam is a good optimization method, update fomula is: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}\begin{array}{ll}\bold{m}_t&=\beta\bold{m}_{t-1}+(1-\beta)\Delta\bold{x}_t\\\\\\bold{v}_t&=\gamma\bold{v}_{t-1}+(1-\gamma)\Delta\bold{x}^2_t\\\\\bold{x}_t&=\gamma\bold{x}_{t-1}-\eta\frac{\sqrt{1-\gamma^t}}{1-\beta^t}\frac{\bold{m}_t}{\sqrt{\bold{v}_t}+\epsilon}\\%20\end{array}) + +where, ![](http://latex.codecogs.com/png.latex?\bold{m}_t)is the exponential smoothing of gradient![](http://latex.codecogs.com/png.latex?\bold{x}_t), which is momentum, ![](http://latex.codecogs.com/png.latex?\bold{v}_t)is the exponential smoothing of gradient![](http://latex.codecogs.com/png.latex?\bold{x}^2_t), Can be seen as a diagonal approximation of Hessian. In default cases ![](http://latex.codecogs.com/png.latex?\beta=0.9,\gamma=0.99), note + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(t)=\frac{\sqrt{1-\gamma^t}}{1-\beta^t}) + +![](http://latex.codecogs.com/png.latex?f(t))Is a function with an initial value of 1, and a limit of 1, the intermediate process is first reduced and then increased, as shown below: +![adam_coeff](../img/adam_coeff.png) + +That is, in the initial stage of optimization, the gradient is larger, the learning rate is appropriately reduced, and the gradient is reduced and smoothed. In the final stage of optimization, the gradient is small, and appropriately increasing the learning rate helps to jump out of the local optimum. + +There are two types of json expressions, as follows: +```json +"optimizer": "adam", + +"optimizer": { + "type": "adam", + "beta": 0.9, + "gamma": 0.99, + "reg2": 0.01 +} +``` + +## 6. FTRL +FTRL is an online learning algorithm whose goal is to optimize the regret bound. It can be proved to be effective under some conditions of learning rate attenuation. + +Another feature of FTRL is that it can get very sparse solutions, which is better than PGD (proximal gradient descent) and better than other online learning algorithms, such as FOBOS, RDA, etc. + +The algorithm flow of FTRL is as follows: + +![FTRL](../img/ftrl_lr_project.png) + +There are two types of json expressions, as follows: +```json +"optimizer": "ftrl", + +"optimizer": { + "type": "ftrl", + "alpha": 0.1, + "beta": 1.0, + "reg1": 0.01, + "reg2": 0.01 +} +``` +ps: ![](http://latex.codecogs.com/png.latex?\lambda_1,\lambda_2)are a regularization parameters, corresponding to "reg1, reg2" in json + +**some tips:** + +For the deep and wide algorithm, in principle, it is necessary to ensure that the convergence speed of the two sides is not too different. +- The wide part optimizer uses FTRL because FTRL converges relatively slowly, so that it is trained by the depth side.. +- The deep part optimizer uses Adam because Adam converges relatively quickly. Although the deep side uses the fast convergence optimizer, his parameters are too many. + +For the FTRL optimizer, it is designed for online learning. In order to ensure the stability of the model, the amplitude of each update is very small. In the online learning environment, the data is one by one or a small batch and a small batch, in theory, should not Let the small amount of data modify the model too mush. So in the FTRL algorithm, the batch size can not be too large, preferably less than 10000 data.. + +Different optimizer convergence speed comparison: FTRL < SGD < Momentum < AdaGrad ~ AdaDelta < Adam + +Since AdaGrad, AdaDelta, and Adam introduce Hessian diagonal approximations, they can have larger batches to ensure the accuracy of gradients and Hessian diagonal matrices. For simpler first-order optimizers such as FTRL, SGD, and Momentum, they need more iterations, so the batch size can't be too big. So there is: +BatchSize(FTRL) < BatchSize(SGD) < BatchSize(Momentum) < BatchSize(AdaGrad) ~ BatchSize(AdaDelta) < BatchSize(Adam) + +Regarding the learning rate, you can start from 1.0 and increase or decrease in an exponential manner (base 2 or 0.5). You can use the learning curve for early stop. But there are the following principles: SGD, Momentum can use a relatively large learning rate, AdaGrad , AdaDelta, Adam is more sensitive to learning rate, generally speaking less than SGD, Momentum, can start from half of SGD, Momentum learning rate + +About Decay, if epoch is less, decay should not be too large. Generally use standard decay, AdaGrad, AdaDelta, Adam use WarnRestarts + +About regularization. Currently FTRL, SGD, AdaGrad, AdaDelta support L1/L2 regularity, Momentum, Adam only supports L2 regularity. It is recommended to start from no regularization, then add regularization. + + +**For the derivation of adding L1 regularization, please refer to**[optimizer](./optimizer.pdf) diff --git a/docs/basic/transfunc_on_angel _en.md b/docs/basic/transfunc_on_angel _en.md new file mode 100644 index 000000000..0595c6545 --- /dev/null +++ b/docs/basic/transfunc_on_angel _en.md @@ -0,0 +1,35 @@ +# Transfer function in Angel + +## 1. Transfer function in Angel: +name| expression| description +---|---|--- +Sigmoid | ![](http://latex.codecogs.com/png.latex?\frac{1}{1+e^{-x}}) | turn any real number into probability +tanh | ![](http://latex.codecogs.com/png.latex?\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}) | turn any real number into a number between -1 and 1 +relu | ![](http://latex.codecogs.com/png.latex?\max(0,x)) | discard the negative part +dropout | ![](http://latex.codecogs.com/png.latex?rand()%20<%20\eta) | randomly set to 0 +identity| ![](http://latex.codecogs.com/png.latex?x) | original output +The following figure shows some common transfer functions (some not available in Angels): +![trans_func](../img/active_funcs.png) + +## 2. The json representation of the transfer function +### 2.1 transfer function without parameters +```json +"transfunc": "sigmoid", + +"transfunc": { + "type": "tanh" +} +``` + +### 2.2 transfer function with parameters +```json +"transfunc": "dropout", + +"transfunc": { + "type": "dropout", + "proba": 0.5, + "actiontype": "train" +} +``` + +ps: Since the dropout transfer function in training and testing (prediction) is not calculated in the same way, it is necessary to use actiontype to indicate the type of scene, train/inctrain, predict. \ No newline at end of file From 4fd89f07f4329c96f03ac0bed9f576adfe793868 Mon Sep 17 00:00:00 2001 From: paynie Date: Fri, 16 Aug 2019 17:05:40 +0800 Subject: [PATCH 07/20] Add copyright --- .../com/tencent/angel/common/Serialize2.java | 16 ++++++++++++++++ .../deploy/config/AngelPod.scala | 16 ++++++++++++++++ .../deploy/config/Constants.scala | 16 ++++++++++++++++ .../deploy/config/KubernetesConf.scala | 16 ++++++++++++++++ .../deploy/config/KubernetesMasterSpec.scala | 16 ++++++++++++++++ .../deploy/config/KubernetesVolumeSpec.scala | 16 ++++++++++++++++ .../features/BasicAngelExecutorFeatureStep.scala | 16 ++++++++++++++++ .../features/BasicAngelMasterFeatureStep.scala | 16 ++++++++++++++++ .../deploy/features/EnvSecretsFeatureStep.scala | 16 ++++++++++++++++ .../deploy/features/JavaMasterFeatureStep.scala | 16 ++++++++++++++++ .../features/KubernetesFeatureConfigStep.scala | 16 ++++++++++++++++ .../deploy/features/LocalDirsFeatureStep.scala | 16 ++++++++++++++++ .../features/MountSecretsFeatureStep.scala | 16 ++++++++++++++++ .../features/MountVolumesFeatureStep.scala | 16 ++++++++++++++++ .../AngelKubernetesClientApplication.scala | 16 ++++++++++++++++ .../submit/KubernetesAngelMasterBuilder.scala | 16 ++++++++++++++++ .../deploy/submit/LoggingPodStatusWatcher.scala | 16 ++++++++++++++++ .../utils/AngelKubernetesClientFactory.scala | 16 ++++++++++++++++ .../deploy/utils/KubernetesUtils.scala | 16 ++++++++++++++++ .../deploy/utils/ThreadUtils.scala | 16 ++++++++++++++++ .../scheduler/AngelExecutorPodStates.scala | 16 ++++++++++++++++ .../scheduler/AngelExecutorPodsAllocator.scala | 16 ++++++++++++++++ .../AngelExecutorPodsLifecycleManager.scala | 16 ++++++++++++++++ .../AngelExecutorPodsPollingSnapshotSource.scala | 16 ++++++++++++++++ .../scheduler/AngelExecutorPodsSnapshot.scala | 16 ++++++++++++++++ .../AngelExecutorPodsSnapshotsStore.scala | 16 ++++++++++++++++ .../AngelExecutorPodsSnapshotsStoreImpl.scala | 16 ++++++++++++++++ .../AngelExecutorPodsWatchSnapshotSource.scala | 16 ++++++++++++++++ .../KubernetesAngelExecutorBuilder.scala | 16 ++++++++++++++++ .../scheduler/KubernetesClusterManager.scala | 16 ++++++++++++++++ .../KubernetesClusterSchedulerBackend.scala | 16 ++++++++++++++++ .../master/AngelKubernetesApplicationMaster.java | 16 ++++++++++++++++ .../master/matrix/committer/SaveResult.java | 16 ++++++++++++++++ .../com/tencent/angel/model/ModelIOUtils.java | 16 ++++++++++++++++ .../angel/model/io/SnapshotLoaderSaver.java | 16 ++++++++++++++++ .../model/output/format/BinaryCSRFormat.java | 16 ++++++++++++++++ .../model/output/format/IndexAndElement.java | 16 ++++++++++++++++ .../model/output/format/RowBasedFormat.java | 16 ++++++++++++++++ .../angel/ps/KubernetesParameterServerApp.java | 16 ++++++++++++++++ .../tencent/angel/ps/io/PSModelIOExecutor.java | 16 ++++++++++++++++ .../server/data/request/CheckpointPSRequest.java | 16 ++++++++++++++++ .../ps/server/data/request/GetStateRequest.java | 16 ++++++++++++++++ .../angel/ps/server/data/request/PSRequest.java | 16 ++++++++++++++++ .../data/response/CheckpointPSResponse.java | 16 ++++++++++++++++ .../server/data/response/GetStateResponse.java | 16 ++++++++++++++++ .../angel/ps/storage/partition/CSRPartition.java | 16 ++++++++++++++++ .../ps/storage/partition/IServerPartition.java | 16 ++++++++++++++++ .../ps/storage/partition/op/ICSRStorageOp.java | 16 ++++++++++++++++ .../partition/storage/LongCSRStorage.java | 16 ++++++++++++++++ .../vector/func/DoubleElemUpdateFunc.java | 16 ++++++++++++++++ .../storage/vector/func/FloatElemUpdateFunc.java | 16 ++++++++++++++++ .../storage/vector/func/IntElemUpdateFunc.java | 16 ++++++++++++++++ .../storage/vector/func/LongElemUpdateFunc.java | 16 ++++++++++++++++ .../transport/adapter/CheckpointCache.java | 16 ++++++++++++++++ .../transport/adapter/CheckpointRequest.java | 16 ++++++++++++++++ .../java/com/tencent/angel/utils/ArrayUtils.java | 16 ++++++++++++++++ .../angel/worker/KubernetesWorkerApp.java | 16 ++++++++++++++++ .../matrix/CooMatrixTest/CooMatrixTest.scala | 16 ++++++++++++++++ .../matrix/CsrMatrixTest/CsrMatrixTest.scala | 16 ++++++++++++++++ .../java/com/tencent/angel/psagent/CSRTest.java | 16 ++++++++++++++++ .../tencent/angel/psagent/CheckpointTest.java | 16 ++++++++++++++++ .../angel/example/ml/RobustRegLocalExample.java | 16 ++++++++++++++++ .../angel/example/ml/SoftmaxRegLocalExample.java | 16 ++++++++++++++++ .../angel/example/psf/PytorchPSFSubmit.java | 16 ++++++++++++++++ .../angel/example/psf/PytorchPSFTask.java | 16 ++++++++++++++++ .../graph/client/getnodefeats/GetNodeFeats.java | 16 ++++++++++++++++ .../client/getnodefeats/GetNodeFeatsParam.java | 16 ++++++++++++++++ .../client/getnodefeats/GetNodeFeatsResult.java | 16 ++++++++++++++++ .../getnodefeats/PartGetNodeFeatsParam.java | 16 ++++++++++++++++ .../getnodefeats/PartGetNodeFeatsResult.java | 16 ++++++++++++++++ .../graph/client/getnodefeats2/GetNodeFeats.java | 16 ++++++++++++++++ .../client/getnodefeats2/GetNodeFeatsParam.java | 16 ++++++++++++++++ .../client/getnodefeats2/GetNodeFeatsResult.java | 16 ++++++++++++++++ .../getnodefeats2/PartGetNodeFeatsParam.java | 16 ++++++++++++++++ .../getnodefeats2/PartGetNodeFeatsResult.java | 16 ++++++++++++++++ .../graph/client/initNeighbor4/InitNeighbor.java | 16 ++++++++++++++++ .../client/initNeighbor4/InitNeighborParam.java | 16 ++++++++++++++++ .../initNeighbor4/PartInitNeighborParam.java | 16 ++++++++++++++++ .../client/initnodefeats/InitNodeFeats.java | 16 ++++++++++++++++ .../client/initnodefeats/InitNodeFeatsParam.java | 16 ++++++++++++++++ .../initnodefeats/PartInitNodeFeatsParam.java | 16 ++++++++++++++++ .../client/initnodefeats2/InitNodeFeats.java | 16 ++++++++++++++++ .../initnodefeats2/InitNodeFeatsParam.java | 16 ++++++++++++++++ .../initnodefeats2/PartInitNodeFeatsParam.java | 16 ++++++++++++++++ .../client/initnodefeats3/InitNodeFeats.java | 16 ++++++++++++++++ .../initnodefeats3/InitNodeFeatsParam.java | 16 ++++++++++++++++ .../initnodefeats3/InitNodeFeatsPartParam.java | 16 ++++++++++++++++ .../sampleFeats/PartSampleNodeFeatsParam.java | 16 ++++++++++++++++ .../client/sampleFeats/SampleNodeFeats.java | 16 ++++++++++++++++ .../client/sampleFeats/SampleNodeFeatsParam.java | 16 ++++++++++++++++ .../sampleFeats/SampleNodeFeatsResult.java | 16 ++++++++++++++++ .../com/tencent/angel/graph/data/ArrayTest.scala | 16 ++++++++++++++++ .../com/tencent/angel/graph/data/NodeUtils.java | 16 ++++++++++++++++ .../angel/graph/utils/IntIndexComparator.java | 16 ++++++++++++++++ .../utils/LongArrayListIndexComparator.java | 16 ++++++++++++++++ .../angel/graph/utils/LongIndexComparator.java | 16 ++++++++++++++++ .../tencent/angel/graph/GetNodeFeatsTest2.java | 16 ++++++++++++++++ .../tencent/angel/graph/InitNeighborTest.java | 16 ++++++++++++++++ .../tencent/angel/graph/InitNeighborTest2.java | 16 ++++++++++++++++ .../angel/ml/psf/optimizer/KmeansUpdateFunc.java | 16 ++++++++++++++++ .../angel/ml/clustering/kmeans/Kmeans.scala | 16 ++++++++++++++++ .../angel/ml/core/AngelMasterContext.scala | 16 ++++++++++++++++ .../angel/ml/core/PSOptimizerProvider.scala | 16 ++++++++++++++++ .../angel/ml/core/PSVariableManager.scala | 16 ++++++++++++++++ .../angel/ml/core/PSVariableProvider.scala | 16 ++++++++++++++++ .../tencent/angel/ml/core/conf/AngelMLConf.scala | 16 ++++++++++++++++ .../ml/core/optimizer/KmeansOptimizer.scala | 16 ++++++++++++++++ .../tencent/angel/ml/core/utils/PSContext.scala | 16 ++++++++++++++++ .../angel/ml/core/utils/SConfHelper.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/AngelCILSImpl.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/CILSImpl.scala | 16 ++++++++++++++++ .../ml/core/variable/PSBlasMatVariable.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/PSEmbedVariable.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/PSMatVariable.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/PSVariable.scala | 16 ++++++++++++++++ .../angel/ml/core/variable/PSVecVariable.scala | 16 ++++++++++++++++ .../tencent/angel/ml/matrix/psf/aggr/Size.scala | 16 ++++++++++++++++ .../ml/matrix/psf/update/XavierNormal.scala | 16 ++++++++++++++++ .../ml/matrix/psf/update/XavierUniform.scala | 16 ++++++++++++++++ 119 files changed, 1904 insertions(+) diff --git a/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java b/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java index e3d92eb62..ee439149b 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/common/Serialize2.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.common; import java.io.DataInputStream; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala index b20d5f220..047a9d3d1 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/AngelPod.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.config import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, Pod, PodBuilder} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala index 264e8adbb..7ac683466 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/Constants.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.config private[angel] object Constants { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala index f4f50dbcf..e0e6e4bf0 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesConf.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.config import com.tencent.angel.conf.AngelConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala index d0bd6acee..4a652c588 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesMasterSpec.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.config import io.fabric8.kubernetes.api.model.HasMetadata diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala index 65929934e..b9e3bcef8 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/config/KubernetesVolumeSpec.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.config private[angel] sealed trait KubernetesVolumeSpecificConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala index b2b734882..d4ba871e4 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelExecutorFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.conf.AngelConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala index 555358056..a69cefe96 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/BasicAngelMasterFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.conf.AngelConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala index 501025a20..16da46622 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/EnvSecretsFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.kubernetesmanager.deploy.config.{AngelPod, KubernetesConf, KubernetesRoleSpecificConf} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala index 21427ce45..3d6daefd2 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/JavaMasterFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.kubernetesmanager.deploy.config.{AngelPod, KubernetesConf, KubernetesMasterSpecificConf} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/KubernetesFeatureConfigStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/KubernetesFeatureConfigStep.scala index b6f251d48..5bbb0fd71 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/KubernetesFeatureConfigStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/KubernetesFeatureConfigStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.kubernetesmanager.deploy.config.AngelPod diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/LocalDirsFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/LocalDirsFeatureStep.scala index 8afc57efd..b835671e2 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/LocalDirsFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/LocalDirsFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import java.util.UUID diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountSecretsFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountSecretsFeatureStep.scala index 0faabc797..4b1c2c39a 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountSecretsFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountSecretsFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.kubernetesmanager.deploy.config.{AngelPod, KubernetesConf, KubernetesRoleSpecificConf} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountVolumesFeatureStep.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountVolumesFeatureStep.scala index 2337f09ce..dedae7be4 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountVolumesFeatureStep.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/features/MountVolumesFeatureStep.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.features import com.tencent.angel.kubernetesmanager.deploy.config._ diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/AngelKubernetesClientApplication.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/AngelKubernetesClientApplication.scala index 6c86446de..e3b6ebf9b 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/AngelKubernetesClientApplication.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/AngelKubernetesClientApplication.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.submit import java.io.{Closeable, StringWriter} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/KubernetesAngelMasterBuilder.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/KubernetesAngelMasterBuilder.scala index 1f8c2a15d..21d583b59 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/KubernetesAngelMasterBuilder.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/KubernetesAngelMasterBuilder.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.submit import com.tencent.angel.kubernetesmanager.deploy.config.{KubernetesConf, KubernetesMasterSpec, KubernetesMasterSpecificConf, KubernetesRoleSpecificConf} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/LoggingPodStatusWatcher.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/LoggingPodStatusWatcher.scala index d2d893a8e..00b59d91d 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/LoggingPodStatusWatcher.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/submit/LoggingPodStatusWatcher.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.submit import java.util.concurrent.{CountDownLatch, TimeUnit} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/AngelKubernetesClientFactory.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/AngelKubernetesClientFactory.scala index a840cd6ed..9228e10e2 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/AngelKubernetesClientFactory.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/AngelKubernetesClientFactory.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.utils import java.io.File diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/KubernetesUtils.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/KubernetesUtils.scala index f8ffe5734..55e9e6dd8 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/KubernetesUtils.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/KubernetesUtils.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.utils import java.util.NoSuchElementException diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/ThreadUtils.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/ThreadUtils.scala index 984e04b57..ec9594197 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/ThreadUtils.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/deploy/utils/ThreadUtils.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.deploy.utils import java.util.concurrent._ diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodStates.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodStates.scala index 9e475f94a..1b0e3eb0f 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodStates.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodStates.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import io.fabric8.kubernetes.api.model.Pod diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsAllocator.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsAllocator.scala index 22bf7aba7..652e19382 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsAllocator.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsAllocator.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import java.util.concurrent.atomic.{AtomicInteger, AtomicLong} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsLifecycleManager.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsLifecycleManager.scala index d3911a109..501270c7f 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsLifecycleManager.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsLifecycleManager.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import com.tencent.angel.conf.AngelConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsPollingSnapshotSource.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsPollingSnapshotSource.scala index b962f9d02..ac24a3b51 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsPollingSnapshotSource.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsPollingSnapshotSource.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import java.util.concurrent.{Future, ScheduledExecutorService, TimeUnit} diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshot.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshot.scala index 2b1b9da3e..82a05feb1 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshot.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshot.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import com.tencent.angel.kubernetesmanager.deploy.config.Constants diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStore.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStore.scala index ba635464f..e46f33770 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStore.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStore.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import io.fabric8.kubernetes.api.model.Pod diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStoreImpl.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStoreImpl.scala index ed5b9cabb..cb3e25710 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStoreImpl.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsSnapshotsStoreImpl.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import java.util.concurrent._ diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsWatchSnapshotSource.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsWatchSnapshotSource.scala index fc4ff8044..192231358 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsWatchSnapshotSource.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/AngelExecutorPodsWatchSnapshotSource.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import java.io.Closeable diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesAngelExecutorBuilder.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesAngelExecutorBuilder.scala index 853fdd9d0..c91a41684 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesAngelExecutorBuilder.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesAngelExecutorBuilder.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import com.tencent.angel.kubernetesmanager.deploy.config._ diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterManager.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterManager.scala index 4581952aa..45af5014e 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterManager.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterManager.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import com.tencent.angel.conf.AngelConf diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterSchedulerBackend.scala b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterSchedulerBackend.scala index 4ebbad6f1..4d8e3d0bb 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterSchedulerBackend.scala +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scheduler/KubernetesClusterSchedulerBackend.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.kubernetesmanager.scheduler import java.util.concurrent.ExecutorService diff --git a/angel-ps/core/src/main/java/com/tencent/angel/master/AngelKubernetesApplicationMaster.java b/angel-ps/core/src/main/java/com/tencent/angel/master/AngelKubernetesApplicationMaster.java index 1ea1b190a..ce2bf1a04 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/master/AngelKubernetesApplicationMaster.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/master/AngelKubernetesApplicationMaster.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.master; import com.tencent.angel.AngelDeployMode; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/master/matrix/committer/SaveResult.java b/angel-ps/core/src/main/java/com/tencent/angel/master/matrix/committer/SaveResult.java index 7a3c6a760..be8f7c9b2 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/master/matrix/committer/SaveResult.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/master/matrix/committer/SaveResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.master.matrix.committer; public class SaveResult { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/model/ModelIOUtils.java b/angel-ps/core/src/main/java/com/tencent/angel/model/ModelIOUtils.java index 7a6301249..f81dca5f1 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/model/ModelIOUtils.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/model/ModelIOUtils.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.model; import com.tencent.angel.ps.storage.partition.RowBasedPartition; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/model/io/SnapshotLoaderSaver.java b/angel-ps/core/src/main/java/com/tencent/angel/model/io/SnapshotLoaderSaver.java index a775f42e0..6be118aab 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/model/io/SnapshotLoaderSaver.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/model/io/SnapshotLoaderSaver.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.model.io; import com.tencent.angel.ml.math2.vector.IntDoubleVector; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/BinaryCSRFormat.java b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/BinaryCSRFormat.java index 55a85e874..9a5125439 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/BinaryCSRFormat.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/BinaryCSRFormat.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.model.output.format; import com.tencent.angel.model.PSMatrixLoadContext; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/IndexAndElement.java b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/IndexAndElement.java index 135881d8f..8e8d31e9e 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/IndexAndElement.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/IndexAndElement.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.model.output.format; import com.tencent.angel.ps.storage.vector.element.IElement; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/RowBasedFormat.java b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/RowBasedFormat.java index c9edd06c1..313065c36 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/RowBasedFormat.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/model/output/format/RowBasedFormat.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.model.output.format; import com.tencent.angel.model.PSMatrixLoadContext; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/KubernetesParameterServerApp.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/KubernetesParameterServerApp.java index f5e132130..4fc681cf7 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/KubernetesParameterServerApp.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/KubernetesParameterServerApp.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps; import com.tencent.angel.RunningMode; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/io/PSModelIOExecutor.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/io/PSModelIOExecutor.java index a1383a2c0..d1068a9e4 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/io/PSModelIOExecutor.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/io/PSModelIOExecutor.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.io; import com.tencent.angel.conf.AngelConf; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/CheckpointPSRequest.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/CheckpointPSRequest.java index 98eb0a374..c317ba625 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/CheckpointPSRequest.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/CheckpointPSRequest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.server.data.request; import com.tencent.angel.ps.ParameterServerId; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/GetStateRequest.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/GetStateRequest.java index 8241c1f90..fcd0ebdbc 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/GetStateRequest.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/GetStateRequest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.server.data.request; import com.tencent.angel.ps.ParameterServerId; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/PSRequest.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/PSRequest.java index 988c8a03a..56af847e4 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/PSRequest.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/request/PSRequest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.server.data.request; import com.tencent.angel.ps.ParameterServerId; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/CheckpointPSResponse.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/CheckpointPSResponse.java index 9769921ec..f4791560a 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/CheckpointPSResponse.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/CheckpointPSResponse.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.server.data.response; public class CheckpointPSResponse extends Response { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/GetStateResponse.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/GetStateResponse.java index bcff00883..4caa2c158 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/GetStateResponse.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/server/data/response/GetStateResponse.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.server.data.response; public class GetStateResponse extends Response { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/CSRPartition.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/CSRPartition.java index 1e62eb22a..d9b923931 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/CSRPartition.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/CSRPartition.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.partition; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/IServerPartition.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/IServerPartition.java index c84bdd6ed..dabe3faca 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/IServerPartition.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/IServerPartition.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.partition; import com.tencent.angel.ps.storage.partition.op.IGeneralServerPartitionOp; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/op/ICSRStorageOp.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/op/ICSRStorageOp.java index 95438d395..8871234df 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/op/ICSRStorageOp.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/op/ICSRStorageOp.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.partition.op; public interface ICSRStorageOp { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/storage/LongCSRStorage.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/storage/LongCSRStorage.java index 5ea86424e..8feaf4a82 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/storage/LongCSRStorage.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/partition/storage/LongCSRStorage.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.partition.storage; public class LongCSRStorage { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/DoubleElemUpdateFunc.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/DoubleElemUpdateFunc.java index 9386fc4c3..95e14c668 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/DoubleElemUpdateFunc.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/DoubleElemUpdateFunc.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.vector.func; public interface DoubleElemUpdateFunc { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/FloatElemUpdateFunc.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/FloatElemUpdateFunc.java index 0170e65d9..94b61d1b2 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/FloatElemUpdateFunc.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/FloatElemUpdateFunc.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.vector.func; public interface FloatElemUpdateFunc { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/IntElemUpdateFunc.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/IntElemUpdateFunc.java index c406d3c6e..7d9802578 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/IntElemUpdateFunc.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/IntElemUpdateFunc.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.vector.func; public interface IntElemUpdateFunc { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/LongElemUpdateFunc.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/LongElemUpdateFunc.java index 40085398f..e67f64c8e 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/LongElemUpdateFunc.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/func/LongElemUpdateFunc.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ps.storage.vector.func; public interface LongElemUpdateFunc { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointCache.java b/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointCache.java index acea5a0bd..0132255fa 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointCache.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointCache.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.psagent.matrix.transport.adapter; import com.tencent.angel.ml.matrix.psf.update.base.VoidResult; diff --git a/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointRequest.java b/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointRequest.java index d84ff0a9e..0f539f395 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointRequest.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/psagent/matrix/transport/adapter/CheckpointRequest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.psagent.matrix.transport.adapter; public class CheckpointRequest extends UserRequest { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/utils/ArrayUtils.java b/angel-ps/core/src/main/java/com/tencent/angel/utils/ArrayUtils.java index 27ee9bf20..f51ed1182 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/utils/ArrayUtils.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/utils/ArrayUtils.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.utils; public class ArrayUtils { diff --git a/angel-ps/core/src/main/java/com/tencent/angel/worker/KubernetesWorkerApp.java b/angel-ps/core/src/main/java/com/tencent/angel/worker/KubernetesWorkerApp.java index 6e639beda..58b616fb4 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/worker/KubernetesWorkerApp.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/worker/KubernetesWorkerApp.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.worker; import com.tencent.angel.common.location.Location; diff --git a/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CooMatrixTest/CooMatrixTest.scala b/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CooMatrixTest/CooMatrixTest.scala index dd8e61e57..144daacd5 100644 --- a/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CooMatrixTest/CooMatrixTest.scala +++ b/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CooMatrixTest/CooMatrixTest.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.math2.matrix.CooMatrixTest import com.tencent.angel.ml.math2.MFactory diff --git a/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CsrMatrixTest/CsrMatrixTest.scala b/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CsrMatrixTest/CsrMatrixTest.scala index ed479075e..84f8f1c44 100644 --- a/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CsrMatrixTest/CsrMatrixTest.scala +++ b/angel-ps/core/src/test/java/com/tencent/angel/ml/math2/matrix/CsrMatrixTest/CsrMatrixTest.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.math2.matrix.CsrMatrixTest import com.tencent.angel.ml.math2.MFactory diff --git a/angel-ps/core/src/test/java/com/tencent/angel/psagent/CSRTest.java b/angel-ps/core/src/test/java/com/tencent/angel/psagent/CSRTest.java index 23249c16c..5debb7605 100644 --- a/angel-ps/core/src/test/java/com/tencent/angel/psagent/CSRTest.java +++ b/angel-ps/core/src/test/java/com/tencent/angel/psagent/CSRTest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.psagent; import com.tencent.angel.client.AngelClient; diff --git a/angel-ps/core/src/test/java/com/tencent/angel/psagent/CheckpointTest.java b/angel-ps/core/src/test/java/com/tencent/angel/psagent/CheckpointTest.java index bc8a8eae7..40e94621a 100644 --- a/angel-ps/core/src/test/java/com/tencent/angel/psagent/CheckpointTest.java +++ b/angel-ps/core/src/test/java/com/tencent/angel/psagent/CheckpointTest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.psagent; import com.tencent.angel.client.AngelClient; diff --git a/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/RobustRegLocalExample.java b/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/RobustRegLocalExample.java index f8094feb5..83c5deec7 100644 --- a/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/RobustRegLocalExample.java +++ b/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/RobustRegLocalExample.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.example.ml; import com.tencent.angel.conf.AngelConf; diff --git a/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/SoftmaxRegLocalExample.java b/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/SoftmaxRegLocalExample.java index 4c5924d65..e1c90b124 100644 --- a/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/SoftmaxRegLocalExample.java +++ b/angel-ps/examples/src/main/java/com/tencent/angel/example/ml/SoftmaxRegLocalExample.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.example.ml; import com.tencent.angel.conf.AngelConf; diff --git a/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFSubmit.java b/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFSubmit.java index 5a9c043a8..479e42146 100644 --- a/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFSubmit.java +++ b/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFSubmit.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.example.psf; import com.tencent.angel.AppSubmitter; diff --git a/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFTask.java b/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFTask.java index c2c91dd9f..87caa21d0 100644 --- a/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFTask.java +++ b/angel-ps/examples/src/main/java/com/tencent/angel/example/psf/PytorchPSFTask.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.example.psf; import com.tencent.angel.exception.AngelException; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeats.java index 3e54eee94..75d6113a9 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsParam.java index 113764d87..297f2313c 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsResult.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsResult.java index 952514cb1..4dfa47548 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsResult.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/GetNodeFeatsResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats; import com.tencent.angel.ml.math2.vector.IntFloatVector; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsParam.java index 1db07bf0f..aa0398dc2 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsResult.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsResult.java index cd0e0b09a..6c8fd062d 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsResult.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats/PartGetNodeFeatsResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats; import com.tencent.angel.graph.data.NodeUtils; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeats.java index fc15f9b05..cc925e9a9 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats2; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsParam.java index a7a91f0d3..61f281f52 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats2; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsResult.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsResult.java index 959356ba0..bb6d601b5 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsResult.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/GetNodeFeatsResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats2; import com.tencent.angel.ml.math2.vector.IntFloatVector; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsParam.java index 1706c6071..7f6a4f6bc 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats2; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsResult.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsResult.java index 15fca5411..e15a70209 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsResult.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/getnodefeats2/PartGetNodeFeatsResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.getnodefeats2; import com.tencent.angel.graph.data.NodeUtils; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighbor.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighbor.java index 98ad304ca..800d19cd9 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighbor.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighbor.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initNeighbor4; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighborParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighborParam.java index 7e3dd11a4..412741757 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighborParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/InitNeighborParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initNeighbor4; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/PartInitNeighborParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/PartInitNeighborParam.java index ebc39974f..cc285b5ef 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/PartInitNeighborParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initNeighbor4/PartInitNeighborParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initNeighbor4; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeats.java index 098eb86b4..037ba63e9 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeatsParam.java index afe10a893..4a9731bed 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/InitNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/PartInitNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/PartInitNodeFeatsParam.java index fee3748d0..2068fd5a2 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/PartInitNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats/PartInitNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeats.java index c20e645f7..83f258191 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats2; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeatsParam.java index 88e7f6619..d4477d078 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/InitNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats2; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/PartInitNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/PartInitNodeFeatsParam.java index 40c07b3c3..bf4725c5c 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/PartInitNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats2/PartInitNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats2; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeats.java index b93837cef..fccc44ac2 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats3; import com.tencent.angel.graph.data.Node; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsParam.java index 379a9c1e6..d11159e4c 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats3; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsPartParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsPartParam.java index 2faeac6cd..3083b1806 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsPartParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/initnodefeats3/InitNodeFeatsPartParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.initnodefeats3; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/PartSampleNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/PartSampleNodeFeatsParam.java index 5d8032d5c..206a9b31b 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/PartSampleNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/PartSampleNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.sampleFeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeats.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeats.java index 7af8fe8b5..f1c55ef95 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeats.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeats.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.sampleFeats; import com.tencent.angel.graph.client.getnodefeats2.PartGetNodeFeatsResult; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsParam.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsParam.java index 54f3a2aa5..70ee81d57 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsParam.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsParam.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.sampleFeats; import com.tencent.angel.PartitionKey; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsResult.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsResult.java index 20f757d38..538fcebad 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsResult.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/client/sampleFeats/SampleNodeFeatsResult.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.client.sampleFeats; import com.tencent.angel.ml.math2.vector.IntFloatVector; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/ArrayTest.scala b/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/ArrayTest.scala index 3e6bb004a..bf9e8648c 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/ArrayTest.scala +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/ArrayTest.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.data object ArrayTest { diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/NodeUtils.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/NodeUtils.java index 443bab7be..02877cdb4 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/NodeUtils.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/data/NodeUtils.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.data; import com.tencent.angel.ml.math2.VFactory; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/IntIndexComparator.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/IntIndexComparator.java index 2cce2979f..4e163a6a4 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/IntIndexComparator.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/IntIndexComparator.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.utils; import it.unimi.dsi.fastutil.ints.IntComparator; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongArrayListIndexComparator.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongArrayListIndexComparator.java index 8535b60b1..72a96c682 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongArrayListIndexComparator.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongArrayListIndexComparator.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.utils; import it.unimi.dsi.fastutil.ints.IntComparator; diff --git a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongIndexComparator.java b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongIndexComparator.java index 6cce899fd..64aa14b4c 100644 --- a/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongIndexComparator.java +++ b/angel-ps/graph/src/main/java/com/tencent/angel/graph/utils/LongIndexComparator.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph.utils; import it.unimi.dsi.fastutil.ints.IntComparator; diff --git a/angel-ps/graph/src/test/java/com/tencent/angel/graph/GetNodeFeatsTest2.java b/angel-ps/graph/src/test/java/com/tencent/angel/graph/GetNodeFeatsTest2.java index f4cf95490..68cc71919 100644 --- a/angel-ps/graph/src/test/java/com/tencent/angel/graph/GetNodeFeatsTest2.java +++ b/angel-ps/graph/src/test/java/com/tencent/angel/graph/GetNodeFeatsTest2.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph; import com.tencent.angel.client.AngelClient; diff --git a/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest.java b/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest.java index 906a77890..c5adb1f33 100644 --- a/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest.java +++ b/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph; import com.tencent.angel.client.AngelClient; diff --git a/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest2.java b/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest2.java index 379c6b282..adf93c432 100644 --- a/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest2.java +++ b/angel-ps/graph/src/test/java/com/tencent/angel/graph/InitNeighborTest2.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.graph; import com.tencent.angel.client.AngelClient; diff --git a/angel-ps/mllib/src/main/java/com/tencent/angel/ml/psf/optimizer/KmeansUpdateFunc.java b/angel-ps/mllib/src/main/java/com/tencent/angel/ml/psf/optimizer/KmeansUpdateFunc.java index 648508a68..794ed2843 100644 --- a/angel-ps/mllib/src/main/java/com/tencent/angel/ml/psf/optimizer/KmeansUpdateFunc.java +++ b/angel-ps/mllib/src/main/java/com/tencent/angel/ml/psf/optimizer/KmeansUpdateFunc.java @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.psf.optimizer; import com.tencent.angel.ml.math2.vector.Vector; diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/clustering/kmeans/Kmeans.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/clustering/kmeans/Kmeans.scala index 40989cf04..eaa8dfe4b 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/clustering/kmeans/Kmeans.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/clustering/kmeans/Kmeans.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.clustering.kmeans import com.tencent.angel.ml.core.PSOptimizerProvider diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/AngelMasterContext.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/AngelMasterContext.scala index 1591f73a8..6d8feab43 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/AngelMasterContext.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/AngelMasterContext.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core import com.tencent.angel.client.AngelClient diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSOptimizerProvider.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSOptimizerProvider.scala index 7de5fb77e..dd37bbcb9 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSOptimizerProvider.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSOptimizerProvider.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core import com.tencent.angel.mlcore.conf.{MLCoreConf, SharedConf} diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableManager.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableManager.scala index d0d712323..95eb7edb2 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableManager.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableManager.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core import com.tencent.angel.client.AngelClient diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableProvider.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableProvider.scala index b1c57633b..f3e503f5b 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableProvider.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/PSVariableProvider.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core import com.tencent.angel.mlcore.conf.SharedConf diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/conf/AngelMLConf.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/conf/AngelMLConf.scala index 0a57e8245..915bab2a5 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/conf/AngelMLConf.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/conf/AngelMLConf.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.conf import com.tencent.angel.ml.math2.utils.RowType diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/optimizer/KmeansOptimizer.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/optimizer/KmeansOptimizer.scala index e44a3b32f..f4d815263 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/optimizer/KmeansOptimizer.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/optimizer/KmeansOptimizer.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.optimizer import java.util.concurrent.Future diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/PSContext.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/PSContext.scala index 1f85854ab..b7145e804 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/PSContext.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/PSContext.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.utils import com.tencent.angel.ml.matrix.MatrixContext diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/SConfHelper.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/SConfHelper.scala index 5687dd588..ce008a677 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/SConfHelper.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/utils/SConfHelper.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.utils import java.io.File diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/AngelCILSImpl.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/AngelCILSImpl.scala index 106ff7d9a..e2947d950 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/AngelCILSImpl.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/AngelCILSImpl.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable import java.util diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/CILSImpl.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/CILSImpl.scala index 9465498df..dd0942b39 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/CILSImpl.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/CILSImpl.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable import com.tencent.angel.ml.matrix.MatrixContext diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSBlasMatVariable.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSBlasMatVariable.scala index a0096291f..6bf51700e 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSBlasMatVariable.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSBlasMatVariable.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable import com.tencent.angel.ml.matrix.MatrixContext diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSEmbedVariable.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSEmbedVariable.scala index 654ec916e..16fc00a1b 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSEmbedVariable.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSEmbedVariable.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable import java.lang.{Long => JLong} diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSMatVariable.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSMatVariable.scala index a96fa044c..992a1aefd 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSMatVariable.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSMatVariable.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable import com.tencent.angel.ml.matrix.MatrixContext diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVariable.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVariable.scala index 04e6dc2ce..f55f7d095 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVariable.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVariable.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable diff --git a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVecVariable.scala b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVecVariable.scala index 02fef6c34..86e23a86e 100644 --- a/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVecVariable.scala +++ b/angel-ps/mllib/src/main/scala/com/tencent/angel/ml/core/variable/PSVecVariable.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.core.variable diff --git a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/aggr/Size.scala b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/aggr/Size.scala index 4d56245ac..42898f2e4 100644 --- a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/aggr/Size.scala +++ b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/aggr/Size.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.matrix.psf.aggr import com.tencent.angel.ml.math2.vector.{IntKeyVector, LongKeyVector} diff --git a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierNormal.scala b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierNormal.scala index 44f9cda67..4b9d9dc37 100644 --- a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierNormal.scala +++ b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierNormal.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.matrix.psf.update import com.tencent.angel.ml.matrix.psf.update.enhance.MMUpdateParam diff --git a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierUniform.scala b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierUniform.scala index 9a60c4e1a..1fe6eef42 100644 --- a/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierUniform.scala +++ b/angel-ps/psf/src/main/java/com/tencent/angel/ml/matrix/psf/update/XavierUniform.scala @@ -1,3 +1,19 @@ +/* + * Tencent is pleased to support the open source community by making Angel available. + * + * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in + * compliance with the License. You may obtain a copy of the License at + * + * https://opensource.org/licenses/Apache-2.0 + * + * Unless required by applicable law or agreed to in writing, software distributed under the License + * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + * or implied. See the License for the specific language governing permissions and limitations under + * the License. + * + */ package com.tencent.angel.ml.matrix.psf.update import java.util.{Random => JRandom} From d5c94a232f4a22829f847cd59864fa4c5d549dcd Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 19:20:26 +0800 Subject: [PATCH 08/20] update run_on_kubernetes_en.md doc and k8s scripts. --- .../kubernetesmanager/scripts/Dockerfile | 9 +- .../kubernetesmanager/scripts/entrypoint.sh | 112 +++++------------- docs/deploy/run_on_kubernetes_en.md | 10 +- 3 files changed, 38 insertions(+), 93 deletions(-) diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile index 21d1e14c7..41d828f18 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile @@ -1,6 +1,5 @@ -FROM spark:testing -RUN rm /opt/entrypoint.sh -COPY entrypoint.sh /opt -COPY angel /opt/angel -COPY lib /usr/lib +FROM tencentangel/angel:v3.0.0 + +#User-defined action + ENTRYPOINT ["/opt/entrypoint.sh"] diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh index a1ea2c6d2..90fcc1561 100755 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh @@ -19,119 +19,63 @@ if [ -z "$uidentry" ] ; then fi fi -SPARK_K8S_CMD="$1" -case "$SPARK_K8S_CMD" in - driver | driver-py | driver-r | executor | master | ps | worker) +ANGEL_ON_K8S_CMD="$1" +case "$ANGEL_ON_K8S_CMD" in + master | ps | worker) shift 1 ;; "") ;; *) - echo "Non-spark-on-k8s command provided, proceeding in pass-through mode..." - exec /sbin/tini -s -- "$@" + echo "Non-angel-on-k8s command provided, proceeding in pass-through mode..." + exec "$@" ;; esac ANGEL_HOME=/opt/angel -ANGEL_CLASSPATH="$ANGEL_CLASSPATH:${ANGEL_HOME}/lib/*" -SPARK_CLASSPATH="$SPARK_CLASSPATH:${SPARK_HOME}/jars/*" -env | grep SPARK_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt -readarray -t SPARK_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt +ANGEL_CLASSPATH="$ANGEL_CLASSPATH:${ANGEL_HOME}/lib/*:$CLASSPATH" +env | grep ANGEL_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt +readarray -t ANGEL_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt -if [ -n "$SPARK_EXTRA_CLASSPATH" ]; then - SPARK_CLASSPATH="$SPARK_CLASSPATH:$SPARK_EXTRA_CLASSPATH" +if [ -n "$ANGEL_EXTRA_CLASSPATH" ]; then + ANGEL_CLASSPATH="$ANGEL_CLASSPATH:$ANGEL_EXTRA_CLASSPATH" fi -if [ -n "$PYSPARK_FILES" ]; then - PYTHONPATH="$PYTHONPATH:$PYSPARK_FILES" -fi - -PYSPARK_ARGS="" -if [ -n "$PYSPARK_APP_ARGS" ]; then - PYSPARK_ARGS="$PYSPARK_APP_ARGS" -fi - -R_ARGS="" -if [ -n "$R_APP_ARGS" ]; then - R_ARGS="$R_APP_ARGS" -fi - -if [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "2" ]; then - pyv="$(python -V 2>&1)" - export PYTHON_VERSION="${pyv:7}" - export PYSPARK_PYTHON="python" - export PYSPARK_DRIVER_PYTHON="python" -elif [ "$PYSPARK_MAJOR_PYTHON_VERSION" == "3" ]; then - pyv3="$(python3 -V 2>&1)" - export PYTHON_VERSION="${pyv3:7}" - export PYSPARK_PYTHON="python3" - export PYSPARK_DRIVER_PYTHON="python3" -fi - -case "$SPARK_K8S_CMD" in - driver) - CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" - ) - ;; - driver-py) +case "$ANGEL_ON_K8S_CMD" in + ps) CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" $PYSPARK_PRIMARY $PYSPARK_ARGS - ) - ;; - driver-r) - CMD=( - "$SPARK_HOME/bin/spark-submit" - --conf "spark.driver.bindAddress=$SPARK_DRIVER_BIND_ADDRESS" - --deploy-mode client - "$@" $R_PRIMARY $R_ARGS + ${JAVA_HOME}/bin/java + "${ANGEL_EXECUTOR_JAVA_OPTS[@]}" + -Xms$ANGEL_EXECUTOR_MEMORY + -Xmx$ANGEL_EXECUTOR_MEMORY + -cp "$ANGEL_CLASSPATH" + com.tencent.angel.ps.KubernetesParameterServerApp ) ;; - executor) + worker) CMD=( ${JAVA_HOME}/bin/java - "${SPARK_EXECUTOR_JAVA_OPTS[@]}" - -Xms$SPARK_EXECUTOR_MEMORY - -Xmx$SPARK_EXECUTOR_MEMORY - -cp "$SPARK_CLASSPATH" - org.apache.spark.executor.CoarseGrainedExecutorBackend - --driver-url $SPARK_DRIVER_URL - --executor-id $SPARK_EXECUTOR_ID - --cores $SPARK_EXECUTOR_CORES - --app-id $SPARK_APPLICATION_ID - --hostname $SPARK_EXECUTOR_POD_IP + "${ANGEL_EXECUTOR_JAVA_OPTS[@]}" + -Xms$ANGEL_EXECUTOR_MEMORY + -Xmx$ANGEL_EXECUTOR_MEMORY + -cp "$ANGEL_CLASSPATH" + com.tencent.angel.worker.KubernetesWorkerApp ) ;; - ps) - CMD=( - ${JAVA_HOME}/bin/java - "${SPARK_EXECUTOR_JAVA_OPTS[@]}" - -Xms$ANGEL_EXECUTOR_MEMORY - -Xmx$ANGEL_EXECUTOR_MEMORY - -cp "$ANGEL_CLASSPATH" - com.tencent.angel.ps.KubernetesParameterServerApp - ) - ;; master) CMD=( ${JAVA_HOME}/bin/java - "${SPARK_EXECUTOR_JAVA_OPTS[@]}" + "${ANGEL_MASTER_JAVA_OPTS[@]}" -Xms$ANGEL_MASTER_MEMORY -Xmx$ANGEL_MASTER_MEMORY -cp "$ANGEL_CLASSPATH" com.tencent.angel.master.AngelKubernetesApplicationMaster ) - ;; + ;; *) - echo "Unknown command: $SPARK_K8S_CMD" 1>&2 + echo "Unknown command: $ANGEL_ON_K8S_CMD" 1>&2 exit 1 esac # Execute the container CMD under tini for better hygiene -exec /sbin/tini -s -- "${CMD[@]}" +exec /sbin/tini -s -- "${CMD[@]}" \ No newline at end of file diff --git a/docs/deploy/run_on_kubernetes_en.md b/docs/deploy/run_on_kubernetes_en.md index f73d54e92..404cadc92 100644 --- a/docs/deploy/run_on_kubernetes_en.md +++ b/docs/deploy/run_on_kubernetes_en.md @@ -22,7 +22,7 @@ The environment requirements for running Angel on Kubernetes include: * Docker images Kubernetes requires users to supply images that can be deployed into containers within pods. The images are built to be run in a container runtime environment that Kubernetes supports. Docker is a container runtime environment that is frequently used with Kubernetes. We provide a basic image for the user, you can pull it down and run or customize your own image through it. Follow the pull command: - `docker pull 202102/angel:v3.0.0` + `docker pull tencentangel/angel:v3.0.0` * Distributed File System we both support cephfs and hdfs distributed file system, to ensure that the kubernetes cluster's pods are accessible @@ -52,7 +52,7 @@ The environment requirements for running Angel on Kubernetes include: -Dangel.kubernetes.master= \ -Dangel.app.submit.class=com.tencent.angel.ml.core.graphsubmit.GraphRunner \ -Dangel.action.type=train \ - -Dangel.kubernetes.container.image=202102/angel:v3.0.0 \ + -Dangel.kubernetes.container.image=tencentangel/angel:v3.0.0 \ -Dangel.kubernetes.master.volumes.hostPath.test.mount.path=/angel/mountpath \ -Dangel.kubernetes.master.volumes.hostPath.test.options.path=/cephfs/basepath/to/angel \ -Dangel.kubernetes.executor.volumes.hostPath.test.mount.path=/angel/mountpath \ @@ -144,7 +144,7 @@ The following configurations are specific to Angel on Kubernetes. | --- | --- | --- | | angel.kubernetes.master | none | kubernetes cluster api server url | | angel.kubernetes.namespace | default | The namespace that will be used for running the angel master and angel ps/worker pods. | -| angel.kubernetes.container.image | none | Container image to use for the Angel application. This is usually of the form example.com/repo/angel:v3.0.0. This configuration is required and must be provided by the user, we provide a base image `docker pull 202102/angel:v3.0.0` . | +| angel.kubernetes.container.image | none | Container image to use for the Angel application. This is usually of the form example.com/repo/angel:v3.0.0. This configuration is required and must be provided by the user, we provide a base image `docker pull tencentangel/angel:v3.0.0` . | | angel.kubernetes.container.image.pullPolicy | IfNotPresent | Container image pull policy used when pulling images within Kubernetes, valid values are `Always`, `Never`, and `IfNotPresent`. | | angel.kubernetes.serviceaccount | default | Service account that is used when running the angel master pod. The angel master pod uses this service account when requesting angel ps/worker pods from the API server | | angel.kubernetes.master.label.[LabelName] | none | Add the label specified by LabelName to the angel master pod. | @@ -164,4 +164,6 @@ The following configurations are specific to Angel on Kubernetes. | angel.kubernetes.master.secrets.[SecretName] | none | Add the Kubernetes Secret named SecretName to the angel master pod on the path specified in the value. | | angel.kubernetes.executor.secrets.[SecretName] | none | Add the Kubernetes Secret named SecretName to the angel ps/worker pod on the path specified in the value. | -All angel on kubernetes configuration information can be referred to `AngelConf` class. \ No newline at end of file +All angel on kubernetes configuration information can be referred to `AngelConf` class. + +**Note**: If the you want to make your own image from the base image we provided, the `entrypoint.sh` and `Dockerfile` in source code `kubernetesmanager/scripts`. \ No newline at end of file From 372e52f4355ef963e1448ad4768b704b8b1a7250 Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 19:40:34 +0800 Subject: [PATCH 09/20] update doc. --- .../com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh | 4 ++-- docs/deploy/run_on_kubernetes_en.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh index 90fcc1561..0ae2b25c1 100755 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh @@ -10,10 +10,11 @@ set +e uidentry=$(getent passwd $myuid) set -e +ANGEL_HOME=/opt/angel # If there is no passwd entry for the container UID, attempt to create one if [ -z "$uidentry" ] ; then if [ -w /etc/passwd ] ; then - echo "$myuid:x:$myuid:$mygid:anonymous uid:$SPARK_HOME:/bin/false" >> /etc/passwd + echo "$myuid:x:$myuid:$mygid:anonymous uid:$ANGEL_HOME:/bin/false" >> /etc/passwd else echo "Container ENTRYPOINT failed to add passwd entry for anonymous UID" fi @@ -31,7 +32,6 @@ case "$ANGEL_ON_K8S_CMD" in exec "$@" ;; esac -ANGEL_HOME=/opt/angel ANGEL_CLASSPATH="$ANGEL_CLASSPATH:${ANGEL_HOME}/lib/*:$CLASSPATH" env | grep ANGEL_JAVA_OPT_ | sort -t_ -k4 -n | sed 's/[^=]*=\(.*\)/\1/g' > /tmp/java_opts.txt readarray -t ANGEL_EXECUTOR_JAVA_OPTS < /tmp/java_opts.txt diff --git a/docs/deploy/run_on_kubernetes_en.md b/docs/deploy/run_on_kubernetes_en.md index 404cadc92..baac02f2d 100644 --- a/docs/deploy/run_on_kubernetes_en.md +++ b/docs/deploy/run_on_kubernetes_en.md @@ -51,7 +51,7 @@ The environment requirements for running Angel on Kubernetes include: -Dangel.deploy.mode=KUBERNETES \ -Dangel.kubernetes.master= \ -Dangel.app.submit.class=com.tencent.angel.ml.core.graphsubmit.GraphRunner \ - -Dangel.action.type=train \ + -Daction.type=train \ -Dangel.kubernetes.container.image=tencentangel/angel:v3.0.0 \ -Dangel.kubernetes.master.volumes.hostPath.test.mount.path=/angel/mountpath \ -Dangel.kubernetes.master.volumes.hostPath.test.options.path=/cephfs/basepath/to/angel \ From c724f8de61f147a967ec1390aff7272a96df0bf9 Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 19:43:48 +0800 Subject: [PATCH 10/20] update doc. --- docs/deploy/run_on_kubernetes_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/deploy/run_on_kubernetes_en.md b/docs/deploy/run_on_kubernetes_en.md index baac02f2d..829878e4d 100644 --- a/docs/deploy/run_on_kubernetes_en.md +++ b/docs/deploy/run_on_kubernetes_en.md @@ -166,4 +166,4 @@ The following configurations are specific to Angel on Kubernetes. All angel on kubernetes configuration information can be referred to `AngelConf` class. -**Note**: If the you want to make your own image from the base image we provided, the `entrypoint.sh` and `Dockerfile` in source code `kubernetesmanager/scripts`. \ No newline at end of file +**Note**: If you want to make your own image from the base image we provided, the `entrypoint.sh` and `Dockerfile` sample in source code `kubernetesmanager/scripts`. \ No newline at end of file From ccb1aae02dbae5f566391d2b83de4b65fc91c76d Mon Sep 17 00:00:00 2001 From: paynie Date: Fri, 16 Aug 2019 19:47:07 +0800 Subject: [PATCH 11/20] remove unused files --- dependency | 1770 ---------------------------------------------------- log/log | 1 - 2 files changed, 1771 deletions(-) delete mode 100644 dependency delete mode 100644 log/log diff --git a/dependency b/dependency deleted file mode 100644 index 41908ae6a..000000000 --- a/dependency +++ /dev/null @@ -1,1770 +0,0 @@ -[INFO] Scanning for projects... -[INFO] ------------------------------------------------------------------------ -[INFO] Reactor Build Order: -[INFO] -[INFO] angel [pom] -[INFO] angel-ps [pom] -[INFO] angel-ps-core [jar] -[INFO] angel-ps-psf [jar] -[INFO] angel-ps-tools [jar] -[INFO] angel-ps-mllib [jar] -[INFO] angel-ps-examples [jar] -[INFO] angel-ps-graph [jar] -[INFO] -[INFO] ----------------------< com.tencent.angel:angel >----------------------- -[INFO] Building angel 3.0.0 [1/8] -[INFO] --------------------------------[ pom ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel --- -[INFO] com.tencent.angel:angel:pom:3.0.0 -[INFO] -[INFO] ---------------------< com.tencent.angel:angel-ps >--------------------- -[INFO] Building angel-ps 3.0.0 [2/8] -[INFO] --------------------------------[ pom ]--------------------------------- -Downloading from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-math/0.1.0-SNAPSHOT/maven-metadata.xml -Progress (1): 2.0 kB Downloaded from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-math/0.1.0-SNAPSHOT/maven-metadata.xml (2.0 kB at 241 B/s) -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps --- -[INFO] com.tencent.angel:angel-ps:pom:3.0.0 -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- junit:junit:jar:4.11:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] ------------------< com.tencent.angel:angel-ps-core >------------------- -[INFO] Building angel-ps-core 3.0.0 [3/8] -[INFO] --------------------------------[ jar ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-core --- -[INFO] com.tencent.angel:angel-ps-core:jar:3.0.0 -[INFO] +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.1:compile -[INFO] | \- commons-lang:commons-lang:jar:2.4:compile -[INFO] +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13.1:test -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13.1:test -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:test -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:test -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:test -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:test -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:test -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:test -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:test -[INFO] +- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.13:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.7:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.0:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | \- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- junit:junit:jar:4.11:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] -------------------< com.tencent.angel:angel-ps-psf >------------------- -[INFO] Building angel-ps-psf 3.0.0 [4/8] -[INFO] --------------------------------[ jar ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-psf --- -[INFO] com.tencent.angel:angel-ps-psf:jar:3.0.0 -[INFO] +- com.tencent.angel:angel-ps-core:jar:3.0.0:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- junit:junit:jar:4.4:test -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.1:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] ------------------< com.tencent.angel:angel-ps-tools >------------------ -[INFO] Building angel-ps-tools 3.0.0 [5/8] -[INFO] --------------------------------[ jar ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-tools --- -[INFO] com.tencent.angel:angel-ps-tools:jar:3.0.0 -[INFO] +- com.tencent.angel:angel-ps-core:jar:3.0.0:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- junit:junit:jar:4.11:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] ------------------< com.tencent.angel:angel-ps-mllib >------------------ -[INFO] Building angel-ps-mllib 3.0.0 [6/8] -[INFO] --------------------------------[ jar ]--------------------------------- -Downloading from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-mlcore/0.1.0-SNAPSHOT/maven-metadata.xml -Progress (1): 2.0 kB Downloaded from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-mlcore/0.1.0-SNAPSHOT/maven-metadata.xml (2.0 kB at 3.6 kB/s) -Downloading from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-format/0.1.0-SNAPSHOT/maven-metadata.xml -Progress (1): 1.2 kB Downloaded from snapshots-repo: https://oss.sonatype.org/content/repositories/snapshots/com/tencent/angel/angel-format/0.1.0-SNAPSHOT/maven-metadata.xml (1.2 kB at 2.2 kB/s) -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-mllib --- -[INFO] com.tencent.angel:angel-ps-mllib:jar:3.0.0 -[INFO] +- junit:junit:jar:4.4:test -[INFO] +- com.tencent.angel:angel-ps-core:jar:3.0.0:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- com.tencent.angel:angel-ps-tools:jar:3.0.0:compile -[INFO] +- com.tencent.angel:angel-ps-psf:jar:3.0.0:compile -[INFO] +- com.tencent.angel:angel-mlcore:jar:0.1.0-SNAPSHOT:compile -[INFO] | +- org.json4s:json4s-core_2.11:jar:3.2.11:compile -[INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.6:compile -[INFO] | | \- org.scala-lang:scalap:jar:2.11.0:compile -[INFO] | | \- org.scala-lang:scala-compiler:jar:2.11.0:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.2.11:compile -[INFO] | +- org.json4s:json4s-native_2.11:jar:3.2.11:compile -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] | \- com.tencent.angel:angel-format:jar:0.1.0-SNAPSHOT:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.1:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:compile -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] ----------------< com.tencent.angel:angel-ps-examples >----------------- -[INFO] Building angel-ps-examples 3.0.0 [7/8] -[INFO] --------------------------------[ jar ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-examples --- -[INFO] com.tencent.angel:angel-ps-examples:jar:3.0.0 -[INFO] +- com.tencent.angel:angel-ps-core:jar:3.0.0:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- com.tencent.angel:angel-ps-mllib:jar:3.0.0:compile -[INFO] | +- com.tencent.angel:angel-ps-tools:jar:3.0.0:compile -[INFO] | \- com.tencent.angel:angel-mlcore:jar:0.1.0-SNAPSHOT:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.2.11:compile -[INFO] | +- org.json4s:json4s-native_2.11:jar:3.2.11:compile -[INFO] | \- com.tencent.angel:angel-format:jar:0.1.0-SNAPSHOT:compile -[INFO] +- com.tencent.angel:angel-ps-psf:jar:3.0.0:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- junit:junit:jar:4.11:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] -[INFO] ------------------< com.tencent.angel:angel-ps-graph >------------------ -[INFO] Building angel-ps-graph 3.0.0 [8/8] -[INFO] --------------------------------[ jar ]--------------------------------- -[INFO] -[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ angel-ps-graph --- -[INFO] com.tencent.angel:angel-ps-graph:jar:3.0.0 -[INFO] +- com.tencent.angel:angel-ps-core:jar:3.0.0:compile -[INFO] | +- org.apache.velocity:velocity:jar:1.7:compile -[INFO] | +- com.github.fommil.netlib:all:pom:1.1.2:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_ref-java:jar:1.1:compile -[INFO] | | | \- com.github.fommil:jniloader:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-win-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_ref-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-osx-x86_64:jar:natives:1.1:compile -[INFO] | | | \- com.github.fommil.netlib:native_system-java:jar:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-x86_64:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-i686:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-linux-armhf:jar:natives:1.1:compile -[INFO] | | +- com.github.fommil.netlib:netlib-native_system-win-x86_64:jar:natives:1.1:compile -[INFO] | | \- com.github.fommil.netlib:netlib-native_system-win-i686:jar:natives:1.1:compile -[INFO] | \- io.fabric8:kubernetes-client:jar:3.0.0:compile -[INFO] | +- io.fabric8:kubernetes-model:jar:2.0.0:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.5:compile -[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.8.1:compile -[INFO] | | \- com.squareup.okio:okio:jar:1.13.0:compile -[INFO] | +- com.squareup.okhttp3:logging-interceptor:jar:3.8.1:compile -[INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.7:compile -[INFO] | | \- org.yaml:snakeyaml:jar:1.15:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.7.7:compile -[INFO] | +- io.fabric8:zjsonpatch:jar:0.3.0:compile -[INFO] | \- com.github.mifmif:generex:jar:1.0.1:compile -[INFO] | \- dk.brics.automaton:automaton:jar:1.11-8:compile -[INFO] +- junit:junit:jar:4.4:test -[INFO] +- it.unimi.dsi:fastutil:jar:7.1.0:compile -[INFO] +- com.tencent.angel:angel-math:jar:0.1.0-SNAPSHOT:compile -[INFO] | \- commons-logging:commons-logging:jar:1.1.1:compile -[INFO] +- org.apache.spark:spark-core_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile -[INFO] | | +- org.apache.avro:avro-ipc:jar:1.7.7:compile -[INFO] | | \- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile -[INFO] | +- com.twitter:chill_2.11:jar:0.8.0:compile -[INFO] | +- com.twitter:chill-java:jar:0.8.0:compile -[INFO] | +- org.apache.xbean:xbean-asm5-shaded:jar:4.4:compile -[INFO] | +- org.apache.hadoop:hadoop-client:jar:2.2.0:compile -[INFO] | +- org.apache.spark:spark-launcher_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-network-common_2.11:jar:2.1.0:compile -[INFO] | | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile -[INFO] | +- org.apache.spark:spark-network-shuffle_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-unsafe_2.11:jar:2.1.0:compile -[INFO] | +- net.java.dev.jets3t:jets3t:jar:0.7.1:compile -[INFO] | +- org.apache.curator:curator-recipes:jar:2.4.0:compile -[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile -[INFO] | +- org.apache.commons:commons-lang3:jar:3.5:compile -[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile -[INFO] | +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile -[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile -[INFO] | +- log4j:log4j:jar:1.2.17:compile -[INFO] | +- com.ning:compress-lzf:jar:1.0.3:compile -[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.2.6:compile -[INFO] | +- net.jpountz.lz4:lz4:jar:1.3.0:compile -[INFO] | +- org.roaringbitmap:RoaringBitmap:jar:0.5.11:compile -[INFO] | +- commons-net:commons-net:jar:2.2:compile -[INFO] | +- org.glassfish.jersey.core:jersey-client:jar:2.22.2:compile -[INFO] | | +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile -[INFO] | | +- org.glassfish.hk2:hk2-api:jar:2.4.0-b34:compile -[INFO] | | | +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b34:compile -[INFO] | | | \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b34:compile -[INFO] | | +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b34:compile -[INFO] | | \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b34:compile -[INFO] | | \- org.javassist:javassist:jar:3.18.1-GA:compile -[INFO] | +- org.glassfish.jersey.core:jersey-common:jar:2.22.2:compile -[INFO] | | +- javax.annotation:javax.annotation-api:jar:1.2:compile -[INFO] | | +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.2:compile -[INFO] | | \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile -[INFO] | +- org.glassfish.jersey.core:jersey-server:jar:2.22.2:compile -[INFO] | | +- org.glassfish.jersey.media:jersey-media-jaxb:jar:2.22.2:compile -[INFO] | | \- javax.validation:validation-api:jar:1.1.0.Final:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet:jar:2.22.2:compile -[INFO] | +- org.glassfish.jersey.containers:jersey-container-servlet-core:jar:2.22.2:compile -[INFO] | +- com.clearspring.analytics:stream:jar:2.7.0:compile -[INFO] | +- io.dropwizard.metrics:metrics-core:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-json:jar:3.1.2:compile -[INFO] | +- io.dropwizard.metrics:metrics-graphite:jar:3.1.2:compile -[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile -[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.11:jar:2.6.5:compile -[INFO] | | \- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.5:compile -[INFO] | +- org.apache.ivy:ivy:jar:2.4.0:compile -[INFO] | +- oro:oro:jar:2.0.8:compile -[INFO] | +- net.razorvine:pyrolite:jar:4.13:compile -[INFO] | +- org.apache.spark:spark-tags_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.commons:commons-crypto:jar:1.0.0:compile -[INFO] | \- org.spark-project.spark:unused:jar:1.0.0:compile -[INFO] +- org.apache.spark:spark-mllib_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-streaming_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-sql_2.11:jar:2.1.0:compile -[INFO] | | +- com.univocity:univocity-parsers:jar:2.2.1:compile -[INFO] | | +- org.apache.spark:spark-sketch_2.11:jar:2.1.0:compile -[INFO] | | +- org.apache.spark:spark-catalyst_2.11:jar:2.1.0:compile -[INFO] | | | +- org.codehaus.janino:janino:jar:3.0.0:compile -[INFO] | | | +- org.codehaus.janino:commons-compiler:jar:3.0.0:compile -[INFO] | | | \- org.antlr:antlr4-runtime:jar:4.5.3:compile -[INFO] | | +- org.apache.parquet:parquet-column:jar:1.8.1:compile -[INFO] | | | +- org.apache.parquet:parquet-common:jar:1.8.1:compile -[INFO] | | | \- org.apache.parquet:parquet-encoding:jar:1.8.1:compile -[INFO] | | \- org.apache.parquet:parquet-hadoop:jar:1.8.1:compile -[INFO] | | +- org.apache.parquet:parquet-format:jar:2.3.0-incubating:compile -[INFO] | | \- org.apache.parquet:parquet-jackson:jar:1.8.1:compile -[INFO] | +- org.apache.spark:spark-graphx_2.11:jar:2.1.0:compile -[INFO] | +- org.apache.spark:spark-mllib-local_2.11:jar:2.1.0:compile -[INFO] | \- org.jpmml:pmml-model:jar:1.2.15:compile -[INFO] | \- org.jpmml:pmml-schema:jar:1.2.15:compile -[INFO] +- com.google.guava:guava:jar:18.0:compile -[INFO] +- com.yahoo.datasketches:sketches-core:jar:0.8.1:compile -[INFO] | \- com.yahoo.datasketches:memory:jar:0.8.1:compile -[INFO] +- org.ehcache:sizeof:jar:0.3.0:compile -[INFO] +- org.slf4j:slf4j-api:jar:1.6.2:compile -[INFO] +- com.esotericsoftware:kryo-shaded:jar:4.0.0:compile -[INFO] | +- com.esotericsoftware:minlog:jar:1.3.0:compile -[INFO] | \- org.objenesis:objenesis:jar:2.2:compile -[INFO] +- de.javakaffee:kryo-serializers:jar:0.42:compile -[INFO] | \- com.esotericsoftware:kryo:jar:4.0.0:compile -[INFO] | \- com.esotericsoftware:reflectasm:jar:1.11.3:compile -[INFO] | \- org.ow2.asm:asm:jar:5.0.4:compile -[INFO] +- org.slf4j:slf4j-log4j12:jar:1.6.2:compile -[INFO] +- commons-pool:commons-pool:jar:1.6:compile -[INFO] +- org.json4s:json4s-jackson_2.11:jar:3.5.3:compile -[INFO] | \- org.json4s:json4s-core_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-ast_2.11:jar:3.5.3:compile -[INFO] | +- org.json4s:json4s-scalap_2.11:jar:3.5.3:compile -[INFO] | \- com.thoughtworks.paranamer:paranamer:jar:2.8:compile -[INFO] +- org.apache.commons:commons-math:jar:2.2:compile -[INFO] +- org.mockito:mockito-core:jar:1.10.19:test -[INFO] | \- org.hamcrest:hamcrest-core:jar:1.1:test -[INFO] +- org.powermock:powermock-module-junit4:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-module-junit4-common:jar:1.6.5:test -[INFO] | +- org.powermock:powermock-core:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-reflect:jar:1.6.5:test -[INFO] +- org.powermock:powermock-api-mockito:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-mockito-common:jar:1.6.5:test -[INFO] | \- org.powermock:powermock-api-support:jar:1.6.5:test -[INFO] +- com.google.protobuf:protobuf-java:jar:2.5.0:compile -[INFO] +- org.cloudera.htrace:htrace-core:jar:2.05:compile -[INFO] | \- org.mortbay.jetty:jetty-util:jar:6.1.26:compile -[INFO] +- io.netty:netty-all:jar:4.1.17.Final:compile -[INFO] +- io.netty:netty:jar:3.9.9.Final:compile -[INFO] +- net.agkn:hll:jar:1.6.0:compile -[INFO] +- org.apache.hadoop:hadoop-common:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:2.7.3:compile -[INFO] | | \- jdk.tools:jdk.tools:jar:1.8:system -[INFO] | +- commons-cli:commons-cli:jar:1.2:compile -[INFO] | +- xmlenc:xmlenc:jar:0.52:compile -[INFO] | +- commons-httpclient:commons-httpclient:jar:3.1:compile -[INFO] | +- commons-codec:commons-codec:jar:1.4:compile -[INFO] | +- commons-io:commons-io:jar:2.4:compile -[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile -[INFO] | +- javax.servlet:servlet-api:jar:2.5:compile -[INFO] | +- org.mortbay.jetty:jetty:jar:6.1.26:compile -[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:runtime -[INFO] | +- com.sun.jersey:jersey-core:jar:1.9:compile -[INFO] | +- com.sun.jersey:jersey-json:jar:1.9:compile -[INFO] | | \- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile -[INFO] | +- com.sun.jersey:jersey-server:jar:1.9:compile -[INFO] | | \- asm:asm:jar:3.1:compile -[INFO] | +- commons-lang:commons-lang:jar:2.6:compile -[INFO] | +- commons-configuration:commons-configuration:jar:1.6:compile -[INFO] | | +- commons-digester:commons-digester:jar:1.8:compile -[INFO] | | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile -[INFO] | | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile -[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -[INFO] | +- org.apache.avro:avro:jar:1.7.4:compile -[INFO] | +- com.jcraft:jsch:jar:0.1.42:compile -[INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile -[INFO] | +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile -[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile -[INFO] | \- org.apache.commons:commons-compress:jar:1.4.1:compile -[INFO] | \- org.tukaani:xz:jar:1.0:compile -[INFO] +- org.apache.hadoop:hadoop-auth:jar:2.7.3:compile -[INFO] | +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile -[INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile -[INFO] | +- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile -[INFO] | | +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile -[INFO] | | \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile -[INFO] | \- org.apache.curator:curator-framework:jar:2.7.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-distributedshell:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-applications-unmanaged-am-launcher:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-common:jar:2.7.3:compile -[INFO] | +- javax.xml.bind:jaxb-api:jar:2.2.2:compile -[INFO] | | +- javax.xml.stream:stax-api:jar:1.0-2:compile -[INFO] | | \- javax.activation:activation:jar:1.1:compile -[INFO] | +- com.sun.jersey:jersey-client:jar:1.9:compile -[INFO] | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.13:compile -[INFO] | +- org.codehaus.jackson:jackson-xc:jar:1.9.13:compile -[INFO] | +- com.google.inject.extensions:guice-servlet:jar:3.0:compile -[INFO] | +- com.google.inject:guice:jar:3.0:compile -[INFO] | | +- javax.inject:javax.inject:jar:1:compile -[INFO] | | \- aopalliance:aopalliance:jar:1.0:compile -[INFO] | \- com.sun.jersey.contribs:jersey-guice:jar:1.9:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.3:compile -[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.3:compile -[INFO] | \- org.codehaus.jettison:jettison:jar:1.1:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3:compile -[INFO] | +- org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.3:compile -[INFO] | \- org.apache.zookeeper:zookeeper:test-jar:tests:3.4.6:compile -[INFO] +- org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.7.3:compile -[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:compile -[INFO] | \- xerces:xercesImpl:jar:2.9.1:compile -[INFO] | \- xml-apis:xml-apis:jar:1.3.04:compile -[INFO] +- org.apache.hadoop:hadoop-hdfs-nfs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.3:compile -[INFO] | \- org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-hs-plugins:jar:2.7.3:compile -[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.3:compile -[INFO] +- org.scala-lang:scala-library:jar:2.11.8:compile -[INFO] +- org.scalatest:scalatest_2.11:jar:3.0.3:test -[INFO] | +- org.scalactic:scalactic_2.11:jar:3.0.3:test -[INFO] | +- org.scala-lang:scala-reflect:jar:2.11.8:compile -[INFO] | +- org.scala-lang.modules:scala-xml_2.11:jar:1.0.5:compile -[INFO] | \- org.scala-lang.modules:scala-parser-combinators_2.11:jar:1.0.4:test -[INFO] +- org.scalanlp:breeze_2.11:jar:0.13:compile -[INFO] | +- org.scalanlp:breeze-macros_2.11:jar:0.13:compile -[INFO] | +- com.github.fommil.netlib:core:jar:1.1.2:compile -[INFO] | +- net.sourceforge.f2j:arpack_combined_all:jar:0.1:compile -[INFO] | +- net.sf.opencsv:opencsv:jar:2.3:compile -[INFO] | +- com.github.rwl:jtransforms:jar:2.4.0:compile -[INFO] | +- org.spire-math:spire_2.11:jar:0.13.0:compile -[INFO] | | +- org.spire-math:spire-macros_2.11:jar:0.13.0:compile -[INFO] | | \- org.typelevel:machinist_2.11:jar:0.6.1:compile -[INFO] | \- com.chuusai:shapeless_2.11:jar:2.3.2:compile -[INFO] | \- org.typelevel:macro-compat_2.11:jar:1.1.1:compile -[INFO] +- net.sf.py4j:py4j:jar:0.10.4:compile -[INFO] +- com.google.code.gson:gson:jar:2.8.5:compile -[INFO] \- org.apache.commons:commons-math3:jar:3.6.1:compile -[INFO] ------------------------------------------------------------------------ -[INFO] Reactor Summary for angel 3.0.0: -[INFO] -[INFO] angel .............................................. SUCCESS [ 0.591 s] -[INFO] angel-ps ........................................... SUCCESS [ 10.081 s] -[INFO] angel-ps-core ...................................... SUCCESS [ 0.440 s] -[INFO] angel-ps-psf ....................................... SUCCESS [ 0.272 s] -[INFO] angel-ps-tools ..................................... SUCCESS [ 0.262 s] -[INFO] angel-ps-mllib ..................................... SUCCESS [ 1.393 s] -[INFO] angel-ps-examples .................................. SUCCESS [ 0.258 s] -[INFO] angel-ps-graph ..................................... SUCCESS [ 0.262 s] -[INFO] ------------------------------------------------------------------------ -[INFO] BUILD SUCCESS -[INFO] ------------------------------------------------------------------------ -[INFO] Total time: 13.827 s -[INFO] Finished at: 2019-08-05T14:05:17+08:00 -[INFO] ------------------------------------------------------------------------ diff --git a/log/log b/log/log deleted file mode 100644 index 64c82ad77..000000000 --- a/log/log +++ /dev/null @@ -1 +0,0 @@ -19/01/24 14:21:10 WARN math2.MatrixExecutors : angel.math.matrix.op.parallel.worker.num is not set, just use default worker number:8 From f1649ca2714efa02df7b5c59b73b3eecd16ec644 Mon Sep 17 00:00:00 2001 From: paynie Date: Fri, 16 Aug 2019 20:14:06 +0800 Subject: [PATCH 12/20] change version to 3.0.0 --- angel-ps/bin/angel-local-submit | 2 +- angel-ps/bin/angel-submit | 2 +- angel-ps/bin/angel-submit-with-jarvis | 2 +- angel-ps/core/pom.xml | 2 +- angel-ps/examples/pom.xml | 2 +- angel-ps/graph/pom.xml | 2 +- angel-ps/jarvis/pom.xml | 2 +- angel-ps/mllib/pom.xml | 2 +- angel-ps/pom.xml | 2 +- angel-ps/psf/pom.xml | 2 +- angel-ps/tools/pom.xml | 2 +- dist/pom.xml | 2 +- pom.xml | 2 +- 13 files changed, 13 insertions(+), 13 deletions(-) diff --git a/angel-ps/bin/angel-local-submit b/angel-ps/bin/angel-local-submit index 029e8ef27..24a62070b 100644 --- a/angel-ps/bin/angel-local-submit +++ b/angel-ps/bin/angel-local-submit @@ -24,7 +24,7 @@ bin=`cd "$bin"; pwd` lib="$bin"/../lib export ANGEL_HOME="$bin/../" -export ANGEL_VERSION=3.0.0-SNAPSHOT +export ANGEL_VERSION=3.0.0 for f in "$lib"/*.jar; do diff --git a/angel-ps/bin/angel-submit b/angel-ps/bin/angel-submit index ef846b7c2..0ae31cc3b 100755 --- a/angel-ps/bin/angel-submit +++ b/angel-ps/bin/angel-submit @@ -13,7 +13,7 @@ # or implied. See the License for the specific language governing permissions and limitations under # the License. -export ANGEL_VERSION=3.0.0-SNAPSHOT +export ANGEL_VERSION=3.0.0 export PATH=${JAVA_HOME}/bin:$PATH if [ "${HADOOP_HOME}" != "" ]; then diff --git a/angel-ps/bin/angel-submit-with-jarvis b/angel-ps/bin/angel-submit-with-jarvis index 4dfb28ac7..ec47f0727 100644 --- a/angel-ps/bin/angel-submit-with-jarvis +++ b/angel-ps/bin/angel-submit-with-jarvis @@ -17,7 +17,7 @@ # export HADOOP_HOME="" # export JAVA_HOME="" -export ANGEL_VERSION=3.0.0-SNAPSHOT +export ANGEL_VERSION=3.0.0 export PATH=$JAVA_HOME/bin:$PATH if [ "$HADOOP_HOME" = "" ]; then diff --git a/angel-ps/core/pom.xml b/angel-ps/core/pom.xml index b12d24f46..29ff7146e 100644 --- a/angel-ps/core/pom.xml +++ b/angel-ps/core/pom.xml @@ -22,7 +22,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 angel-ps-core angel-ps-core diff --git a/angel-ps/examples/pom.xml b/angel-ps/examples/pom.xml index 1a217591e..8751f7004 100644 --- a/angel-ps/examples/pom.xml +++ b/angel-ps/examples/pom.xml @@ -23,7 +23,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 angel-ps-examples diff --git a/angel-ps/graph/pom.xml b/angel-ps/graph/pom.xml index ef3d8f630..f69952ebd 100644 --- a/angel-ps/graph/pom.xml +++ b/angel-ps/graph/pom.xml @@ -5,7 +5,7 @@ angel-ps com.tencent.angel - 3.0.0-SNAPSHOT + 3.0.0 4.0.0 diff --git a/angel-ps/jarvis/pom.xml b/angel-ps/jarvis/pom.xml index e7bcd687c..c0846b24d 100644 --- a/angel-ps/jarvis/pom.xml +++ b/angel-ps/jarvis/pom.xml @@ -21,7 +21,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 ../pom.xml angel-ps-jarvis diff --git a/angel-ps/mllib/pom.xml b/angel-ps/mllib/pom.xml index fcece934c..0db890d65 100644 --- a/angel-ps/mllib/pom.xml +++ b/angel-ps/mllib/pom.xml @@ -22,7 +22,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 ../pom.xml diff --git a/angel-ps/pom.xml b/angel-ps/pom.xml index d3ba373e0..f5008338a 100644 --- a/angel-ps/pom.xml +++ b/angel-ps/pom.xml @@ -22,7 +22,7 @@ com.tencent.angel angel - 3.0.0-SNAPSHOT + 3.0.0 ../pom.xml diff --git a/angel-ps/psf/pom.xml b/angel-ps/psf/pom.xml index 547a84331..e1a887ac1 100644 --- a/angel-ps/psf/pom.xml +++ b/angel-ps/psf/pom.xml @@ -23,7 +23,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 ../pom.xml angel-ps-psf diff --git a/angel-ps/tools/pom.xml b/angel-ps/tools/pom.xml index fbaaa450f..8056ab482 100644 --- a/angel-ps/tools/pom.xml +++ b/angel-ps/tools/pom.xml @@ -23,7 +23,7 @@ com.tencent.angel angel-ps - 3.0.0-SNAPSHOT + 3.0.0 angel-ps-tools diff --git a/dist/pom.xml b/dist/pom.xml index 1ad682227..3376f15a4 100644 --- a/dist/pom.xml +++ b/dist/pom.xml @@ -24,7 +24,7 @@ com.tencent.angel angel - 3.0.0-SNAPSHOT + 3.0.0 angel-dist diff --git a/pom.xml b/pom.xml index 31fcb4112..7b34ea6ce 100644 --- a/pom.xml +++ b/pom.xml @@ -21,7 +21,7 @@ com.tencent.angel angel pom - 3.0.0-SNAPSHOT + 3.0.0 angel 2.7.3 From 81ff4bb3d404be07d257791a12dbecd7e343d8fe Mon Sep 17 00:00:00 2001 From: ouyangwen-it Date: Fri, 16 Aug 2019 21:22:43 +0800 Subject: [PATCH 13/20] update scripts for k8s. --- .../kubernetesmanager/scripts/Dockerfile | 23 ++++++++++++++++--- .../kubernetesmanager/scripts/entrypoint.sh | 1 - 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile index 41d828f18..5b4c93e11 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/Dockerfile @@ -1,5 +1,22 @@ -FROM tencentangel/angel:v3.0.0 +FROM openjdk:8-alpine -#User-defined action +ARG angel_dist_path=./angel-3.0.0 +ARG entrypoint_path=./entrypoint.sh -ENTRYPOINT ["/opt/entrypoint.sh"] +RUN set -ex && \ + apk upgrade --no-cache && \ + apk add --no-cache bash tini libc6-compat linux-pam && \ + mkdir -p /opt/ && \ + rm /bin/sh && \ + ln -sv /bin/bash /bin/sh && \ + echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su && \ + chgrp root /etc/passwd && chmod ug+rw /etc/passwd + +COPY ${angel_dist_path} /opt/angel +COPY ${entrypoint_path} /opt/ + +ENV ANGEL_HOME /opt/angel + +WORKDIR /opt/angel/work-dir + +ENTRYPOINT [ "/opt/entrypoint.sh" ] diff --git a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh index 0ae2b25c1..8c0b9305f 100755 --- a/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh +++ b/angel-ps/core/src/main/java/com/tencent/angel/kubernetesmanager/scripts/entrypoint.sh @@ -10,7 +10,6 @@ set +e uidentry=$(getent passwd $myuid) set -e -ANGEL_HOME=/opt/angel # If there is no passwd entry for the container UID, attempt to create one if [ -z "$uidentry" ] ; then if [ -w /etc/passwd ] ; then From 121c5f06d03a3d4224dc506aa6fb0cdfc37e6051 Mon Sep 17 00:00:00 2001 From: Yun Dai Date: Sat, 17 Aug 2019 12:05:56 +0800 Subject: [PATCH 14/20] [doc] add contributors --- Community.md | 51 +++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 8 deletions(-) diff --git a/Community.md b/Community.md index f80768583..82dcb4dcc 100644 --- a/Community.md +++ b/Community.md @@ -25,7 +25,7 @@ The TSC may add additional members to the TSC by a 2/3 majority vote. - [fitzwang](https://github.com/wangcaihua) - **Fitz Wang** <[fitzwang.ml@gmail.com](fitzwang.ml@gmail.com) > - [leleyu](https://github.com/leleyu)- **Lele Yu** <[yulele214@gmail.com](yulele214@gmail.com) > - [lbluesjjw](https://github.com/bluesjjw)- **Jeremy Jiang** <[blue.jwjiang@gmail.com](blue.jwjiang@gmail.com) > - + ### TSC Meeting - The TSC meets regularly in a voice meeting call @@ -53,10 +53,45 @@ Committers should be familiar with the guidelines for new contributors in [CONT - - - - - - - +## Contributor + +We would like to sincerely thank the following community members for their contributions to Angel. + +- [paynie](https://github.com/paynie) - **Paynie Xiao** +- [andyyehoo](https://github.com/andyyehoo) - **Andy Huang** +- [leleyu](https://github.com/leleyu) - **Lele Yu** +- [shunanzhang](https://github.com/shunanzhang) - **Shunan Zhang** +- [lbluesjjw](https://github.com/bluesjjw) - **Jeremy Jiang** +- [TAAAN](https://github.com/TAAAN) - **Yunkun Tan** +- [weiwee](https://github.com/weiwee) - **Wenbin Wei** +- [rachelsunrh](https://github.com/rachelsunrh) - **Rachel Sun** +- [ZuwenYou](https://github.com/ZuwenYou) - **Zuwen You** +- [fitzwang](https://github.com/wangcaihua) - **Fitz Wang** +- [luxin0123](https://github.com/luxin0123) - **Ally Yu** +- [xujie32](https://github.com/xujie32) - **Joy Xu** +- [FLYINGQQ](https://github.com/FLYINGQQ) +- [yundai424](https://github.com/yundai424) - **Ariel Dai** +- [hbghhy](https://github.com/hbghhy) +- [duoyu119](https://github.com/duoyu119) - **Duo Yu** +- [facaiy](https://github.com/facaiy) - **Facai Yan** +- [f7753](https://github.com/f7753) - **Lewin Ma** +- [ouyangwen-it](https://github.com/ouyangwen-it) - **Gdp Ouyang** +- [ericzhang-cn](https://github.com/ericzhang-cn) - **Eric Zhang** +- [ljch2018](https://github.com/ljch2018) +- [liuzhanhao](https://github.com/liuzhanhao) - **Zhanhao (Jasper) Liu** +- [AugustoYao](https://github.com/AugustoYao) - **Augusto Yao** +- [20100507](https://github.com/20100507) +- [yd19tc](https://github.com/yd19tc) +- [witgo](https://github.com/witgo) - **Guoqiang Li** +- [Morrissss](https://github.com/Morrissss) +- [sunchenglong](https://github.com/sunchenglong) - **Chenglong Sun** +- [quhailong](https://github.com/quhailong) - **Hailong Qu** +- [patrikwang](https://github.com/patrikwang) +- [mtunique](https://github.com/mtunique) - **Tao Meng** +- [ibingoogle](https://github.com/ibingoogle) +- [ghandzhipeng](https://github.com/ghandzhipeng) - **Zhipeng Zhang** +- [Suanec](https://github.com/Suanec) +- [sustcsonglin](https://github.com/sustcsonglin) +- [endymecy](https://github.com/endymecy) +- [StrikeW](https://github.com/StrikeW) +- [wqh17101](https://github.com/wqh17101) \ No newline at end of file From 0098184a26f31cd22178e790a5cf8574f15a7b83 Mon Sep 17 00:00:00 2001 From: oneal Date: Sun, 18 Aug 2019 22:17:55 +0800 Subject: [PATCH 15/20] add English description --- docs/basic/batchsize_decay_en.md | 91 +++++++ docs/basic/computinggraph_on_angel_en.md | 233 ++++++++++++++++ docs/basic/layers_on_angel_en.md | 331 +++++++++++++++++++++++ 3 files changed, 655 insertions(+) create mode 100644 docs/basic/batchsize_decay_en.md create mode 100644 docs/basic/computinggraph_on_angel_en.md create mode 100644 docs/basic/layers_on_angel_en.md diff --git a/docs/basic/batchsize_decay_en.md b/docs/basic/batchsize_decay_en.md new file mode 100644 index 000000000..fad269f7e --- /dev/null +++ b/docs/basic/batchsize_decay_en.md @@ -0,0 +1,91 @@ +# Learning Rate Decay in Angel + +Angel refers to TensorFlow to implement a variety of learning rate Decay schemes. Users can choose Decay schemes according to their needs. Before describing the specific Decay schemes, first learn how Decay in Angel was introduced and when Decay was implemented. + +For the first problem, Decay was introduced in the `GraphLearner'class, with the following code for initialization: +```scala +val ssScheduler: StepSizeScheduler = StepSizeScheduler(SharedConf.getStepSizeScheduler, lr0) +``` +Where `StepSize Scheduler'is the base class of all Decays, object with the same name is the workshop of all Decays, and `SharedConf. getStepSize Scheduler' can obtain the specified decay type by reading the value of `ml. opt. decay. class. name'(default is StandardDecay). +For the second question, Angel offers two options: +- A mini-batch once Decay +- A epoch once Decay + +Through the parameter of "ml.opt.decay.on.batch", when it is true, a mini-batch once Decay, and an epoch once Decay when it is flase (default way). Specific code in the `GraphLearner'class of train method and trainOneEpoch method. + +## 1. ConstantLearningRate +This is the simplest way to Decay, that is, without Decay, the learning rate remains unchanged throughout the training process. + +Configuration sample: +``` +ml.opt.decay.class.name=ConstantLearningRate +``` + +## 2. StandardDecay +The standard Decay scheme, the formula is as follows: + +![](http://latex.codecogs.com/png.latex?lr_{t}=\frac{lr_0}{\sqrt{1+\alpha\cdot%20t}}) + +Configuration sample: +``` +ml.opt.decay.class.name=StandardDecay +ml.opt.decay.alpha=0.001 +``` +## 3. CorrectionDecay +Modified Decay, this scheme is suitable for Momentum, it is designed for Momentum, please do not use it for other optimizers such as Adam. The calculation formula is: + +![](http://latex.codecogs.com/png.latex?lr_{t}=\frac{lr_0}{\sqrt{1+\alpha\cdot%20t}}\cdot\frac{1-\beta}{1-\beta^t}) + +The first part is StandardDecay, which is the normal Decay, and the second part is the correction item, designed for Momentum, which is the reciprocal of the sum of the motion coefficients. Where $\beta$ must be equal to the momentum in the optimizer. Generally set to 0.9 . + +There are two points to note about the use of this Decay: +- The momentum calculation formula should be: velocity = momentum * velocity + gradient, the implementation of Momentum in Angel is also this way. +- Requires a mini-batch once Decay, because it is synchronized with the update of the parameter. + +Configuration sample: +``` +ml.opt.decay.class.name=CorrectionDecay +ml.opt.decay.alpha=0.001 +ml.opt.decay.beta=0.9 +``` +## 4. WarmRestarts +This is a more advanced Decay scheme, which is representative of Decay in the cycle. The standard calculation formula is as follows: +![](http://latex.codecogs.com/png.latex?lr_t=lr_{min}+\frac{1}{2}\cdot\frac{lr_{max}-lr_{min}}{1+\cos{(\frac{t}{interval}\pi)}}) + +For the standard calculation formula, we made the following improvements. +- Attenuate ![](http://latex.codecogs.com/png.latex?lr_{max}) +- Step by step to increase ![](http://latex.codecogs.com/png.latex?interval) + +Configuration sample: +``` +ml.opt.decay.class.name=WarmRestarts +ml.opt.decay.alpha=0.001 +``` + +Where![](http://latex.codecogs.com/png.latex?interval) is set by `ml.opt.decay.intervals`, as follows: +```scala +class WarmRestarts(var etaMax: Double, etaMin: Double, alpha: Double) extends StepSizeScheduler { + + var current: Double = 0 + var numRestart: Int = 0 + var interval: Int = SharedConf.get().getInt(MLConf.ML_OPT_DECAY_INTERVALS, 100) + + override def next(): Double = { + current += 1 + val value = etaMin + 0.5 * (etaMax - etaMin) * (1 + math.cos(current / interval * math.Pi)) + if (current == interval) { + current = 0 + interval *= 2 + numRestart += 1 + etaMax = etaMax / math.sqrt(1.0 + numRestart * alpha) + } + + value + } + + override def isIntervalBoundary: Boolean = { + current == 0 + } + +} +``` diff --git a/docs/basic/computinggraph_on_angel_en.md b/docs/basic/computinggraph_on_angel_en.md new file mode 100644 index 000000000..ec5c023c8 --- /dev/null +++ b/docs/basic/computinggraph_on_angel_en.md @@ -0,0 +1,233 @@ +# Computational graph in Angel + +## 1. What is a computational graph? +Computational graph are commonly used in mainstream deep learning frameworks, such as Tensorflow, Caffe and Maxnet. In fact, large data processing tools such as Spark also use computational graph to schedule tasks. In order to better support deep learning algorithms, Angel also supports computational graph. Angel's computational graph are lighter than Tensorflow's. The main manifestations are as follows: +- **Coarse Granularity**: The 'node' in Angel's calculation graph is layer, not `operator. Tensorflow and so on use `operation` as the node in the graph, which is very flexible and suitable for secondary development (package), but also It gives the machine learning and development developers a steeper learning curve and a larger workload, so the old version of Tensorflow has also been criticized as "API too low-level, low development efficiency", and later Tensorflow version provides layer-based (layer High-level API. Given this, Angel only provides coarse-grained computational graphs. +- **Feature Intersection**: For recommended system-related algorithms, the characteristics of Embedding often have to pass some crossovers (**Note**: the feature intersection here is different from the artificial intersection in the feature engineering, where the feature intersection is performed by the output of Embedding). Then enter DNN. These feature intersections are more cumbersome on Tensorflow, Caffe, Torch, etc., and this feature cross layer is directly provided on Angel. +- **Automatic Generation Network**: Angel can read the Json file to generate a deep network. This is a reference to Caffe, users can generate their own network without writing code, greatly reducing the workload. + +It should be pointed out that Angel does not currently support CNN, RNN, etc., and only focuses on commonly used algorithms in the recommended field. + +## 2. Construction of Computational Graph + +### 2.1 Basic Structure of Layer +To understand how a computational graph is constructed, you first need to understand the structure of its constituent element, Layer (for more detailed information on layers, refer to [Layer in Angel](./layers_on_angel.md).) as follows: +```scala +abstract class Layer(val name: String, val outputDim: Int)(implicit val graph: AngelGraph) + extends Serializable { + var status: STATUS.Value = STATUS.Null + val input = new ListBuffer[Layer]() + val consumer = new ListBuffer[Layer]() + + def addInput(layer: Layer): Unit = { + input.append(layer) + } + + def addConsumer(layer: Layer): Unit = { + consumer.append(layer) + } + + def calOutput(): Matrix = ??? + + def gatherGrad(): Matrix = ??? +} +``` +This abstract class has a clear description of most of the layers, as follows: +- status: The nodes in the Angel calculation graph are stateful and are processed by a state machine, as described in the next section. +- input: Used to record the input of this node/layer, represented by a ListBuffer, a layer can have multiple input layers, you can call addInput(layer: Layer) multiple times. +- outputDim: There can only be one output in Angel, and outputDim is used to specify the dimension of the output +- consumer: Layer has only one output, but the output node can be consumed many times, so it is represented by ListBuffer. When building a graph, call the addConsumer (layer: Layer) of the input layer to tell which layers of the output layer consume it. + +In fact, the specific operation of building a graph has been completed in the base class of inputlayer/linearlayer/joinlayer. User-defined layers need not be concerned, as follows: +```scala +abstract class InputLayer(name: String, outputDim: Int)(implicit graph: AngelGraph) + extends Layer(name, outputDim)(graph) { + graph.addInput(this) + + def calBackward(): Matrix +} + +abstract class JoinLayer(name: String, outputDim: Int, val inputLayers: Array[Layer])(implicit graph: AngelGraph) + extends Layer(name, outputDim)(graph) { + inputLayers.foreach { layer => + layer.addConsumer(this) + this.addInput(layer) + } + + def calGradOutput(idx: Int): Matrix +} + +abstract class LinearLayer(name: String, outputDim: Int, val inputLayer: Layer)(implicit graph: AngelGraph) + extends Layer(name, outputDim)(graph) { + inputLayer.addConsumer(this) + this.addInput(inputLayer) + + def calGradOutput(): Matrix +} +``` +Note: LossLayer is a special Linear Layer, so it is not given here. + +### 2.2 The Basic Structure of AngelGraph +A complex graph is constructed by input/consumer. Although the graph can be traversed from any node in the graph, verge nodes are stored in AngelGraph for convenience. It is easy to operate the graph as follows: +```scala +class AngelGraph(val placeHolder: PlaceHolder, val conf: SharedConf) extends Serializable { + + def this(placeHolder: PlaceHolder) = this(placeHolder, SharedConf.get()) + + private val inputLayers = new ListBuffer[InputLayer]() + private var lossLayer: LossLayer = _ + private val trainableLayer = new ListBuffer[Trainable]() + + + def addInput(layer: InputLayer): Unit = { + inputLayers.append(layer) + } + + def setOutput(layer: LossLayer): Unit = { + lossLayer = layer + } + + def getOutputLayer: LossLayer = { + lossLayer + } + + def addTrainable(layer: Trainable): Unit = { + trainableLayer.append(layer) + } + + def getTrainable: ListBuffer[Trainable] = { + trainableLayer + } +``` +Verge has two major categories: +- inputLayer: The input of this type of node is data. The storage of such nodes in AngelGraph is convenient for reverse calculation. Just call the input layer's `calBackward`. In order to add inputLayer, Angel requires all inputLayer to call AngelGraph's addInput method to add itself to AngelGraph. In fact, this has been done in the base class of InputLayer, the user added inputLayer does not have to care about this. +- lossLayer: Angel does not support multitasking at present, so there is only one lossLayer. This kind of node is mainly convenient for forward calculation. Just call its `predict` or `calOutput`. Since losslayer is a subclass of linearlayer, user-defined lossLayer can be used. Manually call `setOutput(layer: LossLayer)`, but there are not many opportunities for users to add a losslayer. More is to add lossfunc. + +With inputLayers, lossLayer, it is very convenient to traverse the graph from AngelGraph. The forward calculation only needs to call the losslayer's `predict` method, and the reverse calculation just calls the input layer's `calBackward`. However, the gradient calculation, the parameter update is inconvenient, in order to facilitate the parameters, AngelGraph added a trainableLayer variable to hold the layer with parameters. + +## 2.3 Data entry: PlaceHolder +The layer's input/consumer is used to construct the edge of the graph (the relationship of nodes). The special node (inputlayer/losslayer/trainablelayer) is saved in AngelGraph for forward and backward calculation and parameter update. How is the final data input? -- by PlaceHolder + +The PlaceHolder in Angel is passed to the Graph in the construction of AngelGraph, and the Graph is passed to the Layer as an implicit parameter, so the placeholder (data) can be accessed in all Layers. + +Currently, only one PlaceHolder is allowed in Angel. This restriction will be removed in the future, allowing multiple data inputs. PlaceHolder only stores one mini-batch data. The main methods are as follows: +```scala +class PlaceHolder(val conf: SharedConf) extends Serializable { + def feedData(data: Array[LabeledData]): Unit + def getFeats: Matrix + def getLabel: Matrix + def getBatchSize: Int + def getFeatDim: Long + def getIndices: Vector +} +``` +After the data of the Array[LabeledData] type is given to the placeholder by `feedData`, it can be obtained from it: +- feature +- feature dimension +- label +- batchSize +- feature index + + + +## 3. The operation principle of the calculation graph in Angel +The topology of the calculation graph was built in the previous section. This section describes how it works. + +### 3.1 Running state machine +Angel's state machine has the following states: +- Null: Initial state, the Graph will be placed in this state after each feedData. +- Forward: This state indicates that the forward calculation is complete. +- Backward: This state indicates that the backward calculation has been completed, but the gradient of the parameter has not yet been calculated. +- Gradient: This state indicates that the gradient has been calculated and the gradient has been pushed onto the PS. +- Update: This state indicates that the model update is complete. + +These states are in turn, as shown in the following figure: + +![state machine](../img/status.png) + +The introduction of state machine mainly guarantees the sequential operation and reduces repetitive calculation. For example, multiple layers consume the output of the same layer. When calculating, the state can be judged based on the state, as long as it is calculated once. The state machine is embodied in the code as follows: +```scala +def feedData(data: Array[LabeledData]): Unit = { + deepFirstDown(lossLayer.asInstanceOf[Layer])( + (lay: Layer) => lay.status != STATUS.Null, + (lay: Layer) => lay.status = STATUS.Null + ) + + placeHolder.feedData(data) +} + +override def calOutput(): Matrix = { + status match { + case STATUS.Null => + // do come forward calculation + status = STATUS.Forward + case _ => + } + output +} + + +override def calBackward(): Matrix = { + status match { + case STATUS.Forward => + val gradTemp = gatherGrad() + // do backward calculation + status = STATUS.Backward + case _ => + } + + backward +} + +override def pushGradient(): Unit = { + status match { + case STATUS.Backward => + // calculate gradient and push to PS + status = STATUS.Gradient + case _ => + } +} + +override def update(epoch: Int = 0): Unit = { + status match { + case STATUS.Gradient => + optimizer.update(weightId, 1, epoch) + status = STATUS.Update + case _ => + throw new AngelException("STATUS Error, please calculate Gradient frist!") + } +} +``` + +### 3.2 The Training Process of Graph in Angel +The specific code is `GraphLearner', which gives the framework. +```scala +def trainOneEpoch(epoch: Int, iter: Iterator[Array[LabeledData]], numBatch: Int): Double = { + var batchCount: Int = 0 + var loss: Double = 0.0 + while (iter.hasNext) { + graph.feedData(iter.next()) + graph.pullParams() + loss = graph.calLoss() // forward + graph.calBackward() // backward + graph.pushGradient() // pushgrad + PSAgentContext.get().barrier(ctx.getTaskId.getIndex) + if (ctx.getTaskId.getIndex == 0) { + graph.update(epoch * numBatch + batchCount) // update parameters on PS + } + PSAgentContext.get().barrier(ctx.getTaskId.getIndex) + batchCount += 1 + + LOG.info(s"epoch $epoch batch $batchCount is finished!") + } + + loss + } +``` +Proceed as follows: +- feedData: This process will set the state of the Graph to Null +- Pull parameter: According to the data, only the parameters needed for the current mini-batch calculation are pulled, so Angel can train very high-dimensional models. +- Forward calculation: Starting with Losslayer, the `calOutput'method of its inputlayer is invoked cascadely, and output is calculated in turn. After calculation, its state is set to `forward'. For the case where the state is `forward', the result of the last calculation is returned directly, so as to avoid repeated calculation. +- Backward Computation: Inputlayer of Graph is called sequentially, so that the `CalGradOutput'method of the first layer is cascaded to complete backward calculation. After calculation, the state of Graph is set to `backward'. If the state is `backward', the result of the last calculation is returned directly, thus avoiding repeated calculation. +- Gradient calculation and updating: In this step, the gradient of parameters can be calculated by calling `pushGradient'of `trainable'. This method calculates the gradient first, then pushes the gradient onto PS, and finally sets the state to `gradient'.` +- Gradient Updating: Gradient updates are performed on the PS, as long as a gradient-updated PSF is sent, so only one Workor is required to send it (in Spark on Angel via the Driver). Different optimizers are updated differently, in Angel. The core essence of the optimizer is a PSF. Before the parameter update, a synchronization is required to ensure that all gradients are pushed and completed. The parameter update is also done once, ensuring that all the parameters pulled by the worker are up-to-date and will be set to `update`. \ No newline at end of file diff --git a/docs/basic/layers_on_angel_en.md b/docs/basic/layers_on_angel_en.md new file mode 100644 index 000000000..9af2792a7 --- /dev/null +++ b/docs/basic/layers_on_angel_en.md @@ -0,0 +1,331 @@ + +# Layers in Angel + +Most of the algorithms in Angel are based on [computational graph](./computational_graph_on_angel.md). The nodes in Angel are layers. According to the topological structure of layers, they can be divided into three categories: +- verge: Edge nodes, with only input or output layers, such as input and loss layers + - input layer: Mainly has SimpleInputLayer, Embedding + - Loss layer: Mainly using SimpleLossLayer, SoftmaxLossLayer +- linear: Layer with one input and one output + - Fully connected layer: that is FCLayer + - Feature intersection layer: There are many such layers, different algorithms use different feature crossing methods, and can also be used in combination. The main ones are:: + - BiInnerCross: Feature crossing method used in PNN(inner) + - BiOutterCross: Feature crossing method used in PNN(outter)(Currently not implemented) + - BiInnerSumCross: Second-order feature implicit crossover method used in FM + - BiIntecationCross: Feature crossing method used in NFM +- join: There are two or more inputs, one output layer, and there are many such layers, mainly: + - ConcatLayer: Put together multiple input layers and enter a Dense matrix + - SumPooling: Add the input elements correspondingly and output them + - MulPooling: Multiply input elements to output + - DotPooling: Multiply the corresponding elements first, then add them by row, and enter a matrix of n rows and one column. + + +## 1. Input Layer +There are two types of input layers in Angel: +- SimpleInputLayer +- Embedding + +### 1.1 SimpleInpuyLayer +As the name implies, it accepts input. Class constructors are as follows: +```scala +class SimpleInputLayer(name: String, outputDim: Int, transFunc: TransFunc, override val optimizer: Optimizer)(implicit graph: AngelGraph) + extends InputLayer(name, outputDim)(graph) with Trainable +``` +Its main features are: +- Receiving Dense/Sparse Input +- When the input is dense, the internal parameters are dense, the parameters are stored in an array continuously, and the BLAS library is called to complete the calculation; when the input is sparse, the internal parameters are stored in RowBasedMatrix, each row is a sparse vector, and Angel internal mathematical library is used for calculation. +- You need to specify outputDim to specify the transfer function and optimizer + +The completed calculation is expressed as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(x)=tranfunc(x\bold{w}+bias)) + +A typical JSON expression is: +```json +{ + "name": "wide", + "type": "Simpleinputlayer", + "outputdim": 10, + "transfunc": "identity", + "optimizer": "adam" +}, +``` + +### 1.2 Embedding +Embedding is common to many deep learning algorithms. The constructors of the class are as follows: +```scala +class Embedding(name: String, outputDim: Int, val numFactors: Int, override val optimizer: Optimizer)(implicit graph: AngelGraph) + extends InputLayer(name, outputDim)(graph) with Trainable +``` +In addition to name and optimizer, the other two parameters of Embedding are as follows: +- outputDim: It refers to the output of lookup. Angel's Embedding currently assumes that each sample has the same number of fields, and each field is One-hot. The first condition is true in most cases, but the second condition is more Strict, in some cases not established, will be relaxed in the future +- numFactors: It refers to the dimension of the Embedding vector. The size of the Embedding matrix is obtained as follows: The dimension of the input data is stored in the system. This dimension is obtained as the number of columns of the Embedding matrix, and numFactors is the number of rows of the Embedding matrix ( Note: Although the internal implementation is a bit different, this understanding is ok) + +Embedding is a table in the abstract sense, and provides a lookup table (lookup/calOutput). Angel's Embedding is special in that it allows some operations after checking the table, so it consists of two steps: +- Look up table: According to the index, find the corresponding column in the table +- Calculation assembly: Sometimes the data is not one-hot, multiply the found vector by a value. + +The results of sparse vector embedding with values of 1 (one-hot encoding, expressed in dummy format) and floating point (expressed in libsvm format) are shown below: + +![model](http://latex.codecogs.com/png.latex?\dpi{120}(1,5,40,\cdots,10000)\rightarrow(\bold{v}_1,\bold{v}_5,\bold{v}_{40},\cdots,\bold{v}_{10000})) + +![model](http://latex.codecogs.com/png.latex?\dpi{120}(1:0.3,5:0.7,40:1.9,\cdots,10000:3.2)\rightarrow(0.3\bold{v}_1,0.7\bold{v}_5,1.9\bold{v}_{40},\cdots,3.2\bold{v}_{10000})) + +A typical JSON expression is: +```json +{ + "name": "embedding", + "type": "Embedding", + "numfactors": 8, + "outputdim": 104, + "optimizer": { + "type": "momentum", + "momentum": 0.9, + "reg2": 0.01 + } +}, +``` + +## 2. Linear Layer +A linear layer is a layer that has one input and one output. It mainly includes a fully coupled layer (FCLayer) and a series of feature crossing layers. + +### 2.1 FCLayer +FCLayer layer is the most common layer in DNN. Its calculation can be expressed by the following formula: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(x)=tranfunc(x\bold{w}+bias)) + +The constructor in Angel is as follows: +```scala +class FCLayer(name: String, outputDim: Int, inputLayer: Layer, transFunc: TransFunc, override val optimizer: Optimizer + )(implicit graph: AngelGraph) extends LinearLayer(name, outputDim, inputLayer)(graph) with Trainable +``` +From the constructor and calculation formula, it is very similar to DenseInputLayer/SparseInputLayer. The difference is that the input of the former is a Layer, and the latter directly inputs data (do not specify the input Layer in the constructor). + +In parameter data storage, FCLayer, like DenseInput Layer, also uses dense method to calculate with BLAS. + +Since FCLayer is usually used in multiple stacks, there are some simplifications in the configuration of the data, that is, the parameter reduction of multiple stacked FCLayer. Here is an example: +```json +{ + "name": "fclayer", + "type": "FCLayer", + "outputdims": [ + 100, + 100, + 1 + ], + "transfuncs": [ + "relu", + "relu", + "identity" + ], + "inputlayer": "embedding" +}, +``` + +There are three FCLayer stacked together, the input is the input of the first layer, the output is the output of the last layer, the outputdim of each layer, and the transfunc is represented by the column, namely: +- outputdims: The output Dim of each FCLayer is given in the form of a list. +- transfuncs: Give each FCLayer's transfunc as a list + +Note: You can also specify an optimizer for the superimposed FCLayer. At this point, all layers have the same optimizer. If you write separately, it is: +```json +{ + "name": "fclayer_0", + "type": "FCLayer", + "outputdim": 100, + "transfuncs": "relu", + "inputlayer": "embedding" +}, +{ + "name": "fclayer_1", + "type": "FCLayer", + "outputdim": 100, + "transfuncs": "relu", + "inputlayer": "fclayer_0" +}, +{ + "name": "fclayer", + "type": "FCLayer", + "outputdim": 1, + "transfuncs": "identity", + "inputlayer": "fclayer_1" +}, +``` + +### 2.2 BiInnerSumCross +The formula for calculating the feature crossing layer is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k)=\sum_i^k\sum_{j=i+1}^k\bold{u}_i^T\bold{u}_j) + +Where ![](http://latex.codecogs.com/png.latex?(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k)) is the output of Embedding. Specifically, the Embedding result is a two-in-one inner product, and then summed. Therefore, BiInnerSumCross has no parameters, is untrainable, and has an output dimension of 1. + +The constructor is as follows: +```scala +class BiInnerSumCross(name: String, inputLayer: Layer)( + implicit graph: AngelGraph) extends LinearLayer(name, 1, inputLayer)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "biinnersumcross", + "type": "BiInnerSumCross", + "inputlayer": "embedding", + "outputdim": 1 +}, +``` + + +### 2.3 BiInnerCross +The formula for calculating the feature crossing layer is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k)=(\bold{u}_1^T\bold{u}_2,\bold{u}_1^T\bold{u}_3,\bold{u}_1^T\bold{u}_4,\cdots,\bold{u}_{k-1}^T\bold{u}_k)) + +Note：![](http://latex.codecogs.com/png.latex?(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k))is the output of Embedding. Specifically, Embedding results do two or two inner products, so the dimension of output is![](http://latex.codecogs.com/png.latex?\dpi{60}C_k^2=\frac{k(k-1)}{2}). It can be seen that BiInnerCross has no parameters, is untrainable, and has an output dimension of ![](http://latex.codecogs.com/png.latex?\dpi{80}C_k^2). + +The constructor is as follows: +```scala +class BiInnerCross(name: String, outputDim: Int, inputLayer: Layer)( + implicit graph: AngelGraph) extends LinearLayer(name, outputDim, inputLayer)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "biInnerCross", + "type": "BiInnerCross", + "outputdim": 78, + "inputlayer": "embedding" +}, +``` + +### 2.4 BiInteactionCross +The formula for calculating the feature crossing layer is as follows: + +![model](http://latex.codecogs.com/png.latex?\dpi{150}f(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k)=\sum_i^k\sum_{j=i+1}^k\bold{u}_i\otimes\bold{u}_j) + +Notes: ![](http://latex.codecogs.com/png.latex?(\bold{u}_1,\bold{u}_2,\cdots,\bold{u}_k))is the output result of Embedding. Specifically, the result of Embedding is the product of two corresponding elements![](http://latex.codecogs.com/png.latex?\bold{u}_i\otimes\bold{u}_j), and then the sum, so the dimension of output is the same as ![](http://latex.codecogs.com/png.latex?\bold{u}_k), and has nothing to do with the dimension of input data. Thus, BiInteaction Cross has no parameters and is untrainable. + +The constructor is as follows: +```scala +class BiInteractionCross(name: String, outputDim: Int, inputLayer: Layer)( + implicit graph: AngelGraph) extends LinearLayer(name, outputDim, inputLayer)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "biinteractioncross", + "type": "BiInteractionCross", + "outputdim": 8, + "inputlayer": "embedding" +}, +``` +## 3. Join Layer +The join layer refers to a layer with multiple inputs and one output, mainly including: +- ConcatLayer: Combine multiple input layers and enter a Dense matrix +- SumPooling: Add the input elements correspondingly and output them +- MulPooling: Multiply input elements to output +- DotPooling: Multiply the corresponding elements first, then add them by row, and output a matrix of n rows and one column + +### 3.1 ConcatLayer +Combine multiple input layers and enter a Dense matrix. The constructor is as follows: +```scala +class ConcatLayer(name: String, outputDim: Int, inputLayers: Array[Layer])(implicit graph: AngelGraph) + extends JoinLayer(name, outputDim, inputLayers)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "concatlayer", + "type": "ConcatLayer", + "outputdim": 182, + "inputlayers": [ + "embedding", + "biInnerCross" + ] +}, +``` +There are multiple input layers, specified by inputlayers, in the form of a list. + +### 3.2 SumPoolingLayer +The input elements are added together to output a Dense matrix. The constructor is as follows: +```scala +class SumPooling(name: String, outputDim: Int, inputLayers: Array[Layer])(implicit graph: AngelGraph) + extends JoinLayer(name, outputDim, inputLayers)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "sumPooling", + "type": "SumPooling", + "outputdim": 1, + "inputlayers": [ + "wide", + "fclayer" + ] +}, +``` +There are multiple input layers, which are specified in the form of lists, using inputlayers. + +### 3.3 MulPoolingLayer +After multiplying the input elements, a Dense matrix is output. The constructor is as follows: +```scala +class MulPooling(name: String, outputDim: Int, inputLayers: Array[Layer])(implicit graph: AngelGraph) + extends JoinLayer(name, outputDim, inputLayers)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "mulPooling", + "type": "MulPooling", + "outputdim": 1, + "inputlayers": [ + "wide", + "fclayer" + ] +}, +``` +There are multiple input layers, which are specified in the form of lists, using inputlayers. + +### 3.4 DotPoolingLayer +First multiply the corresponding elements, then add them by row, and output a matrix of n rows and columns. The constructor is as follows: +```scala +class DotPooling(name: String, outputDim: Int, inputLayers: Array[Layer])(implicit graph: AngelGraph) + extends JoinLayer(name, outputDim, inputLayers)(graph) +``` + +An example of the json parameter is as follows: +```json +{ + "name": "dotPooling", + "type": "DotPooling", + "outputdim": 1, + "inputlayers": [ + "wide", + "fclayer" + ] +}, +``` +There are multiple input layers, which are specified in the form of lists, using inputlayers. + +## 4. Loss layer +At the top of the network, there is only the input layer, there is no output layer, used to calculate the loss. For the loss function, please refer to [Angel中的损失函数](./lossfunc_on_angel.md) + +### 4.1 SimpleLossLayer +The constructor of SimpleLossLayer is as follows: +```scala +class SimpleLossLayer(name: String, inputLayer: Layer, lossFunc: LossFunc)( + implicit graph: AngelGraph) extends LinearLayer(name, 1, inputLayer)(graph) with LossLayer +``` + +An example of the json parameter is as follows: +```json +{ + "name": "simplelosslayer", + "type": "Simplelosslayer", + "lossfunc": "logloss", + "inputlayer": "sumPooling" +} +``` \ No newline at end of file From b7b9ab6ef0e3373d9c1dc336d684708e43f76a7a Mon Sep 17 00:00:00 2001 From: Yun Dai Date: Mon, 19 Aug 2019 10:15:30 +0800 Subject: [PATCH 16/20] [doc] add contributors --- Community.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Community.md b/Community.md index 82dcb4dcc..6ba5c8a86 100644 --- a/Community.md +++ b/Community.md @@ -1,4 +1,4 @@ -Welcome to the Angel community! This COMMUNITY.md explains where to get help, and different ways to be part of the community. +Welcome to the Angel community! This document explains where to get help, and different ways to be part of the community. The Angel project community comprises a group of core committer and a sub-group that forms the Technical Steering Committee (the “TSC”) which governs the project. @@ -94,4 +94,6 @@ We would like to sincerely thank the following community members for their contr - [sustcsonglin](https://github.com/sustcsonglin) - [endymecy](https://github.com/endymecy) - [StrikeW](https://github.com/StrikeW) -- [wqh17101](https://github.com/wqh17101) \ No newline at end of file +- [wqh17101](https://github.com/wqh17101) +- [Oneal65]() - **Oneal Liu** +- [cheney2015]() - **Cristal Chen** \ No newline at end of file From 76b99ff46fae5a5d3c5fb74192d2adfbaf081534 Mon Sep 17 00:00:00 2001 From: paynie Date: Mon, 19 Aug 2019 19:29:38 +0800 Subject: [PATCH 17/20] Remove unused logs --- .../ps/storage/vector/storage/LongFloatVectorStorage.java | 3 --- 1 file changed, 3 deletions(-) diff --git a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/storage/LongFloatVectorStorage.java b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/storage/LongFloatVectorStorage.java index c76457978..99e3801bc 100644 --- a/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/storage/LongFloatVectorStorage.java +++ b/angel-ps/core/src/main/java/com/tencent/angel/ps/storage/vector/storage/LongFloatVectorStorage.java @@ -98,8 +98,6 @@ public void indexGet(IndexType indexType, int indexSize, ByteBuf in, ByteBuf out @Override public void update(RowType updateType, ByteBuf buf, UpdateOp op) { - LOG.info("update type = " + updateType + ", rowId=" + vector.getRowId() + ", size=" + vector.getSize()); - long startTs = System.currentTimeMillis(); switch (updateType) { case T_FLOAT_SPARSE_LONGKEY: case T_FLOAT_SPARSE_LONGKEY_COMPONENT: @@ -151,7 +149,6 @@ public void update(RowType updateType, ByteBuf buf, UpdateOp op) { "Unsupport operation: update " + updateType + " to " + this.getClass().getName()); } } - LOG.info("update use time = " + (System.currentTimeMillis() - startTs)); } From ad39f7abd92de034341b2bdb42159c0fee8267b8 Mon Sep 17 00:00:00 2001 From: paynie Date: Tue, 20 Aug 2019 09:42:34 +0800 Subject: [PATCH 18/20] remove unused file --- .../angel/master/MasterServiceTest.java | 263 ------------------ 1 file changed, 263 deletions(-) delete mode 100644 angel-ps/core/src/test/java/com/tencent/angel/master/MasterServiceTest.java diff --git a/angel-ps/core/src/test/java/com/tencent/angel/master/MasterServiceTest.java b/angel-ps/core/src/test/java/com/tencent/angel/master/MasterServiceTest.java deleted file mode 100644 index ae910d0cd..000000000 --- a/angel-ps/core/src/test/java/com/tencent/angel/master/MasterServiceTest.java +++ /dev/null @@ -1,263 +0,0 @@ -/* - * Tencent is pleased to support the open source community by making Angel available. - * - * Copyright (C) 2017-2018 THL A29 Limited, a Tencent company. All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in - * compliance with the License. You may obtain a copy of the License at - * - * https://opensource.org/licenses/Apache-2.0 - * - * Unless required by applicable law or agreed to in writing, software distributed under the License - * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express - * or implied. See the License for the specific language governing permissions and limitations under - * the License. - * - */ - - -package com.tencent.angel.master; - -import com.tencent.angel.client.AngelClient; -import com.tencent.angel.client.AngelClientFactory; -import com.tencent.angel.common.location.Location; -import com.tencent.angel.conf.AngelConf; -import com.tencent.angel.conf.MatrixConf; -import com.tencent.angel.ipc.TConnection; -import com.tencent.angel.ipc.TConnectionManager; -import com.tencent.angel.localcluster.LocalClusterContext; -import com.tencent.angel.master.task.AMTask; -import com.tencent.angel.master.task.AMTaskManager; -import com.tencent.angel.master.worker.attempt.WorkerAttempt; -import com.tencent.angel.ml.matrix.MatrixContext; -import com.tencent.angel.ml.math2.utils.RowType; -import com.tencent.angel.protobuf.ProtobufUtil; -import com.tencent.angel.protobuf.generated.MLProtos; -import com.tencent.angel.protobuf.generated.MLProtos.LocationProto; -import com.tencent.angel.protobuf.generated.MLProtos.Pair; -import com.tencent.angel.protobuf.generated.PSAgentMasterServiceProtos; -import com.tencent.angel.protobuf.generated.WorkerMasterServiceProtos.*; -import com.tencent.angel.ps.PSAttemptId; -import com.tencent.angel.ps.ParameterServerId; -import com.tencent.angel.worker.Worker; -import com.tencent.angel.worker.WorkerAttemptId; -import com.tencent.angel.worker.WorkerGroupId; -import com.tencent.angel.worker.WorkerId; -import com.tencent.angel.worker.task.TaskId; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.fs.LocalFileSystem; -import org.apache.hadoop.mapreduce.lib.input.CombineTextInputFormat; -import org.apache.log4j.PropertyConfigurator; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; -import org.mockito.runners.MockitoJUnitRunner; - -import java.util.Map; - -import static org.junit.Assert.*; - -@RunWith(MockitoJUnitRunner.class) public class MasterServiceTest { - private static final Log LOG = LogFactory.getLog(MasterServiceTest.class); - private static final String LOCAL_FS = LocalFileSystem.DEFAULT_FS; - private static final String TMP_PATH = System.getProperty("java.io.tmpdir", "/tmp"); - private AngelClient angelClient; - private WorkerGroupId group0Id; - private WorkerId worker0Id; - private WorkerAttemptId worker0Attempt0Id; - private TaskId task0Id; - private TaskId task1Id; - private ParameterServerId psId; - private PSAttemptId psAttempt0Id; - - static { - PropertyConfigurator.configure("../conf/log4j.properties"); - } - - - @Before public void setup() throws Exception { - try { - // set basic configuration keys - Configuration conf = new Configuration(); - conf.setBoolean("mapred.mapper.new-api", true); - conf.setBoolean(AngelConf.ANGEL_JOB_OUTPUT_PATH_DELETEONEXIST, true); - conf.set(AngelConf.ANGEL_TASK_USER_TASKCLASS, DummyTask.class.getName()); - - // use local deploy mode and dummy dataspliter - conf.set(AngelConf.ANGEL_DEPLOY_MODE, "LOCAL"); - conf.setBoolean(AngelConf.ANGEL_AM_USE_DUMMY_DATASPLITER, true); - conf.set(AngelConf.ANGEL_INPUTFORMAT_CLASS, CombineTextInputFormat.class.getName()); - conf.set(AngelConf.ANGEL_SAVE_MODEL_PATH, LOCAL_FS + TMP_PATH + "/out"); - conf.set(AngelConf.ANGEL_TRAIN_DATA_PATH, LOCAL_FS + TMP_PATH + "/in"); - conf.set(AngelConf.ANGEL_LOG_PATH, LOCAL_FS + TMP_PATH + "/log"); - - conf.setInt(AngelConf.ANGEL_WORKERGROUP_NUMBER, 1); - conf.setInt(AngelConf.ANGEL_PS_NUMBER, 1); - conf.setInt(AngelConf.ANGEL_WORKER_TASK_NUMBER, 2); - - conf.setInt(AngelConf.ANGEL_WORKER_HEARTBEAT_INTERVAL_MS, 1000); - conf.setInt(AngelConf.ANGEL_PS_HEARTBEAT_INTERVAL_MS, 1000); - - // get a angel client - angelClient = AngelClientFactory.get(conf); - - // add matrix - MatrixContext mMatrix = new MatrixContext(); - mMatrix.setName("w1"); - mMatrix.setRowNum(1); - mMatrix.setColNum(100000); - mMatrix.setMaxRowNumInBlock(1); - mMatrix.setMaxColNumInBlock(50000); - mMatrix.setRowType(RowType.T_INT_DENSE); - mMatrix.set(MatrixConf.MATRIX_OPLOG_ENABLEFILTER, "false"); - mMatrix.set(MatrixConf.MATRIX_HOGWILD, "true"); - mMatrix.set(MatrixConf.MATRIX_AVERAGE, "false"); - mMatrix.set(MatrixConf.MATRIX_OPLOG_TYPE, "DENSE_INT"); - angelClient.addMatrix(mMatrix); - - MatrixContext mMatrix2 = new MatrixContext(); - mMatrix2.setName("w2"); - mMatrix2.setRowNum(1); - mMatrix2.setColNum(100000); - mMatrix2.setMaxRowNumInBlock(1); - mMatrix2.setMaxColNumInBlock(50000); - mMatrix2.setRowType(RowType.T_DOUBLE_DENSE); - mMatrix2.set(MatrixConf.MATRIX_OPLOG_ENABLEFILTER, "false"); - mMatrix2.set(MatrixConf.MATRIX_HOGWILD, "false"); - mMatrix2.set(MatrixConf.MATRIX_AVERAGE, "false"); - mMatrix2.set(MatrixConf.MATRIX_OPLOG_TYPE, "DENSE_DOUBLE"); - angelClient.addMatrix(mMatrix2); - - angelClient.startPSServer(); - angelClient.run(); - Thread.sleep(5000); - group0Id = new WorkerGroupId(0); - worker0Id = new WorkerId(group0Id, 0); - worker0Attempt0Id = new WorkerAttemptId(worker0Id, 0); - task0Id = new TaskId(0); - task1Id = new TaskId(1); - psId = new ParameterServerId(0); - psAttempt0Id = new PSAttemptId(psId, 0); - } catch (Exception x) { - LOG.error("setup failed ", x); - throw x; - } - } - - @Test public void testMasterService() throws Exception { - try { - LOG.info("===========================testMasterService==============================="); - Worker worker = LocalClusterContext.get().getWorker(worker0Attempt0Id).getWorker(); - Location masterLoc = - LocalClusterContext.get().getMaster().getAppMaster().getAppContext().getMasterService() - .getLocation(); - TConnection connection = TConnectionManager.getConnection(worker.getConf()); - MasterProtocol master = connection.getMasterService(masterLoc.getIp(), masterLoc.getPort()); - - int psAgentId = master - .getPSAgentId(null, PSAgentMasterServiceProtos.GetPSAgentIdRequest.getDefaultInstance()) - .getPsAgentId(); - - // worker register - WorkerAttemptId worker1Attempt0Id = - new WorkerAttemptId(new WorkerId(new WorkerGroupId(1), 0), 0); - WorkerRegisterRequest registeRequest = - WorkerRegisterRequest.newBuilder().setPsAgentId(psAgentId) - .setWorkerAttemptId(ProtobufUtil.convertToIdProto(worker1Attempt0Id)) - .setLocation(LocationProto.newBuilder().setIp("10.10.10.10").setPort(10000).build()) - .build(); - WorkerRegisterResponse registerResponse = master.workerRegister(null, registeRequest); - assertTrue(registerResponse.getCommand() == WorkerCommandProto.W_SHUTDOWN); - - WorkerReportRequest.Builder reportBuilder = WorkerReportRequest.newBuilder(); - Pair.Builder kvBuilder = Pair.newBuilder(); - TaskStateProto.Builder taskBuilder = TaskStateProto.newBuilder(); - - reportBuilder.setWorkerAttemptId(ProtobufUtil.convertToIdProto(worker0Attempt0Id)); - - taskBuilder.setProgress(0.20f); - taskBuilder.setState("RUNNING"); - taskBuilder.setTaskId(ProtobufUtil.convertToIdProto(task0Id)); - kvBuilder.setKey("task_key1"); - kvBuilder.setValue("100"); - taskBuilder.addCounters(kvBuilder.build()); - kvBuilder.setKey("task_key2"); - kvBuilder.setValue("200"); - taskBuilder.addCounters(kvBuilder.build()); - reportBuilder.addTaskReports(taskBuilder.build()); - - taskBuilder.setProgress(0.30f); - taskBuilder.setState("RUNNING"); - taskBuilder.setTaskId(ProtobufUtil.convertToIdProto(task1Id)); - kvBuilder.setKey("task_key1"); - kvBuilder.setValue("1000"); - taskBuilder.addCounters(kvBuilder.build()); - kvBuilder.setKey("task_key2"); - kvBuilder.setValue("2000"); - taskBuilder.addCounters(kvBuilder.build()); - reportBuilder.addTaskReports(taskBuilder.build()); - - kvBuilder.setKey("worker_key1"); - kvBuilder.setValue("100"); - reportBuilder.addPairs(kvBuilder.build()); - kvBuilder.setKey("worker_key2"); - kvBuilder.setValue("200"); - reportBuilder.addPairs(kvBuilder.build()); - - WorkerReportResponse reportResponse = master.workerReport(null, reportBuilder.build()); - assertTrue(reportResponse.getCommand() == WorkerCommandProto.W_SUCCESS); - assertEquals(reportResponse.getActiveTaskNum(), 2); - - AngelApplicationMaster angelAppMaster = LocalClusterContext.get().getMaster().getAppMaster(); - WorkerAttempt worker0Attempt = - angelAppMaster.getAppContext().getWorkerManager().getWorker(worker0Attempt0Id.getWorkerId()) - .getWorkerAttempt(worker0Attempt0Id); - assertTrue(worker0Attempt != null); - Map workerMetrics = worker0Attempt.getMetrics(); - String valueForWorkerKey1 = workerMetrics.get("worker_key1"); - String valueForWorkerKey2 = workerMetrics.get("worker_key2"); - assertNotNull(valueForWorkerKey1); - assertNotNull(valueForWorkerKey2); - assertEquals(valueForWorkerKey1, "100"); - assertEquals(valueForWorkerKey2, "200"); - - AMTaskManager amTaskManager = angelAppMaster.getAppContext().getTaskManager(); - AMTask task0 = amTaskManager.getTask(task0Id); - AMTask task1 = amTaskManager.getTask(task1Id); - assertTrue(task0 != null); - assertTrue(task1 != null); - Map task0Metrics = task0.getMetrics(); - Map task1Metrics = task1.getMetrics(); - String valueForTask0Key1 = task0Metrics.get("task_key1"); - String valueForTask0Key2 = task0Metrics.get("task_key2"); - String valueForTask1Key1 = task1Metrics.get("task_key1"); - String valueForTask1Key2 = task1Metrics.get("task_key2"); - assertTrue(valueForTask0Key1 != null); - assertTrue(valueForTask0Key2 != null); - assertTrue(valueForTask1Key1 != null); - assertTrue(valueForTask1Key2 != null); - assertEquals(valueForTask0Key1, "100"); - assertEquals(valueForTask0Key2, "200"); - assertEquals(valueForTask1Key1, "1000"); - assertEquals(valueForTask1Key2, "2000"); - assertEquals(task0.getProgress(), 0.20f, 0.000001); - assertEquals(task1.getProgress(), 0.30f, 0.000001); - } catch (Exception x) { - LOG.error("run testMasterService failed ", x); - throw x; - } - } - - @After public void stop() throws Exception { - try { - LOG.info("stop local cluster"); - angelClient.stop(); - } catch (Exception x) { - LOG.error("stop failed ", x); - throw x; - } - } -} From b5d611b132e663550de126f3c0d8ce643d30551e Mon Sep 17 00:00:00 2001 From: paynie Date: Tue, 20 Aug 2019 14:29:41 +0800 Subject: [PATCH 19/20] change dependency jar version --- angel-ps/conf/angel-site.xml | 2 +- angel-ps/core/pom.xml | 16 ---------------- angel-ps/mllib/pom.xml | 2 +- angel-ps/pom.xml | 4 ++-- docs/algo/sona/line_sona_en.md | 2 +- 5 files changed, 5 insertions(+), 21 deletions(-) diff --git a/angel-ps/conf/angel-site.xml b/angel-ps/conf/angel-site.xml index c1d792ba5..fe8be09e7 100644 --- a/angel-ps/conf/angel-site.xml +++ b/angel-ps/conf/angel-site.xml @@ -18,7 +18,7 @@ angel.job.libjars - ${ANGEL_HOME}/lib/jniloader-1.1.jar,${ANGEL_HOME}/lib/native_system-java-1.1.jar,${ANGEL_HOME}/lib/arpack_combined_all-0.1.jar,${ANGEL_HOME}/lib/all-1.1.2.pom,${ANGEL_HOME}/lib/core-1.1.2.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/jackson-annotations-2.6.5.jar,${ANGEL_HOME}/lib/jackson-core-2.7.7.jar,${ANGEL_HOME}/lib/jackson-core-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-databind-2.7.7.jar,${ANGEL_HOME}/lib/jackson-jaxrs-1.9.13.jar,${ANGEL_HOME}/lib/jackson-mapper-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-module-paranamer-2.6.5.jar,${ANGEL_HOME}/lib/jackson-module-scala_2.11-2.6.5.jar,${ANGEL_HOME}/lib/jackson-xc-1.9.13.jar,${ANGEL_HOME}/lib/json4s-ast_2.11-3.2.11.jar,${ANGEL_HOME}/lib/json4s-core_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-scalap_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-jackson_2.11-3.5.3.jar,${ANGEL_HOME}/lib/netty-all-4.1.17.Final.jar,${ANGEL_HOME}/lib/angel-ps-mllib-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-tools-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/scala-reflect-2.11.8.jar,${ANGEL_HOME}/lib/memory-0.8.1.jar,${ANGEL_HOME}/lib/sketches-core-0.8.1.jar,${ANGEL_HOME}/lib/commons-pool-1.6.jar,${ANGEL_HOME}/lib/kryo-shaded-4.0.0.jar,${ANGEL_HOME}/lib/kryo-serializers-0.42.jar,${ANGEL_HOME}/lib/scala-library-2.11.8.jar,${ANGEL_HOME}/lib/angel-ps-core-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-psf-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/fastutil-7.1.0.jar,${ANGEL_HOME}/lib/sizeof-0.3.0.jar,${ANGEL_HOME}/lib/minlog-1.3.0.jar,${ANGEL_HOME}/lib/breeze_2.11-0.13.jar,${ANGEL_HOME}/lib/angel-format-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/angel-math-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/angel-mlcore-0.1.0-SNAPSHOT.jar,${ANGEL_HOME}/lib/libopenblas.so,${ANGEL_HOME}/lib/commons-math-2.2.jar + ${ANGEL_HOME}/lib/jniloader-1.1.jar,${ANGEL_HOME}/lib/native_system-java-1.1.jar,${ANGEL_HOME}/lib/arpack_combined_all-0.1.jar,${ANGEL_HOME}/lib/all-1.1.2.pom,${ANGEL_HOME}/lib/core-1.1.2.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-armhf-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-i686-1.1-natives.jar,${ANGEL_HOME}/lib/netlib-native_system-linux-x86_64-1.1-natives.jar,${ANGEL_HOME}/lib/jackson-annotations-2.6.5.jar,${ANGEL_HOME}/lib/jackson-core-2.7.7.jar,${ANGEL_HOME}/lib/jackson-core-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-databind-2.7.7.jar,${ANGEL_HOME}/lib/jackson-jaxrs-1.9.13.jar,${ANGEL_HOME}/lib/jackson-mapper-asl-1.9.13.jar,${ANGEL_HOME}/lib/jackson-module-paranamer-2.6.5.jar,${ANGEL_HOME}/lib/jackson-module-scala_2.11-2.6.5.jar,${ANGEL_HOME}/lib/jackson-xc-1.9.13.jar,${ANGEL_HOME}/lib/json4s-ast_2.11-3.2.11.jar,${ANGEL_HOME}/lib/json4s-core_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-scalap_2.11-3.5.3.jar,${ANGEL_HOME}/lib/json4s-jackson_2.11-3.5.3.jar,${ANGEL_HOME}/lib/netty-all-4.1.17.Final.jar,${ANGEL_HOME}/lib/angel-ps-mllib-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-tools-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/scala-reflect-2.11.8.jar,${ANGEL_HOME}/lib/memory-0.8.1.jar,${ANGEL_HOME}/lib/sketches-core-0.8.1.jar,${ANGEL_HOME}/lib/commons-pool-1.6.jar,${ANGEL_HOME}/lib/kryo-shaded-4.0.0.jar,${ANGEL_HOME}/lib/kryo-serializers-0.42.jar,${ANGEL_HOME}/lib/scala-library-2.11.8.jar,${ANGEL_HOME}/lib/angel-ps-core-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/angel-ps-psf-${ANGEL_VERSION}.jar,${ANGEL_HOME}/lib/fastutil-7.1.0.jar,${ANGEL_HOME}/lib/sizeof-0.3.0.jar,${ANGEL_HOME}/lib/minlog-1.3.0.jar,${ANGEL_HOME}/lib/breeze_2.11-0.13.jar,${ANGEL_HOME}/lib/angel-format-0.1.1.jar,${ANGEL_HOME}/lib/angel-math-0.1.1.jar,${ANGEL_HOME}/lib/angel-mlcore-0.1.1.jar,${ANGEL_HOME}/lib/libopenblas.so,${ANGEL_HOME}/lib/commons-math-2.2.jar diff --git a/angel-ps/core/pom.xml b/angel-ps/core/pom.xml index 29ff7146e..a14ed9356 100644 --- a/angel-ps/core/pom.xml +++ b/angel-ps/core/pom.xml @@ -64,22 +64,6 @@ kubernetes-client 3.0.0 - - com.tencent.angel - angel-math - - 0.1.0-SNAPSHOT - - - com.tencent.angel - angel-format - 0.1.0-SNAPSHOT - - - com.tencent.angel - angel-format - 0.1.0-SNAPSHOT - diff --git a/angel-ps/mllib/pom.xml b/angel-ps/mllib/pom.xml index 0db890d65..ce96d643a 100644 --- a/angel-ps/mllib/pom.xml +++ b/angel-ps/mllib/pom.xml @@ -60,7 +60,7 @@ com.tencent.angel angel-mlcore - 0.1.0-SNAPSHOT + 0.1.1 org.json4s diff --git a/angel-ps/pom.xml b/angel-ps/pom.xml index f5008338a..487afc680 100644 --- a/angel-ps/pom.xml +++ b/angel-ps/pom.xml @@ -49,12 +49,12 @@ com.tencent.angel angel-math - 0.1.0-SNAPSHOT + 0.1.1 com.tencent.angel angel-format - 0.1.0-SNAPSHOT + 0.1.1 com.tencent.angel diff --git a/docs/algo/sona/line_sona_en.md b/docs/algo/sona/line_sona_en.md index a47aad0ec..bc82f51ce 100644 --- a/docs/algo/sona/line_sona_en.md +++ b/docs/algo/sona/line_sona_en.md @@ -93,6 +93,6 @@ $SPARK_HOME/bin/spark-submit \ --executor-cores 4 \ --executor-memory 10g \ --class org.apache.spark.angel.examples.graph.LINEExample2 \ - ./lib/angelml-0.1.0-SNAPSHOT.jar + ./lib/angelml-0.1.1.jar input:$input output:$output embedding:128 negative:5 epoch:100 stepSize:0.01 batchSize:1000 numParts:2 subSample:false remapping:false order:2 interval:5 ``` \ No newline at end of file From 6e385a50622ac43f83953838e90c6083d1a92d2f Mon Sep 17 00:00:00 2001 From: paynie Date: Tue, 20 Aug 2019 14:33:45 +0800 Subject: [PATCH 20/20] fix UT --- .../mllib/src/test/java/com/tencent/angel/ml/afm/AFMTest.java | 2 +- .../mllib/src/test/java/com/tencent/angel/ml/dcn/DCNTest.java | 2 +- .../java/com/tencent/angel/ml/deepandwide/WideDeepTest.java | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/afm/AFMTest.java b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/afm/AFMTest.java index c389a056c..0cbc09c77 100644 --- a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/afm/AFMTest.java +++ b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/afm/AFMTest.java @@ -64,7 +64,7 @@ private void setSystemConf() throws Exception { conf.setInt(AngelConf.ANGEL_PS_NUMBER, 1); //String savePath = LOCAL_FS + TMP_PATH + "/model/wideDeep"; - String savePath = "file:///E:/model/AFM"; + String savePath = LOCAL_FS + TMP_PATH + "/model/AFM"; String logPath = LOCAL_FS + TMP_PATH + "/AFMlog"; // Set save model path conf.set(AngelConf.ANGEL_SAVE_MODEL_PATH, savePath); diff --git a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/dcn/DCNTest.java b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/dcn/DCNTest.java index 4bf76ee20..797f0bdf2 100644 --- a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/dcn/DCNTest.java +++ b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/dcn/DCNTest.java @@ -64,7 +64,7 @@ private void setSystemConf() throws Exception { conf.setInt(AngelConf.ANGEL_PS_NUMBER, 1); //String savePath = LOCAL_FS + TMP_PATH + "/model/wideDeep"; - String savePath = "file:///E:/model/DCN"; + String savePath = LOCAL_FS + TMP_PATH + "/model/DCN"; String logPath = LOCAL_FS + TMP_PATH + "/DCNlog"; // Set save model path conf.set(AngelConf.ANGEL_SAVE_MODEL_PATH, savePath); diff --git a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/deepandwide/WideDeepTest.java b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/deepandwide/WideDeepTest.java index 18606818f..9888bda6d 100644 --- a/angel-ps/mllib/src/test/java/com/tencent/angel/ml/deepandwide/WideDeepTest.java +++ b/angel-ps/mllib/src/test/java/com/tencent/angel/ml/deepandwide/WideDeepTest.java @@ -65,7 +65,7 @@ private void setSystemConf() throws Exception { conf.setInt(AngelConf.ANGEL_PS_NUMBER, 1); //String savePath = LOCAL_FS + TMP_PATH + "/model/wideDeep"; - String savePath = "file:///E:/model/wideDeep"; + String savePath = LOCAL_FS + TMP_PATH + "/model/wideDeep"; String logPath = LOCAL_FS + TMP_PATH + "/wideDeeplog"; // Set save model path conf.set(AngelConf.ANGEL_SAVE_MODEL_PATH, savePath);