Release Distribution base class and Add Normal, Uniform class #26355

pangyoki · 2020-08-17T08:30:33Z

PR types

Others

PR changes

APIs

Describe

Implement Probability Distribution related classes.

Release Distribution abstract base class and its sample, log_prob, entropy, probs methods.
Move Normal class and Uniform class in paddle/fluid/layers/distributions.py path to paddle/distribution.py.
Add parameter name of each class API. name is initialized in __init__ function, and every function of the class has its own name.
Add probs method for Normal class and Uniform class.

LutaoChu · 2020-08-19T04:09:30Z

python/paddle/distribution.py

+    broadcasting (e.g., `high - low` is a valid operation).
+
+    Args:
+        low(float|list|numpy.ndarray|Variable): The lower boundary of uniform distribution.The data type is float32


Variable -》 Tensor ，the following is the same

pytorch文档没写支持int，但实际是支持的。考虑int的情况较常用，建议也支持一下

LutaoChu · 2020-08-19T04:11:08Z

python/paddle/distribution.py

+    * :math:`Z`: is the normalizing constant.
+
+    The parameters `low` and `high` must be shaped in a way that supports
+    broadcasting (e.g., `high - low` is a valid operation).


加一个broadcast semantic的链接

LutaoChu · 2020-08-19T04:12:46Z

python/paddle/distribution.py

+        .. code-block:: python
+
+          import numpy as np
+          from paddle.fluid import layers


避免出现paddle.fluid

LutaoChu · 2020-08-19T04:13:25Z

python/paddle/distribution.py

+          # Complete example
+          value_npdata = np.array([0.8], dtype="float32")
+          value_tensor = layers.create_tensor(dtype="float32")
+          layers.assign(value_npdata, value_tensor)


layers接口替换为新版本的paddle接口

Done, use paddle.to_tensor method

LutaoChu · 2020-08-19T04:34:20Z

python/paddle/distribution.py

+                zero_tmp, zero_tmp.shape, min=0., max=1., seed=seed)
+            output = uniform_random_tmp * (zero_tmp + self.high - self.low
+                                           ) + self.low
+            return nn.reshape(output, output_shape)


return应该要加个name吧，return nn.reshape(output, output_shape, name=name)

LutaoChu · 2020-08-19T06:16:20Z

python/paddle/distribution.py

+
+        """
+        name = self.name + '_probs'
+        return ops.exp(self.log_prob(value))


建议直接用概率定义计算

LutaoChu · 2020-08-19T06:17:50Z

python/paddle/distribution.py

+    * :math:`Z`: is the normalization constant.
+
+    Args:
+        loc(float|list|numpy.ndarray|Variable): The mean of normal distribution.The data type is float32.


同Uniform，Variable -》Tensor，建议支持int

LutaoChu · 2020-08-19T06:45:00Z

python/paddle/distribution.py

+          # Complete example
+          value_npdata = np.array([0.8], dtype="float32")
+          value_tensor = layers.create_tensor(dtype="float32")
+          layers.assign(value_npdata, value_tensor)


value_tensor建议用paddle.to_tensor创建

LutaoChu · 2020-08-19T06:48:04Z

python/paddle/distribution.py

+
+        """
+        name = self.name + '_probs'
+        return ops.exp(self.log_prob(value))


同Uniform，建议直接计算pdf

LutaoChu · 2020-08-19T07:17:00Z

python/paddle/fluid/tests/unittests/test_distribution.py

+        self.assertEqual(uniform1.name, name)
+
+        uniform2 = Uniform(0.0, 1.0)
+        self.assertEqual(uniform2.name, 'Uniform')


补充method name的单测用例

LutaoChu · 2020-08-20T08:31:50Z

python/paddle/distribution.py

+    [broadcasting](https://www.paddlepaddle.org.cn/documentation/docs/en/develop/beginners_guide/basic_concept/broadcasting_en.html) (e.g., `high - low` is a valid operation).
+
+    Args:
+        low(int|float|list|numpy.ndarray|Tensor): The lower boundary of uniform distribution.The data type is float32 or int


漏了list|numpy.ndarray|Tensor

Done, refer to PR #26535

LutaoChu · 2020-08-20T08:32:04Z

python/paddle/distribution.py

+
+    Args:
+        low(int|float|list|numpy.ndarray|Tensor): The lower boundary of uniform distribution.The data type is float32 or int
+        high(int|float|list|numpy.ndarray|Tensor): The higher boundary of uniform distribution.The data type is float32 or int


Done, refer to PR #26535

LutaoChu · 2020-08-20T08:47:40Z

python/paddle/distribution.py

+            lb = tensor.cast(lb_bool, dtype=value.dtype)
+            ub = tensor.cast(ub_bool, dtype=value.dtype)
+            return elementwise_sub(
+                nn.log(lb * ub), nn.log(self.high - self.low), name=name)


动态图不需要加name，cast和elementwise_sub最好用core.ops

Done, refer to PR #26535

LutaoChu · 2020-08-20T08:48:02Z

python/paddle/distribution.py

+            ub_bool = value < self.high
+            lb = tensor.cast(lb_bool, dtype=value.dtype)
+            ub = tensor.cast(ub_bool, dtype=value.dtype)
+            return elementwise_div((lb * ub), (self.high - self.low), name=name)


Done, refer to PR #26535

LutaoChu · 2020-08-20T09:30:02Z

python/paddle/distribution.py

+
+    Mathematical details
+
+    The probability density function (pdf) is,


去掉逗号

Done, refer to PR #26535

LutaoChu · 2020-08-20T09:30:39Z

python/paddle/distribution.py

+
+    Args:
+        loc(int|float|list|numpy.ndarray|Tensor): The mean of normal distribution.The data type is float32 or int.
+        scale(int|float|list|numpy.ndarray|Tensor): The std of normal distribution.The data type is float32 or int.


data type缺少list|numpy.ndarray|Tensor

Done, refer to PR #26535

XiaoguangHu01

LGTM

LutaoChu

LGTM

saxon-zh

LGTM

pangyoki force-pushed the add_Distribution_class_branch branch 4 times, most recently from e9443c2 to 97ed688 Compare August 18, 2020 17:55

LutaoChu reviewed Aug 19, 2020

View reviewed changes

pangyoki force-pushed the add_Distribution_class_branch branch 2 times, most recently from 30b37f2 to 8232ad8 Compare August 19, 2020 09:36

pangyoki added 16 commits August 19, 2020 13:17

fixed static module

76d505e

solve conflict

1dfd7d0

Add Distribution base class, Uniform class and Normal class

070370c

release Distribution class and Normal, Uniform class

ee1f7cf

Add Doc args explaination

d14307e

save distributions.py and process in distribution.py

62fd09d

delete useless function in test_distribution

b01bb5f

Add NormalNumpy test class

6738485

Add probs in NormalNumpy

5615fc6

add distribution to paddle init

29fc2c3

Add Distribution base class and name attribute unittest

cc01f37

Change Doc

3db669f

Change Doc

bed23af

adjust format

7e529e7

Fixed Doc Code

48b7663

implement probs and change Variable to Tensor

8188031

pangyoki force-pushed the add_Distribution_class_branch branch from 8232ad8 to 8188031 Compare August 19, 2020 13:17

pangyoki added 4 commits August 19, 2020 17:14

Add name for all functions and add name unittest

5f16898

support int datatype

e49fa5d

Add dynamic mode

cbf6452

optimize test_distribution static and dygraph

a0324e2

LutaoChu reviewed Aug 20, 2020

View reviewed changes

XiaoguangHu01 approved these changes Aug 21, 2020

View reviewed changes

raindrops2sea approved these changes Aug 21, 2020

View reviewed changes

zhiqiu self-requested a review August 21, 2020 02:37

zhiqiu approved these changes Aug 21, 2020

View reviewed changes

LutaoChu approved these changes Aug 21, 2020

View reviewed changes

saxon-zh approved these changes Aug 21, 2020

View reviewed changes

zhiqiu merged commit 644dfd7 into PaddlePaddle:develop Aug 21, 2020

pangyoki mentioned this pull request Aug 21, 2020

fix Distribution class problem #26535

Merged

pangyoki mentioned this pull request Aug 31, 2020

fix dtype not matching bug in log_prob and probs method of Distribution class #26767

Merged

pangyoki changed the title ~~Release Distribution base class and Normal, Uniform class~~ Release Distribution base class and Add Normal, Uniform class Oct 16, 2020


		Mathematical details

		The probability density function (pdf) is,

Release Distribution base class and Add Normal, Uniform class #26355

Release Distribution base class and Add Normal, Uniform class #26355

Conversation

pangyoki commented Aug 17, 2020 • edited Loading

PR types

PR changes

Describe

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

LutaoChu left a comment

Choose a reason for hiding this comment

saxon-zh left a comment

Choose a reason for hiding this comment

pangyoki commented Aug 17, 2020 •

edited

Loading