深度学习系列4: 为什么你需要使用嵌入层 #2018

TobiasLee · 2017-08-07T11:41:07Z

close: #1954

TobiasLee · 2017-08-07T11:41:33Z

@sqrthree 翻译完成

LJ147 · 2017-08-09T03:48:56Z

校对认领

linhe0x0 · 2017-08-09T03:48:59Z

@LJ147 好的呢 🍺

LJ147 · 2017-08-09T18:46:32Z

TODO/deep-learning-4-embedding-layers.md

-1. Content-based filtering. This type of filtering is based on data about the item/product. For example, we have our users fill out a survey on what movies they like. If they say that they like sci-fi movies we recommend them sci-fi movies. In this case al lot of meta-information has to be available for all items.
-2. Collaborative filtering: Let’s find other people like you, see what they liked and assume you like the same things. People like you = people who rated movies that you watched in a similar way. In a large dataset this has proven to work a lot better than the meta-data approach. Essentially asking people about their behavior is less good compared to looking at their actual behavior. Discussing this further is something for the psychologists among us.
+1. 基于内容过滤，这种过滤是基于物品或者是产品的数据的。比如说，我们让用户填一份他们喜欢的电影的表格。如果他们说喜欢科幻电影，那么我们就给他推荐科幻电影。这种方法需要大量的对应产品的元数据。
+2. 协同过滤：找到像你的人，假设你们的爱好相同，看看他们喜欢什么。像你的人，以为着他们对你看过的电影有着相似的评价。在大数据集中，这个方法和元数据方法相比，效果更好。另外一点很重要的是，询问用户他们的行为和观察他们实际的行为之间是有出入的，这其中更深沉次的原因需要心理学家来解释。


协同过滤：找到像你的人，假设你们的爱好相同，看看他们喜欢什么。像你的人，以为着他们对你看过的电影有着相似的评价。 ->
协同过滤：找到和你相似的人，假设你们的爱好相同，看看他们喜欢什么。和你相似的人，以为着他们对你看过的电影有着相似的评价。

LJ147 · 2017-08-09T19:03:07Z

TODO/deep-learning-4-embedding-layers.md

-3. [More on CNNs & Handling Overfitting](https://medium.com/towards-data-science/deep-learning-3-more-on-cnns-handling-overfitting-2bd5d99abe5d)
+1. [设置 AWS & 图像识别](https://github.com/xitu/gold-miner/blob/master/TODO/deep-learning-1-setting-up-aws-image-recognition.md)
+2. [卷积神经网络](https://github.com/xitu/gold-miner/blob/master/TODO/deep-learning-2-convolutional-neural-networks.md)
+3. [More on CNNs & Handling Overfitting](https://github.com/xitu/gold-miner/blob/master/TODO/deep-learning-3-more-on-cnns-handling-overfitting.md)


More on CNNs & Handling Overfitting ->

CNNs 以及应对过拟合的详细探讨

LJ147 · 2017-08-09T19:05:46Z

TODO/deep-learning-4-embedding-layers.md


-Upon introduction the concept of the embedding layer can be quite foreign. For example, the Keras documentation provides no explanation other than “Turns positive integers (indexes) into dense vectors of fixed size”. A quick Google search might not get you much further either since these type of documentations are the first things to pop-up. However, in a sense Keras’ documentation describes all that happens. So why should you use an embedding layer? Here are the two main reasons:
+介绍嵌入层这个概念可能会让它变得更加难以理解。比如说，Keras 文档里是这么写的：“把正整数（索引）转换为固定大小的密集向量”。你 Google 搜索也无济于事，因为文档通常会排在搜索结果的第一个。然而，从某种意义上来说，Keras 文档的描述是正确的。那么为什么你应该使用嵌入层呢？这里有两个主要原因：


你 Google 搜索也无济于事，因为文档通常会排在搜索结果的第一个 ->
你 Google 搜索也无济于事，因为文档通常会排在搜索结果的第一个（你还是不能获得更加详细的解释）

LJ147

一校完成

LJ147 · 2017-08-09T19:06:27Z

@sqrthree 一校完成

linhe0x0

LGTM

changkun · 2017-08-10T14:08:28Z

@sqrthree 校对认领

linhe0x0 · 2017-08-10T14:08:30Z

@changkun 妥妥哒 🍻

changkun

@lileizhenshuai 校对完毕

译文质量很高，只有个别极小的翻译意见。

changkun · 2017-08-10T14:10:40Z

TODO/deep-learning-4-embedding-layers.md


-Embedding layers can even be used to deal with the sparse matrix problem in recommender systems. Since the deep learning course (fast.ai) uses recommender systems to introduce embedding layers I want to explore them here as well.
+嵌入层甚至可以在稀疏的矩阵问题中，比如推荐系统，得到应用。在 fast.ai 的深度学习课程中利用推荐系统来介绍了嵌入层，这里我也想谈谈推荐系统。


「嵌入层甚至可以在稀疏的矩阵问题中，比如推荐系统，得到应用。」=> 「嵌入层甚至可以用来处理推荐系统中的系数矩阵问题。」

changkun · 2017-08-10T14:21:42Z

TODO/deep-learning-4-embedding-layers.md


-Upon introduction the concept of the embedding layer can be quite foreign. For example, the Keras documentation provides no explanation other than “Turns positive integers (indexes) into dense vectors of fixed size”. A quick Google search might not get you much further either since these type of documentations are the first things to pop-up. However, in a sense Keras’ documentation describes all that happens. So why should you use an embedding layer? Here are the two main reasons:
+介绍嵌入层这个概念可能会让它变得更加难以理解。比如说，Keras 文档里是这么写的：“把正整数（索引）转换为固定大小的密集向量”。你 Google 搜索也无济于事，因为文档通常会排在搜索结果的第一个（你还是不能获得更加详细的解释）。然而，从某种意义上来说，Keras 文档的描述是正确的。那么为什么你应该使用嵌入层呢？这里有两个主要原因：


「把正整数（索引）转换为固定大小的密集向量」=>「把正整数（索引）转换为固定长度的的稠密向量」

changkun · 2017-08-10T14:31:55Z

TODO/deep-learning-4-embedding-layers.md


 ---

-Welcome to part 4 of this series on deep learning. As you might have noticed there has been a slight delay between the first three entries and this post. The initial goal of this series was to write along with the fast.ai course on deep learning. However, the concepts of the later lectures are often overlapping so I decided to finish the course first. This way I get to provide a more detailed overview of these topics. In this blog I want to cover a concept that spans multiple lectures of the course (4–6) and that has proven very useful to me in practice:Embedding Layers.
+欢迎阅读深度学习系列的第四部分，你可能注意到这一篇和前面三篇文章之间隔了一小段时间。我写这个系列最初的目的是想记录 fast.ai 上的深度学习课程，然而后面一部分课程的内容有所重叠，所以我决定先把课程完成，这样我能够更为细致地从整体上来把握这些主题。在这篇文章里我想介绍一个在很多节课中都有涉及，并且在实践中被证明非常有用的一个概念：嵌入层。


「嵌入层」=>「嵌入层（Embedding layer）」
尽管此译法已经很常用了，但在首次出现的时候注明一下主题相关的概念性英文名词是有必要的

changkun · 2017-08-10T14:32:39Z

TODO/deep-learning-4-embedding-layers.md


-1. One-hot encoded vectors are high-dimensional and sparse. Let’s assume that we are doing Natural Language Processing (NLP) and have a dictionary of 2000 words. This means that, when using one-hot encoding, each word will be represented by a vector containing 2000 integers. And 1999 of these integers are zeros. In a big dataset this approach is not computationally efficient.
-2. The vectors of each embedding get updated while training the neural network. If you have seen the image at the top of this post you can see how similarities between words can be found in a multi-dimensional space. This allows us to visualize relationships between words, but also between everything that can be turned into a vector through an embedding layer.
+1. 独热编码（one-hot encoding）向量是高维且稀疏的。假如说我们在做自然语言处理（NLP）的工作，并且有一个包含 2000 个单词的字典。这意味着当我们使用独热编码时，每个单词由一个含有 2000 个整数的向量来表示，并且其中的 1999 个整数都是 0。在大数据集下这种方法的计算效率是很低的。


「独热编码（one-hot encoding）」=>「独热编码（One-hot encoding）」

changkun · 2017-08-10T14:41:33Z

TODO/deep-learning-4-embedding-layers.md


-This concept might still be a bit vague. Let’s have a look at what an embedding layer does with an example of words. Nevertheless, the origin of embeddings comes from word embeddings. You can look up [word2vec](https://arxiv.org/pdf/1301.3781.pdf) if you are interested in reading more. Let’s take this sentence as an example (do not take it to seriously):
+嵌入层这个概念可能还是有点模糊，让我们再来看嵌入层作用在词语上的一个例子。嵌入这个概念源自于词嵌入，如果你想了解更多的话可以看看 [word2vec](https://arxiv.org/pdf/1301.3781.pdf) 这篇论文。让我们用下面这个句子作为例子（随便举的一个例子，不用太认真对待）：


「词嵌入」=>「词嵌入（Word embedding）」

changkun · 2017-08-10T14:42:46Z

TODO/deep-learning-4-embedding-layers.md


-These previous examples showed that word embeddings are very important in the world of Natural Language Processing. They allow us to capture relationships in language that are very difficult to capture otherwise. However, embedding layers can be used to embed many more things than just words. In my current research project I am using embedding layers to embed online user behavior. In this case I am assigning indices to user behavior like ‘page view on page type X on portal Y’ or ‘scrolled X pixels’. These indices are then used for constructing a sequence of user behavior.
+前面的例子体现了词嵌入在自然语言处理领域重要的地位，它能够帮助我们找到很难察觉的词语之间的关系。然而，嵌入层不仅仅可以用在词语上。在我现在的一个研究项目里，我用嵌入层来嵌入网上用户的使用行为：我为用户行为分配索引，比如“浏览类型 X 的网页（在门户 Y 上）”或“滚动了X像素”。然后，用这些索引构建用户的行为序列。


「滚动了X像素」=>「滚动了 X 像素」

changkun · 2017-08-10T14:46:02Z

TODO/deep-learning-4-embedding-layers.md


-1. Content-based filtering. This type of filtering is based on data about the item/product. For example, we have our users fill out a survey on what movies they like. If they say that they like sci-fi movies we recommend them sci-fi movies. In this case al lot of meta-information has to be available for all items.
-2. Collaborative filtering: Let’s find other people like you, see what they liked and assume you like the same things. People like you = people who rated movies that you watched in a similar way. In a large dataset this has proven to work a lot better than the meta-data approach. Essentially asking people about their behavior is less good compared to looking at their actual behavior. Discussing this further is something for the psychologists among us.
+1. 基于内容过滤，这种过滤是基于物品或者是产品的数据的。比如说，我们让用户填一份他们喜欢的电影的表格。如果他们说喜欢科幻电影，那么我们就给他推荐科幻电影。这种方法需要大量的对应产品的元数据。


「基于内容过滤，」=>「基于内容过滤：」

TobiasLee · 2017-08-11T01:24:20Z

@sqrthree 修改完毕

linhe0x0 · 2017-08-11T12:53:14Z

TODO/deep-learning-4-embedding-layers.md


-This concept might still be a bit vague. Let’s have a look at what an embedding layer does with an example of words. Nevertheless, the origin of embeddings comes from word embeddings. You can look up [word2vec](https://arxiv.org/pdf/1301.3781.pdf) if you are interested in reading more. Let’s take this sentence as an example (do not take it to seriously):
+嵌入层这个概念可能还是有点模糊，让我们再来看嵌入层作用在词语上的一个例子。嵌入这个概念源自于词嵌入（Word embedding），如果你想了解更多的话可以看看 [word2vec](https://arxiv.org/pdf/1301.3781.pdf) 这篇论文。让我们用下面这个句子作为例子（随便举的一个例子，不用太认真对待）：

 > “deep learning is very deep”


这句是故意保留的吗？

TobiasLee · 2017-08-14T13:41:49Z

已发布 https://juejin.im/post/599183c6f265da3e2e5717d2

Update deep-learning-4-embedding-layers.md

e1dea60

linhe0x0 added the 校对认领 label Aug 7, 2017

linhe0x0 mentioned this pull request Aug 7, 2017

深度学习系列4: 为什么你需要使用嵌入层 #1954

Closed

linhe0x0 added the AI label Aug 8, 2017

linhe0x0 added the 正在校对 label Aug 9, 2017

LJ147 reviewed Aug 9, 2017

View reviewed changes

根据一校意见修改

a5267c0

linhe0x0 reviewed Aug 10, 2017

View reviewed changes

linhe0x0 removed the 校对认领 label Aug 10, 2017

changkun reviewed Aug 10, 2017

View reviewed changes

根据二校意见修改

354c42c

linhe0x0 reviewed Aug 11, 2017

View reviewed changes

Update deep-learning-4-embedding-layers.md

fefdb08

linhe0x0 approved these changes Aug 14, 2017

View reviewed changes

linhe0x0 merged commit 1198f0f into xitu:master Aug 15, 2017

linhe0x0 added 翻译完成 and removed 正在校对 labels Aug 15, 2017

cdadar pushed a commit to cdadar/gold-miner that referenced this pull request Dec 8, 2017

深度学习系列4: 为什么你需要使用嵌入层 (xitu#2018)

92cda83

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

深度学习系列4: 为什么你需要使用嵌入层 #2018

深度学习系列4: 为什么你需要使用嵌入层 #2018

TobiasLee commented Aug 7, 2017 •

edited by linhe0x0

Loading

TobiasLee commented Aug 7, 2017

LJ147 commented Aug 9, 2017

linhe0x0 commented Aug 9, 2017

LJ147 Aug 9, 2017 •

edited

Loading

LJ147 Aug 9, 2017 •

edited

Loading

LJ147 Aug 9, 2017

LJ147 left a comment

LJ147 commented Aug 9, 2017

linhe0x0 left a comment

changkun commented Aug 10, 2017

linhe0x0 commented Aug 10, 2017

changkun left a comment

changkun Aug 10, 2017

changkun Aug 10, 2017

changkun Aug 10, 2017

changkun Aug 10, 2017

changkun Aug 10, 2017

changkun Aug 10, 2017

changkun Aug 10, 2017

TobiasLee commented Aug 11, 2017

linhe0x0 Aug 11, 2017

TobiasLee commented Aug 14, 2017


		Upon introduction the concept of the embedding layer can be quite foreign. For example, the Keras documentation provides no explanation other than “Turns positive integers (indexes) into dense vectors of fixed size”. A quick Google search might not get you much further either since these type of documentations are the first things to pop-up. However, in a sense Keras’ documentation describes all that happens. So why should you use an embedding layer? Here are the two main reasons:
		介绍嵌入层这个概念可能会让它变得更加难以理解。比如说，Keras 文档里是这么写的：“把正整数（索引）转换为固定大小的密集向量”。你 Google 搜索也无济于事，因为文档通常会排在搜索结果的第一个。然而，从某种意义上来说，Keras 文档的描述是正确的。那么为什么你应该使用嵌入层呢？这里有两个主要原因：


		Embedding layers can even be used to deal with the sparse matrix problem in recommender systems. Since the deep learning course (fast.ai) uses recommender systems to introduce embedding layers I want to explore them here as well.
		嵌入层甚至可以在稀疏的矩阵问题中，比如推荐系统，得到应用。在 fast.ai 的深度学习课程中利用推荐系统来介绍了嵌入层，这里我也想谈谈推荐系统。


		This concept might still be a bit vague. Let’s have a look at what an embedding layer does with an example of words. Nevertheless, the origin of embeddings comes from word embeddings. You can look up [word2vec](https://arxiv.org/pdf/1301.3781.pdf) if you are interested in reading more. Let’s take this sentence as an example (do not take it to seriously):
		嵌入层这个概念可能还是有点模糊，让我们再来看嵌入层作用在词语上的一个例子。嵌入这个概念源自于词嵌入，如果你想了解更多的话可以看看 [word2vec](https://arxiv.org/pdf/1301.3781.pdf) 这篇论文。让我们用下面这个句子作为例子（随便举的一个例子，不用太认真对待）：


		These previous examples showed that word embeddings are very important in the world of Natural Language Processing. They allow us to capture relationships in language that are very difficult to capture otherwise. However, embedding layers can be used to embed many more things than just words. In my current research project I am using embedding layers to embed online user behavior. In this case I am assigning indices to user behavior like ‘page view on page type X on portal Y’ or ‘scrolled X pixels’. These indices are then used for constructing a sequence of user behavior.
		前面的例子体现了词嵌入在自然语言处理领域重要的地位，它能够帮助我们找到很难察觉的词语之间的关系。然而，嵌入层不仅仅可以用在词语上。在我现在的一个研究项目里，我用嵌入层来嵌入网上用户的使用行为：我为用户行为分配索引，比如“浏览类型 X 的网页（在门户 Y 上）”或“滚动了X像素”。然后，用这些索引构建用户的行为序列。

深度学习系列4: 为什么你需要使用嵌入层 #2018

深度学习系列4: 为什么你需要使用嵌入层 #2018

Conversation

TobiasLee commented Aug 7, 2017 • edited by linhe0x0 Loading

TobiasLee commented Aug 7, 2017

LJ147 commented Aug 9, 2017

linhe0x0 commented Aug 9, 2017

LJ147 Aug 9, 2017 • edited Loading

Choose a reason for hiding this comment

LJ147 Aug 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LJ147 left a comment

Choose a reason for hiding this comment

LJ147 commented Aug 9, 2017

linhe0x0 left a comment

Choose a reason for hiding this comment

changkun commented Aug 10, 2017

linhe0x0 commented Aug 10, 2017

changkun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TobiasLee commented Aug 11, 2017

Choose a reason for hiding this comment

TobiasLee commented Aug 14, 2017

TobiasLee commented Aug 7, 2017 •

edited by linhe0x0

Loading

LJ147 Aug 9, 2017 •

edited

Loading

LJ147 Aug 9, 2017 •

edited

Loading