forked from Haggaim/haggaim.github.io
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
executable file
·433 lines (378 loc) · 36.5 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><![endif]-->
<title>Matan's Homepage</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<!--
<meta name="description" content="Haggai's web page" />
<meta name="author" content="tamarillo" />
-->
<!-- favicons -->
<!-- <link rel="shortcut icon" href="images/templatemo_favicon.ico"> -->
<!-- bootstrap core CSS -->
<link href="css/bootstrap.min.css" rel="stylesheet" />
<!-- fancybox CSS -->
<link href="css/jquery.lightbox.css" rel="stylesheet" />
<!-- flex slider CSS -->
<link href="css/flexslider.css" rel="stylesheet" />
<!-- custom styles for this template -->
<link href="css/templatemo_style.css" rel="stylesheet" />
<!-- HTML5 shim and Respond.js IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
<script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
<![endif]-->
</head>
<body>
<header>
<div class="container">
<div class="row">
<div class="col-md-3 hidden-xs"></div>
<div class="col-xs-3 col-xs-offset-20 visible-xs">
<a href="#" id="mobile_menu"><span class="glyphicon glyphicon-align-justify"></span></a>
</div>
<div class="col-xs-24 visible-xs" id="mobile_menu_list">
<ul>
<li><a href="#templatemo_about">About</a></li>
<!-- <li><a href="#templatemo_slideshow">Slideshow</a></li> -->
<li><a href="#templatemo_publications">Publications</a></li>
<li><a href="/blog/" onclick="location.replace('https://matanatz.github.io/blog'),'_top'">Blog</a></li>
</ul>
</div>
<div class="col-md-16 col-sm-18 hidden-xs" id="templatemo-nav-bar">
<ul class="nav navbar-right">
<li><a href="#templatemo_about">About</a></li>
<!-- <li><a href="#templatemo_slideshow">Slideshow</a></li> -->
<li><a href="#templatemo_publications">Publications</a></li>
</ul>
</div>
</div>
</div>
</header><!-- end of templatemo_header -->
<section id="templatemo_about">
<div class="container">
<div class="row">
<div class="col-md-2"></div>
<div id="my_photo" class="col-md-4 col-sm-7 col-xs-24">
<img src="images/matan.jpg" alt="image 1"/>
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Matan Atzmon</h2>
<p>
I am a research scientist at NVIDIA, part of the <a href=https://nv-tlabs.github.io>Toronto AI Lab</a>.
Prior to that, I completed my Ph.D. studies at the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science under the supervision of <a href="http://www.wisdom.weizmann.ac.il/~ylipman/">Prof. Yaron Lipman</a>.
<br>
My main research interests are developing 3D deep learning methods, learning with weak supervision, generative modeling, and equivariant network design.
<br>
</p>
<br>
<p>
<b>Email:</b> matzmon(at)nvidia(dot)com, <a href="https://scholar.google.com/citations?user=BXNft08AAAAJ&hl=en">Google scholar page</a>, <a href="https://github.com/matanatz">GitHub page</a> <a href="https://twitter.com/matanatzmon">Twitter</a><br>
</p>
</div>
</div><!-- end of row -->
</div>
</section><!-- end of templatemo_about -->
<section id="templatemo_publications">
<div class="container">
<hr>
<div class="row">
<h1>Publications</h1>
</div>
</div>
<!--rematching -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/rematching/intro2.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>ReMatching Dynamic Reconstruction Flow</h2>
<p>
Sara Oblak, Despoina Paschalidou, Sanja Fidler, Matan Atzmon<br>
<i>Preprint</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Reconstructing dynamic scenes from image inputs is a fundamental computer vision task with many downstream applications. Despite recent advancements, existing approaches still struggle to achieve high-quality reconstructions from unseen viewpoints and timestamps. This work introduces the ReMatching framework, designed to improve generalization quality by incorporating deformation priors into dynamic reconstruction models. Our approach advocates for velocity-field-based priors, for which we suggest a matching procedure that can seamlessly supplement existing dynamic reconstruction pipelines. The framework is highly adaptable and can be applied to various dynamic representations. Moreover, it supports integrating multiple types of model priors and enables combining simpler ones to create more complex classes. Our evaluations on popular benchmarks involving both synthetic and real-world dynamic scenes demonstrate a clear improvement in reconstruction accuracy of current state-of-the-art models."> Abstract</a>
<a href="https://arxiv.org/abs/2411.00705" class="btn btn-default">Arxiv </a> <a href="https://research.nvidia.com/labs/toronto-ai/ReMatchingDynamicReconstructionFlow/" class="btn btn-default">Project </a>
</div>
</div><!-- end of row -->
<!--spacemesh -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/spacemesh/teaser.jpg" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes</h2>
<p>
Tianchang Shen, Zhaoshuo Li, Marc Law, Matan Atzmon, Sanja Fidler, James Lucas, Jun Gao, Nicholas Sharp<br>
<i>SIGGRAPH Asia 2024</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Meshes are ubiquitous in visual computing and simulation, yet most existing machine learning techniques represent meshes only indirectly, e.g. as the level set of a scalar field or deformation of a template, or as a disordered triangle soup lacking local structure. This work presents a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh vertex, which implies the discrete mesh. In particular, our vertex embeddings generate cyclic neighbor relationships in a halfedge mesh representation, which gives a guarantee of edge-manifoldness and the ability to represent general polygonal meshes. This representation is well-suited to machine learning and stochastic optimization, without restriction on connectivity or topology. We first explore the basic properties of this representation, then use it to fit distributions of meshes from large datasets. The resulting models generate diverse meshes with tessellation structure learned from the dataset population, with concise details and high-quality mesh elements. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair."> Abstract</a>
<a href="https://arxiv.org/abs/2409.20562" class="btn btn-default">Arxiv </a> <a href="https://research.nvidia.com/labs/toronto-ai/space-mesh/" class="btn btn-default">Project </a>
</div>
</div><!-- end of row -->
<!--apen -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/apen/fig.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Approximately Piecewise E (3) Equivariant Point Networks</h2>
<p>
Matan Atzmon, Jiahui Huang, Francis Williams, Or Litany<br>
<i>International Conference on Learning Representations (ICLR) 2024</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Integrating a notion of symmetry into point cloud neural networks is a provably effective way to improve their generalization capability. Of particular interest are E(3) equivariant point cloud networks where Euclidean transformations applied to the inputs are preserved in the outputs. Recent efforts aim to extend networks that are E(3) equivariant, to accommodate inputs made of multiple parts, each of which exhibits local E(3) symmetry. In practical settings, however, the partitioning into individually transforming regions is unknown a priori. Errors in the partition prediction would unavoidably map to errors in respecting the true input symmetry. Past works have proposed different ways to predict the partition, which may exhibit uncontrolled errors in their ability to maintain equivariance to the actual partition. To this end, we introduce APEN: a general framework for constructing approximate piecewise-E(3) equivariant point networks. Our primary insight is that functions that are equivariant with respect to a finer partition will also maintain equivariance in relation to the true partition. Leveraging this observation, we propose a design where the equivariance approximation error at each layers can be bounded solely in terms of (i) uncertainty quantification of the partition prediction, and (ii) bounds on the probability of failing to suggest a proper subpartition of the ground truth one. We demonstrate the effectiveness of APEN using two data types exemplifying part-based symmetry: (i) real-world scans of room scenes containing multiple furniture-type objects; and, (ii) human motions, characterized by articulated parts exhibiting rigid movement. Our empirical results demonstrate the advantage of integrating piecewise E(3) symmetry into network design, showing a distinct improvement in generalization compared to prior works for both classification and segmentation tasks."> Abstract</a>
<a href="https://arxiv.org/abs/2402.08529" class="btn btn-default">Arxiv </a> <a href="https://research.nvidia.com/labs/toronto-ai/apen/index.html" class="btn btn-default">Project </a> <a href="https://www.youtube.com/watch?v=gHqQMaZZIGQ" class="btn btn-default">Short Video </a> <a href="https://www.youtube.com/watch?v=7apFfjpQA6o" class="btn btn-default">Seminar Recording </a>
</div>
</div><!-- end of row -->
<!--Neural kernel surface reconstruction -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/nksr/nksr.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Neural Kernel Surface Reconstruction</h2>
<p>
Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, Francis Williams<br>
<i>Computer Vision and Pattern Recognition (CVPR) 2023, highlight paper</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="We present a novel method for reconstructing a 3D implicit surface from a large-scale, sparse, and noisy point cloud. Our approach builds upon the recently introduced Neural Kernel Fields (NKF) representation. It enjoys similar generalization capabilities to NKF, while simultaneously addressing its main limitations: (a) We can scale to large scenes through compactly supported kernel functions, which enable the use of memory-efficient sparse linear solvers. (b) We are robust to noise, through a gradient fitting solve. (c) We minimize training requirements, enabling us to learn from any dataset of dense oriented points, and even mix training data consisting of objects and scenes at different scales. Our method is capable of reconstructing millions of points in a few seconds, and handling very large scenes in an out-of-core fashion. We achieve state-of-the-art results on reconstruction benchmarks consisting of single objects, indoor scenes and outdoor scenes."> Abstract</a>
<a href="https://arxiv.org/abs/2305.19590" class="btn btn-default">Arxiv </a> <a href="https://research.nvidia.com/labs/toronto-ai/NKSR/" class="btn btn-default">Project </a> <a href="https://www.youtube.com/watch?v=J1V5F2z-dWY" class="btn btn-default">Video </a>
</div>
</div><!-- end of row -->
<!--phd -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/thesis/img/img.001.jpeg" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Ph.D. Thesis</h2>
<p>
Matan Atzmon<br>
<i>Weizmann Institute of Science, 2022</i> <br>
</p>
<a href="https://drive.google.com/file/d/1OUrtKns2dktl1q6N0fubh0RHWmjYp140/view?usp=share_link" class="btn btn-default">Paper </a>
</div>
</div><!-- end of row -->
<!--equiv shapes -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/equiv_shapes/1.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Frame Averaging for Equivariant Shape Space Learning</h2>
<p>
Matan Atzmon, Koki Nagano, Sanja Fidler, Sameh Khamis, Yaron Lipman<br>
<i>Computer Vision and Pattern Recognition (CVPR) 2022</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="The task of shape space learning involves mapping a train set of shapes to and from a latent representation space with good generalization properties. Often, real-world collections of shapes have symmetries, which can be defined as transformations that do not change the essence of the shape. A natural way to incorporate symmetries in shape space learning is to ask that the mapping to the shape space (encoder) and mapping from the shape space (decoder) are equivariant to the relevant symmetries. In this paper, we present a framework for incorporating equivariance in encoders and decoders by introducing two contributions: (i) adapting the recent Frame Averaging (FA) framework for building generic, efficient, and maximally expressive Equivariant autoencoders; and (ii) constructing autoencoders equivariant to piecewise Euclidean motions applied to different parts of the shape. To the best of our knowledge, this is the first fully piecewise Euclidean equivariant autoencoder construction. Training our framework is simple: it uses standard reconstruction losses, and does not require the introduction of new losses. Our architectures are built of standard (backbone) architectures with the appropriate frame averaging to make them equivariant. Testing our framework on both rigid shapes dataset using implicit neural representations, and articulated shape datasets using mesh-based neural networks show state of the art generalization to unseen test shapes, improving relevant baselines by a large margin. In particular, our method demonstrates significant improvement in generalizing to unseen articulated poses."> Abstract</a>
<a href="https://arxiv.org/abs/2112.01741" class="btn btn-default">Arxiv </a> <a href="https://nv-tlabs.github.io/equivariant" class="btn btn-default">Project </a> <a href="https://www.youtube.com/watch?v=Lft6r5oVyXM" class="btn btn-default">Video </a>
</div>
</div><!-- end of row -->
<!--frame -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/frames/frames.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Frame Averaging for Invariant and Equivariant Network Design</h2>
<p>
Omri Puny*, Matan Atzmon*, Heli Ben-Hamu*, Edward J. Smith, Ishan Misra, Aditya Grover, Yaron Lipman (*equal contribution)<br>
<i>International Conference on Learning Representations (ICLR) 2022, oral presentation</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Many machine learning tasks involve learning functions that are known to be invariant or equivariant to certain symmetries of the input data. However, it is often challenging to design neural network architectures that respect these symmetries while being expressive and computationally efficient. For example, Euclidean motion invariant/equivariant graph or point cloud neural networks. We introduce Frame Averaging (FA), a general purpose and systematic framework for adapting known (backbone) architectures to become invariant or equivariant to new symmetry types. Our framework builds on the well known group averaging operator that guarantees invariance or equivariance but is intractable. In contrast, we observe that for many important classes of symmetries, this operator can be replaced with an averaging operator over a small subset of the group elements, called a frame. We show that averaging over a frame guarantees exact invariance or equivariance while often being much simpler to compute than averaging over the entire group. Furthermore, we prove that FA-based models have maximal expressive power in a broad setting and in general preserve the expressive power of their backbone architectures. Using frame averaging, we propose a new class of universal Graph Neural Networks (GNNs), universal Euclidean motion invariant point cloud networks, and Euclidean motion invariant Message Passing (MP) GNNs. We demonstrate the practical effectiveness of FA on several applications including point cloud normal estimation, beyond 2-WL graph separation, and n-body dynamics prediction, achieving state-of-the-art results in all of these benchmarks."> Abstract</a>
<a href="https://arxiv.org/abs/2110.03336" class="btn btn-default">Arxiv </a>
<a href="https://github.com/omri1348/Frame-Averaging" class="btn btn-default">Code </a>
</div>
</div><!-- end of row -->
<!--augment -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/aug/prob_inset.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields</h2>
<p>
Matan Atzmon, David Novotny, Andrea Vedaldi, Yaron Lipman<br>
<i>Technical report</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Implicit neural representation is a recent approach to learn shape collections as zero level-sets of neural networks, where each shape is represented by a latent code. So far, the focus has been shape reconstruction, while shape generalization was mostly left to generic encoder-decoder or auto-decoder regularization. In this paper we advocate deformation-aware regularization for implicit neural representations, aiming at producing plausible deformations as latent code changes. The challenge is that implicit representations do not capture correspondences between different shapes, which makes it difficult to represent and regularize their deformations. Thus, we propose to pair the implicit representation of the shapes with an explicit, piecewise linear deformation field, learned as an auxiliary function. We demonstrate that, by regularizing these deformation fields, we can encourage the implicit neural representation to induce natural deformations in the learned shape space, such as as-rigid-as-possible deformations."> Abstract</a>
<a href="https://arxiv.org/abs/2108.08931" class="btn btn-default">Arxiv </a>
</div>
</div><!-- end of row -->
<!--sal__ -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/sal++/cars.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>SALD: Sign Agnostic Learning with Derivatives</h2>
<p>
Matan Atzmon and Yaron Lipman<br>
<i> International Conference on Learning Representations (ICLR) 2021 </i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Learning 3D geometry directly from raw data, such as point clouds, triangle soups, or unoriented meshes is still a challenging task that feeds many downstream computer vision and graphics applications. In this paper, we introduce SALD: a method for learning implicit neural representations of shapes directly from raw data. We generalize sign agnostic learning (SAL) to include derivatives: given an unsigned distance function to the input raw data, we advocate a novel sign agnostic regression loss, incorporating both pointwise values and gradients of the unsigned distance function. Optimizing this loss leads to a signed implicit function solution, the zero level set of which is a high quality and valid manifold approximation to the input 3D data. The motivation behind SALD is that incorporating derivatives in a regression loss leads to a lower sample complexity, and consequently better fitting. In addition, we prove that SAL enjoys a minimal length property in 2D, favoring minimal length solutions. More importantly, we are able to show that this property still holds for SALD, i.e., with derivatives included. We demonstrate the efficacy of SALD for shape space learning on two challenging datasets: ShapeNet that contains inconsistent orientation and non-manifold meshes, and D-Faust that contains raw 3D scans (triangle soups). On both these datasets, we present state-of-the-art results."> Abstract</a>
<a href="https://arxiv.org/abs/2006.05400" class="btn btn-default">Arxiv </a>
<a href="https://github.com/matanatz/SALD"class="btn btn-default">Code</a>
<a href="https://www.youtube.com/watch?v=Q1QMcsukN4k" class="btn btn-default"> Video </a>
</div>
</div><!-- end of row -->
</div>
<!--iso -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/iso/iso.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Isometric Autoencoders</h2>
<p>
Amos Gropp, Matan Atzmon, Yaron Lipman<br>
<i>Technical report</i> <br>
</p>
<a class="btn btn-default abstract" ptitle=" High dimensional data is often assumed to be concentrated on or near a low-dimensional manifold. Autoencoders (AE) is a popular technique to learn representations of such data by pushing it through a neural network with a low dimension bottleneck while minimizing a reconstruction error. Using high capacity AE often leads to a large collection of minimizers, many of which represent a low dimensional manifold that fits the data well but generalizes poorly. Two sources of bad generalization are: extrinsic, where the learned manifold possesses extraneous parts that are far from the data; and intrinsic, where the encoder and decoder introduce arbitrary distortion in the low dimensional parameterization. An approach taken to alleviate these issues is to add a regularizer that favors a particular solution; common regularizers promote sparsity, small derivatives, or robustness to noise. In this paper, we advocate an isometry (i.e., local distance preserving) regularizer. Specifically, our regularizer encourages: (i) the decoder to be an isometry; and (ii) the encoder to be the decoder's pseudo-inverse, that is, the encoder extends the inverse of the decoder to the ambient space by orthogonal projection. In a nutshell, (i) and (ii) fix both intrinsic and extrinsic degrees of free om and provide a non-linear generalization to principal component analysis (PCA). Experimenting with the isometry regularizer on dimensionality reduction tasks produces useful low-dimensional data representations."> Abstract</a>
<a href="https://arxiv.org/abs/2006.09289" class="btn btn-default">Arxiv </a>
</div>
</div><!-- end of row -->
<!--udr -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/udr/IDR_new.gif" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance</h2>
<p>
Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Ronen Basri, Yaron Lipman<br>
<i>34th Annual Conference on Neural Information Processing Systems (NeurIPS 2020), spotlight</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="In this work we address the challenging problem of multiview 3D surface reconstruction. We introduce Implicit Differentiable Renderer (IDR): a neural network architecture that simultaneously learns the unknown geometry, camera parameters, and a neural renderer that approximates the light reflected from the surface towards the camera. The geometry is represented as a zero level-set of a neural network, while the neural renderer, derived from the rendering equation, is capable of (implicitly) modeling a wide set of lighting conditions and materials. We trained our network on real world 2D images of objects with different material properties, lighting conditions, and noisy camera initializations from the DTU MVS dataset. We found our model to produce state of the art 3D surface reconstructions with high fidelity, resolution and detail."> Abstract</a>
<a href="https://arxiv.org/abs/2003.09852" class="btn btn-default">Arxiv </a>
<a href="https://github.com/lioryariv/idr" class="btn btn-default">Code </a>
<a href="https://lioryariv.github.io/idr/" class="btn btn-default">Project Page </a>
</div>
</div><!-- end of row -->
</div>
<!--igr -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/igr/igr.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Implicit Geometric Regularization for Learning Shapes</h2>
<p>
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, Yaron Lipman<br>
<i> International Conference on Machine Learning (ICML) 2020 </i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Representing shapes as level sets of neural networks has been recently proved to be useful for different shape analysis and reconstruction tasks. So far, such representations were computed using either: (i) pre-computed implicit shape representations; or (ii) loss functions explicitly defined over the neural level sets. In this paper we offer a new paradigm for computing high fidelity implicit neural representations directly from raw data (ie, point clouds, with or without normal information). We observe that a rather simple loss function, encouraging the neural network to vanish on the input point cloud and to have a unit norm gradient, possesses an implicit geometric regularization property that favors smooth and natural zero level set surfaces, avoiding bad zero-loss solutions. We provide a theoretical analysis of this property for the linear case, and show that, in practice, our method leads to state of the art implicit neural representations with higher level-of-details and fidelity compared to previous methods."> Abstract</a>
<a href="https://arxiv.org/abs/2002.10099" class="btn btn-default">Arxiv </a>
<a href="https://github.com/amosgropp/IGR" class="btn btn-default">Code</a>
</div>
</div><!-- end of row -->
</div>
<!--sal -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/sal/sal.gif" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>SAL: Sign Agnostic Learning of Shapes from Raw Data </h2>
<p>
Matan Atzmon and Yaron Lipman<br>
<i> Computer Vision and Pattern Recognition (CVPR) 2020, oral presentation </i> <br>
</p>
<a class="btn btn-default abstract" ptitle="Recently, neural networks have been used as implicit representations for surface reconstruction, modelling, learning, and generation. So far, training neural networks to be implicit representations of surfaces required training data sampled from a ground-truth signed implicit functions such as signed distance or occupancy functions, which are notoriously hard to compute. In this paper we introduce Sign Agnostic Learning (SAL), a deep learning approach for learning implicit shape representations directly from raw, unsigned geometric data, such as point clouds and triangle soups. We have tested SAL on the challenging problem of surface reconstruction from an un-oriented point cloud, as well as end-to-end human shape space learning directly from raw scans dataset, and achieved state of the art reconstructions compared to current approaches. We believe SAL opens the door to many geometric deep learning applications with real-world data, alleviating the usual painstaking, often manual pre-process. "> Abstract</a>
<a href="https://arxiv.org/abs/1911.10414" class="btn btn-default">Arxiv </a>
<a href="https://github.com/matanatz/SAL" class="btn btn-default"> Code </a>
<a href="https://youtu.be/vx3jl72qqO0" class="btn btn-default"> Video </a>
</div>
</div><!-- end of row -->
<!--levelsets -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/level_sets/2019_neural_levelsets.png" alt="">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Controlling Neural Level Sets </h2>
<p>
Matan Atzmon, Niv Haim, Lior Yariv, Ofer Israelov, Haggai Maron, Yaron Lipman <br>
<i> 33rd Annual Conference on Neural Information Processing Systems (NeurIPS 2019) </i> <br>
</p>
<a class="btn btn-default abstract" ptitle="The level sets of neural networks represent fundamental properties such as decision boundaries of classifiers and are used to model non-linear manifold data such as curves and surfaces. Thus, methods for controlling the neural level sets could find many applications in machine learning. In this paper we present a simple and scalable approach to directly control level sets of a deep neural network. Our method consists of two parts: (i) sampling of the neural level sets, and (ii) relating the samples’ positions to the network parameters. The latter is achieved by a sample network that is constructed by adding a single fixed linear layer to the original network. In turn, the sample network can be used to incorporate the level set samples into a loss function of interest. We have tested our method on three different learning tasks: raining networks robust to adversarial attacks, improving generalization to unseen data, and curve and surface reconstruction from point clouds. Notably, we increase robust accuracy to the level of standard classification accuracy in off-the-shelf networks, improving it by 2% in MNIST and 27% in CIFAR10 compared to state-of-the-art methods. For surface reconstruction, we produce high fidelity surfaces directly from raw 3D point clouds."> Abstract</a>
<a href="https://arxiv.org/abs/1905.11911" class="btn btn-default">Arxiv </a>
<a href="https://github.com/matanatz/ControllingNeuralLevelsets" class="btn btn-default">Code </a>
<a href="https://github.com/matanatz/ControllingNeuralLevelsets/blob/master/Controlling_Neural_Level_Sets_Poster.pdf" class="btn btn-default">Poster </a>
</div>
</div><!-- end of row -->
<!--PCNN -->
<div class="row" id="templatemo_publications_LargeScaleBD">
<div class="col-md-1"></div>
<div class="col-md-5 col-sm-7 col-xs-24">
<img src="projects/PCNN/rep.png" alt="image 1">
</div>
<div class="col-md-1"></div>
<div class="col-md-16">
<h2>Point Convolutional Neural Networks by Extension Operators</h2>
<p>
Matan Atzmon*, Haggai Maron* and Yaron Lipman (*equal contribution)<br>
<i>ACM SIGGRAPH 2018</i> <br>
</p>
<a class="btn btn-default abstract" ptitle="This paper presents Point Convolutional Neural Networks (PCNN): a novel framework for applying convolutional neural networks to point clouds. The framework consists of two operators: extension and restriction, mapping point cloud functions to volumetric functions and vise-versa. A point cloud convolution is defined by pull-back of the Euclidean volumetric convolution via an extension-restriction mechanism. The point cloud convolution is computationally efficient, invariant to the order of points in the point cloud, robust to different samplings and varying densities, and translation invariant, that is the same convolution kernel is used at all points. PCNN generalizes image CNNs and allows readily adapting their architectures to the point cloud setting. Evaluation of PCNN on three central point cloud learning benchmarks convincingly outperform competing point cloud learning methods, and the vast majority of methods working with more informative shape representations such as surfaces and/or normals"> Abstract</a>
<a href="https://arxiv.org/abs/1803.10091" class="btn btn-default"> Arxiv </a>
<a href="https://github.com/matanatz/pcnn" class="btn btn-default">GitHub</a>
<a href="projects/PCNN/pointConv_sigraph_final_pdf.pdf" class="btn btn-default">Slides</a>
<a href="https://dl.acm.org/action/downloadSupplement?doi=10.1145%2F3197517.3201301&file=a71-atzmon.mp4" class="btn btn-default">Video</a>
</div>
</section><!-- end of templatemo_publications -->
<br>
<br>
<!--<script src="https://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false"></script> -->
<script src="js/jquery.min.js"></script>
<script src="js/jquery.scrollto.min.js"></script>
<script src="js/jquery.easing.min.js"></script>
<script src="js/bootstrap.min.js"></script>
<script src="js/jquery.lightbox.min.js"></script>
<script src="js/jquery.flexslider.js"></script>
<script src="js/jquery.singlePageNav.min.js"></script>
<script src="js/templatemo_script.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','../../www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-52088992-1', 'weizmann.ac.il');
ga('require', 'linkid', 'linkid.html');
ga('send', 'pageview');
</script>
</body>
<!-- Mirrored from www.wisdom.weizmann.ac.il/~shaharko/ by HTTrack Website Copier/3.x [XR&CO'2014], Wed, 06 Apr 2016 11:56:56 GMT -->
</html>