API Doc Improvements - Tracking

[domdivakaruni]

Let's fix our docs!

The effort to spruce up MXNet's Python API docs is picking up steam but could use your help to add that extra oomph! Many of the APIs now have proper explanations and good examples, but we still have much more to go!

Here’s how you can join in and help:

1. Pick a few functions (or at least 1 😄 ) from the list below

   • Ensure you only pick ones that don’t have a current owner assigned and you can submit within 2 business days. Post a comment to the issue with the ones you’ve selected.

2. Look up current documentation

   • Find whats currently documented for the function at the mxnet.io python API docs or within the C++ code in git.

3. Write up:

   • A clear and concise description of the function and its behavior.
   • List and describe each parameter with the valid input values, whether it is required or optional, and default value if the parameter is optional.
   • Illustrate the function and parameter behavior using examples
   • Refer to these examples for guidance:- Embedding , ROIPooling , Reshape

4. Submit your work

   • Creating a new issue with [API docs submission] ‘your function name(s)’ in the title and paste your write-up markdown
     OR
   • Creating a pull request for a change to the appropriate .cc or .py file. Instructions here: How to make a pull request for docs

API List

#	Function Name	Package	Contributor	Status
1	IRHeader	recordio
2	pack	recordio
3	pack_img	recordio
4	unpack	recordio
5	unpack_img	recordio
6	BilinearSampler	ndarray	@Lyken17
7	Correlation	ndarray	@Lyken17
8	GridGenerator	ndarray	@Lyken17
9	IdentityAttachKLSparseReg	ndarray	@Lyken17
10	SpatialTransformer	ndarray	@Lyken17
11	UpSampling	ndarray	@Lyken17
12	adam_update	ndarray
13	ones_like	ndarray	@yuruofeifei
14	random_exponential	ndarray	@asmushetzel
15	random_gamma	ndarray	@asmushetzel
16	random_generalized_negative_binomial	ndarray	@asmushetzel
17	random_negative_binomial	ndarray	@asmushetzel
18	random_poisson	ndarray	@asmushetzel
19	rmsprop_update	ndarray	@jiajiechen
20	rmspropalex_update	ndarray	@jiajiechen
21	sample_exponential	ndarray	@asmushetzel
22	sample_gamma	ndarray	@asmushetzel
23	sample_generalized_negative_binomial	ndarray	@asmushetzel
24	sample_negative_binomial	ndarray	@asmushetzel
25	sample_normal	ndarray	@asmushetzel
26	sample_poisson	ndarray	@asmushetzel
27	sample_uniform	ndarray	@asmushetzel
28	sgd_mom_update	ndarray
29	sgd_update	ndarray
30	smooth_l1	ndarray
31	softmax_cross_entropy	ndarray	@yuruofeifei
32	waitall	ndarray
33	zeros_like	ndarray	@yuruofeifei
34	Caffe	metric
35	Torch	metric
36	check_label_shapes	metric
37	Bilinear	initializer
38	FusedRNN	initializer
39	LSTMBias	initializer
40	LogValidationMetricsCallback	callback
41	_init_torch_module	torch
42	_make_torch_function	torch
43	CustomOp.assign	operator
44	NameManager	name
45	Monitor	monitor
46	CastAug	image
47	CenterCropAug	image
48	ColorJitterAug	image
49	ColorNormalizeAug	image
50	CreateAugmenter	image
51	HorizontalFlipAug	image
52	LightingAug	image
53	RandomCropAug	image
54	RandomOrderAug	image
55	RandomSizedCropAug	image
56	ResizeAug	image
57	center_crop	image
58	color_normalize	image
59	fixed_crop	image
60	imdecode	image
61	random_crop	image
62	random_size_crop	image
63	resize_short	image	@jhwei
64	scale_down	image	@jhwei
65	ImageIter.augmentation_transform	image
66	ImageIter.imdecode	image
67	ImageIter.postprocess_data	image
68	ImageIter.read_image	image
69	ImageIter.check_data_shape	image
70	ImageIter.check_valid_image	image
71	Executor._get_dict	executor
72	_monitor_callback_wrapper	executor
73	MultiBoxDetection	contrib
74	MultiBoxPrior	contrib
75	MultiBoxTarget	contrib
76	Proposal	contrib
77	count_sketch	contrib
78	fft	contrib
79	ifft	contrib
80	set_is_training	contrib
81	test	contrib
82	train	contrib
83	MXNetError	base
84	mx_float_p	base
85	py_str	base
86	print_summary	visualization
87	Symbol.attr	symbol
88	Symbol.attr_dict	symbol
89	Symbol.debug_str	symbol
90	Symbol.grad	symbol
91	Symbol.list_attr	symbol
92	MXRecordIO.close	recordio
93	MXRecordIO.open	recordio
94	Optimizer.set_lr_mult	optimizer
95	Optimizer.set_lr_scale	optimizer
96	Optimizer.set_wd_mult	optimizer
97	Optimizer.update	optimizer
98	LinearRegressionOutput	ndarray	@naidu0830
99	NDArray._at	ndarray	@reminisce
100	NDArray._slice	ndarray	@reminisce
101	NDArray._sync_copyfrom	ndarray	@reminisce
102	gamma	ndarray	@zhenwendai
103	gammaln	ndarray	@zhenwendai
104	identity	ndarray	@zhenwendai
105	load	ndarray	@zhenwendai
106	moveaxis	ndarray	@zhenwendai
107	DataDesc	io
108	DataDesc.get_batch_axis	io
109	DataDesc.get_list	io
110	DataIter.getpad	io
111	MXDataIter	io
112	ImageDetRecordIter	io
113	Initializer._legacy_init	initializer
114	Initializer.dumps	initializer
115	ProgressBar	callback	@saurabh3949
116	log_train_metric	callback
117	module_checkpoint	callback
118	_parse_aux_states	test_utils
119	_parse_location	test_utils
120	check_symbolic_backward	test_utils	@reminisce
121	check_symbolic_forward	test_utils	@reminisce
122	np_reduce	test_utils
123	numeric_grad	test_utils
124	print_max_err_loc	test_utils
125	BucketSentenceIter	rnn
126	encode_sentences	rnn
127	do_rnn_checkpoint	rnn
128	load_rnn_checkpoint	rnn
129	save_rnn_checkpoint	rnn
130	BaseRNNCell.begin_state	rnn
131	BaseRNNCell.pack_weights	rnn
132	BaseRNNCell.unpack_weights	rnn
133	BaseRNNCell.unroll	rnn
134	BidirectionalCell	rnn
135	DropoutCell	rnn
136	FusedRNNCell	rnn
137	GRUCell	rnn
138	LSTMCell	rnn
139	ModifierCell	rnn
140	RNNCell	rnn
141	RNNParams	rnn
142	RNNParams.get	rnn
143	SequentialRNNCell	rnn
144	SequentialRNNCell.add	rnn
145	ZoneoutCell	rnn
146	BaseRNNCell._get_activation	rnn
147	FusedRNNCell._slice_weights	rnn
148	dump_profile	profiler
149	profiler_set_config	profiler
150	profiler_set_state	profiler
151	CustomOp	operator
152	CustomOp.backward	operator
153	CustomOp.forward	operator
154	CustomOpProp	operator
155	CustomOpProp.infer_shape	operator
156	CustomOpProp.list_arguments	operator
157	CustomOpProp.list_auxiliary_states	operator
158	CustomOpProp.list_outputs	operator
159	NDArrayOp.declare_backward_	operator
160	NumpyOp	operator
161	PythonOp	operator
162	PythonOp.backward	operator
163	PythonOp.forward	operator
164	PythonOp.get_symbol	operator
165	PythonOp.infer_shape	operator
166	PythonOp.list_arguments	operator
167	PythonOp.list_outputs	operator
168	PythonOp.need_top_grad	operator
169	register	operator
170	CustomOpProp.infer_type	operator
171	NameManager.get	name
172	Prefix	name
173	Monitor.install	monitor
174	Monitor.tic	monitor
175	Monitor.toc	monitor
176	Monitor.toc_print	monitor
177	getLogger	log
178	KVStore.load_optimizer_states	kvstore	@eric-haibin-lin
179	KVStore.save_optimizer_states	kvstore	@eric-haibin-lin
180	- [x ] KVStore.set_optimizer	kvstore	@eric-haibin-lin
181	create	kvstore	@eric-haibin-lin
182	ImageIter	image
183	ImageIter.next	image
184	ImageIter.next_sample	image
185	ImageIter.reset	image
186	Executor	executor	@kevinthesun
187	Executor.backward	executor	@kevinthesun
188	Executor.copy_params_from	executor	@kevinthesun
189	Executor.reshape	executor	@kevinthesun
190	Executor.set_monitor_callback	executor	@kevinthesun
191	Executor._get_outputs	executor
192	Executor.debug_str	executor	@kevinthesun
193	compute_gradient	contrib
194	grad_and_loss	contrib
195	mark_variables	contrib
196	set_recording	contrib
197	LogMetricsCallback	contrib
198	TrainingStateScope	contrib
199	build_param_doc	base
200	c_array	base	@yuruofeifei
201	c_str	base	@yuruofeifei
202	check_call	base
203	ctypes2buffer	base
204	ctypes2numpy_shared	base
205	_load_lib	base
206	_notify_shutdown	base
207	add_fileline_to_docstring	base
208	AttrScope	attribute
209	AttrScope.current	attribute
210	AttrScope.get	attribute

[piiswrong]

NumpyOP pythonop and ndarrayop are deprecated. We only support customop now

[rongali-naidu]

I will document "LinearRegressionOutput" API

[asmushetzel]

I will document all samplers and randoms as I anyway enhanced or authored them, namely

random_gamma
random_exponential
random_poisson
random_negative_binomial
random_generalized_negative_binomial

sample_uniform
sample_normal
sample_gamma
sample_exponential
sample_poisson
sample_negative_binomial
sample_generalized_negative_binomial

I may need more than 2 business days but should be definitely done by end of next week.

[moontails]

I will document Functions from 125 to 147 in the rnn package.

If I get more time, I will also work on 161-168

[domdivakaruni]

@moontails that is tremendous thank you! Do you plan on making PR's or creating Issues with your functions in markdown? When you are done with your first function send it in so we can help with a review before you get rolling with the rest. great stuff!

[moontails]

@domdivakaruni I was thinking of creating PR's.

Thats a good idea! I will submit a PR when am done with the changes for the first function and gather all feedback before diving in on the rest :)

[zhenwendai]

I can document some of ndarray operators:

102 gamma ndarray
103 gammaln ndarray
104 identity ndarray
105 load ndarray
106 moveaxis ndarray

[domdivakaruni]

Hi @moontails do you want to call ownership of all 22 rnn functions at once or do you want to stake your claim in batches as you make progress?

[yuruofeifei]

I will work on zeros_like, ones_like, softmax_cross_entropy, c_array, c_str

[jhwei]

I can work on resize_short, scale_down.

[domdivakaruni]

@yuruofeifei @jhwei @zhenwendai good stuff!

[moontails]

@domdivakaruni either works fine for me. I had hoped to send out a PR yesterday to get some feedback before working on the rest, but ran into some issues setting up the repo (with the whole installing mxnet and make lint)

[domdivakaruni]

@moontails did you get it worked out? Let me know what you ran into.
@nswamy fyi

[nswamy]

@moontails whats the issue that you are running into?
@domdivakaruni can we have folks picks few APIs at once and add more to their bucket as they make progress?

[Lyken17]

I can document 6~11.

Question : Should we use reST or Markdown for improvement through github issues? This page says Markdown, but pull request pages requires reST.

[nswamy]

@Lyken17 please prefer to use reST and submit a PR. If you want to just submit doc outside of the the code(python doc string or describe method of C++ code) use either reST or Markdown and submit an issue.

[nswamy]

@zhenwendai, The docs for these APIs are already updated. Could you please pick a different set?
102 gamma ndarray
103 gammaln ndarray
104 identity ndarray

[srviest]

I will work on 58 color_normalize. BTW, where are the definitions of 78 fft & 79 ifft?

[jiayue666]

I can document 2-5.

[szha]

I can share 130-145 with @moontails. Let me know which ones would you prefer that I do.

@piiswrong do we need to document 146 BaseRNNCell._get_activation and 147 FusedRNNCell._slice_weights? These are not intended for use other than by the APIs.

[domdivakaruni]

@moontails are you still working on these APIs? What can you share with @szha? can you please let us know as soon as you can? Thanks!

[moontails]

@domdivakaruni @szha I am sorry guys, I still havent been able to get it to work on my Macbook.

I will let @szha start checking things off from 130-145 and get back on track if I am able to unblock myself.

[asmushetzel]

@moontails Don't know which problem you ran into but also had to resolve some hiccups on my mac. Resolved it by:

• Complete fresh clone of the whole mxnet
• Not running all build/installs for this repo. Just the documentation creation as described in the doc referenced in section 4. at the beginning of this thread.
• In file tests/ci_build/install/ubuntu_install_python.sh bypassed the certificate checking by
  wget https://bootstrap.pypa.io/get-pip.py --no-check-certificate

That got me going. I used a separate mxnet repo for standard building/installation and functional testing (I had to touch a bit more than just the "describe" section of the operators). Don't know whether this was necessary, but at least it worked that way.

Did punt on lint on my mac as I couldn't get it to work. Rather did submit the pull request and fixed the flagged lint issues based on the automatic tests. It's straightforward.

[eric-haibin-lin]

I'm going to take the four APIs for kvstore 178 - 181

[nswamy]

@Lyken17 any luck updating the doc for the APIs that you picked ?

[Lyken17]

Hi @nswamy

I just updated GridGenerator and submit it to issue #6147.

I meet troubles when documenting ndarray.UpSampling. Even myself cannot find the proper way to use it, according to issues #2823 #4134 #1412, it seems to be an incomplete implementation. I also looked up several FCN mxnet implementations, they all use deconvolution instead of UpSampling.

[nswamy]

@Lyken17, Thanks for posting the issue, I will convert this to actual documentation in code.
You can skip UpSampling for now, we might need to add a note that it is incomplete. @mli, @piiswrong what do you suggest?

For other APIs that you work on, can you please edit the docs in the code directly? If building and rendering takes time or you get into an issue, you can skip that step and just follow the tips to avoid any lint errors.

[apaleyes]

Hi @nswamy

I've just posted a pr for 118 and 119: #6314

One thing though is that I wasn't able to get rendering going on my Mac. "make lint" was fine, but building docs was always failing. I'll post a separate issue perhaps to get some advice on that.