Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Json documentation generation #1775

Merged
merged 5 commits into from
Jul 22, 2016
Merged

WIP: Json documentation generation #1775

merged 5 commits into from
Jul 22, 2016

Conversation

daspecster
Copy link
Contributor

@daspecster daspecster commented May 5, 2016 •
edited
Loading

scripts/generate_json_docs.py is now generating parsable JSON for the gcloud-common site generator.
As well as building a types.json file.

Several things are still in progress...

  • Extracting code examples is not working consistently.
  • The table of contents needs to be updated. (just switched to listing all classes)
  • Class linking should be added. i.e. class:gcloud.datastore._generated.query_pb2.Query
  • Source Code links are broken for base class docs
  • Not yet sure how to handle __init__.py's
  • "Getting Started" and "Guide" type pages need to be ported.(done for what we have but should add more).
  • Running generate_json_docs.py needs to be added to the build process and documented better.
  • Find all directives that are not filtered and filter/handle them (for now anyway).
  • Possibly add auto-href generation for link in text for easier navigation out site docs.
  • Figure out how to handle snippets. This should probably be in a different PR after this launches
  • Bring in Code snippets Add ability to pull snippets when generating JSON documentation #1951
  • Trim back TOC.
  • Switch to live gh-pages committing

@daspecster daspecster added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 5, 2016
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@googlebot googlebot added the cla: no This human has *not* signed the Contributor License Agreement. label May 5, 2016
@dhermes
Copy link
Contributor

dhermes commented May 5, 2016

@daspecster From https://api.github.com/repos/GoogleCloudPlatform/gcloud-python/pulls/1775/commits it appears some of the commits have @jgeewax and @callmehiphop emails in them (which is why the CLA bot is angry).

dhermes
dhermes reviewed May 5, 2016
View reviewed changes
docs/json/toc.json
@@ -0,0 +1,160 @@
{

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@dhermes
Copy link
Contributor

dhermes commented May 5, 2016

High level comment: It looks like you've had to write the majority of the parser from scratch (i.e. have your own representations of Module / Class / etc.) The pdoc.import_module behavior seems to just be a fancy AST parser?

@daspecster
Copy link
Contributor Author

Yeah, I think I'm probably going to drop pdoc and just use inspect.

@daspecster
Copy link
Contributor Author

Also, @dhermes for the ci bot, should I just squash those commits? I didn't want to leave out the log of their contributions.

@dhermes
Copy link
Contributor

dhermes commented May 5, 2016

I don't particularly mind about the CI, though those initial commits aren't necessarily needed for your work.

jgeewax
jgeewax reviewed May 5, 2016
View reviewed changes
scripts/generate_json_docs.py

def generate_doc_types_json(modules, types_file_path):
doc_types_list = [{
"id": "gcloud",

This comment was marked as spam.

Sign in to view

@jgeewax
Copy link
Contributor

jgeewax commented May 5, 2016

Nice -- this is awesome. Thanks for cranking on it.

@jgeewax jgeewax added the docs label May 5, 2016
@googlebot
Copy link

CLAs look good, thanks!

@googlebot googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels May 5, 2016
@tseaver tseaver mentioned this pull request May 6, 2016
Merged
@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

@googlebot googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Jun 6, 2016
@tseaver
Copy link
Contributor

tseaver commented Jun 6, 2016

The "continuation line" lint failures are obnoxious: finding the style which passes both pep8 and pylint is always a bit of trial and error for me. The others are fairly straightforward fixes.

@tseaver
Copy link
Contributor

tseaver commented Jul 15, 2016

@daspecster Those bits of narrative are a definite improvement.

@tseaver
Copy link
Contributor

tseaver commented Jul 16, 2016

LGTM

@daspecster
Copy link
Contributor Author

Ok thanks! Should we trim the TOC back a bit?

@daspecster
Copy link
Contributor Author

@dhermes, I'll fix that formatting issue and I also just noticed that I broke the sub nav links with my last push.

@daspecster daspecster force-pushed the json-docs branch 3 times, most recently from 7610b13 to 820f2ac Compare July 20, 2016 18:46
@daspecster
Copy link
Contributor Author

Ok! Have another look through and LMKWYT

http://daspecster.github.io/gcloud-python/

@daspecster
Copy link
Contributor Author

Sorry, forgot to tag @tseaver and @dhermes on that last comment.

@tseaver
Copy link
Contributor

tseaver commented Jul 20, 2016

The docstring examples on the datastore landing page still look borked:

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/datastore

The snippet captions appear to have regressed:

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/bigquery

http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub

@daspecster
First pass at Python doc generator...(originally JJ's commit)
45f5189
@daspecster
Initial update to generate json docs
de3d16f 
Testing deploy with 1:1 class:json.

Add overview.html

Deploy script working now in testing.

Fix code blocks.

Add href support.

Add snippets to base service module. closes googleapis#1951

Fix types generation.

Fix linking.

Fix external github links.

Update gcloud-commit site files.

Swtich to live site generation.

clean up manifest for starters.

Remove old example detection code.

Update TOC.
@daspecster
Update site and fix typos.
803266b
@daspecster
Copy link
Contributor Author

daspecster commented Jul 20, 2016 •
edited
Loading

I think I somehow squashed the snippet title code out. I'll try and redo it real quick.

@daspecster daspecster mentioned this pull request Jul 20, 2016
Merged
@daspecster
Copy link
Contributor Author

@tseaver fixed! http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub
I fixed the .. doctest:: thing with #2006.

Not sure how that got lost, I think I may have lost the commit in rebasing or something.

@tseaver
Copy link
Contributor

tseaver commented Jul 21, 2016

@daspecster The http://daspecster.github.io/gcloud-python/stable/bigquery-usage.html page is still showing the pre-snippet examples: maybe that will get fixed when you merge?

@daspecster
Copy link
Contributor Author

@tseaver Oh sorry, I pushed gh-pages when I was doing #2009 and #2008.
It's back now though. http://daspecster.github.io/gcloud-python/#/docs/master/gcloud/pubsub

@daspecster
Copy link
Contributor Author

Let me know if there is anything that's a show stopper here. I would like to try and get this merged today if possible.

@tseaver
Copy link
Contributor

tseaver commented Jul 22, 2016

LGTM

@daspecster
Copy link
Contributor Author

I'm going to merge this at 2pm EST unless I hear otherwise. Just to give everyone a chance to chime in.

@daspecster daspecster merged commit ba6b13c into googleapis:master Jul 22, 2016
@daspecster
Copy link
Contributor Author

Travis master build that I'm watching for this https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/146717128

This was referenced Aug 3, 2016
Closed
Merged
@daspecster daspecster deleted the json-docs branch January 24, 2017 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@tseaver tseaver

Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@daspecster @googlebot @dhermes @jgeewax @tseaver @callmehiphop