Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download raw build log #3585

Merged
merged 44 commits into from
May 30, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
ae03632
Add raw_log property
stsewd Feb 7, 2018
fea2137
Move log to separate template
stsewd Feb 8, 2018
200b358
Add renderer for text/plain media type
stsewd Feb 8, 2018
a7cfd45
Add endpoint /log for getting raw log
stsewd Feb 8, 2018
717d70e
Add suggestion for raw log link
stsewd Feb 8, 2018
a414abb
Fix linter issue
stsewd Feb 8, 2018
8b6e266
Linter
stsewd Feb 8, 2018
33af7f2
Fix typo
stsewd Feb 10, 2018
18545db
Fix method inside BuildViewSet
stsewd Feb 10, 2018
9f6da49
Change Raw log to View raw
stsewd Feb 10, 2018
a1f22a7
Add url for view raw
stsewd Feb 10, 2018
1c46ce0
Check for valiud format before renderer
stsewd Feb 10, 2018
d4eaa37
Fix test
stsewd Feb 10, 2018
31d3999
Mark command and output as safe
stsewd Feb 10, 2018
73b7a12
Add test
stsewd Feb 10, 2018
9be631d
Isort
stsewd Feb 10, 2018
16aa10b
Use render_to_string
stsewd Feb 16, 2018
3cd340d
Update log template
stsewd Feb 16, 2018
d91dd60
Update tests
stsewd Feb 16, 2018
4ca3c47
Improve renderer class
stsewd Feb 16, 2018
47315c8
Remove black listed url
stsewd Feb 16, 2018
aca0ef8
Add black listed url again
stsewd Feb 16, 2018
75a8042
Show view raw only to staff users
stsewd Feb 16, 2018
5c7629c
Try ro reuse previous endpoint
stsewd Feb 16, 2018
949b42d
Check against None value
stsewd Feb 19, 2018
d9f229f
Rename renderer
stsewd Feb 19, 2018
ce7e16c
Remove old code
stsewd Feb 19, 2018
40f259d
Show build state
stsewd Feb 19, 2018
a029a08
Update tests
stsewd Feb 19, 2018
c3b02f4
Remove raw_log property
stsewd Feb 19, 2018
03e642b
New url
stsewd Feb 19, 2018
39b3607
Update tests
stsewd Feb 20, 2018
7bfd92c
Fix tests
stsewd Feb 20, 2018
2af04a8
Fix newline
stsewd Feb 20, 2018
59af03a
Better status check
stsewd Feb 20, 2018
990145c
Decode strings
stsewd Feb 20, 2018
52aafc5
Better protection for None
stsewd Feb 20, 2018
dc2cbdd
Fix information provided by data
stsewd Feb 21, 2018
86d7bde
More tests
stsewd Feb 21, 2018
29696ed
Remove trash from rebase
stsewd May 24, 2018
c8dc506
Improve log format
stsewd May 24, 2018
a037062
Add slugs to api response
stsewd May 24, 2018
bd03a78
Show link to everyone
stsewd May 24, 2018
1415270
More tests
stsewd May 24, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions readthedocs/restapi/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ class BuildSerializer(serializers.ModelSerializer):
"""Build serializer for user display, doesn't display internal fields."""

commands = BuildCommandSerializer(many=True, read_only=True)
project_slug = serializers.ReadOnlyField(source='project.slug')
version_slug = serializers.ReadOnlyField(source='version.slug')
Copy link
Member Author

@stsewd stsewd May 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to add this new fields here since we can't access this inside the template as an object but as the result of the serializer.

docs_url = serializers.ReadOnlyField(source='version.get_absolute_url')
state_display = serializers.ReadOnlyField(source='get_state_display')

Expand Down
14 changes: 14 additions & 0 deletions readthedocs/restapi/templates/restapi/log.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Read the Docs build information
Build id: {{ build.id }}
Project: {{ build.project_slug }}
Version: {{ build.version_slug }}
Commit: {{ build.commit }}
Date: {{ build.date }}
State: {{ build.state }}
Success: {% if build.state == 'finished' %}{{ build.success }}{% else %}Unknown{% endif %}

{% for command in build.commands %}
[rtd-command-info] start-time: {{ command.start_time }}, end-time: {{ command.end_time }}, duration: {{ command.run_time }}, exit-code: {{ command.exit_code }}
{{ command.command|safe }}
{{ command.output|safe }}
{% endfor %}
27 changes: 25 additions & 2 deletions readthedocs/restapi/views/model_views.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@
from allauth.socialaccount.models import SocialAccount
from django.shortcuts import get_object_or_404
from rest_framework import decorators, permissions, status, viewsets
from django.template.loader import render_to_string
from rest_framework.decorators import detail_route
from rest_framework.renderers import JSONRenderer
from rest_framework.renderers import BaseRenderer, JSONRenderer
from rest_framework.response import Response

from readthedocs.builds.constants import BRANCH, TAG
Expand All @@ -34,6 +35,28 @@
log = logging.getLogger(__name__)


class PlainTextBuildRenderer(BaseRenderer):

"""
Custom renderer for text/plain format.

charset is 'utf-8' by default.
"""

media_type = 'text/plain'
format = 'txt'

def render(self, data, accepted_media_type=None, renderer_context=None):
renderer_context = renderer_context or {}
response = renderer_context.get('response')
if not response or response.exception:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is no response probably is something wrong, anyway, this is just an extra protection since renderer_context "always" has this.

return data.get('detail', '').encode(self.charset)
data = render_to_string(
'restapi/log.txt', {'build': data}
)
return data.encode(self.charset)


class UserSelectViewSet(viewsets.ModelViewSet):

"""
Expand Down Expand Up @@ -213,7 +236,7 @@ class VersionViewSet(UserSelectViewSet):

class BuildViewSetBase(UserSelectViewSet):
permission_classes = [APIRestrictedPermission]
renderer_classes = (JSONRenderer,)
renderer_classes = (JSONRenderer, PlainTextBuildRenderer)
serializer_class = BuildSerializer
admin_serializer_class = BuildAdminSerializer
model = Build
Expand Down
136 changes: 135 additions & 1 deletion readthedocs/rtd_tests/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
from rest_framework import status
from rest_framework.test import APIClient

from readthedocs.builds.models import Build, Version
from readthedocs.builds.models import Build, BuildCommandResult, Version
from readthedocs.integrations.models import Integration
from readthedocs.oauth.models import RemoteOrganization, RemoteRepository
from readthedocs.projects.models import Feature, Project
Expand Down Expand Up @@ -277,6 +277,140 @@ def test_make_build_commands(self):
self.assertEqual(build['commands'][0]['run_time'], 5)
self.assertEqual(build['commands'][0]['description'], 'foo')

def test_get_raw_log_success(self):
build = get(Build, project_id=1, version_id=1, builder='foo')
get(
BuildCommandResult,
build=build,
command='python setup.py install',
output='Installing dependencies...'
)
get(
BuildCommandResult,
build=build,
command='git checkout master',
output='Switched to branch "master"'
)
client = APIClient()

api_user = get(User, user='test', password='test')
client.force_authenticate(user=api_user)
resp = client.get('/api/v2/build/{0}.txt'.format(build.pk))
self.assertEqual(resp.status_code, 200)

self.assertIn('Read the Docs build information', resp.content.decode())
self.assertIn('Build id: {}'.format(build.id), resp.content.decode())
self.assertIn('Project: {}'.format(build.project.slug), resp.content.decode())
self.assertIn('Version: {}'.format(build.version.slug), resp.content.decode())
self.assertIn('Commit: {}'.format(build.commit), resp.content.decode())
self.assertIn('Date: ', resp.content.decode())
self.assertIn('State: finished', resp.content.decode())
self.assertIn('Success: True', resp.content.decode())
self.assertIn('[rtd-command-info]', resp.content.decode())
self.assertIn(
'python setup.py install\nInstalling dependencies...',
resp.content.decode()
)
self.assertIn(
'git checkout master\nSwitched to branch "master"',
resp.content.decode()
)

def test_get_raw_log_building(self):
build = get(
Build, project_id=1, version_id=1,
builder='foo', success=False,
exit_code=1, state='building',
)
get(
BuildCommandResult,
build=build,
command='python setup.py install',
output='Installing dependencies...',
exit_code=1,
)
get(
BuildCommandResult,
build=build,
command='git checkout master',
output='Switched to branch "master"'
)
client = APIClient()

api_user = get(User, user='test', password='test')
client.force_authenticate(user=api_user)
resp = client.get('/api/v2/build/{0}.txt'.format(build.pk))
self.assertEqual(resp.status_code, 200)

self.assertIn('Read the Docs build information', resp.content.decode())
self.assertIn('Build id: {}'.format(build.id), resp.content.decode())
self.assertIn('Project: {}'.format(build.project.slug), resp.content.decode())
self.assertIn('Version: {}'.format(build.version.slug), resp.content.decode())
self.assertIn('Commit: {}'.format(build.commit), resp.content.decode())
self.assertIn('Date: ', resp.content.decode())
self.assertIn('State: building', resp.content.decode())
self.assertIn('Success: Unknow', resp.content.decode())
self.assertIn('[rtd-command-info]', resp.content.decode())
self.assertIn(
'python setup.py install\nInstalling dependencies...',
resp.content.decode()
)
self.assertIn(
'git checkout master\nSwitched to branch "master"',
resp.content.decode()
)

def test_get_raw_log_failure(self):
build = get(
Build, project_id=1, version_id=1,
builder='foo', success=False, exit_code=1
)
get(
BuildCommandResult,
build=build,
command='python setup.py install',
output='Installing dependencies...',
exit_code=1,
)
get(
BuildCommandResult,
build=build,
command='git checkout master',
output='Switched to branch "master"'
)
client = APIClient()

api_user = get(User, user='test', password='test')
client.force_authenticate(user=api_user)
resp = client.get('/api/v2/build/{0}.txt'.format(build.pk))
self.assertEqual(resp.status_code, 200)

self.assertIn('Read the Docs build information', resp.content.decode())
self.assertIn('Build id: {}'.format(build.id), resp.content.decode())
self.assertIn('Project: {}'.format(build.project.slug), resp.content.decode())
self.assertIn('Version: {}'.format(build.version.slug), resp.content.decode())
self.assertIn('Commit: {}'.format(build.commit), resp.content.decode())
self.assertIn('Date: ', resp.content.decode())
self.assertIn('State: finished', resp.content.decode())
self.assertIn('Success: False', resp.content.decode())
self.assertIn('[rtd-command-info]', resp.content.decode())
self.assertIn(
'python setup.py install\nInstalling dependencies...',
resp.content.decode()
)
self.assertIn(
'git checkout master\nSwitched to branch "master"',
resp.content.decode()
)

def test_get_invalid_raw_log(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One extra test to see if the response.exception works :)

client = APIClient()

api_user = get(User, user='test', password='test')
client.force_authenticate(user=api_user)
resp = client.get('/api/v2/build/{0}.txt'.format(404))
self.assertEqual(resp.status_code, status.HTTP_404_NOT_FOUND)


class APITests(TestCase):
fixtures = ['eric.json', 'test_data.json']
Expand Down
8 changes: 8 additions & 0 deletions readthedocs/templates/builds/build_detail.html
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,14 @@
</a>
</li>
</div>

<div data-bind="visible: finished()">
<li>
<a href="{% url "build-detail" build.pk "txt" %}">
{% trans "View raw" %}
</a>
</li>
</div>
</ul>

<div class="build-id">
Expand Down