Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix the Cheetah template renderer in salt.utils.templates, and unit-tests for Jinja, Cheetah, Mako, Genshi, wempy templates, #51718

Merged
merged 15 commits into from
Dec 18, 2019
Merged

Fix the Cheetah template renderer in salt.utils.templates, and unit-tests for Jinja, Cheetah, Mako, Genshi, wempy templates, #51718

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
50cd57d
Fix the Cheetah template renderer in salt.utils.templates.
arizvisa Feb 19, 2019
bd60898
Modified the render_cheetah_tmpl() function in salt.util.templates to…
arizvisa Feb 20, 2019
0e6e793
Ugh..Added unit tests for rendering all of the template types as unit…
arizvisa Oct 16, 2019
b93200c
Fixed lint issues in the new unit tests for tests.unit.utils.test_tem…
arizvisa Oct 16, 2019
be70b9d
Added all of the template packages (Cheetah3, Genshi, Mako, wempy) to…
arizvisa Oct 26, 2019
8734cd9
Revert "Added all of the template packages (Cheetah3, Genshi, Mako, w…
arizvisa Oct 26, 2019
d95774e
Added all of the template packages (Cheetah3, Genshi, Mako, wempy) to…
arizvisa Oct 26, 2019
c6fd7ca
Revert "Added all of the template packages (Cheetah3, Genshi, Mako, w…
arizvisa Oct 26, 2019
a944de4
Added all of the available template packages (Cheetah3, Genshi, Mako,…
arizvisa Oct 26, 2019
56143af
Fixed a copy-pasta error in the unit.utils.test_templates test that r…
arizvisa Oct 26, 2019
7a3f696
Re-fixed the copy-pasta error in the wempy_variable test from unit.ut…
arizvisa Oct 28, 2019
41a3023
Revert "Added all of the available template packages (Cheetah3, Gensh…
arizvisa Dec 17, 2019
017acd2
Added all of the available template packages (Cheetah3, Genshi, Mako,…
arizvisa Dec 17, 2019
cd3744d
Updated all of the testing requirements using pre-commit.
arizvisa Dec 17, 2019
72861ef
Moved the context in tests.unit.utils.test_templates within the scope…
arizvisa Dec 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions requirements/static/darwin.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,9 @@ supervisor==3.3.5; python_version < '3'
virtualenv
watchdog
yamlordereddictloader

# Available template libraries that can be used
Genshi
Cheetah3==3.1.0
Mako
wempy; python_version <'3'
6 changes: 6 additions & 0 deletions requirements/static/linux.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,9 @@ timelib
tornado<5.0
virtualenv
watchdog

# Available template libraries that can be used
Genshi
Cheetah3==3.1.0
Mako
wempy; python_version <'3'
2 changes: 2 additions & 0 deletions requirements/static/py2.7/darwin.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,5 @@ zc.lockfile==1.4 # via cherrypy
zipp==0.6.0 # via importlib-metadata
# Passthrough dependencies from pkg/osx/req.txt
pyobjc==5.1.2
# Passthrough dependencies from pkg/osx/req.txt
pyobjc==5.1.2
9 changes: 5 additions & 4 deletions requirements/static/py3.4/linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5 # via tornado
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl-match-hostname==3.7.0.1 # via docker, websocket-client
backports.ssl-match-hostname==3.7.0.1 # via docker
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +20,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.4 # via cherrypy
cherrypy==17.3.0
contextlib2==0.5.5 # via cherrypy
Expand All @@ -32,6 +32,7 @@ docker==3.7.2
docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitpython==2.1.11
google-auth==1.6.3 # via kubernetes
Expand All @@ -51,6 +52,7 @@ kazoo==2.6.1
keyring==5.7.1
kubernetes==3.0.0
lxml==4.3.3 # via junos-eznc, ncclient
mako==1.1.0
markupsafe==1.1.1
mock==3.0.5 ; python_version < "3.6"
more-itertools==5.0.0
Expand Down Expand Up @@ -101,8 +103,7 @@ scp==0.13.2 # via junos-eznc
selectors2==2.0.1 # via ncclient
setproctitle==1.1.10
setuptools-scm==3.2.0
singledispatch==3.4.0.3 # via tornado
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, websocket-client
smmap2==2.0.5 # via gitdb2
strict-rfc3339==0.7
tempora==1.14.1 # via portend
Expand Down
4 changes: 3 additions & 1 deletion requirements/static/py3.5/darwin.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl_match_hostname==3.7.0.1
backports_abc==0.5
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +21,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.5 # via cherrypy
cherrypy==17.4.1
click==7.0
Expand All @@ -35,6 +36,7 @@ docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
enum34==1.1.6
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitdb==0.6.4
gitpython==2.1.11
Expand Down
8 changes: 4 additions & 4 deletions requirements/static/py3.5/linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5 # via tornado
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl-match-hostname==3.7.0.1 # via websocket-client
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +19,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.4 # via cherrypy
cherrypy==17.3.0
contextlib2==0.5.5 # via cherrypy
Expand All @@ -32,6 +31,7 @@ docker==3.7.2
docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitpython==2.1.11
google-auth==1.6.3 # via kubernetes
Expand All @@ -51,6 +51,7 @@ kazoo==2.6.1
keyring==5.7.1
kubernetes==3.0.0
lxml==4.3.3 # via junos-eznc, ncclient
mako==1.1.0
markupsafe==1.1.1
mock==3.0.5 ; python_version < "3.6"
more-itertools==5.0.0
Expand Down Expand Up @@ -99,8 +100,7 @@ salttesting==2017.6.1
scp==0.13.2 # via junos-eznc
setproctitle==1.1.10
setuptools-scm==3.2.0
singledispatch==3.4.0.3 # via tornado
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pathlib2, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, websocket-client
smmap2==2.0.5 # via gitdb2
strict-rfc3339==0.7
tempora==1.14.1 # via portend
Expand Down
2 changes: 2 additions & 0 deletions requirements/static/py3.5/windows.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.5 # via cherrypy
cherrypy==17.4.1
colorama==0.4.1 # via pytest
Expand All @@ -32,6 +33,7 @@ docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
enum34==1.1.6
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitdb==0.6.4
gitpython==2.1.10
Expand Down
4 changes: 3 additions & 1 deletion requirements/static/py3.6/darwin.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl_match_hostname==3.7.0.1
backports_abc==0.5
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +21,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.5 # via cherrypy
cherrypy==17.4.1
click==7.0
Expand All @@ -35,6 +36,7 @@ docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
enum34==1.1.6
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitdb==0.6.4
gitpython==2.1.11
Expand Down
8 changes: 4 additions & 4 deletions requirements/static/py3.6/linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5 # via tornado
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl-match-hostname==3.7.0.1 # via websocket-client
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +19,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.4 # via cherrypy
cherrypy==17.3.0
contextlib2==0.5.5 # via cherrypy
Expand All @@ -32,6 +31,7 @@ docker==3.7.2
docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitpython==2.1.11
google-auth==1.6.3 # via kubernetes
Expand All @@ -51,6 +51,7 @@ kazoo==2.6.1
keyring==5.7.1
kubernetes==3.0.0
lxml==4.3.3 # via junos-eznc, ncclient
mako==1.1.0
markupsafe==1.1.1
mock==2.0.0 # via moto
more-itertools==5.0.0
Expand Down Expand Up @@ -99,8 +100,7 @@ salttesting==2017.6.1
scp==0.13.2 # via junos-eznc
setproctitle==1.1.10
setuptools-scm==3.2.0
singledispatch==3.4.0.3 # via tornado
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, websocket-client
smmap2==2.0.5 # via gitdb2
strict-rfc3339==0.7
tempora==1.14.1 # via portend
Expand Down
2 changes: 2 additions & 0 deletions requirements/static/py3.6/windows.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.5 # via cherrypy
cherrypy==17.4.1
colorama==0.4.1 # via pytest
Expand All @@ -32,6 +33,7 @@ docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
enum34==1.1.6
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitdb==0.6.4
gitpython==2.1.10
Expand Down
4 changes: 3 additions & 1 deletion requirements/static/py3.7/darwin.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl_match_hostname==3.7.0.1
backports_abc==0.5
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +21,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.5 # via cherrypy
cherrypy==17.4.1
click==7.0
Expand All @@ -35,6 +36,7 @@ docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
enum34==1.1.6
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitdb==0.6.4
gitpython==2.1.11
Expand Down
8 changes: 4 additions & 4 deletions requirements/static/py3.7/linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,7 @@ asn1crypto==0.24.0 # via cryptography
atomicwrites==1.3.0 # via pytest
attrs==19.1.0 # via pytest
aws-xray-sdk==0.95 # via moto
backports-abc==0.5 # via tornado
backports.functools-lru-cache==1.5 # via cheroot
backports.ssl-match-hostname==3.7.0.1 # via websocket-client
bcrypt==3.1.6 # via paramiko
boto3==1.9.132
boto==2.49.0
Expand All @@ -21,6 +19,7 @@ cachetools==3.1.0 # via google-auth
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4 # via requests
cheetah3==3.1.0
cheroot==6.5.4 # via cherrypy
cherrypy==17.3.0
contextlib2==0.5.5 # via cherrypy
Expand All @@ -32,6 +31,7 @@ docker==3.7.2
docutils==0.14 # via botocore
ecdsa==0.13.3 # via python-jose
future==0.17.1 # via python-jose
genshi==0.7.3
gitdb2==2.0.5 # via gitpython
gitpython==2.1.11
google-auth==1.6.3 # via kubernetes
Expand All @@ -51,6 +51,7 @@ kazoo==2.6.1
keyring==5.7.1
kubernetes==3.0.0
lxml==4.3.3 # via junos-eznc, ncclient
mako==1.1.0
markupsafe==1.1.1
mock==2.0.0 # via moto
more-itertools==5.0.0
Expand Down Expand Up @@ -99,8 +100,7 @@ salttesting==2017.6.1
scp==0.13.2 # via junos-eznc
setproctitle==1.1.10
setuptools-scm==3.2.0
singledispatch==3.4.0.3 # via tornado
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, singledispatch, tempora, websocket-client
six==1.12.0 # via bcrypt, cheroot, cherrypy, cryptography, docker, docker-pycreds, google-auth, junos-eznc, kazoo, kubernetes, mock, more-itertools, moto, ncclient, packaging, pygit2, pynacl, pyopenssl, pytest, python-dateutil, python-jose, pyvmomi, responses, salttesting, tempora, websocket-client
smmap2==2.0.5 # via gitdb2
strict-rfc3339==0.7
tempora==1.14.1 # via portend
Expand Down
6 changes: 6 additions & 0 deletions requirements/static/windows.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,9 @@ virtualenv
timelib
tornado<5.0
wmi==1.4.9

# Available template libraries that can be used
Genshi
Cheetah3==3.1.0
Mako
wempy; python_version <'3'
21 changes: 20 additions & 1 deletion salt/utils/templates.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,7 +520,26 @@ def render_cheetah_tmpl(tmplstr, context, tmplpath=None):
Render a Cheetah template.
'''
from Cheetah.Template import Template
return salt.utils.data.decode(Template(tmplstr, searchList=[context]))

# Compile the template and render it into the class
tclass = Template.compile(tmplstr)
data = tclass(namespaces=[context])

# Figure out which method to call based on the type of tmplstr
if six.PY3 and isinstance(tmplstr, six.string_types):
# This should call .__unicode__()
res = str(data)
elif six.PY2 and isinstance(tmplstr, six.text_type):
# Expicitly call .__unicode__()
res = data.__unicode__()
elif isinstance(tmplstr, six.binary_type):
# This should call .__str()
res = str(data)
else:
raise SaltRenderError('Unknown type {!s} for Cheetah template while trying to render.'.format(type(tmplstr)))

# Now we can decode it to the correct encoding
return salt.utils.data.decode(res)
# pylint: enable=3rd-party-module-not-gated


Expand Down
Loading