Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(wsgi): remove some appsec code from wsgi contrib #6326

Merged
merged 80 commits into from
Jul 17, 2023
Merged
Show file tree
Hide file tree
Changes from 77 commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
9dc7fc0
early sketches of the core api
emmettbutler Jun 1, 2023
e4533f4
early sketches of the core api
emmettbutler Jun 1, 2023
5b3d274
fix test assertion about called context end handler
emmettbutler Jun 2, 2023
0f8639b
add a basic multithread listener test
emmettbutler Jun 2, 2023
562024c
test concurrent event dispatch
emmettbutler Jun 2, 2023
aca7c1e
test context parenting across threads
emmettbutler Jun 2, 2023
b6708c8
hacking on core api
emmettbutler Jun 5, 2023
7932691
reminder comment
emmettbutler Jun 7, 2023
4613257
make a test that fails with data leaking between asyncio tasks
emmettbutler Jun 20, 2023
c6ea21a
make test pass by using a contextvar
emmettbutler Jun 20, 2023
23d3dba
reset contextvar on context end
emmettbutler Jun 20, 2023
f4f888c
turn ddtrace.internal._context into a thin wrapper around the core api
emmettbutler Jun 20, 2023
df519aa
make appsec context use a core context
emmettbutler Jun 20, 2023
7587563
core api tests passing with more sensible use of contextvars
emmettbutler Jun 20, 2023
9be88a2
make test_telemetry_metrics_attack test pass by putting context data …
emmettbutler Jun 20, 2023
b47ab26
hacking progress - replacing _ASM contextvar
emmettbutler Jun 22, 2023
1f51779
Revert "hacking progress - replacing _ASM contextvar"
emmettbutler Jun 22, 2023
26a114f
use core instead of asm contextvar in datahandler
emmettbutler Jun 22, 2023
614b16b
fully replace _ASM contextvar
emmettbutler Jun 22, 2023
dd7f972
store everything on the entire context ancestor tree to maintain comp…
emmettbutler Jun 22, 2023
f82dc70
give every span an executioncontext
emmettbutler Jun 22, 2023
35a07e1
deduplicate root context id
emmettbutler Jun 22, 2023
895175d
update test that accesses private stuff
emmettbutler Jun 22, 2023
eb4f007
close span-held executioncontext correctly
emmettbutler Jun 23, 2023
d61805b
bugfixes
emmettbutler Jun 23, 2023
6b07a6b
make _context wrapper code trivial
emmettbutler Jun 23, 2023
69f8239
do typechecking properly
emmettbutler Jun 23, 2023
50e3652
replace _context with core
emmettbutler Jun 23, 2023
303a186
make eventhub context-local
emmettbutler Jun 23, 2023
3823003
delete test that was based on a faulty premise
emmettbutler Jun 23, 2023
906a8c8
make test work in py2
emmettbutler Jun 23, 2023
4a84d9d
unused imports
emmettbutler Jun 23, 2023
df5f56f
formatting
emmettbutler Jun 23, 2023
c9fd843
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jun 23, 2023
6c21948
flake8
emmettbutler Jun 23, 2023
42deb69
mypy
emmettbutler Jun 23, 2023
43e9517
don't die on valueerrors from context reset'
emmettbutler Jun 23, 2023
5b62e6f
fix line numbers in test expectations
emmettbutler Jun 23, 2023
929a530
fix set_items
emmettbutler Jun 23, 2023
a3a8f8b
regenerate flask snapshots
emmettbutler Jun 23, 2023
9231e25
remove unused context management code from Span
emmettbutler Jun 23, 2023
d3676aa
undo flask snapshots
emmettbutler Jun 23, 2023
2c1c7e6
undo file deletions
emmettbutler Jun 23, 2023
281237e
remove outdated comment
emmettbutler Jun 23, 2023
ef15279
use core API instead of span context data in otel
emmettbutler Jun 23, 2023
a656c96
some adjustments to try and fix unclear failing tests
emmettbutler Jun 23, 2023
e2c652b
make assertion a little more permissive to account for apparent incon…
emmettbutler Jun 23, 2023
d584690
silly wild guess: exit the context before callbacks instead of after
emmettbutler Jun 23, 2023
8660805
Revert "silly wild guess: exit the context before callbacks instead o…
emmettbutler Jun 23, 2023
e867304
hypothesis: managing _CURRENT_CONTEXT is slow enough that it causes t…
emmettbutler Jun 26, 2023
128a93a
try skipping the context manager entirely
emmettbutler Jun 26, 2023
0b04c52
fix ctx_api test to work with how spans are handled
emmettbutler Jun 26, 2023
69f49e3
encapsulate parenting logic to improve maintainability
emmettbutler Jun 26, 2023
7aa900b
don't use spans for cross-process context sharing
emmettbutler Jun 26, 2023
f5aa235
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jun 26, 2023
69d29d1
don't use contextmanager hackery when there's a simpler API available
emmettbutler Jun 26, 2023
0585486
attempt some optimizations on internal.core
emmettbutler Jun 28, 2023
db1d8ae
when spans are involved, use the old method of context storage becaus…
emmettbutler Jun 28, 2023
3939908
prechecks in core
emmettbutler Jun 28, 2023
f156796
don't look for nonexistent children attribute in tests
emmettbutler Jun 28, 2023
53ebc9d
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jun 28, 2023
0c30690
remove old assertions
emmettbutler Jun 28, 2023
046ffb8
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 5, 2023
2b4bf4d
precheck
emmettbutler Jul 5, 2023
0d2813e
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 6, 2023
d735b79
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 7, 2023
e71ea47
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 7, 2023
48f0d0e
update core from latest dev branch
emmettbutler Jul 7, 2023
033d96c
mypy
emmettbutler Jul 7, 2023
1a1afb7
fix failing tests
emmettbutler Jul 7, 2023
62f7735
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 7, 2023
9c9ee0b
chore(ci-visibility): test suite skipping using ``pytest_collection_m…
gnufede Jul 10, 2023
597be87
Merge branch '1.x' into emmett.butler/core-api
emmettbutler Jul 10, 2023
ac60364
move a bit of asm-specific logic out of wsgi contrib
emmettbutler Jul 11, 2023
0c1743c
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
emmettbutler Jul 12, 2023
96ca10c
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
emmettbutler Jul 13, 2023
fabebf3
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
emmettbutler Jul 14, 2023
7c8105e
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
emmettbutler Jul 14, 2023
2947218
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
emmettbutler Jul 14, 2023
7db176f
Merge branch '1.x' into emmett.butler/wsgi-no-appsec
ZStriker19 Jul 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions ddtrace/appsec/_asm_request_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def __init__(self):

def finalise(self):
if self.active:
env = core.get_item("asm_env")
env = self.execution_context.get_item("asm_env")
# assert _CONTEXT_ID.get() == self._id
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved
callbacks = GLOBAL_CALLBACKS.get(_CONTEXT_CALL, []) + env.callbacks.get(_CONTEXT_CALL)
if callbacks is not None:
Expand Down Expand Up @@ -319,12 +319,26 @@ def asm_request_context_manager(
The ASM context manager
"""
if config._appsec_enabled:
resources = _DataHandler()
asm_request_context_set(remote_ip, headers, headers_case_sensitive, block_request_callable)
resources = _on_context_started(remote_ip, headers, headers_case_sensitive, block_request_callable)
try:
yield resources
finally:
resources.finalise()
core.set_item("asm_env", None)
_on_context_ended(resources)
else:
yield None


def _on_context_started(remote_ip, headers, headers_case_sensitive, block_request_callable):
resources = _DataHandler()
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved
asm_request_context_set(remote_ip, headers, headers_case_sensitive, block_request_callable)
core.on("wsgi.block_decided", _on_block_decided)
return resources


def _on_context_ended(resources):
resources.finalise()
core.set_item("asm_env", None)


def _on_block_decided(callback):
set_value(_CALLBACKS, "flask_block", callback)
15 changes: 5 additions & 10 deletions ddtrace/contrib/wsgi/wsgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,8 @@
import sys
from typing import TYPE_CHECKING

from ddtrace.appsec import _asm_request_context
from ddtrace.internal.schema.span_attribute_schema import SpanDirection

from ...appsec._constants import SPAN_DATA_NAMES
from ..trace_utils import _get_request_header_user_agent
from ..trace_utils import _set_url_tag

Expand All @@ -27,6 +25,9 @@

import ddtrace
from ddtrace import config
from ddtrace.appsec import _asm_request_context
from ddtrace.appsec import utils
from ddtrace.appsec._constants import SPAN_DATA_NAMES
from ddtrace.ext import SpanKind
from ddtrace.ext import SpanTypes
from ddtrace.ext import http
Expand All @@ -38,7 +39,6 @@
from ddtrace.vendor import wrapt

from .. import trace_utils
from ...appsec import utils
from ...appsec._constants import WAF_CONTEXT_NAMES
from ...constants import SPAN_KIND
from ...internal import core
Expand Down Expand Up @@ -165,29 +165,25 @@ def __call__(self, environ, start_response):
headers = get_request_headers(environ)
closing_iterator = ()
not_blocked = True
with _asm_request_context.asm_request_context_manager(
environ.get("REMOTE_ADDR"), headers, headers_case_sensitive=True
):
with _asm_request_context.asm_request_context_manager(environ.get("REMOTE_ADDR"), headers, True):
req_span = self.tracer.trace(
self._request_span_name,
service=trace_utils.int_service(self._pin, self._config),
span_type=SpanTypes.WEB,
)

if self.tracer._appsec_enabled:
# [IP Blocking]
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved
if core.get_item(WAF_CONTEXT_NAMES.BLOCKED, span=req_span):
ctype, content = self._make_block_content(environ, headers, req_span)
start_response("403 FORBIDDEN", [("content-type", ctype)])
closing_iterator = [content]
not_blocked = False

# [Suspicious Request Blocking on request]
def blocked_view():
ctype, content = self._make_block_content(environ, headers, req_span)
return content, 403, [("content-type", ctype)]

_asm_request_context.set_value(_asm_request_context._CALLBACKS, "flask_block", blocked_view)
core.dispatch("wsgi.block_decided", [blocked_view])
emmettbutler marked this conversation as resolved.
Show resolved Hide resolved

if not_blocked:
req_span.set_tag_str(COMPONENT, self._config.integration_name)
Expand All @@ -212,7 +208,6 @@ def blocked_view():
req_span.finish()
raise
if self.tracer._appsec_enabled and core.get_item(WAF_CONTEXT_NAMES.BLOCKED, span=req_span):
# [Suspicious Request Blocking on request or response]
_, content = self._make_block_content(environ, headers, req_span)
closing_iterator = [content]

Expand Down
Loading