Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: read the request body from the temporary file if it was cached. #1863

Merged
merged 7 commits into from
Jul 29, 2020
Merged

bugfix: read the request body from the temporary file if it was cached. #1863

merged 7 commits into from
Jul 29, 2020

Conversation

membphis
Copy link
Member

@membphis membphis commented Jul 17, 2020

What this PR does / why we need it:

as title, fixed #1784

perf: update the jsonschema to v0.9 .

used hash object to check if there was a duplicated value in the array object,
the current way is 10~100 times than the old way.

Pre-submission checklist:

  • Did you explain what problem does this PR solve? Or what new features have been added?
  • Have you added corresponding test cases?
  • Have you modified the corresponding document?
  • Is this PR backward compatible?

@membphis membphis mentioned this pull request Jul 17, 2020
4 tasks
@@ -47,7 +47,7 @@ dependencies = {
"luafilesystem = 1.7.0-2",
"lua-tinyyaml = 0.1",
"lua-resty-prometheus = 1.1",
"jsonschema = 0.8",
"jsonschema = 0.9",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not modify this line

@@ -288,7 +288,7 @@ GET /t
GET /t
--- error_code: 400
--- response_body
{"error_msg":"invalid configuration: property \"upstream\" validation failed: property \"checks\" validation failed: property \"active\" validation failed: property \"healthy\" validation failed: property \"http_statuses\" validation failed: expected unique items but items 2 and 1 are equal"}
{"error_msg":"invalid configuration: property \"upstream\" validation failed: property \"checks\" validation failed: property \"active\" validation failed: property \"healthy\" validation failed: property \"http_statuses\" validation failed: expected unique items but items 1 and 2 are equal"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change this test case?

@membphis
Copy link
Member Author

https://github.com/apache/incubator-apisix/pull/1863/files#diff-00625723b6e737f3cdb18af67165b70fR1899-R1901

@moonming in this test case, we found two bugs.

  1. big request body for admin API
  2. slower to detect whether the objects in the test array are unique (jsonschema).

image

@moonming
Copy link
Member

moonming commented Jul 20, 2020 via email

if req_body then
if #req_body > 1024 * 1024 * 1.5 then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to detect the size (use stat, for example) before reading it. It doesn't make sense to do the check if we have already read the very BIG file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed.

if req_body then
if #req_body > 1024 * 1024 * 1.5 then
core.log.error("maximum request body is 1.5mb, but got ", #req_body)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, we can use MiB to indicate the size is 1024 base.



local function get_file(file_name)
local f = io_open(file_name, 'r')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to check the second error returned value before using the f.

local function get_file(file_name)
local f = io_open(file_name, 'r')
if f then
local req_body = f:read("*all")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

@membphis membphis merged commit 20207c8 into apache:master Jul 29, 2020
@membphis membphis deleted the bug-big-route-size branch July 29, 2020 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: /admin/routes return {"error_msg":"missing configurations"}
4 participants