Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: refactor error log for manager api #689

Merged
merged 22 commits into from
Nov 6, 2020
Merged

chore: refactor error log for manager api #689

merged 22 commits into from
Nov 6, 2020

Conversation

nic-chen
Copy link
Member

@nic-chen nic-chen commented Nov 4, 2020 •
edited
Loading

close #678

  1. Support custom error log level
  2. Support saving to a local file, and can customize the file path

membphis
membphis reviewed Nov 4, 2020
View reviewed changes
Copy link
Member

@membphis membphis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add more log by different new PRs, correct is enough.

Missing tests, we need a way to confirm that the error log can be written to a local file.

api/conf/conf.go Outdated Show resolved Hide resolved
api/filter/logging.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Nov 4, 2020 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (v2.0@b4b646b). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             v2.0     #689   +/-   ##
=======================================
  Coverage        ?   43.16%           
=======================================
  Files           ?       18           
  Lines           ?     1237           
  Branches        ?        0           
=======================================
  Hits            ?      534           
  Misses          ?      614           
  Partials        ?       89

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4b646b...30d8b73. Read the comment docs.

membphis
membphis requested changes Nov 4, 2020
View reviewed changes
api/conf/conf.json Outdated Show resolved Hide resolved
api/log/zap.go Show resolved Hide resolved
api/test/shell/cli_test.sh Show resolved Hide resolved
.github/workflows/cli_test.yml Show resolved Hide resolved
api/conf/conf.json Outdated Show resolved Hide resolved
@membphis
Copy link
Member

membphis commented Nov 5, 2020

api/logs/.gitkeep to api/logs/placeholder.txe, more readable.

@membphis
Copy link
Member

membphis commented Nov 5, 2020

@nic-chen after you fixed the CI, then we can turn this PR to review ready ^_^

@membphis membphis marked this pull request as ready for review November 5, 2020 13:00
@nic-chen
Copy link
Member Author

nic-chen commented Nov 5, 2020

Relative path for log file depends on this pr #693

@johzchen
fix typo
f3c5dc7
@johzchen
Merge branch 'v2.0' into log-refactor
74a89c4 
# Conflicts:
#	api/conf/conf.go
#	api/main.go
membphis
membphis reviewed Nov 6, 2020
View reviewed changes
api/conf/conf.json Outdated Show resolved Hide resolved
@membphis
Copy link
Member

membphis commented Nov 6, 2020

api/logs/.gitkeep to api/logs/placeholder.txe, more readable.

ping @nic-chen

@membphis
Copy link
Member

membphis commented Nov 6, 2020

CI is failed, please fix them first

@nic-chen
Copy link
Member Author

nic-chen commented Nov 6, 2020

api/logs/.gitkeep to api/logs/placeholder.txe, more readable.

ping @nic-chen

OK

@nic-chen
Copy link
Member Author

nic-chen commented Nov 6, 2020

CI is failed, please fix them first

working on it

membphis
membphis approved these changes Nov 6, 2020
View reviewed changes
Copy link
Member

@membphis membphis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@juzhiyuan juzhiyuan merged commit 3a405f7 into apache:v2.0 Nov 6, 2020
LiteSun added a commit that referenced this pull request Nov 21, 2020
@LiteSun @nic-chen
@juzhiyuan @jiayx @johzchen @liuxiran @membphis @gxthrj @idbeta @starsz @EnableAsync @ShiningRush
release: release 2.0-rc version. (#598)
b989a0f 
* Update CHANGELOG.md

* Update CHANGELOG.zh-CN.md

* Update CHANGELOG.md

* Update CHANGELOG.zh-CN.md

* Update CHANGELOG.md

* feat: update deploy docs

* feat: separate build and run to 2 scripts (#600)

* feat: separate build and run

* doc: update doc about build and run

* doc: update doc about build and run

* Update schema-sync.sh

* Update develop.md

* Update develop.zh-CN.md

Co-authored-by: 琚致远 <juzhiyuan@apache.org>

* feat: remove unused dist folder

* doc: add etcd api version tips (#604)

* doc: add etcd version tips(#602)

* Update README.md

* Update README.zh-CN.md

Co-authored-by: 琚致远 <juzhiyuan@apache.org>

* Update deploy.md

* Update deploy.zh-CN.md

* feat(docs): update deploy docs

* feat(doc): added tip in deploy

* feat(doc): update run part in deploy

* feat(doc): added FAQ about how to redeploy

* fix: 2.0 release issues (#606)

* fix: remove json schema generation  from ci

* fix: use json schema generated from APISIX v2.0

* fix: ant-design/ant-design/issues/27396

* fix: using relative path to read conf (#617)

* fix: using  relative path

* fix path

* fix path

* feat: support custom server host, port and DAG lib path (#625)

* feat: support custom server port and dag lib path

* test: custom host

* test: add test case

* fix bug

* test

* test

* test

* feat support config etcd endpoints

* Update conf.json

* Update conf.go

* Update test-api.yml

* Update deploy.md

* Update deploy.zh-CN.md

* Update deploy.md

* Update conf.json

Co-authored-by: 琚致远 <juzhiyuan@apache.org>

* feat(doc): update config file

* feat(doc): update Note

* feat(docs): update NOTE

* feat(doc): update ETCD endpoints

* Update deploy.md

* feat: refactor folders (#629)

* feat: refactor api & frontend

* feat: remove some actions

* feat: added - v2.0

* feat: added ignored files for checker

* feat: trigger ci

* feat: trigger CI

* feat: added conf.json

* fix: build dashboard

* feat: remove demo temp

* feat: update build.sh

* chore: improve issue template and vscode (#660)

* test: add e2e test for field hosts in `route` api (#612)

* feat: ETCD cluster and APISIX cluster for CI

* feat: add test cases

* feat: add upstream service in docker compose

* fix: comment

* test: add test cases

* fix: code format

* test: add ci

* fix ci

* fix ci

* fix ci

* fix: remove consumer test

* test: sleep for sync

* test: e2e as an independent subproject

* fix CI error

* test: run docker

* fix: remove json schema generate script in docker build

* fix: check host and hosts config together

* fix ci

* test: add test cases for `host` in route

* remove useless code

* fix: using relative path to read conf (#617)

* fix: using  relative path

* fix path

* fix path

* fix conf path

* fix ci error

* fix etcd ip

* fix: code format

* run backend e2e test ci on v2 branch

* fix: code format

* fix: code format

* fix: CI error

* test: remove deploy CI again

* remove useless codes

* fix: go fmt

* test: don't use `go fmt`

* fix: code format

* fix: var name

* fix lint

* fix CI error

* debug

* fix: docker container name

* fix CI error

* fix CI error

* test: add more test cases

* fix CI error

* chore: remove useless code

* fix: go fmt

* fix: refactor test code

* fix: check body

* fix small issue

* fix: update docker compose

* fix CI

* test

* test

* test: build image first

* test: revert subnet ip in docker compose

* fix by review

* fix: docker compose

* test: add test cases

* fix: remove useless code

* fix: code format

* test: code format

* feat: added e2e test for Login page (#619)

* feat: added Front-end e2e test YAML file

* feat: added login e2e test

* feat: update Login.e2e.js

* Update Login.e2e.js

* feat: added e2e readme

* feat: added licence

* feat: added  start-server-and-test package

* feat: update login test case

* Update frontend-e2e-test.yml

* feat: added logout test case

* Update frontend-e2e-test.yml

* feat: added login failed with empty input

* feat: update CI

* feat: update text

* feat: added public.js

* feat: change logout timeout

* feat: Added e2e test documentation link to development.md

* Update develop.md

* Update develop.zh-CN.md

* Update README.md

Co-authored-by: 琚致远 <juzhiyuan@apache.org>

* CI: collect golang unit test code coverage (#654)

* fix: promethus incorrect value when update route (#666)

* feat: deploy with Docker (#657)

* feat: added Docker

* feat: added License header

* feat: added line in docs

* Update test-docker.yml

* feat(doc): update docs

* feat: added extra line

* feat: added Go Proxy in Dockerfile

* feat: update Dockerfile

* chore: update docs

* fix: copy correct files

* feat: improve Dockerfile

* Revert "feat: improve Dockerfile"

This reverts commit c68a4c4.

* fix run fail

* feat: update docs

* fix: compatible with Golang conf

* Squashed commit of the following:

commit 94450bf
Author: litesun <7sunmiao@gmail.com>
Date:   Tue Nov 3 10:19:04 2020 +0800

    fix: promethus incorrect value when update route (#666)

commit de8bdbf
Author: nic-chen <33000667+nic-chen@users.noreply.github.com>
Date:   Tue Nov 3 07:50:29 2020 +0800

    CI: collect golang unit test code coverage (#654)

commit 722c0fd
Author: litesun <7sunmiao@gmail.com>
Date:   Tue Nov 3 00:03:06 2020 +0800

    feat: added e2e test for Login page (#619)

    * feat: added Front-end e2e test YAML file

    * feat: added login e2e test

    * feat: update Login.e2e.js

    * Update Login.e2e.js

    * feat: added e2e readme

    * feat: added licence

    * feat: added  start-server-and-test package

    * feat: update login test case

    * Update frontend-e2e-test.yml

    * feat: added logout test case

    * Update frontend-e2e-test.yml

    * feat: added login failed with empty input

    * feat: update CI

    * feat: update text

    * feat: added public.js

    * feat: change logout timeout

    * feat: Added e2e test documentation link to development.md

    * Update develop.md

    * Update develop.zh-CN.md

    * Update README.md

    Co-authored-by: 琚致远 <juzhiyuan@apache.org>

commit 915ce83
Author: nic-chen <33000667+nic-chen@users.noreply.github.com>
Date:   Mon Nov 2 21:13:34 2020 +0800

    test: add e2e test for field hosts in `route` api (#612)

    * feat: ETCD cluster and APISIX cluster for CI

    * feat: add test cases

    * feat: add upstream service in docker compose

    * fix: comment

    * test: add test cases

    * fix: code format

    * test: add ci

    * fix ci

    * fix ci

    * fix ci

    * fix: remove consumer test

    * test: sleep for sync

    * test: e2e as an independent subproject

    * fix CI error

    * test: run docker

    * fix: remove json schema generate script in docker build

    * fix: check host and hosts config together

    * fix ci

    * test: add test cases for `host` in route

    * remove useless code

    * fix: using relative path to read conf (#617)

    * fix: using  relative path

    * fix path

    * fix path

    * fix conf path

    * fix ci error

    * fix etcd ip

    * fix: code format

    * run backend e2e test ci on v2 branch

    * fix: code format

    * fix: code format

    * fix: CI error

    * test: remove deploy CI again

    * remove useless codes

    * fix: go fmt

    * test: don't use `go fmt`

    * fix: code format

    * fix: var name

    * fix lint

    * fix CI error

    * debug

    * fix: docker container name

    * fix CI error

    * fix CI error

    * test: add more test cases

    * fix CI error

    * chore: remove useless code

    * fix: go fmt

    * fix: refactor test code

    * fix: check body

    * fix small issue

    * fix: update docker compose

    * fix CI

    * test

    * test

    * test: build image first

    * test: revert subnet ip in docker compose

    * fix by review

    * fix: docker compose

    * test: add test cases

    * fix: remove useless code

    * fix: code format

    * test: code format

commit 94d0245
Author: 琚致远 <juzhiyuan@apache.org>
Date:   Mon Nov 2 13:17:42 2020 +0800

    chore: improve issue template and vscode (#660)

* Revert "Squashed commit of the following:"

This reverts commit 93d38ee.

* feat: remove output from ignore files

Co-authored-by: nic-chen <johz@163.com>

* feat: use web instead of frontend (#674)

* feat: use web instead of frontend

* feat: rename frontend to web

* feat: remove all frontend to web

* feat: support get plugin schema based on schema_type (#651)

* feat: support get plugin schema based on schema_type

* fix: ci errors

* fix: run error casued by api-breaker.lua

* fix: get schema_type from query and add some test

* fix: update schema.json file

* fix: update validate to support schematype

* fix: properties:{} validate failed

* fix: some code errors refer to the review

* test: add linter for manager api (#655)

* test: add go lint

* fix lint

* fix by review

* fix errors

* test: run lint on v2.0

* fix: text format

* fix: os check

* fix typo

* fix: fmt --> log

* fix log

* fix: unnecessary nil check around range for lint

* fix: trigger lint for push to branch v2.0

* feat: update plugin to 1.0.10

* fix: enable HTTPS setting unsuccessful in Route (#692)

* fix: wrong stepHeader after edit MatchingRules

* feat: disable plugin orchestration when select forcehttps

* feat: update Route transform

* feat: cache Front-end e2e CI node_modules (#696)

* chore: refactor `conf` of `manager api` (#693)

* feat: refactor conf

* fix default listen port

* fix build and run scripts

* fix: docs

* chore: remove useless file

* fix docker for test

* fix CI

* fix CI

* fix ci

* fix: `-c` conf dir  -->  `-p` work dir

* fix go test error

* fix conf

* fix: revert changes

* fix: remove useless comment

* fix: remove useless comment

* doc: add comment for config

* doc: todo

* fix: config format

* fix: if secret use default value, should generate a random string to replace it.

* fix comment style

* fix: change web dir in docker file

* doc: update comments

* fix:bug that dirty data exists after updating route and wrong mod for prod env (#704)

* fix: route bug, dirty data exists after updating

* fix mod for env

* fix mod for env

* fix error log

* fix error log

* chore: refactor error log for `manager api` (#689)

* chore: refactor log

* fix: custom log by conf

* feat: add error log

* fix default config

* fix CI fail

* fix: should not save log to file by default

* test: add test case

* test: add test case

* fix CI fail

* fix error

* fix CI

* fix error

* fix according to reviews

* test: more test cases

* fix error

* chore: use `/dev/stdout` as default log file path

* fix typo

* fix docker for logs dir

* fix CI fail

* fix: delete useless files

* fix: change file name

* bugfix: dashboard 2.0 failed to fetch ssl certificate not found  (#719)

* fix: set ssl status, since it's default 0

* add test cases

* test: add test cases

* fix: update test cases

* fix: test case fail

* test: remove hosts in hosts setting

* fix: according review

* fix: according review

* chore: add comment

* fix: update order (#744)

* feat(Consumer): use username instead of id (#742)

* feat(Consumer): use username instead of id

* feat: remove duplicated var

* chore: exit if any error and specify the download file name when download by `wget`. (#751)

fix #646

* fix: an error will occur if `pass_host` is set to `node` when creating upstream (#750)

* fix: bug #749

* fix: bug #749

* test: add test cases

* fix: CI

* fix: CI fail

* test: add e2e test cases for upstream (#738)

* fix: react warnings (#747)

* fix: update json schema (#754)

* chore: move the Dockerfile to `test` folder, because it was used for testing (#753)

* feat: convert uri to uris (#740)

* feat: deploy with docker (#701)

* feat: added deploy with docker CD

* feat: docker deploy test

* fix: CI

* fix: path of `config.yaml`

* fix: CI fail

* docs: update doc for docker deploy

* fix: typo

* fix: add EOF && trigger  CD

Co-authored-by: nic-chen <johz@163.com>
Co-authored-by: kv <gxthrj@163.com>

* fix: invalid values from the manager-api (#736)

* feat: return None when timestamp is invalid

* chore: added TODO

* docs: add makefile && modify  develop and deploy  docs (#729)

* docs: add makefile && modify  develop and deploy  docs

* docs: Make the makefile clearer

* docs: modify frontend to web

* doc: two blankline between commands

* doc: remove blanklien EOF

* docs: remove make run/stop in makefile

* docs: make readme readable

* docs: declear in readme

* docs: make readme readable

* docs: remove dependencies in readme

* docs: check english desc

* docs: style adjust

* docs: sync makefile desc

* docs: remove blank

* docs: style adjust

* docs: remove ENV=local in deploy

* fix: modify lint ci

* docs: unify go-lint

* docs: check all text

* docs: makefile Aligned

* docs: update style

* docs: add markdown code style

* docs: fix  syntax

* docs: remove Self-referencing

* docs: user guide

* docs: use header

* docs: style fix

* docs: add desc

* docs: modify 'pack' to 'package'

* docs: unify manager-api

* docs: mv Install to Installation

* docs: node to Node.js

* docs: source codes to Source Codes

* docs: Source Codes

* fix: mkdir -p ./output/logs in makefile build

* feat: support specifying APISIX path to generate json schema (#765)

* feat: support specifying APISIX path to generate json schema

* doc: update doc about schema sync

* fix: doc

* fix: errors

* fix: error according to test

* test

* no message

* test

* fix: remove debug

* test: add consumer e2e test (#735)

* add consumer e2e test
add public method "PartialBody"

* test: add end with EOL

* add test data plane to case 2
delete some useless code

* modify code style

* fix: makefile build error (#767)

* fix: used dashboard add consumer of jwt, would have an error when get the jwt token (#768)

* feat: update changelog (#771)

* Update CHANGELOG.md

* Update CHANGELOG.zh-CN.md

* docs: update change log

Co-authored-by: nic-chen <johz@163.com>

* fix: consumer schema for auth plugin (#770)

* fix: consumer schema for auth plugin

* fix: update by jsonschema

* fix: json schema

* fix: doc (#772)

* fix: doc (#774)

* feat: remove CD for PR

* fix: closed WatchResponse channel when cancel function is called (#779) (#795)

* ci: fix CI naming (#799)

* fix: make cli test more compatible (#798)

* test: fix e2e test unstable (#800)

* fix: double scroll bar in plugin page (#801)

* feat(docs): improve README & Deploy (#785)

* feat(doc): update README & deploy

* feat(docs): added FAQ

* fix: linkx

* feat(docs): update deploy with docker

* feat(docs): update FAQ

* feat: update README

* feat: update README

* feta: update README

* fix: link

* feat: update deploy with docker

* feat(docs): added tip for some users

* feat(doc): improve deploy with docker

* feat(docs): improve deploy with docker

* feat(doc): update run with docker

* feat(doc): improve deploy

* feat(doc): added more info

* feat(doc): added more info

* feat(doc): remove extra info

* feat: added more detailed

* fix: CI fail according `api/conf/conf.yaml ` is  changed.

* fix: revert changed

* feat(docs): improve English version

* feat(docs): use frontend instead, just like backend

* feat(docs): remove extra statements

* feat(docs): update typo

* feat(docs): remove startup with message

* feat: update CI's name

Co-authored-by: nic-chen <johz@163.com>

* feat: skip puppeteer chromium download when build (#808)

* feat(Makefile): update release-src (#816)

* ci: fix CI fail (#818)

* ci: fix CI fail

* ci: fix CI fail

* ci: fix CI fail

* test: add e2e test for config route with service_id or upstream_id (#810)

* test: add e2e test for config route with service_id or upstream_id

* test: fix route test

* test: fix route test

* test: fix route test

* test: fix route test format

* test: fix route test format

* test: fix test format

* test: add test on data plane

* test: fix test conflict

* feat: install signal handler for graceful shutdown (#737) (#796)

* Revert "test: add consumer e2e test (#735)" (#829)

This reverts commit c140f41.

* feat: add a hanlder unit test for upstream and remove init

* append license

* revert unreviewed pr (#841)

Co-authored-by: nic-chen <33000667+nic-chen@users.noreply.github.com>
Co-authored-by: 琚致远 <juzhiyuan@apache.org>
Co-authored-by: jiayx <jiayx@users.noreply.github.com>
Co-authored-by: nic-chen <johz@163.com>
Co-authored-by: liuxiran <belovedxixi@126.com>
Co-authored-by: YuanSheng Wang <membphis@gmail.com>
Co-authored-by: kv <gxthrj@163.com>
Co-authored-by: idbeta <idbeta@gmail.com>
Co-authored-by: Peter Zhu <starszcan@gmail.com>
Co-authored-by: EnableAsync <43645467+EnableAsync@users.noreply.github.com>
Co-authored-by: ShiningRush <277040271@qq.com>
@nic-chen nic-chen deleted the log-refactor branch June 16, 2021 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@membphis membphis membphis approved these changes

@juzhiyuan juzhiyuan juzhiyuan approved these changes

@gxthrj gxthrj Awaiting requested review from gxthrj

@moonming moonming Awaiting requested review from moonming

@spacewander spacewander Awaiting requested review from spacewander

Assignees
No one assigned
Labels
backend
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@nic-chen @codecov-io @membphis @juzhiyuan @johzchen