Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Query anonymization doesn't work in fallback log #2803

Open
LantaoJin opened this issue Jul 4, 2024 · 1 comment · May be fixed by #2804
Open

[BUG] Query anonymization doesn't work in fallback log #2803

LantaoJin opened this issue Jul 4, 2024 · 1 comment · May be fixed by #2804
Labels
bug Something isn't working

Comments

@LantaoJin
Copy link
Member

What is the bug?
Query anonymization is not working in the fallback log

How can one reproduce the bug?
Write a simple query and add ?format=json to trigger falling back to old engine.

POST /_plugins/_sql?format=json
{
  "query" : """
  SELECT min(DistanceMiles), avg(DistanceMiles), max(DistanceMiles)
    FROM opensearch_dashboards_sample_data_flights
      GROUP BY Carrier
  """
}

Or write a query which is unsupported in V2, e.g JOIN

POST /_plugins/_sql
{
  "query" : """
  SELECT s.FlightNum, s.OriginCityName, d.DestCityName
    FROM opensearch_dashboards_sample_data_flights s
      JOIN opensearch_dashboards_sample_data_flights d
        ON s.DestCityName = d.OriginCityName
  """
}

See this line in the logs

Request SQLQueryRequest(jsonContent={"query":"\n SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier\n "}, query=
SELECT min(DistanceMiles),avg(DistanceMiles),max(DistanceMiles) FROM opensearch_dashboards_sample_data_flights GROUP BY Carrier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

Request SQLQueryRequest(jsonContent={"query":"\n SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName\n "}, query=
SELECT s.FlightNum, s.OriginCityName, d.DestCityName FROM opensearch_dashboards_sample_data_flights s JOIN opensearch_dashboards_sample_data_flights d ON s.DestCityName = d.OriginCityName
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

What is the expected behavior?
Logs should show

Request SQLQueryRequest(query=
SELECT MIN ( identifier ), AVG ( identifier ), MAX ( identifier ) FROM table GROUP BY identifier
, path=/_plugins/_sql, format=json, params={pretty=true, format=json}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

Request SQLQueryRequest(query=
SELECT identifier, identifier, identifier FROM table s JOIN table d ON identifier = identifier
, path=/_plugins/_sql, format=jdbc, params={pretty=true}, sanitize=true, cursor=Optional.empty) is not supported and falling back to old SQL engine

What is your host/environment?

  • Version 2.15
  • Plugins SQL

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

@LantaoJin LantaoJin added bug Something isn't working untriaged labels Jul 4, 2024
@LantaoJin LantaoJin linked a pull request Jul 4, 2024 that will close this issue
3 tasks
@dblock
Copy link
Member

dblock commented Jul 22, 2024

[Catch All Triage w/ 1, 2, 3]

@dblock dblock removed the untriaged label Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants