CLI "invalid character" error message when issuing valid jobs create command #1769
Comments
|
Thanks for reporting the issue. I suspect this is caused by a mismatch between the payload and the expected schema. But clearly this is something the CLI should be able to gracefully convert to. @andrewnester Do you think this is the same as #1764? |
|
Yes, I believe it's the same and should be addressed by detecting type mismatches |
github-merge-queue bot
pushed a commit
that referenced
this issue
Oct 11, 2024
## Changes Added JSON input validation for CLI commands. Now when invalid JSON passed as a payload to CLI commands, CLI performs input normalisation and detects if there are any mismatches such as incorrect types, unknown fields and etc. This diagnostic information is printed in standard error output and does not block command execution, so the change is backward compatible. Fixes #1769 #1764 #1625 #1560 ## Tests Added unit tests ``` andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti}' Error: error decoding JSON at (inline):1:2: unexpected EOF andrew.nester@HFW9Y94129 ~ % databricks jobs create --json '{"seeti": true}' Warning: unknown field: seeti in (inline):1:9 Error: Job settings must be specified. ``` --------- Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
|
The fix has been merged in #1771 and will be released in the next CLI release |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the issue
I am issuing a jobs create command that I grabbed from the workspace UI itself (ellipsis on existing job > View JSON > move to Create tab and then "Copy Databricks CLI command".
Getting the following error message:
Error: invalid character '\'' looking for beginning of valueSteps to reproduce the behavior
databricks jobs create --profile dev --json '{"name":"dummy_job3","email_notifications":{"no_alert_for_skipped_runs":false},"webhook_notifications":{},"timeout_seconds":0,"max_concurrent_runs":1,"tasks":[{"task_key":"nb1","run_if":"ALL_SUCCESS","notebook_task":{"notebook_path":"/Workspace/Users/REDACTED/dummy_notebooks/notebook1","source":"WORKSPACE"},"job_cluster_key":"Job_cluster","timeout_seconds":0,"email_notifications":{}}],"job_clusters":[{"job_cluster_key":"Job_cluster","new_cluster":{"spark_version":"14.3.x-scala2.12","spark_conf":{"spark.sql.hive.metastore.jars":"maven","spark.hadoop.javax.jdo.option.ConnectionDriverName":"org.mariadb.jdbc.Driver","databricks.loki.fileStatusCache.enabled":"false","spark.hadoop.javax.jdo.option.ConnectionPassword":"{{secrets/metastore/dev-password}}","spark.hadoop.javax.jdo.option.ConnectionURL":"{{secrets/metastore/connectionuri}}","spark.hadoop.javax.jdo.option.ConnectionUserName":"{{secrets/metastore/dev-userid}}","spark.sql.hive.metastore.version":"3.1.0"},"gcp_attributes":{"use_preemptible_executors":false,"google_service_account":"REDACTED","availability":"ON_DEMAND_GCP","zone_id":"auto"},"node_type_id":"n2-standard-4","driver_node_type_id":"n2-standard-4","custom_tags":{"Team":"REDACTED","Project":"REDACTED","Domain":"REDACTED","Priority":"P5"},"spark_env_vars":{"SNOWFLAKE_SPARK_CONNECTOR_VERSION":"2.12"},"enable_elastic_disk":false,"policy_id":"0008B1A76BC0A34A","data_security_mode":"SINGLE_USER","runtime_engine":"STANDARD","autoscale":{"min_workers":1,"max_workers":1}}}],"queue":{"enabled":true},"run_as":{"user_name":"REDACTED"}}'Expected Behavior
Job created successfully
Actual Behavior
Error: invalid character '\'' looking for beginning of valueOS and CLI version
Databricks CLI v0.228.0
OS is Windows 11 Enterprise
Against a GCP databricks (not sure if that is relevant)
Is this a regression?
Yes
Debug Logs
output.txt
The text was updated successfully, but these errors were encountered: