Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source Bigquery: com.google.cloud.bigquery.BigQueryException: Arrays of arrays are not supported at #4993

Closed
marcosmarxm opened this issue Jul 26, 2021 · 1 comment · Fixed by #4981
Closed

Source Bigquery: com.google.cloud.bigquery.BigQueryException: Arrays of arrays are not supported at #4993

marcosmarxm opened this issue Jul 26, 2021 · 1 comment · Fixed by #4981
Assignees
@DoNotPanicUA
Labels
connectors/source/bigquery connectors/sources-database type/bug Something isn't working

Comments

@marcosmarxm
Copy link
Member

Enviroment

  • Airbyte version: example is 0.27.4-alpha
  • OS Version / Instance: Not informed.
  • Deployment: Docker
  • Source Connector and version: BigQuery 0.1.0
  • Destination Connector and version: AWS S3 0.1.9
  • Severity: Very Low / Low / Medium / High / Critical
  • Step where error happened: Sync job

Current Behavior

BigQuery connector failed if table has array of arrays.

Expected Behavior

Should work or documentation must point this limitation.

Logs

LOG 

2021-07-22 21:32:09 INFO () WorkerRun(call):62 - Executing worker wrapper. Airbyte version: 0.27.4-alpha
2021-07-22 21:32:09 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.27.4-alpha
2021-07-22 21:32:09 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 3 attempt id: 2
2021-07-22 21:32:09 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {analytics_252052248.events_20210721=full_refresh - overwrite}
2021-07-22 21:32:09 INFO () DefaultAirbyteDestination(start):78 - Running destination...
2021-07-22 21:32:09 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:32:09 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-s3:0.1.9 exists...
2021-07-22 21:32:09 INFO () LineGobbler(voidCall):85 - airbyte/destination-s3:0.1.9 was found locally.
2021-07-22 21:32:09 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/destination-s3:0.1.9 write --config destination_config.json --catalog destination_catalog.json
2021-07-22 21:32:09 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:32:09 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-bigquery:0.1.0 exists...
2021-07-22 21:32:09 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/source-bigquery:0.1.0 read --config source_config.json --catalog source_catalog.json --state input_state.json
2021-07-22 21:32:09 INFO () LineGobbler(voidCall):85 - airbyte/source-bigquery:0.1.0 was found locally.
2021-07-22 21:32:09 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started.
2021-07-22 21:32:09 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join.
2021-07-22 21:32:09 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started.
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):81 - {} - Running integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.s.b.BigQuerySource(main):183 - {} - starting source: class io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):85 - {} - Command: WRITE
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):86 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):95 - {} - Running integration: io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {read=null, catalog=source_catalog.json, state=input_state.json, config=source_config.json}
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):99 - {} - Command: READ
2021-07-22 21:32:12 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:12 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):100 - {} - Integration config: IntegrationConfig{command=READ, configPath='source_config.json', catalogPath='source_catalog.json', statePath='input_state.json'}
2021-07-22 21:32:13 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:13 �[32mINFO�[m i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):42 - {} - S3 format config: {"format_type":"Parquet","page_size_kb":1024,"block_size_mb":128,"compression_codec":"SNAPPY","dictionary_encoding":true,"max_padding_size_mb":8,"dictionary_page_size_kb":1024}
2021-07-22 21:32:13 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:13 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-07-22 21:32:13 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:13 �[32mINFO�[m i.a.i.s.r.CdcStateManager(<init>):46 - {} - Initialized CDC state with: null
2021-07-22 21:32:13 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:13 �[32mINFO�[m i.a.i.s.r.StateManager(createCursorInfoForStream):137 - {} - No cursor field set in catalog but not present in state. Stream: AirbyteStreamNameNamespacePair{name='events_20210721', namespace='analytics_252052248'}, New Cursor Field: event_server_timestamp_offset. Resetting cursor value
2021-07-22 21:32:15 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:15 �[32mINFO�[m i.a.i.d.s.w.ProductionWriterFactory(create):61 - {} - Avro schema for stream events_20210721: {"type":"record","name":"events_20210721","namespace":"analytics_252052248","fields":[{"name":"_airbyte_ab_id","type":{"type":"string","logicalType":"uuid"}},{"name":"_airbyte_emitted_at","type":{"type":"long","logicalType":"timestamp-millis"}},{"name":"geo","type":["null","string"],"default":null},{"name":"items","type":["null","string"],"default":null},{"name":"device","type":["null","string"],"default":null},{"name":"user_id","type":["null","string"],"default":null},{"name":"app_info","type":["null","string"],"default":null},{"name":"platform","type":["null","string"],"default":null},{"name":"user_ltv","type":["null","string"],"default":null},{"name":"ecommerce","type":["null","string"],"default":null},{"name":"stream_id","type":["null","string"],"default":null},{"name":"event_date","type":["null","string"],"default":null},{"name":"event_name","type":["null","string"],"default":null},{"name":"event_params","type":["null","string"],"default":null},{"name":"privacy_info","type":["null","string"],"default":null},{"name":"traffic_source","type":["null","string"],"default":null},{"name":"user_pseudo_id","type":["null","string"],"default":null},{"name":"event_timestamp","type":["null","double"],"default":null},{"name":"user_properties","type":["null","string"],"default":null},{"name":"event_dimensions","type":["null","string"],"default":null},{"name":"event_value_in_usd","type":["null","double"],"default":null},{"name":"event_bundle_sequence_id","type":["null","double"],"default":null},{"name":"event_previous_timestamp","type":["null","double"],"default":null},{"name":"user_first_touch_timestamp","type":["null","double"],"default":null},{"name":"event_server_timestamp_offset","type":["null","double"],"default":null}]}
2021-07-22 21:32:15 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:15 �[32mINFO�[m i.a.i.d.s.p.S3ParquetWriter(<init>):72 - {} - Full S3 path for stream 'events_20210721': wombo-data/bq-test/analytics_252052248/events_20210721/2021_07_22_1626989535043_0.parquet
2021-07-22 21:32:15 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:15 �[33mWARN�[m o.a.h.u.NativeCodeLoader(<clinit>):60 - {} - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-07-22 21:32:16 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:16 �[33mWARN�[m o.a.h.m.i.MetricsConfig(loadFirst):134 - {} - Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2021-07-22 21:32:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:18 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):89 - {} - Overwrite mode
2021-07-22 21:32:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:18 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):98 - {} - Purging non-empty output path for stream 'events_20210721' under OVERWRITE mode...
2021-07-22 21:32:18 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:18 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):102 - {} - Deleted 1 file(s) for stream 'events_20210721'.
2021-07-22 21:32:30 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:30 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(queryTableFullRefresh):460 - {} - Queueing query for table: events_20210721
2021-07-22 21:32:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:48 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):200 - {} - Closing database connection pool.
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - Exception in thread "main" java.lang.ClassCastException: class com.google.cloud.bigquery.FieldValueList cannot be cast to class java.lang.String (com.google.cloud.bigquery.FieldValueList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.cloud.bigquery.FieldValue.getStringValue(FieldValue.java:110)
2021-07-22 21:32:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:48 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):202 - {} - Closed database connection pool.
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.fillObjectNode(BigQueryUtils.java:78)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$setJsonField$1(BigQueryUtils.java:93)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.setJsonField(BigQueryUtils.java:93)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$rowToJson$0(BigQueryUtils.java:61)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.rowToJson(BigQueryUtils.java:61)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryDatabase.lambda$query$1(BigQueryDatabase.java:125)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.lambda$tryAdvance$0(CollectSpliterators.java:115)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.tryAdvance(CollectSpliterators.java:114)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:62)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:83)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
2021-07-22 21:32:48 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:184)
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):141 - Source thread complete.
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join.
2021-07-22 21:32:49 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:49 �[32mINFO�[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded.
2021-07-22 21:32:49 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:49 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):118 - {} - Uploading remaining data for stream 'events_20210721'.
2021-07-22 21:32:49 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:49 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):120 - {} - Upload completed for stream 'events_20210721'.
2021-07-22 21:32:49 INFO () JsonSchemaValidator(test):76 - JSON schema validation failed. 
errors: $: null found, object expected
2021-07-22 21:32:49 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):83 - Validation failed: null
2021-07-22 21:32:49 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:49 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):122 - {} - Completed integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):144 - Destination thread complete.
2021-07-22 21:32:49 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:32:49 ERROR () DefaultReplicationWorker(run):148 - Sync worker failed.
io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
	at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:135) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:52) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):172 - sync summary: io.airbyte.config.ReplicationAttemptSummary@363bdb60[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989529401,endTime=1626989569691]
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):181 - Source did not output any state messages
2021-07-22 21:32:49 WARN () DefaultReplicationWorker(run):189 - State capture: No new state, falling back on input state: io.airbyte.config.State@32f0b984[state={}]
2021-07-22 21:32:49 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-07-22 21:32:49 INFO () RetryingTemporalAttemptExecution(get):118 - Last output present: true. Should attempt again: true
2021-07-22 21:32:49 INFO () RetryingTemporalAttemptExecution(get):123 - Starting attempt: 1 of 3
2021-07-22 21:32:49 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.27.4-alpha
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 3 attempt id: 2
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {analytics_252052248.events_20210721=full_refresh - overwrite}
2021-07-22 21:32:49 INFO () DefaultAirbyteDestination(start):78 - Running destination...
2021-07-22 21:32:49 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:32:49 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-s3:0.1.9 exists...
2021-07-22 21:32:49 INFO () LineGobbler(voidCall):85 - airbyte/destination-s3:0.1.9 was found locally.
2021-07-22 21:32:49 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/destination-s3:0.1.9 write --config destination_config.json --catalog destination_catalog.json
2021-07-22 21:32:49 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:32:49 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-bigquery:0.1.0 exists...
2021-07-22 21:32:49 INFO () LineGobbler(voidCall):85 - airbyte/source-bigquery:0.1.0 was found locally.
2021-07-22 21:32:49 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/source-bigquery:0.1.0 read --config source_config.json --catalog source_catalog.json --state input_state.json
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started.
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join.
2021-07-22 21:32:49 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started.
2021-07-22 21:32:52 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:52 �[32mINFO�[m i.a.i.s.b.BigQuerySource(main):183 - {} - starting source: class io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:32:52 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:52 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):81 - {} - Running integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):95 - {} - Running integration: io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {read=null, catalog=source_catalog.json, state=input_state.json, config=source_config.json}
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):99 - {} - Command: READ
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):85 - {} - Command: WRITE
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):100 - {} - Integration config: IntegrationConfig{command=READ, configPath='source_config.json', catalogPath='source_catalog.json', statePath='input_state.json'}
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):86 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.s.r.CdcStateManager(<init>):46 - {} - Initialized CDC state with: null
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.s.r.StateManager(createCursorInfoForStream):137 - {} - No cursor field set in catalog but not present in state. Stream: AirbyteStreamNameNamespacePair{name='events_20210721', namespace='analytics_252052248'}, New Cursor Field: event_server_timestamp_offset. Resetting cursor value
2021-07-22 21:32:53 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:53 �[32mINFO�[m i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):42 - {} - S3 format config: {"format_type":"Parquet","page_size_kb":1024,"block_size_mb":128,"compression_codec":"SNAPPY","dictionary_encoding":true,"max_padding_size_mb":8,"dictionary_page_size_kb":1024}
2021-07-22 21:32:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:55 �[32mINFO�[m i.a.i.d.s.w.ProductionWriterFactory(create):61 - {} - Avro schema for stream events_20210721: {"type":"record","name":"events_20210721","namespace":"analytics_252052248","fields":[{"name":"_airbyte_ab_id","type":{"type":"string","logicalType":"uuid"}},{"name":"_airbyte_emitted_at","type":{"type":"long","logicalType":"timestamp-millis"}},{"name":"geo","type":["null","string"],"default":null},{"name":"items","type":["null","string"],"default":null},{"name":"device","type":["null","string"],"default":null},{"name":"user_id","type":["null","string"],"default":null},{"name":"app_info","type":["null","string"],"default":null},{"name":"platform","type":["null","string"],"default":null},{"name":"user_ltv","type":["null","string"],"default":null},{"name":"ecommerce","type":["null","string"],"default":null},{"name":"stream_id","type":["null","string"],"default":null},{"name":"event_date","type":["null","string"],"default":null},{"name":"event_name","type":["null","string"],"default":null},{"name":"event_params","type":["null","string"],"default":null},{"name":"privacy_info","type":["null","string"],"default":null},{"name":"traffic_source","type":["null","string"],"default":null},{"name":"user_pseudo_id","type":["null","string"],"default":null},{"name":"event_timestamp","type":["null","double"],"default":null},{"name":"user_properties","type":["null","string"],"default":null},{"name":"event_dimensions","type":["null","string"],"default":null},{"name":"event_value_in_usd","type":["null","double"],"default":null},{"name":"event_bundle_sequence_id","type":["null","double"],"default":null},{"name":"event_previous_timestamp","type":["null","double"],"default":null},{"name":"user_first_touch_timestamp","type":["null","double"],"default":null},{"name":"event_server_timestamp_offset","type":["null","double"],"default":null}]}
2021-07-22 21:32:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:55 �[32mINFO�[m i.a.i.d.s.p.S3ParquetWriter(<init>):72 - {} - Full S3 path for stream 'events_20210721': wombo-data/bq-test/analytics_252052248/events_20210721/2021_07_22_1626989575235_0.parquet
2021-07-22 21:32:55 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:55 �[33mWARN�[m o.a.h.u.NativeCodeLoader(<clinit>):60 - {} - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-07-22 21:32:56 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:56 �[33mWARN�[m o.a.h.m.i.MetricsConfig(loadFirst):134 - {} - Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2021-07-22 21:32:57 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:57 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):89 - {} - Overwrite mode
2021-07-22 21:32:57 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:57 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):98 - {} - Purging non-empty output path for stream 'events_20210721' under OVERWRITE mode...
2021-07-22 21:32:58 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:32:58 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):102 - {} - Deleted 1 file(s) for stream 'events_20210721'.
2021-07-22 21:33:08 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:08 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(queryTableFullRefresh):460 - {} - Queueing query for table: events_20210721
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):200 - {} - Closing database connection pool.
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):202 - {} - Closed database connection pool.
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - Exception in thread "main" java.lang.ClassCastException: class com.google.cloud.bigquery.FieldValueList cannot be cast to class java.lang.String (com.google.cloud.bigquery.FieldValueList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.cloud.bigquery.FieldValue.getStringValue(FieldValue.java:110)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.fillObjectNode(BigQueryUtils.java:78)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$setJsonField$1(BigQueryUtils.java:93)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.setJsonField(BigQueryUtils.java:93)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$rowToJson$0(BigQueryUtils.java:61)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.rowToJson(BigQueryUtils.java:61)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryDatabase.lambda$query$1(BigQueryDatabase.java:125)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.lambda$tryAdvance$0(CollectSpliterators.java:115)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.tryAdvance(CollectSpliterators.java:114)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:62)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:83)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
2021-07-22 21:33:27 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:184)
2021-07-22 21:33:27 INFO () DefaultReplicationWorker(run):141 - Source thread complete.
2021-07-22 21:33:27 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join.
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded.
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):118 - {} - Uploading remaining data for stream 'events_20210721'.
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):120 - {} - Upload completed for stream 'events_20210721'.
2021-07-22 21:33:27 INFO () JsonSchemaValidator(test):76 - JSON schema validation failed. 
errors: $: null found, object expected
2021-07-22 21:33:27 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):83 - Validation failed: null
2021-07-22 21:33:27 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:27 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):122 - {} - Completed integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):144 - Destination thread complete.
2021-07-22 21:33:28 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:33:28 ERROR () DefaultReplicationWorker(run):148 - Sync worker failed.
io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
	at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:135) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:52) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):172 - sync summary: io.airbyte.config.ReplicationAttemptSummary@1721934f[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989569711,endTime=1626989608001]
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):181 - Source did not output any state messages
2021-07-22 21:33:28 WARN () DefaultReplicationWorker(run):189 - State capture: No new state, falling back on input state: io.airbyte.config.State@32f0b984[state={}]
2021-07-22 21:33:28 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-07-22 21:33:28 INFO () RetryingTemporalAttemptExecution(get):118 - Last output present: true. Should attempt again: true
2021-07-22 21:33:28 INFO () RetryingTemporalAttemptExecution(get):123 - Starting attempt: 2 of 3
2021-07-22 21:33:28 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.27.4-alpha
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 3 attempt id: 2
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {analytics_252052248.events_20210721=full_refresh - overwrite}
2021-07-22 21:33:28 INFO () DefaultAirbyteDestination(start):78 - Running destination...
2021-07-22 21:33:28 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:33:28 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-s3:0.1.9 exists...
2021-07-22 21:33:28 INFO () LineGobbler(voidCall):85 - airbyte/destination-s3:0.1.9 was found locally.
2021-07-22 21:33:28 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/destination-s3:0.1.9 write --config destination_config.json --catalog destination_catalog.json
2021-07-22 21:33:28 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:33:28 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-bigquery:0.1.0 exists...
2021-07-22 21:33:28 INFO () LineGobbler(voidCall):85 - airbyte/source-bigquery:0.1.0 was found locally.
2021-07-22 21:33:28 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/3/2 --network host --log-driver none airbyte/source-bigquery:0.1.0 read --config source_config.json --catalog source_catalog.json --state input_state.json
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started.
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join.
2021-07-22 21:33:28 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started.
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):81 - {} - Running integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):85 - {} - Command: WRITE
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):86 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.s.b.BigQuerySource(main):183 - {} - starting source: class io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):95 - {} - Running integration: io.airbyte.integrations.source.bigquery.BigQuerySource
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {read=null, catalog=source_catalog.json, state=input_state.json, config=source_config.json}
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):99 - {} - Command: READ
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):100 - {} - Integration config: IntegrationConfig{command=READ, configPath='source_config.json', catalogPath='source_catalog.json', statePath='input_state.json'}
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[32mINFO�[m i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):42 - {} - S3 format config: {"format_type":"Parquet","page_size_kb":1024,"block_size_mb":128,"compression_codec":"SNAPPY","dictionary_encoding":true,"max_padding_size_mb":8,"dictionary_page_size_kb":1024}
2021-07-22 21:33:31 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:31 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-07-22 21:33:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:32 �[32mINFO�[m i.a.i.s.r.CdcStateManager(<init>):46 - {} - Initialized CDC state with: null
2021-07-22 21:33:32 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:32 �[32mINFO�[m i.a.i.s.r.StateManager(createCursorInfoForStream):137 - {} - No cursor field set in catalog but not present in state. Stream: AirbyteStreamNameNamespacePair{name='events_20210721', namespace='analytics_252052248'}, New Cursor Field: event_server_timestamp_offset. Resetting cursor value
2021-07-22 21:33:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:33 �[32mINFO�[m i.a.i.d.s.w.ProductionWriterFactory(create):61 - {} - Avro schema for stream events_20210721: {"type":"record","name":"events_20210721","namespace":"analytics_252052248","fields":[{"name":"_airbyte_ab_id","type":{"type":"string","logicalType":"uuid"}},{"name":"_airbyte_emitted_at","type":{"type":"long","logicalType":"timestamp-millis"}},{"name":"geo","type":["null","string"],"default":null},{"name":"items","type":["null","string"],"default":null},{"name":"device","type":["null","string"],"default":null},{"name":"user_id","type":["null","string"],"default":null},{"name":"app_info","type":["null","string"],"default":null},{"name":"platform","type":["null","string"],"default":null},{"name":"user_ltv","type":["null","string"],"default":null},{"name":"ecommerce","type":["null","string"],"default":null},{"name":"stream_id","type":["null","string"],"default":null},{"name":"event_date","type":["null","string"],"default":null},{"name":"event_name","type":["null","string"],"default":null},{"name":"event_params","type":["null","string"],"default":null},{"name":"privacy_info","type":["null","string"],"default":null},{"name":"traffic_source","type":["null","string"],"default":null},{"name":"user_pseudo_id","type":["null","string"],"default":null},{"name":"event_timestamp","type":["null","double"],"default":null},{"name":"user_properties","type":["null","string"],"default":null},{"name":"event_dimensions","type":["null","string"],"default":null},{"name":"event_value_in_usd","type":["null","double"],"default":null},{"name":"event_bundle_sequence_id","type":["null","double"],"default":null},{"name":"event_previous_timestamp","type":["null","double"],"default":null},{"name":"user_first_touch_timestamp","type":["null","double"],"default":null},{"name":"event_server_timestamp_offset","type":["null","double"],"default":null}]}
2021-07-22 21:33:33 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:33 �[32mINFO�[m i.a.i.d.s.p.S3ParquetWriter(<init>):72 - {} - Full S3 path for stream 'events_20210721': wombo-data/bq-test/analytics_252052248/events_20210721/2021_07_22_1626989613459_0.parquet
2021-07-22 21:33:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:34 �[33mWARN�[m o.a.h.u.NativeCodeLoader(<clinit>):60 - {} - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-07-22 21:33:34 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:34 �[33mWARN�[m o.a.h.m.i.MetricsConfig(loadFirst):134 - {} - Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
2021-07-22 21:33:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:36 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):89 - {} - Overwrite mode
2021-07-22 21:33:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:36 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):98 - {} - Purging non-empty output path for stream 'events_20210721' under OVERWRITE mode...
2021-07-22 21:33:36 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:36 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(initialize):102 - {} - Deleted 1 file(s) for stream 'events_20210721'.
2021-07-22 21:33:48 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:33:48 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(queryTableFullRefresh):460 - {} - Queueing query for table: events_20210721
2021-07-22 21:34:08 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:08 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):200 - {} - Closing database connection pool.
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - Exception in thread "main" java.lang.ClassCastException: class com.google.cloud.bigquery.FieldValueList cannot be cast to class java.lang.String (com.google.cloud.bigquery.FieldValueList is in unnamed module of loader 'app'; java.lang.String is in module java.base of loader 'bootstrap')
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.cloud.bigquery.FieldValue.getStringValue(FieldValue.java:110)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.fillObjectNode(BigQueryUtils.java:78)
2021-07-22 21:34:08 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:08 �[32mINFO�[m i.a.i.s.r.AbstractRelationalDbSource(lambda$read$2):202 - {} - Closed database connection pool.
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$setJsonField$1(BigQueryUtils.java:93)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.setJsonField(BigQueryUtils.java:93)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.lambda$rowToJson$0(BigQueryUtils.java:61)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryUtils.rowToJson(BigQueryUtils.java:61)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.db.bigquery.BigQueryDatabase.lambda$query$1(BigQueryDatabase.java:125)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.lambda$tryAdvance$0(CollectSpliterators.java:115)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.CollectSpliterators$1.tryAdvance(CollectSpliterators.java:114)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.LazyAutoCloseableIterator.computeNext(LazyAutoCloseableIterator.java:62)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:42)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.CompositeIterator.computeNext(CompositeIterator.java:83)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.commons.util.DefaultAutoCloseableIterator.computeNext(DefaultAutoCloseableIterator.java:58)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)
2021-07-22 21:34:08 ERROR () LineGobbler(voidCall):85 - 	at io.airbyte.integrations.source.bigquery.BigQuerySource.main(BigQuerySource.java:184)
2021-07-22 21:34:08 INFO () DefaultReplicationWorker(run):141 - Source thread complete.
2021-07-22 21:34:08 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join.
2021-07-22 21:34:08 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:08 �[32mINFO�[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded.
2021-07-22 21:34:08 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:08 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):118 - {} - Uploading remaining data for stream 'events_20210721'.
2021-07-22 21:34:09 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:09 �[32mINFO�[m i.a.i.d.s.w.BaseS3Writer(close):120 - {} - Upload completed for stream 'events_20210721'.
2021-07-22 21:34:09 INFO () JsonSchemaValidator(test):76 - JSON schema validation failed. 
errors: $: null found, object expected
2021-07-22 21:34:09 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):83 - Validation failed: null
2021-07-22 21:34:09 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-07-22 21:34:09 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):122 - {} - Completed integration: io.airbyte.integrations.destination.s3.S3Destination
2021-07-22 21:34:09 INFO () DefaultReplicationWorker(run):144 - Destination thread complete.
2021-07-22 21:34:09 INFO () EnvConfigs(getEnvOrDefault):302 - WORKER_ENVIRONMENT not found or empty, defaulting to DOCKER
2021-07-22 21:34:09 ERROR () DefaultReplicationWorker(run):148 - Sync worker failed.
io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
	at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:135) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:52) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.27.4-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
2021-07-22 21:34:09 INFO () DefaultReplicationWorker(run):172 - sync summary: io.airbyte.config.ReplicationAttemptSummary@39861079[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989608019,endTime=1626989649237]
2021-07-22 21:34:09 INFO () DefaultReplicationWorker(run):181 - Source did not output any state messages
2021-07-22 21:34:09 WARN () DefaultReplicationWorker(run):189 - State capture: No new state, falling back on input state: io.airbyte.config.State@32f0b984[state={}]
2021-07-22 21:34:09 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-07-22 21:34:09 INFO () RetryingTemporalAttemptExecution(get):112 - Max retries reached: 3
2021-07-22 21:34:09 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):202 - attempt summaries: [io.airbyte.config.ReplicationOutput@2977b8be[replicationAttemptSummary=io.airbyte.config.ReplicationAttemptSummary@363bdb60[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989529401,endTime=1626989569691],state=io.airbyte.config.State@32f0b984[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@173ce451[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@1cbb083[stream=io.airbyte.protocol.models.AirbyteStream@2c292bb3[name=events_20210721,jsonSchema={"type":"object","properties":{"geo":{"type":"string"},"items":{"type":"string"},"device":{"type":"string"},"user_id":{"type":"string"},"app_info":{"type":"string"},"platform":{"type":"string"},"user_ltv":{"type":"string"},"ecommerce":{"type":"string"},"stream_id":{"type":"string"},"event_date":{"type":"string"},"event_name":{"type":"string"},"event_params":{"type":"string"},"privacy_info":{"type":"string"},"traffic_source":{"type":"string"},"user_pseudo_id":{"type":"string"},"event_timestamp":{"type":"number"},"user_properties":{"type":"string"},"event_dimensions":{"type":"string"},"event_value_in_usd":{"type":"number"},"event_bundle_sequence_id":{"type":"number"},"event_previous_timestamp":{"type":"number"},"user_first_touch_timestamp":{"type":"number"},"event_server_timestamp_offset":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=analytics_252052248,additionalProperties={}],syncMode=full_refresh,cursorField=[event_server_timestamp_offset],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}]], io.airbyte.config.ReplicationOutput@380fc4c0[replicationAttemptSummary=io.airbyte.config.ReplicationAttemptSummary@1721934f[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989569711,endTime=1626989608001],state=io.airbyte.config.State@32f0b984[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@4861b1fc[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@14725fdb[stream=io.airbyte.protocol.models.AirbyteStream@71e39a55[name=events_20210721,jsonSchema={"type":"object","properties":{"geo":{"type":"string"},"items":{"type":"string"},"device":{"type":"string"},"user_id":{"type":"string"},"app_info":{"type":"string"},"platform":{"type":"string"},"user_ltv":{"type":"string"},"ecommerce":{"type":"string"},"stream_id":{"type":"string"},"event_date":{"type":"string"},"event_name":{"type":"string"},"event_params":{"type":"string"},"privacy_info":{"type":"string"},"traffic_source":{"type":"string"},"user_pseudo_id":{"type":"string"},"event_timestamp":{"type":"number"},"user_properties":{"type":"string"},"event_dimensions":{"type":"string"},"event_value_in_usd":{"type":"number"},"event_bundle_sequence_id":{"type":"number"},"event_previous_timestamp":{"type":"number"},"user_first_touch_timestamp":{"type":"number"},"event_server_timestamp_offset":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=analytics_252052248,additionalProperties={}],syncMode=full_refresh,cursorField=[event_server_timestamp_offset],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}]], io.airbyte.config.ReplicationOutput@378c696d[replicationAttemptSummary=io.airbyte.config.ReplicationAttemptSummary@39861079[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989608019,endTime=1626989649237],state=io.airbyte.config.State@32f0b984[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@7227c031[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@742dd529[stream=io.airbyte.protocol.models.AirbyteStream@19f1d8cb[name=events_20210721,jsonSchema={"type":"object","properties":{"geo":{"type":"string"},"items":{"type":"string"},"device":{"type":"string"},"user_id":{"type":"string"},"app_info":{"type":"string"},"platform":{"type":"string"},"user_ltv":{"type":"string"},"ecommerce":{"type":"string"},"stream_id":{"type":"string"},"event_date":{"type":"string"},"event_name":{"type":"string"},"event_params":{"type":"string"},"privacy_info":{"type":"string"},"traffic_source":{"type":"string"},"user_pseudo_id":{"type":"string"},"event_timestamp":{"type":"number"},"user_properties":{"type":"string"},"event_dimensions":{"type":"string"},"event_value_in_usd":{"type":"number"},"event_bundle_sequence_id":{"type":"number"},"event_previous_timestamp":{"type":"number"},"user_first_touch_timestamp":{"type":"number"},"event_server_timestamp_offset":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=analytics_252052248,additionalProperties={}],syncMode=full_refresh,cursorField=[event_server_timestamp_offset],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}]]]
2021-07-22 21:34:09 INFO () SyncWorkflow$ReplicationActivityImpl(replicate):203 - sync summary: io.airbyte.config.StandardSyncOutput@1b9663c9[standardSyncSummary=io.airbyte.config.StandardSyncSummary@cb6c7f0[status=failed,recordsSynced=0,bytesSynced=0,startTime=1626989529401,endTime=1626989649237],state=io.airbyte.config.State@32f0b984[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@7227c031[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@742dd529[stream=io.airbyte.protocol.models.AirbyteStream@19f1d8cb[name=events_20210721,jsonSchema={"type":"object","properties":{"geo":{"type":"string"},"items":{"type":"string"},"device":{"type":"string"},"user_id":{"type":"string"},"app_info":{"type":"string"},"platform":{"type":"string"},"user_ltv":{"type":"string"},"ecommerce":{"type":"string"},"stream_id":{"type":"string"},"event_date":{"type":"string"},"event_name":{"type":"string"},"event_params":{"type":"string"},"privacy_info":{"type":"string"},"traffic_source":{"type":"string"},"user_pseudo_id":{"type":"string"},"event_timestamp":{"type":"number"},"user_properties":{"type":"string"},"event_dimensions":{"type":"string"},"event_value_in_usd":{"type":"number"},"event_bundle_sequence_id":{"type":"number"},"event_previous_timestamp":{"type":"number"},"user_first_touch_timestamp":{"type":"number"},"event_server_timestamp_offset":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[],namespace=analytics_252052248,additionalProperties={}],syncMode=full_refresh,cursorField=[event_server_timestamp_offset],destinationSyncMode=overwrite,primaryKey=[],additionalProperties={}]],additionalProperties={}]]
2021-07-22 21:34:09 ERROR () Exceptions(swallow):84 - Swallowed error.
java.lang.NullPointerException: null value in entry: sync_cpu_request=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:32) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:171) ~[guava-30.1.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:281) ~[guava-30.1.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateSyncMetadata(TrackingMetadata.java:62) ~[io.airbyte.airbyte-scheduler-persistence-0.27.4-alpha.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.generateSyncMetadata(JobTracker.java:211) ~[io.airbyte.airbyte-scheduler-persistence-0.27.4-alpha.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.lambda$trackSync$3(JobTracker.java:134) ~[io.airbyte.airbyte-scheduler-persistence-0.27.4-alpha.jar:?]
	at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.java:82) ~[io.airbyte-airbyte-commons-0.27.4-alpha.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.trackSync(JobTracker.java:121) ~[io.airbyte.airbyte-scheduler-persistence-0.27.4-alpha.jar:?]
	at io.airbyte.scheduler.app.JobSubmitter.trackCompletion(JobSubmitter.java:164) ~[io.airbyte.airbyte-scheduler-app-0.27.4-alpha.jar:?]
	at io.airbyte.scheduler.app.JobSubmitter.lambda$submitJob$2(JobSubmitter.java:139) ~[io.airbyte.airbyte-scheduler-app-0.27.4-alpha.jar:?]
	at io.airbyte.commons.concurrency.LifecycledCallable.onSuccess(LifecycledCallable.java:119) [io.airbyte-airbyte-commons-0.27.4-alpha.jar:?]
	at io.airbyte.commons.concurrency.LifecycledCallable.call(LifecycledCallable.java:99) [io.airbyte-airbyte-commons-0.27.4-alpha.jar:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]

Steps to Reproduce

Are you willing to submit a PR?

Not at the moment.
@marcosmarxm marcosmarxm added the type/bug Something isn't working label Jul 26, 2021
@DoNotPanicUA DoNotPanicUA self-assigned this Jul 26, 2021
@DoNotPanicUA DoNotPanicUA linked a pull request Jul 26, 2021 that will close this issue
🐛 BigQuery source: Fix nested arrays #4981
Merged
7 tasks
@vbhakta8
Copy link

vbhakta8 commented Jul 27, 2021 •
edited
Loading

The OS is Amazon Linux 2, if that matters. This issues was originally reported by me in the Slack community. Thanks for creating the issue 🙏.

@sherifnada sherifnada moved this to Done in GL Roadmap Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DoNotPanicUA DoNotPanicUA

Labels
connectors/source/bigquery connectors/sources-database type/bug Something isn't working
Projects
No open projects
GL Roadmap
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

🐛 BigQuery source: Fix nested arrays airbytehq/airbyte
4 participants
@marcosmarxm @vbhakta8 @DoNotPanicUA @igrankova