Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

分库分表同步报找不到表 #283

Closed
charnet1019 opened this issue Nov 6, 2023 · 8 comments
Closed

分库分表同步报找不到表 #283

charnet1019 opened this issue Nov 6, 2023 · 8 comments
Labels
bug Something isn't working
Milestone

Comments

@charnet1019
Copy link

库名如下:
image

表名如下:
delivery_order_0 ... 60
delivery_order_item_0 ... 60
order_0 ... 30
order_addr_0 ... 30
order_invoice_0 ... 30

点击设置后报错:
image

通过Notebook可以查看到表:
image

日志:

2023-11-06 15:03:13 INFO  executeaction- com.qlangtech.tis.coredefine.module.action.PluginAction:doSubformDetailedClick
2023-11-06 15:03:13 ERROR  c.q.t.m.c.v.TisExceptionInterceptor- toJSONString error
com.alibaba.fastjson.JSONException: toJSONString error
	at com.alibaba.fastjson.JSON.toJSONString(JSON.java:1012)
	at com.qlangtech.tis.trigger.util.JsonUtil.toString(JsonUtil.java:139)
	at com.qlangtech.tis.manage.common.valve.AjaxValve.buildResultStruct(AjaxValve.java:169)
	at com.qlangtech.tis.manage.common.valve.AjaxValve.writeInfo2Client(AjaxValve.java:115)
	at com.qlangtech.tis.manage.common.valve.AjaxValve.writeExecuteResult(AjaxValve.java:96)
	at com.qlangtech.tis.manage.common.valve.AjaxValve.doExecute(AjaxValve.java:78)
	at org.apache.struts2.result.StrutsResultSupport.execute(StrutsResultSupport.java:206)
	at com.opensymphony.xwork2.DefaultActionInvocation.executeResult(DefaultActionInvocation.java:375)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:279)
	at org.apache.struts2.interceptor.debugging.DebuggingInterceptor.intercept(DebuggingInterceptor.java:250)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept(DefaultWorkflowInterceptor.java:179)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(ValidationInterceptor.java:263)
	at org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.doIntercept(AnnotationValidationInterceptor.java:49)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ConversionErrorInterceptor.doIntercept(ConversionErrorInterceptor.java:142)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:137)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:137)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.StaticParametersInterceptor.intercept(StaticParametersInterceptor.java:201)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.MultiselectInterceptor.intercept(MultiselectInterceptor.java:67)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.DateTextFieldInterceptor.intercept(DateTextFieldInterceptor.java:133)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.CheckboxInterceptor.intercept(CheckboxInterceptor.java:89)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.FileUploadInterceptor.intercept(FileUploadInterceptor.java:243)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ModelDrivenInterceptor.intercept(ModelDrivenInterceptor.java:101)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ScopedModelDrivenInterceptor.intercept(ScopedModelDrivenInterceptor.java:142)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ChainingInterceptor.intercept(ChainingInterceptor.java:160)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.PrepareInterceptor.doIntercept(PrepareInterceptor.java:175)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.I18nInterceptor.intercept(I18nInterceptor.java:121)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.interceptor.ServletConfigInterceptor.intercept(ServletConfigInterceptor.java:167)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.AliasInterceptor.intercept(AliasInterceptor.java:203)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor.intercept(ExceptionMappingInterceptor.java:196)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.qlangtech.tis.manage.common.valve.OperationLogInterceptor.doIntercept(OperationLogInterceptor.java:64)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.qlangtech.tis.manage.common.valve.TisExceptionInterceptor.doIntercept(TisExceptionInterceptor.java:89)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.qlangtech.tis.manage.spring.aop.AuthorityCheckAdvice.doIntercept(AuthorityCheckAdvice.java:85)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at com.opensymphony.xwork2.interceptor.ParametersInterceptor.doIntercept(ParametersInterceptor.java:137)
	at com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(MethodFilterInterceptor.java:99)
	at com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocation.java:249)
	at org.apache.struts2.factory.StrutsActionProxy.execute(StrutsActionProxy.java:48)
	at org.apache.struts2.dispatcher.Dispatcher.serviceAction(Dispatcher.java:574)
	at org.apache.struts2.dispatcher.ExecuteOperations.executeAction(ExecuteOperations.java:79)
	at org.apache.struts2.dispatcher.servlet.StrutsServlet.service(StrutsServlet.java:80)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651)
	at com.qlangtech.tis.manage.common.DefaultFilter.doFilter(DefaultFilter.java:169)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:501)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(Thread.java:750)
Caused by: com.alibaba.fastjson.JSONException: serializer write error
	at com.alibaba.fastjson.serializer.SerializeConfig$ObjectSerializerAdapter.write(SerializeConfig.java:73)
	at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:542)
	at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:542)
	at com.alibaba.fastjson2.writer.ObjectWriterImplList.write(ObjectWriterImplList.java:373)
	at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:542)
	at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:542)
	at com.alibaba.fastjson2.writer.ObjectWriterImplMap.write(ObjectWriterImplMap.java:542)
	at com.alibaba.fastjson.JSON.toJSONString(JSON.java:1004)
	... 102 common frames omitted
Caused by: java.io.IOException: java.lang.RuntimeException: java.lang.RuntimeException: com.qlangtech.tis.plugin.datax.SelectedTab.getDftPks()
	at com.qlangtech.tis.trigger.util.JsonUtil$1.write(JsonUtil.java:100)
	at com.alibaba.fastjson.serializer.SerializeConfig$ObjectSerializerAdapter.write(SerializeConfig.java:71)
	... 109 common frames omitted
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: com.qlangtech.tis.plugin.datax.SelectedTab.getDftPks()
	at com.qlangtech.tis.trigger.util.JsonUtil$UnCacheString.getValue(JsonUtil.java:131)
	at com.qlangtech.tis.trigger.util.JsonUtil$1.write(JsonUtil.java:97)
	... 110 common frames omitted
Caused by: java.lang.RuntimeException: com.qlangtech.tis.plugin.datax.SelectedTab.getDftPks()
	at com.qlangtech.tis.extension.util.GroovyShellEvaluate.eval(GroovyShellEvaluate.java:170)
	at com.qlangtech.tis.extension.util.GroovyShellEvaluate.lambda$scriptEval$0(GroovyShellEvaluate.java:130)
	at com.qlangtech.tis.trigger.util.JsonUtil$UnCacheString.getValue(JsonUtil.java:129)
	... 111 common frames omitted
Caused by: java.lang.RuntimeException: TableNotFound{ dbId=mall4cloud_order_x,tableName='purchase_order_item_lang_0'}
	at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.getTableMetadata(BasicDataSourceFactory.java:120)
	at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsReader.getTableMetadata(BasicDataXRdbmsReader.java:179)
	at com.qlangtech.tis.plugin.datax.SelectedTab.lambda$getContextTableColsStream$7(SelectedTab.java:265)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
	at com.qlangtech.tis.extension.impl.SuFormProperties$SuFormGetterContext.getContextAttr(SuFormProperties.java:313)
	at com.qlangtech.tis.plugin.datax.SelectedTab.getContextTableColsStream(SelectedTab.java:263)
	at com.qlangtech.tis.plugin.datax.SelectedTab.getContextTableCols(SelectedTab.java:246)
	at com.qlangtech.tis.plugin.datax.SelectedTab.getDftPks(SelectedTab.java:168)
	at com.qlangtech.tis.plugin.datax.SelectedTab$getDftPks.call(Unknown Source)
	at Script23.run(Script23.groovy:1)
	at com.qlangtech.tis.extension.util.GroovyShellEvaluate.eval(GroovyShellEvaluate.java:168)
	... 113 common frames omitted
Caused by: com.qlangtech.tis.plugin.ds.TableNotFoundException: null
	at com.qlangtech.tis.plugin.ds.DataSourceFactory.parseTableColMeta(DataSourceFactory.java:199)
	at com.qlangtech.tis.plugin.ds.DataSourceFactory.lambda$parseTableColMeta$0(DataSourceFactory.java:331)
	at com.qlangtech.tis.plugin.ds.DataSourceFactory.validateConnection(DataSourceFactory.java:117)
	at com.qlangtech.tis.plugin.ds.DataSourceFactory.parseTableColMeta(DataSourceFactory.java:330)
	at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.parseTableColMeta(BasicDataSourceFactory.java:140)
	at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.lambda$getTableMetadata$1(BasicDataSourceFactory.java:110)
	at com.qlangtech.tis.plugin.ds.DBConfig.vistDbName(DBConfig.java:130)
	at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.getTableMetadata(BasicDataSourceFactory.java:109)
	... 123 common frames omitted
@baisui1981
Copy link
Member

baisui1981 commented Nov 6, 2023

delivery_order_0 ... 60
delivery_order_item_0 ... 60
order_0 ... 30
order_addr_0 ... 30
order_invoice_0 ... 30

根据您所列的表和 异常信息中展示的表名purchase_order_item_lang_0 并不在 这些表中

mall4cloud_order_x,tableName='purchase_order_item_lang_0

是不是哪儿有出入?猜测,是不是您的库里面只有部分表进行了分表 而像 purchase_order_item_lang 这个表并没有进行分表?

@charnet1019
Copy link
Author

charnet1019 commented Nov 6, 2023 via email

@charnet1019
Copy link
Author

charnet1019 commented Nov 7, 2023

delivery_order_0 ... 60
delivery_order_item_0 ... 60
order_0 ... 30
order_addr_0 ... 30
order_invoice_0 ... 30

根据您所列的表和 异常信息中展示的表名purchase_order_item_lang_0 并不在 这些表中

mall4cloud_order_x,tableName='purchase_order_item_lang_0

是不是哪儿有出入?猜测,是不是您的库里面只有部分表进行了分表 而像 purchase_order_item_lang 这个表并没有进行分表?

image
是有做分表的,但其中undo_log没有做,一共有875张表

@charnet1019
Copy link
Author

实际有这么多的分表,已经把没有分表的undo_log排除了,设置任何一个表都会提示找不到表,但在Nodebook中使用tables则可以正常显示所有的分表

image
image

@baisui1981
Copy link
Member

我怀疑purchase_order_item_lang_0 这个分表,并不是每个分库 中都存在,可能某几个分库中不存在的情况下 是会报这个异常的,您确认一下

@charnet1019
Copy link
Author

我怀疑purchase_order_item_lang_0 这个分表,并不是每个分库 中都存在,可能某几个分库中不存在的情况下 是会报这个异常的,您确认一下

每个库中都有对应的表,表的数量和结构都是一样的,设置任何一个分表都会提示对应的表找不到比较奇怪

会不会是因这个分表规则的问题,这个我没有填是用的默认的
image

@baisui1981
Copy link
Member

稍等 ,我本地试试,我微信:16660356

@baisui1981 baisui1981 added the bug Something isn't working label Nov 7, 2023
@baisui1981 baisui1981 added this to the v4.0.0 milestone Nov 7, 2023
@baisui1981
Copy link
Member

baisui1981 commented Nov 7, 2023

找到原因了 mysql8 驱动取表元数据信息 getTables 方法的第二个参数必须要用户传dbname

try (ResultSet tables = metaData1.getTables(null, getDbSchema(), table.getTableName(), null)) {
int count = 0;
List<String> matchEntries = Lists.newArrayList();
while (tables.next()) {
matchEntries.add(tables.getString("TABLE_NAME") + "(" + tables.getString("TABLE_TYPE") + "," + tables.getString("TABLE_SCHEM") + ")");
count++;

try (ResultSet tables = metaData1.getTables(null, getDbSchema(), table.getTableName(), null)) {
}

而 MySQLV8DataSourceFactory 实现中使用了固定的逻辑库名
https://github.com/qlangtech/plugins/blob/c5e4c3787a940fd0f152fc97cfc12c6c97bfd352/tis-datax/tis-ds-mysql-v8-plugin/src/main/java/com/qlangtech/tis/plugin/ds/mysql/MySQLV8DataSourceFactory.java#L81-L86

先改成这样

 @Override
    public String getDBSchema() {

        if (this.splitTableStrategy.isSplittable()) {
            // 如果使用了分表策略就不能返回默认的dbName了,不然会出错
            try {
                String[] dbNames = new String[1];
                this.getDbConfig().vistDbName((config, jdbcUrl, ip, dbName) -> {
                    dbNames[0] = dbName;
                    return true;
                });
                return Objects.requireNonNull(dbNames[0], "dbName can not be null");
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }
        // return null;
        return this.dbName;
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants