-
-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How many rows of data can be copied ? #179
Comments
Hi @sandeepraghu Theoretically unlimited. I have copied from a MSSQL to Azure MSSQL aprox 50million rows at no problem. Try to use the java binary instead of docker. |
Hi @MichalisDBA can you please share the steps for setup java binary and run , i have tried but getting errors.. |
I have a Linux VM running AlmaLinux (RHEL Clone) Let's say you a have a user User home path
Install JAVA
Create a directory
Download ReplicaDB
Extract it
Create a symlink for the replicadb binary
Add JAVA to to user environment
Write this somewhere inside
The Finally read the changes you made to bashrc or logoff from the terminal and login again. Now you can run replicadb in whatever path you have. Test it by running:
|
Hi @MichalisDBA successfully installed , now to run and replicate/copy data which command i should use.. |
Create the appropriate conf file and run
also read the docs are very comprehensive https://osalvador.github.io/ReplicaDB/index.html#full-documentation and try the configuration wizard |
Hi @MichalisDBA yes it working fine and copying data |
Yes you can create cron schedules. For multiple tables you have to create multiple .conf files for the time being. |
HI @MichalisDBA |
Do something like this.
|
okay will do likw this. |
Hi @MichalisDBA 2024-03-16 12:25:53,142 ERROR SQLServerManager:113 Error while performing BulkCopy into dbo.Job |
@sandeepraghu i think this does not have to do with replicadb. Is your sink table being access by other processes or users? Do anyone else insert, update or delete data to the sink table while you run replicadb targeting the sink table? |
Hi @MichalisDBA , please check the command is executing successful , also it showing correct total number of task process i.e. ubuntu@ip-172-16-1-106:~/replica_data/replicadb$ replicadb --options-file conf/tableWorkCenter.conf |
@sandeepraghu Are your destination tables being accessed and data are written from other proccesses? As you mentioned replicadb works. You have to investigate that. To test and see that actually replicadb works fine i would do something like that. SourceDB -> run the select query that you use in replicadb conf but count only the rows -> select count(*) from table where... -> for example 10.000 rows. Run replicadb with that query but for the sink table create a new one so that no other app or proccess know about this table. Check SinkDB table if the count of rows are matched. For me replicadb always transfers the exact rows no matter what and we are talking about million of rows being transfered that i tested. |
Hi @sandeepraghu, In the INCREMENTAL mode, a total of exact rows are replicated, but then a MERGE is performed to the sink table. In a MERGE if the primary key matches an UPDATE will be performed and if it does not exist an INSERT will be performed. Maybe you are counting rows that have been UPDATED? |
How many rows of data can be copied ?
The text was updated successfully, but these errors were encountered: