Error importing large CSV file to CUBRID

Error importing large CSV file to CUBRID

Hi,

Here is a good and easy solution for importing large data into a CURBID database .
Instead of using the default "Import Data…" function in the CM, for large data it is better to import them using "Prepared Insert" statements.

CM provides this handy solution which will allow you to insert data in multiple threads. Then "Insert by read file". In the CM, right click on the table, and instead of "Import Data." choose "Execute Prepared SQL". The following  window  will Popup.

Here you will see the Prepared SQL Statement which was automatically created for your target table including the parameter name/index and its type in the bottom of the window. So, just select the filename and indicate the number of threads to use (default 1). It is recommended to use no more than 7 or 8 threads.

Now to boost the import performance it is recommended to remove the indexing before importing, which can be later restored after all the data has been imported.

So, first create your table, but do not add any UNIQUE, PRIMARY KEY, or INDEX to any of the columns, because they all will trigger the indexing every time a new record is inserted during the import process.

Then do the above mentioned "Prepared SQL" importing. Once the import is done, you can add the PRIMARY KEY, UNIQUE or INDEX to the columns you want. This will trigger the indexing only once.

After using there steps you can improve your data import performance.