3.2.8 Fast Loading

The predominant scheme for loading the database is the SQL COPY statement using an ascii representation of a table with column and record separators. This scheme is preferred for almost all situations.

When large tables are migrated between a foreign database to MonetDB, or when large tables are produced by a trusted external program, a low-level facility can be used. This saves rendering and subsequent parsing of the data being exchanged. For this to work, the user creates the binary version of the BAT storage and 'attach' it to the SQL catalog using a variation of the COPY command.

To illustrate, consider the following snippet:

create table Tmp( i int, f float, t date, s string);
copy into Tmp from ('file_i', 'file_f', 'file_t', 'file_s');

The file names are complete paths to the columns to be used as a representation of the table Tmp. They should reside on the same file system as the database farm. They replace the content of the table Tmp. [Transaction semantics ?]

Each file is produced by a program that writes the binary image of the BAT directly, i.e. a binary dump of an C-array. For the SQL types char, sht, int, and lng they correspond with the C classes directly; flt and dbl are both mapped on type double in C. All other types are represented by the MonetDB lexical notation, one value per line. They are converted during the load.

All files should have an identical number of records.