Alex Nazarovsky

MATLAB, DSP, Julia, Power quality, Engineering, Metrology

Fast CSV Values Export to File. MEX-based Dlmwrite Replacement for MATLAB

MATLAB functions that export values into a .CSV file are very slow. It is said, that their inefficiency is rooted in bad implementation of the string concatenation. It is refered as Schlemiel the Painter’s algorithm.

Nevertheless, sometimes I have to write huge amounts of text data into CSV files. For example I had to generate test signals for power quality measurements algoritms, such as Flicker estimation. So I had written a MEX function to speed up this task. Write process can also be interrupted by pressing Ctrl-C using this wonderful trick (http://www.caam.rice.edu/~wy1/links/mex_ctrl_c_trick/).

Function takes four parameters

mex_WriteMatrix
1
2
3
4
5
6
7
8
9
10
Usage:
     mex_WriteMatrix(filename,matrix,format,delimiter, writemode);

Parameters:
     filename  - full path for CSV file to export
     matrix    - matrix of type 'double' values to be exported
     format    - format of export (sprintf) , e.g. '%10.6f'
     delimiter - delimiter, for example can be ',' or ';'
     writemode - write mode 'w+' for rewriting file 'a+' for appending

You can find the code and compiled mexw64 for function mex_WriteMatrix here

Comparing functions (150 Mb CSV file)
1
2
 mex_WriteMatrix(fname,Z,'%10.10f',',','w+');
 dlmwrite(fname, Z, 'delimiter', ',','precision', '%10.10f');

As you can notice difference in speed is huge! On big files ~ 1GB in size, write speed values are about 17-18 Megabytes/s.

You can use it freely, I hope it will save you a lot of time.