LEVIATHAN v962456e · 962456eee1

Standard Library

namespace csv

Read and write CSV (comma-separated values) text.

since 0.1.0-alpha.1linuxwindowswasm

Overview

csv::rows reads a file lazily as a sequence of rows, each row an array of string fields, following RFC 4180: fields may be quoted, a quoted field may contain the delimiter, quotes (written twice) and line breaks, and rows end at either \n or \r\n. csv::renderRow is the matching writer for one row. A row that is written with renderRow and read back with rows gives exactly the same fields.

Description

The csv namespace reads and writes comma-separated text as defined by RFC 4180, with an optional single-character delimiter such as a tab. It is written in the language and lives in the standard library.

  • csv::rows(file) returns a Seq<Array<string>>: one Array<string> of fields per record. csv::rows(file, delimiter) does the same for another delimiter, for example '\t' for tab-separated text.
  • csv::renderRow(fields) and csv::renderRow(fields, delimiter) produce the text of one record, without a line break at the end; the caller adds it with writeln.

The reader is lazy. Records are parsed one at a time as the sequence is pulled, and the first pull reads a single physical line rather than the whole file. It is also single-pass: it reads from the cursor of the File that the caller opened, and it never closes that file. The caller opens the file (or declares it with using) and closes it when done. Sequence operations such as skip(1) to step over a header row work on the result.

Reading records from standard input

File f = File("/dev/stdin", std::read);
for (Array<string> row in csv::rows(f).skip(1)) {
    string joined = row.joinToString("|");
    console.writeln("${row.length()}: ${joined}");
}
f.close();
Run it ▶
name,quote,n
"Smith, John","say ""hi""",3

Ada,"two
lines",4

3: Smith, John|say "hi"|3
3: Ada|two
lines|4

The same loop reads a file by name; this snippet is only compiled:

Reading a file by name

File f = File("data.csv", std::read);
int count = 0;
for (Array<string> row in csv::rows(f).skip(1)) {
    count = count + 1;
}
f.close();
Run it ▶

Rules

  • A " at the start of a field opens a quoted field. Inside it, "" is one literal quote, and the delimiter and line breaks are ordinary content, including an embedded \r\n, which is preserved byte for byte.
  • A record ends at a line break outside quotes. Both \r\n and a bare \n end a record; neither appears in the fields. A line break inside a quoted field is never altered.
  • A UTF-8 byte order mark at the start of the input is removed.
  • Blank lines between records are skipped. They are not an empty record, and they do not end the input. An input of only blank lines gives an empty sequence.
  • The reader recovers leniently from text that does not follow the quoting grammar: a " that is not at the start of a field, or text after a closing ", is kept as literal text and does not throw.
  • A quoted field that is still open at the end of the input throws a RuntimeException (csv: unterminated quoted field at end of input). This is the one error the reader reports.
  • Records may have different numbers of fields; they are returned as they are.
  • A delimiter must be an ASCII character other than ", \r or \n. A violation throws a RuntimeException immediately, when rows or renderRow is called, not later while reading.
  • renderRow quotes a field when it contains the delimiter, a ", \n or \r, and doubles each " inside it. A row whose only field is the empty string is written as "", because an empty line would be skipped by the reader. Reading back the text of renderRow(r) gives exactly r.

Examples

Writing records

console.writeln(csv::renderRow(["plain", "with, comma", "say \"hi\"", ""]));
console.writeln(csv::renderRow(["two\nlines", "x"]));
console.writeln(csv::renderRow([""]));
console.writeln(csv::renderRow(["a", "b\tc", "d"], '\t'));
console.writeln(csv::renderRow(["a", "b,c"], '\t'));
try {
    csv::renderRow(["a"], '"');
} catch (RuntimeException e) {
    console.writeln(e.message);
}
Run it ▶
plain,"with, comma","say ""hi""",
"two
lines",x
""
a	"b	c"	d
a	b,c
csv: delimiter must be an ASCII character other than quote or line break

Tab-separated input, an unterminated quoted field and the error it raises:

Tab-separated input and the unterminated-field error

File f = File("/dev/stdin", std::read);
try {
    for (Array<string> row in csv::rows(f, '\t')) {
        console.writeln(row.joinToString("|"));
    }
} catch (RuntimeException e) {
    console.writeln(e.message);
}
f.close();
Run it ▶
a	b	c
1	"x	y"	3
"unterminated	4

a|b|c
1|x	y|3
csv: unterminated quoted field at end of input

Blank lines between records are skipped, and records may differ in length:

Blank lines and ragged records

File f = File("/dev/stdin", std::read);
for (Array<string> row in csv::rows(f)) {
    string joined = row.joinToString("|");
    console.writeln("${row.length()}: ${joined}");
}
f.close();
Run it ▶
a,b

1,2

3,4,5

2: a|b
2: 1|2
3: 3|4|5

Examples

Reading rows and writing one back out

File input = File("/dev/stdin", std::read);
for (Array<string> row in csv::rows(input).skip(1)) {
    console.writeln(row.at(0) + " -> " + csv::renderRow(row));
}
input.close();
Run it ▶
name,note
Ann,"likes ""quotes"""
"Smith, John",plain

Ann -> Ann,"likes ""quotes"""
Smith, John -> "Smith, John",plain

Functions

renderRow

renderRow(Array<string> fields) -> string

Format one row as a line of CSV text.

A field is wrapped in double quotes only when it needs it, that is, when it contains a comma, a quote, or a line break; a quote inside a quoted field is written twice. The result has no trailing line break, so print it with writeln. A row holding a single empty field is written as "", so that it is not mistaken for a blank line when read back.

Parameters

fields
The field values of the row.

Returns

The CSV text for the row.

Examples

console.writeln(csv::renderRow(["a", "b", "c"]));
console.writeln(csv::renderRow(["Smith, John", "say \"hi\"", ""]));
console.writeln(csv::renderRow(["two\nlines", "x"]));
console.writeln(csv::renderRow([""]));
Run it ▶
a,b,c
"Smith, John","say ""hi""",
"two
lines",x
""

renderRow(Array<string> fields, char delimiter) -> string

Format one row as a line of delimited text with a custom delimiter.

This behaves like renderRow(fields) with the comma replaced by delimiter; a field is quoted when it contains that delimiter.

Parameters

fields
The field values of the row.
delimiter
The field delimiter; it must be an ASCII character other than ", \n and \r.

Returns

The delimited text for the row.

Throws

RuntimeException
when delimiter is not an allowed character.

See also: rows, renderRow

rows

rows(File f) -> Seq<Array<string>>

Read a comma-separated file as a lazy sequence of rows.

Each row is an array of string fields; no field is converted to a number or trimmed. Rows are read one at a time as the sequence is consumed, so a large file is never loaded whole. Because it reads from the file's current position, the sequence can be consumed only once, and the caller opens and closes the file.

Parsing follows RFC 4180. A field that starts with " is quoted: inside it, "" is a literal quote and the delimiter and line breaks are ordinary text. Records end at \n or \r\n. A leading UTF-8 byte order mark is removed, and blank lines between records are skipped. Rows may have different numbers of fields. Quote characters that do not fit the quoting grammar are kept as literal text instead of causing an error.

Parameters

f
The open file to read.

Returns

The rows of the file, in order.

Throws

RuntimeException
when a quoted field is still open at the end of the input; the error is raised when the sequence reaches that record.

Examples

Reading quoted fields and a header row

File input = File("/dev/stdin", std::read);
for (Array<string> row in csv::rows(input)) {
    console.writeln(row.length().toString() + ": " + row.joinToString("|"));
}
input.close();
Run it ▶
name,quote
"Smith, John","say ""hi"""
Ann,

x,,y

2: name|quote
2: Smith, John|say "hi"
2: Ann|
3: x||y

Skipping the header row

File input = File("/dev/stdin", std::read);
for (Array<string> row in csv::rows(input).skip(1)) {
    console.writeln(row.at(0));
}
input.close();
Run it ▶
id,name
7,Jane
8,Omar

7
8

rows(File f, char delimiter) -> Seq<Array<string>>

Read a file with a custom field delimiter as a lazy sequence of rows.

This behaves like rows(f) with the comma replaced by delimiter, for example a tab for TSV files or a semicolon for CSV files written in some locales.

Parameters

f
The open file to read.
delimiter
The field delimiter; it must be an ASCII character other than ", \n and \r.

Returns

The rows of the file, in order.

Throws

RuntimeException
when delimiter is not an allowed character; this is checked immediately, before any row is read.

See also: renderRow, rows

See also

  • rows — Read a comma-separated file as a lazy sequence of rows.
  • renderRow — Format one row as a line of CSV text.
  • File — An open file.