Skip to content

Conversation

@RoryO
Copy link

@RoryO RoryO commented Jul 17, 2025

There are two related issues with the CSV export that will trip up CSV parsing. If we create a new project, create a new intgrid layer and draw an X shape into it, the resulting CSV is

1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,
0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,
0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,
0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,
0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,
0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,
0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,
0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,
0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,
0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,
0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,
0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1

The first issue is the trailing commas, which is unusual. This can cause issue parsing. The trailing comma represents each line has a last field of null or None. Some CSV parsing libraries can work around this, and it appears Excel and LibreOffice seem to ignore the empty last column. However if you just split or tokenize the line on the comma, as what most simple parsers would do, you end up with a null final element in the row.

The confounding second issue is the trailing commas are inconsistent, as the last line does not have the trailing comma. This thwarts attempts to either preprocess the file by stripping the trailing commas or expect each line to end with null while parsing.

What is perplexing is this looks intentional? The code as written intentionally adds the comma at the end of the row unless the index of the row is the last row, where it does not add the comma.

This changes the CSV output to look like what most people expect: no trailing commas on any row. Also removes the new line at the end of the file though that's neither here nor there. The resulting CSV is

1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1
0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0
0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0
0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0
0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0
0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0
0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0
0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0
0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0
0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0
0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0
0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0
0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0
0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0
0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0
1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1

This will however also likely break any tools which expect the odd imbalanced trailing comma behavior.

@deepnight
Copy link
Owner

deepnight commented Jul 24, 2025

This is a breaking change that should probably only be enabled using a dedicated Project option for a period of time, with a future deprecation notice.
I will integrate your PR in the next build but I keep it here for now 👍
Anyway, thanks for fix 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants