Required column checks should be case-insensitive

[bstandaert-wustl]

I'm looking at a GTFS from Valley Metro whose directions.txt looks like this:

route_id,direction_id,Direction
0,0,North
0,1,South
0A,0,North
0A,1,South
1,0,East
1,1,West
3,0,East
3,1,West
7,0,North

Trying to import this causes an exception:

Error: Missing required value in directions.txt for direction on line 2.

If I edit the column name to lowercase "direction", things work as expected.

Would it make sense for the library to normalize these column names to lowercase on import?

[brendannee]

Thanks for documenting this issue.

You could use the ignoreErrors config parameter to have node-gtfs ignore this: https://github.com/BlinkTagInc/node-gtfs?tab=readme-ov-file#ignoreerrors

Or, you could use the exclude parameter within the agencies array to skip importing directions.txt to work around this: https://github.com/BlinkTagInc/node-gtfs?tab=readme-ov-file#agencies

I'm hesitant to have the library convert to lowercase vs throwing an error. Let me know what you think.

[bstandaert-wustl]

I don't think ignoreErrors does what I want; IIUC that would skip the import and move on to the next agency, but I do want to import this agency's data.

exclude seems like a good solution for this case, since I don't care about the direction names; thanks for pointing that out. I do wonder if similar errors could occur in more critical files with other agencies, where that wouldn't be an option.

If you think converting to lowercase is likely to have negative side effects, perhaps the right solution is to have "permissive"/"strict" import modes to control whether normalization is enabled, but that might be too much complexity.

[brendannee]

Good feedback.

I updated the ignoreErrors config so that it works on a file-by-file level - so that if it hits an error importing one GTFS file like directions.txt, it will ignore that and move to the next file. The README has been updated as well.

https://github.com/BlinkTagInc/node-gtfs/releases/tag/4.17.7

[bstandaert-wustl]

Thanks! I think that's a fairly good solution.