fread detect types algorithm - nrows #773
Description
Hi,
fread has probably an issue: not using nrows limit in detect types algorithm.
Example: file AAA.cvs has 619 rows. We are trying read rows from = 3 to 308 by option nrows=305 and skip=2 , 1 it is header. fread trying detect type without nrows limitation... See command log attached below.
Format example for rows: 3-308
AAA|BBB|CCC
1|2|3
Format example for rows: 309-617
ZZZ|YYY
1|2
> packageVersion("data.table")
[1] ‘1.9.3’Best regards
robert
fread("AAA.cvs" ,nrows= 305 ,skip= 2 ,sep="|" , ,colClasses=colClasses,verbose=T)
# Input contains no \n. Taking this to be a filename to open
# File opened, filesize is 0.000 GB
# File is opened and mapped ok
# Detected eol as \r\n (CRLF) in that order, the Windows standard.
# Looking for supplied sep '|' on line 3 ('skip' has been supplied) ... found ok
# Found 11 columns
# First row with 11 fields occurs on line 3 (either column names or first row of data)
# All the fields on line 3 are character fields. Treating as the column names.
# Count of eol after first data row: 617
# Subtracted 1 for last eol and any trailing empty lines, leaving 616 data rows
# nrow limited to nrows passed in (305)
# Type codes: 41214111441 (first 5 rows)
# Type codes: 41214111441 (+middle 5 rows)
# Error in fread("AAA.cvs", : ' ends field 9 on line 3 when detecting types: AAA|1111