- Traverse Directories in /local/datafeeds
- Get the files: name.csv, name_old.csv - If name_old.csv doesn’t exist, then skip to the script
- Diff the files and create name_remove.csv (products to delete?)
- Run name.csv and name_remove.csv through python script
- Delete name_old.csv and name_delete.csv
- Rename name.csv to name_old.csv
- If failure, alert me somehow?
- Repeat this daily
- Get the data feed from the directory
- Process it by affiliate script
- Get cleaned file for addition
- Connect to DB by PYDBC
- Run and commit changes (insert, update, or delete)
- Close connection
- Pandas Dataframe with the columns matching our product schema.
- [title,price,description,url,image,brand,gender,colors,categories,mid,affiliate]
- Return 1 on success, 0 on failure