Rebase staging from master (#345)

* Release/1.1.3 (#290)

* [MLOP-634] Butterfree dev workflow, set triggers for branches staging and master (#280)

* Change github actions pipelines.

* Change pipeline logic.

* [BUG] Fix Staging GithubActions Pipeline (#283)

* New step on pipelie.

* Some adjusts.

* Apply only wheel. (#285)

* [BUG] Change version on setup.py to PyPI (#286)

* Add new make command to change version.

* Change command order.

* Change desc and variable name.

* Change command name.

* Keep milliseconds when using 'from_ms' argument in timestamp feature (#284)

* changed timestamp resolution

* fix import

* simple refactor

Co-authored-by: Henrique Camargo <[email protected]>

* Change trigger for pipeline staging (#287)

* Change trigger to publish dev pipeline.

* Some fix.

* Create a dev package. (#288)

* [MLOP-633] Butterfree dev workflow, update documentation (#281)

* Update workflow doc.

* Update README

* Add pre-release.

* Fix typo.

* [MLOP-632] Butterfree dev workflow, automate release description (#279)

* release 1.1.4

* update changelog

Co-authored-by: Mayara Moromisato <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>

* Release/1.2.0 (#328)

* [MLOP-636] Create migration classes (#282)

* [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)

* Add interval branch modifications.

* Add interval_runs notebook.

* Add tests.

* Apply style (black, flack8 and mypy).

* Fix tests.

* Change version to create package dev.

* Allow slide selection (#293)

* Fix Slide Duration Typo (#295)

* [MLOP-637] Implement diff method (#292)

* [MLOP-640] Create CLI with migrate command (#298)

* [MLOP-645] Implement query method, cassandra (#291)

* [MLOP-671] Implement get_schema on Spark client (#301)

* [MLOP-648] Implement query method, metastore (#294)

* Fix Validation Step (#302)

* [MLOP-647] [MLOP-646] Apply migrations (#300)

* add apply migration method

* add test apply migration

* add migrate actor with tests

* mypy compliant

* fix test interaction with mocked object

* Rebase and some adjusts.

Co-authored-by: Mayara Moromisato <[email protected]>

* [BUG] Apply create_partitions to historical validate (#303)

* Apply create_partitions to historical validate.

* Remove comments and adjusts.

* [BUG] Fix key path for validate read (#304)

* Fix key path

* bump version

Co-authored-by: AlvaroMarquesAndrade <1a789766b1c4c8b679e80f11fa6d63d42fa4bcdf>

* [FIX] Add Partition types for Metastore (#305)

* [MLOP-639] Track logs in S3 (#306)

* Apply tracking logs and logging config.

* Adjusts in CLI and logging.conf.

* Some adjusts.

* Change version to generate new dev package

* Fix version.

* Apply style.

* Add new assert in the migrate unit test.

* [BUG] Change logging config (#307)

* Change logging config.

* Some adjusts.

* Remove a code smell.

* Change solution for tracking logs (#308)

* Change tracking logs method.

* Change version to generate dev package.

* Change path name in S3

* Read and write consistency level options (#309)

* modify cassandra client to be region aware

* add option for the user to set read and write consistency levels on cassandra config

* add tests

* use env vars instead

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Fix kafka reader. (#310)

* Fix path validate. (#311)

* Add local dc property (#312)

* add local dc property

* update version

* Remove metastore migrate (#313)

* Remove metastore migrate.

* Change version to create a dev package.

* Fix link in our docs. (#315)

* [BUG] Fix Cassandra Connect Session (#316)

* Fix Cassandra Connect Session.

* Apply style.

* Fix migration query. (#318)

* Fix migration query add type key. (#319)

* Fix db-config condition (#321)

* Fix db-config condition.

* Apply style.

* MLOP-642 Document migration in Butterfree (#320)

* update docs

* add more information and reference new cli.md file

* [MLOP-702] Debug mode for Automate Migration (#322)

* Create flag debug-mode.

* Fix tests.

* Fix migrate test.

* [MLOP-727] Improve logging messages (#325)

* Fix logging message for local file

* Remove json import

* [MLOP-728] Improve logging messages (#324)

* Improve logs.

* Revert debug-mode condition.

* Fix method to generate agg feature name. (#326)

* [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)

* Change writer type for interval mode.

* Some adjusts.

* Release 1.2.0

Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>

* Release 1.2.1 (#338)

* Add the missing link for H3 geohash (#330)

* Add the missing link for H3 geohash

* Update the H3 geohash link.

* Update the same link 

Update the same link in in spark_function_and_window.ipynb example

* Update README.md (#331)

* Update Github Actions Workflow runner (#332)

* Update Workflow runner version

* bump flake8-bandit

* chore: bypass false positive for S105

Co-authored-by: Lucas Cardozo <[email protected]>

* Delete sphinx version. (#334)

* Update files to staging (#336)

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Fix kafka reader. (#310)

* Fix path validate. (#311)

* Add local dc property (#312)

* release 1.2.1

Co-authored-by: Jay Vala <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Lucas Fonseca <[email protected]>
Co-authored-by: Lucas Cardozo <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>

* fix: methods

* fix: duplicate

---------

Co-authored-by: hmeretti <[email protected]>
Co-authored-by: Mayara Moromisato <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>
Co-authored-by: Jay Vala <[email protected]>
Co-authored-by: Lucas Fonseca <[email protected]>
Co-authored-by: Lucas Cardozo <[email protected]>

Author: 12 people

CHANGELOG.md

@@ -23,7 +23,6 @@ Preferably use **Added**, **Changed**, **Removed** and **Fixed** topics in each
 
 ## [1.2.0](https://github.com/quintoandar/butterfree/releases/tag/1.2.0)
 
-
 ### Added
 * [MLOP-636] Create migration classes ([#282](https://github.com/quintoandar/butterfree/pull/282))
 * [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets ([#278](https://github.com/quintoandar/butterfree/pull/278))
@@ -50,9 +49,6 @@ Preferably use **Added**, **Changed**, **Removed** and **Fixed** topics in each
 * [BUG] Fix Cassandra Connect Session ([#316](https://github.com/quintoandar/butterfree/pull/316))
 * Fix method to generate agg feature name. ([#326](https://github.com/quintoandar/butterfree/pull/326))
 
-### Added
-* [MLOP-636] Create migration classes ([#282](https://github.com/quintoandar/butterfree/pull/282))
-
 ## [1.1.3](https://github.com/quintoandar/butterfree/releases/tag/1.1.3)
 ### Added
 * [MLOP-599] Apply mypy to ButterFree ([#273](https://github.com/quintoandar/butterfree/pull/273))

requirements.dev.txt

@@ -8,4 +8,4 @@ sphinx-rtd-theme==0.5.2
 recommonmark==0.7.1
 pyarrow>=1.0.0
 setuptools
-wheel
+wheel