Releases: kuhn-ruess/cmdbsyncer
Version 3.10
This pull release introduces several important updates across the codebase, primarily focused on improving authentication mechanisms, enhancing the CMDB mode and host model, and refining Ansible inventory handling. The changes also include general code cleanup and documentation improvements.
Authentication and API Changes:
Replaced the x-login-header authentication header with x-login-user, updating all relevant API endpoints, documentation, and security definitions for consistency and clarity. The authentication logic now uses the User model and checks the password with check_password. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-e6720453fc4b0e74ea6fc341e1c8780da6d918f00807610f1c791d8de26a7d25R7-R24) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-34a0e37e357ff23f2a4807276f25459b957a5a3331fc3275aa1cadf28a386a36L27-R37) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-34a0e37e357ff23f2a4807276f25459b957a5a3331fc3275aa1cadf28a386a36L47-R48)
CMDB Mode and Host Model Enhancements:
Added support for CMDB mode in configuration (CMDB_MODE and CMDB_MODELS), and extended the Host model with new fields (cmdb_fields, cmdb_template, last_import_id). Introduced the CmdbField embedded document for flexible field storage. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-556ffb367cabead355f9000bec3b8d09405a54e33e3349892ecb6a777db90825R113-R114) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-556ffb367cabead355f9000bec3b8d09405a54e33e3349892ecb6a777db90825R123-R138) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafL23-R31) [[4]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafL40-R46) [[5]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafL53-R64)
Improved host import and cleanup logic with the new delete_host_not_found_on_import static method and additional inventory attribute handling. Updated set_account to respect CMDB mode and use the new import ID. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafR88-R116) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafL342-R397) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafR411-R413) [[4]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-16c58a535854a27f7cc607a43cb9b08b8a805a427981a094ecaa1cf0889adfafR436-L381)
Account Model and Types Improvements:
Expanded the list of supported account types and added a template type for CMDB labels. Refactored the Account model to use type instead of typ, providing backward compatibility via a property. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-ca5b141626287331063637b809556852358a35b296a3286f66cd475cf0ac8ba5R11-R33) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-ca5b141626287331063637b809556852358a35b296a3286f66cd475cf0ac8ba5R55) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-ca5b141626287331063637b809556852358a35b296a3286f66cd475cf0ac8ba5L72-R117)
Ansible Inventory and Playbook Updates:
Updated Ansible inventory scripts: replaced the old inventory script with a documentation note, and added a new inventory_single script for faster host-specific inventory generation. Fixed a typo in the cleanup playbook to enable tasks. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-6ef150b198b9cf49c007d9e098710ed083fac123177df92c443fefd84ad32560L1-R1) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-cee9ed78e4992a6f026428583fcf02ff7b486e20d2e195bf435d5117097d72a2R1-R28) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-f4f017cf8b44d51741a832a6d6b482586ceea9e710f2036817fad842b884c0f2L7-R7)
General Codebase Cleanup and Documentation:
Improved documentation and code comments in models and configuration files. Removed unused imports and updated versioning in application/__init__.py. [[1]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-1b4edbe81db1496f2c1e16e23959787445b422d14cfd34bb07682643cc1ce459L10-L13) [[2]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-1b4edbe81db1496f2c1e16e23959787445b422d14cfd34bb07682643cc1ce459L23-L34) [[3]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-de7ec3042dd5439c0ee22413010ab35deae91ab8e1e95d0503bd59d82c98ed75L8-R8) [[4]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-de7ec3042dd5439c0ee22413010ab35deae91ab8e1e95d0503bd59d82c98ed75L59-R59) [[5]](https://github.com/kuhn-ruess/cmdbsyncer/pull/71/files#diff-ade8f5ff16323b428edb6463ffb52d56822294039615645ed370b51dce380cc0L12)
These changes collectively improve security, maintainability, and flexibility for both API consumers and internal data management.
Version 3.9 GUI Updates
The New Version's main visible improvements are in the User Interface.
New Styling Options are now introduced, with which help the Forms for example for Conditions are now more user-friendly.
As a result of Pentesting, the Syncer now Saves Account Passwords Encrypted. It's nothing to do for you, then run ./cmdbsyncer sys self_configure after update (which you should do anyway).
Also, you need to update the requirements (if not using docker) otherwise you will get exceptions on start because of missing modules.
Account Updates
Accounts a basic Config Containers. Now you can create child instances of them.
The Child's will inherit the Credentials and other settings, but you can overwrite there stuff if needed.
This is useful, for example, if you want to use multiple DB Filters for Objects when exporting Checkmk Rules.
For Docker Users:
Cronjobs are now integrated into the Container. No need anymore to call something from outside.
Also: the self_configure functions will happen automatically, no manual step needed.
Other Changes
Fix: Fixed a bug in Checkmk Tag Export which in some cases not created not tag choices
Feat: Rewritten GUI for Tag export to be more user-friendly
Feat: Tag export now longer deletes Tags unless not configured to so
Feat: Checkmk Rule Export can now use the DB bases Filters (set in Accounts)
Feat: Checkmk Downtimes can now be set for just the current day, not only for timeranges as before.
Fix: Checkmk Folder Creation options now can create Titles
Feat: Netbox Custom Fields can now also be used for Contacts
Feat/Fix: Better Caching behavior for internal modules.
Feat: You can now configure Sentry for better logging (default off)
v3.8.4 Bugfix release
What's Changed
Fixes:
- Update Ansible Cache now works in cron manager
- Optimized Gui Host Filters
- Fixed Hosts View Regex Filters for Labels/ Inventory
- Checkmk Folder creation now correctly creates title attribute
- Fixed special dict custom_fields for attribute matching
Full Changelog: v3.8.3...v3.8.4
v3.8.3 Bugfixes and small features
What's Changed
Fixes:
- Host Label Search fixed
- Checkmk Groups Export fixed
- Fixed possible Gui Exceptions for Nonetypes
- Netbox fields export not longer send null where not allowed by netbox
- No longer crash if an import source does not have a Hostname for an entry
- Checkmk HW/SW Import no only imports configured variables
- Host Logging only logs exact changes, not the complete current and old data
Features
- Checkmk Inventorize supports Get Host by Folder
- All HTTP Based plugins can now use the Account Setting verify_cert to set the path to a certificat for authentication
- Netbox Import now imports regions
- Import Plugin for PRTG
- Better Error Messages in GUI for Duplicate Entries
- Inventorize Support for Rest Module
- Ansible Cache generation now uses Status bar if used from command line
- Hostname Validation can now be disabled with config switch
Full Changelog: v3.8.2...v3.8.3
v3.8.2 - Support for Vmware and Jira Cloud
What's Changed
- V3.8.1 by @Bastian-Kuhn in #55
- V3.8.2 Support for VMware Attribute Sync and Import from Jira Cloud by @Bastian-Kuhn in #57
Full Changelog: v3.8.1...v3.8.2
v3.8.1
Minor Bug fixes and Small Features:
DCD Rules now have a Debug Mode
Netbox Dataflow Export is rewritten and faster
Checkmk Import now reads all attributes
Debug Outputs of Request now print JSON and not Python anymore
Fixed Jdisc Import Module
Netbox Exports now honor None, which results in Null in JSON to Netbox.
More Minor Netbox Fixes
Fixed Problem for Exports to Checkmk RAW Edition
Feat: In the Host view, it's now possible to search for Attributes key:value pairs, in case the Value is a String or Integer. T
v3.8.0 New Netbox Integration, Improved Interface, JDisc Support
This Version has a ton of new Features and Bug fixes
First is the new Netbox Integrations, with uses the pynetbox SDK and now supports many aspects of Netbox.
Also, new API Endpoints are integrated, which can be used to Add/ Remove Hosts from the Syncer Database.
Additionally, this Versions Supports imports from JDISC.
Furthermore, the GUI is improved, looks more modern with Syntax Highlighting for Jinja codes.
The Syncer also Stores now the original data types on imports, so it's simpler to use Jinja Functions.
The Logging is now completed in the GUI, and you can use Checkmk Checks to monitor your Syncer jobs.
Debugging is also improved, many commands now support --debug to show the debug mode if needed, which no longer needs to be enabled in the config. With --debug-rules DEVICENAME many modues now support a Rule Debug view.
The Export to Checkmk is also faster then before, and the Checkmk Export functions now automatically detect the Target Checkmk Version and adjust the API Calls if needed.
The next release will be 3.8.1 with some new features. Will be merged soon.
Full List of Changes:
Fix: Checkmk Prevent Invalid Labels from being send to checkmk
Fix: Possibly fix Checkmk DCD Export
Feat: Checkmk DCD optimized view and added --debug switch
Feat: Netbox: Simplified use of list variables, activate them if set without checkbox
Fixed Debug view #60
Fix: Fixed Netbox VM Import #50
Fix: Netbox Device Sync: Do not ignore empty fields anymore
Fix: Netbox devices without ip not longer crash for Interface sync
Feat: Added log
Feat: JDisc device import now supports unnamed devices if configured in account
Feat: Imports now store a label when the object is last seen or last updated
Fix: Fixed sortorder of Crons in GUI
Feat: Checkmk Module now autodetects Checkmk Version, no config setting needed anymore
Fixed: Missing condition type in view
Fix: Raise the correct error message if Hostnames are not of type string
Fix: Platform Manufacturers for Virutal Machines
Fix: Various fixes
Fix: CSV Inventorize, delete Hostname Field
Minor Fix
Feature: New Debug Logs for JDisc imports
Fix: Fixed List Condition
Feat: Changed some more Rendering Styles to new Versions Style
Feat: Simplifed Attribute Matching Engine
Feat: Added Export function for Checkmk Filter Rules
Fixed typo #47
Feat: Optimized Rewrite Attributes view
Feat: Netbox: Device and VM Filters
Feat: Finished Netbox IPAM Prefixs module
Fix: Minor fixes
Feat: Added inital cronjob log line
Feat: Cron jobs now log more messages
Fix: Failures in cron are better flagged
Fixed Condition Matching
Fix: Auto convert of boolen for custom Attributes now it's strict only for Captical True and False strings
Fix: Label match now also takes label converting into account
Fix: Netbox device import is now faster again
Feat: New Debug Mode for Conditions and Better Styling for Dowtimes
Fix: Import of rules exported from GUI no longer skip first line
Fix: get_attributes now returns also empty attributes, so that they can be checked in Jinja Code without exception
Fix: Added missing Export functions for views
Feat: Jdisc now reads bios vesion
Feature: Checkmk Site Update can now define Ansible Custom Variables
Feature: Filter in Host view for Label Keys
Feat: Empty Labels now not longer imported if you use a config switch
Feat: New Debug Options for all modules
Fix: Jinja Helper now return just the original, if no converting is needed
Fix Netbox: Cleanup Device Type Creation, removed Model Field
Fix: String functions in Match class now convert value to string
Breaking Feature: The Syncer now not longer converts every imported Label Value into a String. The Value now keeps the original format, simplifing the rules
Feat: Ansible rules now support Jinja
Fix: Better Error Handling for Netbox VM import
Feat: When importing objects of type host, the hostname will now be validated and thow a exception if not valid. #46
Feat: Netbox now supports Prefixes for Sync
Feat: Netbox Devices and Vms are now decommissioned when not longer present in syncer
Feat: Netbox can now set the cmdbsyncer_id field. Currently enabled for devices and vms
Fix: Netbox no longer 'downgrade' fields with None or Unkowns if it has a better value already
Feat: Netbox Dataflow is now using directly the api
Feat: All Netbox Rules can now exportetd on CLI
Feat: Rules can now exported from GUI. Just Imports need the cli from now on
Feat: Log view can now filter for error and has better icons
Change: LDAP Import now uses objects instead of hosts in cli command names
Feat: Netbox: Better Debugging and Devices use progress bar
Fix: Also show changes if rules are deleted
Fixed requirements
Feat: netbox now uses slugify to create slugs
Feat: Netbox Contact Sync now supports groups
Feat: Netbox: Now supports IP Export with correct references to support duplicate IPs and still assign them to the correct interface
Fix: Netbox: Cleanup of functions and names
Fix: Netbox Virutal Interfaces now longer share cache with dcim interfaces
Fix: Optimized logging
Feat: Further Optimized Netbox Integration
Feat: More rule exports for Netbox rules
Feat: Device Debug Mode is now with --debug-rules which is the new Standard
Feat: Netbox export now supports Virtualization Interfaces
Fix: Netbox: Fixed Custom Fields
Feat: JDISC: All commands now support --debug switch
Fix: Netbox various fixes
Fix: Netbox fixing session
Feat: Netbox IPAM IP: Added Support for VRF
Feat: Optimized Debug Mode
Feat: Netbox Dataflow Plugin now supports Custom Fields
Feat: Netbox: Added Support to export Clusters and Virtual Machines
Feat: Rule Export now supports DCD Rules
Fix: Netbox IPs now supports IP Ignoreing again
Feat: Netbox Interfaces and Devices now support assignment of multiple addresses
Fix: Added missing files
Feat: Show Number of Changes for rules in Navbar
Feat: New Rule modes for Netbox wich can auto expand lists. Can be used if needed in other modules too
Fix: Pass configs again to Checkmk Downtimes
Feat: Netbox now has more Filters and better error Handling
Feat: Netbox now has more shell debug options
Fix: Fixed missing comment
Fix: Netbox Data-Flow function can now hanle multiply outcomes
Fix: Mintor fixes
Feat: Added more parts of modern view to Netbox Rules
Feat: Netbox exports now support list fields like Tags
Feat: Better Styling for Conditions
Feat: Netbox Devices export support Custom Fields again
Fix: Checkmk Exception handling does not longer lead to exception
Fix: Last feature is now checked again the correct attribute list
Feat: Checkmk: Function to Remove Attributes if set in Checkmk but not in Syncer
Fix: Netbox: Don't import devices without a Hostname #43
Fix: Netbox Dataflow Plugin now can export data
Feat: Checkmk: Tags Export nun supports Dry Run
Fix: Checkmk Tags: No longer do unnecessary update
Fix: Checkmk: Fixed extra_folder options
Fix: Netbox Device Export now better Supports Role and other nested fields
Fix: Checkmk: Better validation of a created Folder Path
Feat: Inital Support for Netbox Dataflow Plugin
Fix: Serval Fixes related to Netbox
Feat WIP: Netbox Dataflow Plugin Support
Feat: Netbox added support for multiple Interface/ IPs entities
Feat: Checkmk: New Action Option to not create a Hosts, just update it if it's there
Fix: Checkmk Catch more execptions in case of faulty hosts
Code cleanups and preperation for new functions
Fix: Checkmk Limiter options
Feat: Further Improved Netbox Modules added more Fields
Feat: New Filter in Accounts to Filter based on Object Type on Database Level
Feat: Rewrite of Netbox Integration for more speed and easy setup of planned features
Fixes: Netbox/ Jdisc fixes while test session
Feat: Jdisc support for Executables
Feat: Jdisc Applications: Now support Inventorize Mode and moved away from devices
Fix: Checkmk custom create_folder feature now has no problem with extra_folder_options anymore
Feat: Checkmk Optimized Export Limits Handling
Feat: Checkmk Optimized Export Limits Handling
Fix: Netbox Contact Export, fixed Contact Updates
Feat: Optimized Logging behavior of Checkmk Plugins
Fix: Do not break existing setups
Feat: Further improved Sync to Netbox functions
Feat: Fixes and Labels with Dict content are now auto expanded
Fix: CSV fixed crone mode
Feat: Checkmk: Optimized Multiprozessing with Timeouts in case of VM Freezes
Feat: Http Request now have a retry and a retry timeout.
Fix: Optimized Memory Usage of multiprocessing objects
Feat: Improved import/export rules memory footprint
Feat: Improved import/export rules memory foodprint
Fix: Fixed rewriting of Checkmk Groups
Feat: Reduced UWSGI Helpers in Docker
Feat: Checkmk Tags: Do not longer try to repair
Feat: Netbox Devices: Added Jinja Support to all Fields
Feat: Netbox added Contacts Support
Fix: Api Cron moniotring: Fixed Date Format
Fix: Fixed creation of empty attributes
Feat: Add last_update as Information for RestAPI
Fix: Netbox: Fixed Missing method: extract_data
Fix: Autoupdate Cache if single Items in Inventory change
Fix: Set more default Plugin Names
Fix: Set Default logname for yet undefined plugins
Fix: Fixed possible API crash
Feat: Jinja Expressions can now written with newlines for better usability
Fix: API with Cronjob Information does not crash anymore if a job was never started
Feat: Netbox Device export now has a mode to opt out fields from updating them
Feat: Netbox Device Export now supports Custom Fields
Fixed deletion
Feat: Netbox: Optimized Field Names
Feat: Netbox IP Address Management and better connection to e.g. Interfaces
Feature: Deletion of Hosts in Checkmk can now be disabled in local_config.py
Fix: Fixed requirements for uwgi
Fix: Fixed bug in error Handling in case of Host Movement
Fix: Checkmk Site Management, Addressfield not longer unique
Feat: Improved Netbox and begin of adding new modes
Jdisc imp...
v3.7 Power Release
The most Prominent Features of this Release:
Important: After Update, please commit the changes; otherwise there will be an exception on Checkmk Export
Also note the part "Interaction Needed"
General
- GUI: Simplified Interface, clearer Descriptions in Menu
- Export of Hosts and Tags to Checkmk, use now all available Processing Power to calculate Rules before importing.
- Bulk Operations for Checkmk are now Enable as Default
- Speed speed speed for Checkmk Operations
Supported Versions
- Checkmk: API Calls are Adapted to Checkmk 2.3 and some functions may not work on 2.2
- Checkmk Rules: On 2.2 the rules will delete and created again all the time due an API change in Checkmk. - SET local_config: 'CMK_SUPPORT': '2.2' to make version better compatible to old 2.2
Interaction Needed
- CSV: On Import, Hostnames are only set to lowercase if set in local_config.py. No longer as Default. See Local Config
- MySQL: On Import, Hostnames are only set to lowercase if set in local_config.py. No longer as Default. See Local Config
- Mssql: On Import, Hostnames are only set to lowercase if set in local_config.py. No longer as Default. See Local Config
- Checkmk: Export Rule Value of Folder is deprecated. Replaced by Jinja Support of normal Move Folder Rule.
- General: Jina Placeholder for Hostsname is now always Uppercase HOSTNAME.
- Checkmk: The Checkmk API once allowed, accidentally, that a host could be converted to a cluster. Since that is no longer possible, the Syncer now deletes hosts which should become a cluster to recre
ate them as such. - General: Config introduced "CRYPTOGRAPHY_KEY". Please overwrite it, since it is used to encrypt stored passwords in the database
- CRON: The Maintenance Cronjob had a Typo. After Update, you need to reelect this command in the config of the cron group.
New Features
- Checkmk:Folder names can now set that they will not be lowercase to keep their case. See Checkmk Config
- Checkmk: Folders can now get Attributes and Different Names, managed by Syncer.
- Checkmk: Move to Folder Rule now Supports full Jinja and Replaces Value of Folder rule
- Checkmk: It's possible to create folders but not move the host in
- Checkmk: The Debug Page shows now the Rule Debug was before only was possible in cli
- Checkmk: Export --dry-run and --save-requests to test or just save needed actions to run them later or archive them
- Checkmk: Support to Manage DCD (Dynamic Configuration) Rules
- Checkmk: Support to manage Password Store
- Checkmk: Detailed logging (if enabled) for changes made
- Global: Changes on Import hosts for Labels are now logged inside the Host Objects log
- LDAP: Added Support for Inventorize
Interim Version 3.6
This Version is the last Version before the Multiprocessing Features.
3.5.0-p0 First 3.5.0 Stable release
This Version does only contain some minor new features and improvements.
But it has two incompatible Changes.
- New Python Modules used as result of Security Updates.
- Inventory Data has now a new prefix Style.
One can be resolved without much effort for the most of the users. But the second one
may need an update of your rules since the attribute naming for inventorized data is effected.
Not effected are rules which are based on attributes which are imported by normal plugins.
So please refer to the Documentation for this update:
https://cmdbsyncer.readthedocs.io/en/latest/updates/3-5/