Changelog#

Versions follow Semantic Versioning (<major>.<minor>.<patch>).

Backward incompatible (breaking) changes will only be introduced in major versions with advance notice in the Deprecations section of releases.

[UNRELEASED DRAFT] (2023-09-20)#

Improvements#

  • #62: Add PipelineRunContext.info, the information about where the pipeline is running.

  • #63: Updated codebase to pydantic>=2.0,<3.0

  • #70: Add saf.utils.salt.MasterClient and saf.utils.salt.MinionClient which are asyncio cooperative

  • #73: Several improvements to the project

    • Defer imports of uncommon usage libraries to reduce base requirements(for example, keras, numpy, papermill)

    • The salt-analytics.examples package now uses the same version as salt-analytics-framework

Bug Fixes#

  • #65: Allow SKIP_REQUIREMENTS_INSTALL=0 usage

  • #73: Fix regression which prevented pipelines without processors

Trivial/Internal Changes#

  • #57: Several CI related improvements:

    • Automatic update GH Actions steps versions

    • Different pytest config file for regular test suite and examples test suite

    • Use the tests_root variable instead of hardcoding paths when generating coverage reports

    • Force a newer python-tools-scripts, >=0.17.0

    • Explicitly define REPO_ROOT instead of relying on ptscripts.CWD

  • #58: Define a last CI job which set’s the whole pipeline exit status. Run actionlint and shellcheck against workflows in CI. Report code coverage to codecov.

  • #64: Fix and constraint test requirements

  • #67: Bump to coverage==7.2.7

  • #73: Several trivial/internal updates to the project:

    • Allow the nox -e build target to also build the examples package

    • When updating examples/requirements/all.txt only care about *.txt files

    • Upgrade pre-commit hook versions

0.5.0 (2023-06-01)#

Features#

  • #54: Add a Jupyter notebook processor that allows running parameterized notebooks using papermill

Improvements#

  • #46: Allow multiple collectors to feed data into the processors chain

  • #47: Forwarders always run concurrently now

  • #48: Create a test event collector(generator)

  • #49: Turn the noop processor into the test event processor(generator)

  • #50: Allow defining if a pipeline should restart or not when it ends or when it fails during processing.

  • #52: Chain processors asynchronously

Trivial/Internal Changes#

  • #47: Add a timed test case to verify concurrency

0.4.0 (2023-05-18)#

Improvements#

  • #41: AsyncIO cooperative file reads/writes. Support glob matching on paths.

Bug Fixes#

  • #42: Explicitly create a new loop and assign it to the current thread and avoid a DeprecationWarning

  • #43: Fix the seek to end of file call

  • #44: Import TypedDict from typing_extensions on Python < 3.9.2

0.3.0 (2023-05-17)#

Improvements#

  • #40: Allow the file collector to read from multiple files at a time

0.2.0 (2023-05-16)#

Improvements#

  • #31: Refactored the logs collector into a generic file collector

  • #33: If a processor decides not to return the passed event, no attempts to run the next processor on it or just forward it should be made

  • #34: Log the traceback on the first time a pipeline run raises an exception

  • #37: Processors can now return 1 or more events, they’ll all get forwarded

Bug Fixes#

  • #32: If one of the processors raises an exception, stop processing the event

0.1.1 (2023-05-15)#

Improvements#

  • #30: The CollectedEvent.data type is now Mapping instead of Dict. This allows to use TypedDict’s for that attribute.

Trivial/Internal Changes#

  • #29: Start testing against Salt’s onedir archives.

  • #30: Fix .pre-commit-config.yaml headers

0.1.0 (2023-04-28)#

First public release of the project.