This is mostly housekeeping release — we needed to do a bit of updating and testing for the new Azure Machine Learning nteract-based notebook environment. This is going to be the default execution environment for Azure Sentinel Notebooks. Read more about nteract and about the Azure ML Notebooks feature.
New Features
Friendly Exceptions
We’ve introduced a set of friendly exceptions for common configuration-related problems that users might encounter while using msticpy in notebooks. In most cases, the cause is lack of a config item such as an API key or setting. Hitting one of these exceptions gives you a few lines of explanatory text plus links to documentation on readthedocs to help you fix it.
In most cases the traceback is not relevant (since this is a config, rather than code-based error) so this is not shown for these exception types (although this can be re-enabled).
Time Series Analysis
We added time series decomposition functionality (using Statsmodel STL) so that you can do time series analysis on any data (previously we relied on Kql/Azure Sentinel to do the decomposition/analysis part). This function allows you to perform the analysis and anomaly visual on any time-stamped data in a pandas DataFrame. Ashwin Patil added documentation and a sample notebook describing how to use this.
Anomalous Sequences
Karishma Dixit added some more capability and documentation for the Anomalous Sequence feature. This allows you to identify unusual sequences of actions/activities in user sessions. Karishma wrote about this in a recent blog. Feature documentation is here and a Jupyter notebook demonstrating use of anomalous sequences is also available.
Documentation and Readme updates
We’ve updated the GitHub README to add missing details, correct some things and added a few more images to illustrate the functionality and make it bit more attractive.
Also:
- Added documentation for Azure Sentinel configuration for notebooks — how to use the
config.jsonandmsticpyconfig.yaml. - Updated a lot of the introductory sections to readthedocs to bring up-to-date and align with README
- Replaced some badly-formatted tables in readthedocs docs
- Anomaly Sequence and TimeSeries module/API docs not appear in the API doc tree
Other Changes
- We replaced the use of the ipywidgets Output widget with IPython updatable display() objects. This is primarily for nteract/AML compatability. (The PR for this #71). Most Jupyter notebook users will be familiar with the IPython display() function. What is less well-known is that each call to display creates a display object that can be referenced and updated by later code.
- Added check_version() function that will check the current version against latest on PyPI
