We have released the latest version of MSTICpy to both GitHub and PyPI. This release includes a wide number of new features and capabilities that we wanted to showcase. All these features come with documentation that can be found on our ReadTheDocs site and the detailed release notes can be found here.
Anomaly Sequence Analysis
Karishma Dixit in the MSTIC team developed a Python package to detect unusual sequence of events in Office activity data. She repackaged the code for more generic use in MSTICpy. The anomalous_sequence package can be run against a variety of log events to look for unusual activity sequences. Taking a Pandas DataFrame as input, it can evaluate sequences of commands (essentially event types) or also include the presence of parameters in the event and arguments to those parameters.
To take an Office activity event — an example of a command would be Set-Mailbox, a possible command parameter for Set-Mailbox is AuditEnabled, which has a parameter value of “true” or “false”.
Anomalous_sequence also includes a visualization component to view results on a timeline.
This sub-package can be used to aid threat hunting on various types of session data (for example, Office Exchange logs, AWS cloud trail logs, etc.). Karishma will soon be producing a Jupyter notebook which demonstrates the usage of this new sub-package.
Jupyter Notebook tools
Setting a Jupyter notebook up to run MSTICpy(with package installs, imports and configuration) is lengthy boilerplate code that needs to be run at the start of each notebook. Each time we need to update this pattern we have to edit all of the notebooks. Customers creating their own notebooks have to copy or implement the same logic. We’ve now encapsulated this logic into a single function call in MSTICpy — nbinit.init_notebook(). This does a set of standard imports, checks for and installs additional packages, and sets options. You can specify additions to this such as installing and importing an additional package.
Another common source of problems is ensuring that the msticpyconfig.yaml is set up correctly — specifying the right TI providers, Azure Sentinel and other data providers have correctly formatted configuration. validate_config() will run against your current configuration or a YAML config file specified as a parameter and give you a quick diagnosis of any problems.
Morph Charts
Morph Charts is a visualization tool created by Microsoft Research to provide interactive visualizations of data that can be chained together to help tell a story through data. Whilst Morph Charts is still a research project it has already proved to be a useful feature for understanding data, and identifying anomalies from a security perspective.
We have wrapped the Morph Charts creation process into MSTICpy, allowing you to take your own data, format it using a Morph Charts template and then visualize the charts in a Jupyter notebook. The initial release includes a template to visualize sign in data from Azure Active Directory, however we will expand this in future to support additional data sources.
