We recently released an update to MSTICPy that adds some cool new features as well as some minor fixes.
Warning — this release includes some potentially breaking changes since we have renamed several pivot functions to have shorter, friendlier names. Several Azure Sentinel query functions have shortened names.
Pivot improvements
Pivots are still a relatively new feature in MSTICPy and as such we are still working to improve and expand them, especially as we get feedback from users. In this release we have made changes to improve the usability of the pivots (by shortening names and adding shortcuts), improved the join options (joining pivot results to input data), and added several pandas utility extensions.
The most significant of these changes is adding short names and shortcuts for many queries. This makes calling them much easier.
- Short names added for a number of pivots, particularly in relation to Azure Sentinel data queries.
- Shortcut functions have been added to many entities for common pivot functions: e.g. IpAddress.util.whois() has become IpAddress.whois() (although you can still use the former). You can also create your own shortcut links to pivot functions in containers.
- Pivot shortcuts also work as instance methods — so this is now possible:
my_ip = IpAddress(Address=”157.23.49.1")my_ip.whois()
Joins in Pivots have been expanded in two ways:
- All pivot functions now support joins (previously joins were limited to data query and TI pivots).
- You can now do case insensitive joins by setting the join_ignore_case parameter. This lets you join text columns ignoring case differences. This can be helpful with data like hash strings and domain names, which are case insensitive and can be represented differently. You should note that case insensitive joins do have a performance impact so consider its usage when working with large datasets.
An example of joining pivot data with an existing DataFrame:
IpAddress.util.whois(data=df, column=”ip_address”, join=”inner”)We have also added more accessors for Pandas DataFrames in pivots, these include:
- Simple filtering of data (e.g. Matching a substring or regular expression anywhere in a row).
- Filtering which columns are displayed (list of names or pattern matches).
- Parse json strings in cells to the Python objects.
- Expand lists in cells to separate rows.
- Sort data.
Greynoise TI Provider
GreyNoise recent offered up a free API service that allows users to see if IP addresses appear in GreyNoise datasets. This provides a great way for analysts to quickly triage against GreyNoise data and quickly filter down IP addresses sets.
As well as adding support for the new community API we have also added support for the enterprise API features as well. Whilst usage of the community API doesn’t need a registration or API key access to the enterprise features does. By default, the community API is used but the enterprise API features can be accessed by setting the `ioc_query_type` to either ‘quick’ or ‘full’ (e.g. ti.lookup_ioc(observable=”152.7.2.28”, ioc_query_type=”full”)).
Also on TI providers you can now enable or disable individual providers with `ti.enable_provider()` and `ti.disable_provider()`.
As well as these new additions there are several other new updates and fixes in this release. Head over the to release notes and documentation for more details.
