FIPS Dataset class

This notebook illustrates basic functionality with the FIPSDataset class that holds FIPS 140 dataset

from sec_certs.dataset.fips import FIPSDataset
from sec_certs.sample import FIPSCertificate

Get fresh dataset snapshot from mirror

dset: FIPSDataset = FIPSDataset.from_web_latest()
print(len(dset))

Do some basic dataset serialization

# Dump dataset into json and load it back
dset.to_json("./fips_dataset.json")
new_dset = FIPSDataset.from_json("./fips_dataset.json")
assert dset == new_dset

Simple dataset manipulation

# Get certificates from a single manufacturer
cisco_certs = [cert for cert in dset if "Cisco" in cert.manufacturer]

# Get certificates with some CVE
vulnerable_certs = [cert for cert in dset if cert.heuristics.related_cves]

# Show CVE ids of some vulnerable certificate
print(f"{vulnerable_certs[0].heuristics.related_cves=}")

Dissect single certificate

# Select a certificate and print some attributes
cert: FIPSCertificate = dset["542cacae1d41132a"]

print(f"{cert.web_data.module_name=}")
print(f"{cert.heuristics.cpe_matches=}")
print(f"{cert.web_data.level=}")

Serialize single certificate

cert.to_json("./cert.json")
new_cert: FIPSCertificate = FIPSCertificate.from_json("./cert.json")
assert new_cert == cert

Create new dataset and fully process it

Warning: It’s not good idea to run this from notebook. It may take several hours to finnish. We recommend using from_web_latest() or turning this into a Python script.

dset = FIPSDataset()
dset.get_certs_from_web()
dset.process_auxillary_datasets()
dset.download_all_artifacts()
dset.convert_all_pdfs()
dset.analyze_certificates()