This itself is a significant challenge, since the availability of datasets is extremely rare, because from one side, many such datasets are internal and cannot be shared due to privacy issues, and on the other hand the others are heavily anonymized and do not reflect current trends, or they lack certain statistical characteristics, so a perfect dataset is yet to exist. Running these systems over real labeled network traces with a comprehensive and extensive set of intrusions and abnormal behavior is the most idealistic methodology for testing and evaluation. However, its adoption to real-world applications has been hampered due to system complexity as these systems require a substantial amount of testing, evaluation, and tuning prior to deployment. CSE-CIC-IDS2018 on AWS A collaborative project between the Communications Security Establishment (CSE) & the Canadian Institute for Cybersecurity (CIC)Īnomaly detection has been the main focus of many researchers’ due to its potential in detecting novel attacks.
