This paper presents the architecture of a modular, big-data based IS security management system (ISMS) and elaborates one of its modules - the domain generation algorithm (DGA) generated domain detection module. The presented methods, models and techniques are used in Riga Technical University, and can be used in any other large organization to stand against IS security challenges. The paper describes how organization can construct IS security management system using mostly free and open source tools and reach it's IS security goals by preventing or minimizing consequences of malware with little impact on employee's privacy. The presented DGA detection module provides detection of malicious DNS requests by extracting features from domain names and feeding them into random forest classifier. ISMS doesn't rely solely of DGA detection and instead uses an ensemble of modules and algorithms for increasing the accuracy of the overall system. The presented IS security management system can be employed in real-time environment and its DGA detection module allows to identify infected device as soon as it starts to communicate with the botnet command and control centre to obtain new commands. The presented model has been validated in the production environment and has identified infected devices which were not detected by antivirus software nor by firewall or Intrusion Detection System. Copyright © 2020 by SCITEPRESS - Science and Technology Publications, Lda. All rights reserved.