Summaries - Office of Research & Innovation
Research Summaries
Back Automated Data Analysis for Network Optimization and Threat Detection in Network Architectures
Fiscal Year | 2018 |
Division | Research & Sponsored Programs |
Department | Naval Research Program |
Investigator(s) |
Kragh, Frank E.
Das, Arijit Miller, Donna L. |
Sponsor | NPS Naval Research Program (Navy) |
Summary |
Data Analysis is easy when the data sets are small. This is because manual examination using a single personal computer and very common software tools (C, C++, Python, R, MATLAB, etc.) are more than adequate for the task. Data Analysis is very different when the data sets are larger. The Naval Postgraduate School (NPS) has experimented with cloud computing and manipulation of common open-source cluster computing software tools to adapt them to processing very large data sets (>50 TB) on clusters of up to 100 computers. This allows automated processing of very large datasets to produce various forms of indices that can be directly queried to return the desired analytic results. Machine learning is used to devise algorithms that lend themselves to prediction and data modeling. These analytical data models allow analysts to produce reliable, repeatable, data-driven decisions and results including discovering hidden insights through learning from otherwise obscured trends in the data. Machine learning can be applied to the task at hand by implementing the learning phase and perhaps the classifying/pattern recognition phases on clusters of computers leveraging the same open-source cluster computing tools mentioned above. Traditional machine learning involves a few layers of non-linear processing analogous to the processing done in biological nervous tissue. It has been shown that often much improved results can be had by using many layers. This is referred to as deep learning and is well suited to cluster computing. NPS herein proposes to use modest-sized computer clusters (consisting of heterogeneous common PCs), such as already possessed by network operational centers, open-source cluster computing software tools, and deep machine learning algorithms to produce software prototypes to automatically identify anomalous network activity and possible cyber threats in tactical networks. |
Keywords | |
Publications | Publications, theses (not shown) and data repositories will be added to the portal record when information is available in FAIRS and brought back to the portal |
Data | Publications, theses (not shown) and data repositories will be added to the portal record when information is available in FAIRS and brought back to the portal |