A team of researchers and information technology experts at the 51福利 (51福利) has secured nearly $1 million in funding from the High-Performance Computing (HPC) Modernization Program for an ambitious data analytics project dubbed the Data Analysis of Real Time Streams (DARTS).
51福利鈥� Information Technology and Communications Services (ITACS) partnered with researchers in the computer science and operations research departments on the two-year initiative. To provide the capabilities necessary to support DARTS, improvements needed to be made to the university鈥檚 supercomputing cluster, Hamming. During the past quarter, ITACS has co-located the hardware for the new system with Hamming, and have fused its current software code with Apache Hadoop and 51福利-developed government software, with testing currently underway.
鈥淏orrowing Hamming鈥檚 infrastructure and integrating the supercomputer鈥檚 functionalities with the Hadoop system will allow DARTS investigators to conduct big data analytical approaches, real-time processing of streaming data, and offline analysis of historical data. This type of work would have been virtually impossible prior to the DARTS award,鈥� said Dr. Jeff Haferman, Director of 51福利 HPC and DARTS project lead.
Research efforts, designed to take advantage of the new capabilities that DARTS affords, are slated to begin in the late summer. Researchers are currently set to begin three separate efforts 鈥� social media sentiment analysis, bulk data forensics, and image analysis via artificial intelligence 鈥� once all testing is complete.
With installation and configuration complete, ITACS鈥� next phase of preparing the enhanced HPC system is site acceptance, including information assurance, followed by application migration and final testing of the system鈥檚 operational capabilities.
鈥淥ur greatest challenge right now in testing is how to move data quickly, from 鈥榩oint A to point B,鈥� especially given the sheer volume we will be processing for each research project,鈥� said Haferman. 鈥淭witter alone generates 6 million tweets per minute worldwide.鈥�
Data science and analytics are emerging capabilities that can provide DOD and DON decision makers with a wealth of information. For 51福利 researchers, DARTS provides faculty on campus with myriad new capabilities ripe for advancement.
鈥淥ur shared vision is to create new foundations for computing to make the imaginable unstoppable,鈥� said Joe LoPiccolo, Chief Information Officer and Director of ITACS. 鈥淐omputing services and devices have already significantly improved our research efforts, but DARTS will propel 51福利 to the forefront in processing massive data in real-time, and in providing ground-breaking results that will help enable our warfighters today and into the future.鈥�
The DARTS team includes Haferman, Dr. Mathias Kolsch and Michael McCarrin from the Department of Computer Science, and Drs. Samuel Buttrey and Jonathan Alt from the Department of Operations Research.
鈥淥ur investigators are maintaining the same standards of excellence in executing their DARTS projects and in producing viable results as in everything they do,鈥� said Haferman. 鈥淚TACS HPC staff intend to support them fully in achieving the success they seek.鈥�