BigData Analysis Framework for Detecting Harmful Information
The development of IT technology makes it easy to access portal sites and social media anywhere, and the seriousness of illegal and harmful information such as illegal sites, cyber violence, online game addiction, It is growing and causing various social problems. In addition, images that are hacked with imposters or CCTV are distributed to specific sites, posing a serious threat to personal privacy and information security, and hacked CCTV also serves as a malicious Zombie PC, such as DDos. Therefore, there is a need for a BigData analysis technique for blocking harmful information containing dangerous elements from large-sized social data generated from SNS and blog and search data such as Google. , we collect data from various harmful information sites and social media, then separate and extract metadata and log data by analyzing the structure of collected large data, store it in database based on HBase, When data is extracted through analysis, it is presented to web-based visualized data and applied to relevant companies and institutions to suggest real-time countermeasures.
Keywords - Web Crawler, BigData, Hazard Information