The Internet of Things (IoT) will connect not only computers and mobile devices, but it will also interconnect smart buildings, homes, and cities, as well as electrical grids, gas, and water networks, automobiles, airplanes, etc. IoT will lead to the development of a wide range of advanced information services that need to be processed in real-Time and require data centers with large storage and computing power. The integration of IoT with Cloud and Fog Computing can bring not only the required computational power and storage capacity, but they enable IoT services to be pervasive, cost-effective, and can be accessed from anywhere using any device (mobile or stationary). However, IoT infrastructures and services will introduce grand security challenges due to the significant increase in the attack surface, complexity, heterogeneity and number of resources. In this paper, we present an IoT security framework for smart infrastructures such as Smart Homes (SH) and smart buildings (SB). We also present a general threat model that can be used to develop a security protection methodology for IoT services against cyber-Attacks (known or unknown). Additionally, we show that Anomaly Behavior Analysis (ABA) Intrusion Detection System (ABA-IDS) can detect and classify a wide range of attacks against IoT sensors.