Introduction
The usage of big data and cloud computing are highlighted and explored in this study with regard to a made-up insurance company, Webb’s of Cardiff, and its project of big data, Thingsure. While have been hired as a big data solution consultancy to help Webb with its digital transformation process, to assist with the “collection, storage, and analyzing of big data ”. The main goals of the Thingsure project focus on minimizing insurance fraud, building the profit margins constant as well as offering customers competitive pricing. The initiative plans to use Internet of Things (IoT) gadgets such as Berring’s smart doorbell and Mini Motor’s car dashcam to achieve these objectives. This study attempts to examine solutions that are cloud-based, suggest an architecture that is cloud-based and customized to the project’s requirements, and identify any potential deployment problems. Some of the following objectives are covered in this entire study.
This study will shed light on establishing an effective system that has the ability to collect complex data sets. This study will also participate in in-depth analysis of different data storage options, in the proper way from the point of view of business systems. This will shed light to properly analyze the problems associated with the adoption of enterprise systems. In addition, enterprises such as Webb’s of Cardiff are always searching for creative ways to utilize the power of big data and cloud computing in the rapidly changing climate of digital transformation. In the context of Webb’s insurance company and its innovative responsibility,
Thingsure, this article explores the relationship between these technologies. This report’s goal as a big data solution consultant is to analyze alternative cloud-based solutions to satisfy the project’s requirements, propose a cloud-based architecture for the gathering, storing, and analysis of big data, and critically assess the deployment-related problems. In summary, this study will look at the big data needs for Thingsure, assess cloud-based options, recommend a cloud-based architecture, and assess any risks and problems that could arise while implementing such a system. This study will follow up by offering some suggestions for the future.
Big Data Requirements & Solutions
It is of the utmost importance to identify and evaluate cloud-based big data solutions that can successfully satisfy the requirements of the project in the context of the Thingsure project for Webb’s of Cardiff, which indicates to use of big data and cloud computing for a number of purposes. The primary goals of the project are to offer “reasonable prices, establish sustainable revenue margins, and reduce the possibility of “insurance fraud“. Webb has been searching for solutions that can analyze IoT device data for risk estimation and processing claim information in order to accomplish these objectives. Here, we’ll list several kinds of cloud-based big data solutions and evaluate how effectively they work in this situation.
In addition, many large and medium companies are using big data in their systems to develop operations, offer excellent customer service, build “personalized marketing campaigns” and take other actions that, key purpose of adopting big data to increase profits and make revenue.
AWS (Amazon Web Services) big data Services
For Webb’s Thingsure project, Amazon Web Services (AWS) shines forward as a reliable choice (Ageed et al.,2021). AWS provides a comprehensive ecosystem with its full collection of big data services, which includes “Amazon EMR”, which is used for data processing, Amazon “Simple Storage Service” for expandable storage, and “Amazon Redshift”, which is used for data warehouse services (Taher et al.,2019). The development of an architecture based on the cloud that can efficiently handle the huge quantities of IoT data, which has been generated by “MoniMotor and Brrring” devices is made possible by this combination (Yassine et al.,2019). Additionally, the project’s goal of improving risk estimation techniques gets immediate backing from AWS’s machine learning capabilities, which are an invaluable tool for predictive modeling. Overall, AWS’s “adaptability and durability” make it a popular choice for satisfying the project’s goals in a timely and secure way.
Apart from these, AWS has service limits to protect the company from “unexpected excessive provisioning” as well as protection against fraudulent activities aimed at increasing companies’ bills, as well as endpoint service protection. Services and other third-party connectors frequently come with limitations similar to this.
Microsoft Azure Big Data Solution
The comprehensive set of capabilities that Azure provides is approximately matched to the needs of the Thingsure project for Webb’s of Cardiff. A complete data ecosystem, including “Azure Data Lake Storage”, “Azure HDInsight for data analytics”, and “Azure Synapse Analytics” for data warehouse services, ensures effective data “processing and storage (Sahal et al.,2020)”. An essential instrument for effortless data input from IoT devices, Azure’s IoT Portal makes the data-collecting process simpler. By integrating “Azure Machine Learning”, Webb can get the ability to increase the accuracy of the processing of claims as well as enhance the modeling of risks by employing predictive analytics (Behl et al.,2022). The extensive array of services that are provided by Azure makes it a tempting choice for successfully achieving the project’s big data and cloud computing objectives.
Apart from these, although all big data services provide different benefits still have some limitations such as when a company using Azure, they are not able to access this without internet access to the system in the event of a main system outage. It is also determined that cloud service-provided security still has an argument that no cloud service provider commits 100% security. While a company adopts Azure, they are not able to add more than 40 terms to a single Azure AD organization.
GCP (“Google cloud platform”)
Through its wide range of services, “Google Cloud Platform” (GCP) provides the perfect response for Webb’s “Thingsure project”. The significant IoT data created by the “MoniMotor and Brrring devices” may be effectively processed and analyzed using “Big Query”, a potent data analytics generator (Gupta et al.,2020). Apart from these, the GCP Cloud Storage offers “scalable and dependable” data storage capabilities, which are essential for handling the huge amount of data involved. Additionally, publication or submission allows real-time data broadcasting, enabling Webb to gather data and respond to it immediately (Bisong, and Bisong, 2019). The project’s potential is further boosted by the company’s significant experience in “data analytics and machine learning”, which increases the accuracy of predictions and perfectly matches Webb’s objectives of boosting risk models and accelerating claims processing (Bisong, and Bisong, 2019). In its entirety, GCP is an excellent option for supporting the big data component of the Thingsure program due to its wide range of products and Google’s experience in data-driven solutions.
The ability of the solution that was selected to expand is of the utmost importance because it must be prepared to deal with the heavy “data flow” from IoT devices while satisfying the needs of Webb’s vast worldwide client base. Smooth “data collection and processing” depends on the perfect integration of information with “MoniMotor and Brrring’s sources”. The chosen solution should also have excellent analytics capabilities, including innovative machine learning techniques that are in line with Webb’s goals of improving the processing of claims and risk forecasting. To ensure that the project maintains within the “limits of finances” while meeting revenue targets, cost-effectiveness is an essential consideration. To remove the concerns of Webb’s Chief Information Security Officer, protect data privacy, and strengthen defenses against future incidents, security components need an in-depth review.
Proposed System Architecture
Dashcams from MoniMotor and Brrring’s smart doorbell, respectively, capture data on driving telematics and property entrance activities. Real-time streaming pipelines provide continuous data flow by ingesting IoT data into the cloud. For data streaming, tools such as Apache Kafka are employed. To extract meaningful insights, data is analyzed in both real-time and batch modes. Stream processing and batch processing are handled by Apache Spark and Apache Flink, respectively. Storage of data can be done for rapid retrieval and analysis, IoT data, telematics data, and video footage are stored in a scalable, cloud-based NoSQL database (e.g., Amazon DynamoDB or Google Cloud Firestore).
Figure 1: Architecture of the collection structure of complex data sets
(Source: Created own)
The above figure illustrates the architecture of the collection structure of a complex data set in which the source, data ware house and data Marts are shown. According to the above architecture, it can be said that the cloud-based application can be helpful for storing the matter data and stagging.
To prepare for analysis, data from diverse sources is merged and converted using Apache Nifi or AWS Glue. Data scientists and analysts construct prediction models for risk assessment and fraud detection using cloud-based machine learning services (e.g., AWS SageMaker or Google AI Platform). Data insights are visualized and presented using BI solutions like Tableau or Power BI, allowing Webb’s employees to easily analyze them (Ethan, 2023). To offer policyholders access to their IoT data and associated information, customer-facing applications and portals are hosted in the cloud (e.g., AWS Elastic Beanstalk or Google App Engine). The system architecture is also demonstrated in this section (Refer to Appendix).
The illustration of the enterprise system architecture in which four clients are shown along with representing the identification manager and web application server. However, the server application and application of the third party are also shown to be utilized for the purpose of enhancing the overall organizational sustainability. To secure sensitive customer data and maintain compliance with data privacy rules, robust security mechanisms such as encryption, access limits, and audit trails are deployed. Cloud-native monitoring tools (for example, AWS CloudWatch or Google Cloud Monitoring) watch the system’s health and performance, allowing for proactive problem remediation (Sun, and Huo, 2021). Because the design allows real-time data processing, fast reactions to driving events and property entrance activities are possible. Cloud-based solutions enable Webb’s to manage massive amounts of data generated by IoT devices, guaranteeing that the company’s 10 million+ clients can be served. Data from diverse IoT devices is merged and converted rapidly, resulting in a cohesive dataset for analysis. Machine learning services enable the development of predictive models to assess risk and detect fraud, improving pricing and claims processing accuracy.
Demonstrates the cloud-based architecture in which the interface of the client is at the top whereas the internet is in the middle. In this country of application with management cloud computing storage in structure and service Management. In this context, the above figures are found to illustrate the management and security part that can be used for the purpose of enhancing the accuracy of management in the system. Data visualization technologies make data insights accessible and actionable for Webb’s teams, allowing for enhanced decision-making. Security and compliance are prioritized in the design, answering the CISO’s worries about data breaches and privacy (Ren, 2020). Cloud services provide cost-effective scalability and resource management, which aligns with the CFO’s concerns about project costs. The customer-facing apps improve consumer engagement, which contributes to the objective of delivering competitive pricing. Cloud-native monitoring tools assure the system’s continuing health and performance, harmonizing with the CRO’s reputation management objective.
Project Risks & Issues
Deploying a cloud-based big data solution for Webb’s of Cardiff’s Thingsure project is a hard task fraught with possible pitfalls and obstacles. The issues are discussed below;
Storing sensitive client data in the cloud might expose you to security concerns such as data breaches and unauthorized access. Compliance with data privacy requirements (for example, the GDPR) is critical. Integrating data from several IoT devices, each with its own data format and communication protocol can be complicated and difficult. Handling the huge volume of data generated by IoT devices from 10 million+ users might put a burden on system resources and negatively affect performance.
If cloud expenses are not properly managed, they can spiral out of control, potentially resulting in budget overruns. Relying extensively on a single cloud provider may result in vendor lock-in, making future migration to a different provider difficult. Determining data ownership and guaranteeing data portability across cloud and on-premises systems can be difficult. To properly manage and optimize cloud-based big data solutions, Webb’s IT team may require training and upskilling (Wei et al., 2020). The issue is that ensuring compliance with industry-specific norms and standards can be difficult. As data quantities increase, maintaining data quality, provenance, and governance becomes increasingly crucial.
Solution
Put in place strong encryption techniques for data in transit and at rest. For access control, use cloud-native security services such as AWS Identity and Access Management (IAM) or Azure Active Directory. Conduct security audits and vulnerability assessments on a regular basis. Before sensitive data leaves IoT devices, encrypt it. Use cloud auto-scaling technologies to assign resources dynamically based on demand. To spread data over several cloud services, use data sharing or partitioning. Continuously monitor resource utilization and make necessary adjustments. To keep track of cloud expenditures, use cost monitoring and alerts (Lv, 2022). To discover and reduce resource inefficiencies, use cloud-native cost optimization technologies. Consider a well-structured pricing mechanism that corresponds to consumption trends. Use a multi-cloud approach to lessen your reliance on a single supplier.
In contractual agreements with IoT device makers and cloud providers, clearly specify data ownership and obligations. To ensure data portability, provide data export and backup options. Invest in IT worker training and certification programs. Utilize the cloud provider’s training and documentation resources. Consider collaborating with external cloud professionals or consultants.
The above figure demonstrates the involvement of Python libraries. The Python libraries are imported in such a manner that it can be helpful for the purpose of completing this project along with executing them appropriately (Chenthara et al., 2019). It can be said that the recall school confusion Matrix and the precision school can be determined appropriately live with the help of using the imported Matrix within Python.
The above code is used for the purpose of reading the data in which the data is demonstrated that can be utilized for the purpose of executing.
The data cleaning husband shown in the above figure in which the null values are removed for the purpose of enhancing the overall quality of the data.
The formation of multiple data is shown in the above figure in which monthly charges and total charges are demonstrated.
The setting of zero and one is done for yes and no variables in the data set. According to this variable the phone service and online backup as well as the senior citizen are used for the purpose of setting different variables in the Data cell which have been found to be helpful for the purpose of completing the execution process (Xu et al., 2019).
The visualization is shown in the above figure which has been found to be very useful for the purpose of illustrating the graphs for different variables such as 10-year total charges and monthly charges (Alouffi et al., 2021).
The above figure demonstrates the label encoding in which the value change is defined for the purpose of identifying the object as well as fit transform. However, the different data variables are also shown after the label encoding.
The above figure illustrates the data value setting in which the column data is shown for the purpose of identifying the tenure monthly charge as a total charge. The eggs and why values are also illustrated followed by splitting the data into train and test (Bestak, and Smys, 2019). In this context, the scalar value is used for standard scalar identification whereas the X train and x test data are also demonstrated.
The above figure demonstrates the use of a “support vector machine” in which the support vector machine model is developed where the random state is taken as one and the model SVM is used for the prediction according to the x test data (Gaurav et al., 2019).
The above figure demonstrates the predictive value of the model in which the macro average value is 0.17 whereas the weighted average value of precision is 0.11. On the other hand, the recall value is 0.5 0 and F1’s score value is 0.25.
Conclusion
After completing the entire finding it has been concluded that This research has examined big data and cloud computing in connection with Webb’s of Cardiff and their driven project, Thingsure. This study has made a concerted effort to tackle the essential tasks of collecting, preserving, and analyzing large data sets as the big data solution consultant chosen to support Webb in its digital transformation approach. investigation of this study has focused on Thingsure’s overall objectives, which include lowering “insurance fraud”, “maintaining stable profit margins”, and “offering competitive pricing”.
Throughout this investigation, this study has carefully examined several kinds of cloud-based options, each aiming to meet the specific requirements of the project. In order to achieve these impressive goals, the goal of this project has been to suggest a customized “cloud-based architecture” that can leverage the power of “Internet of Things” devices such as Berring’s “smart doorbell” and MoniMotor’s “automotive dashcam”. In addition, this study thoroughly investigated the “possible difficulties and complexity involved” in implementing such a system and its limitations, taking into thought the issues expressed by Webb’s leadership.
This study has shed light on SVM to create a predictive model for insurance fraud, through implementing this platform to make customers aware of fraud. Companies can adopt many other cloud computing services such as IBM and Alibaba Cloud. Enterprises such as Webb continually search for innovative, creative ways to take advantage of big data and cloud computing in the ever-evolving environment of digital transformation. In this research, the study has made an effort to shed light on the mutually advantageous connection that exists between technological advances and Webb’s goal of transforming the insurance sector through Thingsurel. Apart from these, the successful implementation of the chosen solution depends on the solution’s efficiency of implementation inside Webb’s present structure and R&D ecosystem.
References
Ageed, Z.S., Zeebaree, S.R., Sadeeq, M.M., Kak, S.F., Yahia, H.S., Mahmood, M.R. and Ibrahim, I.M., 2021. Comprehensive survey of big data mining approaches in cloud systems. Qubahan Academic Journal, 1(2), pp.29-38.
Alouffi, B., Hasnain, M., Alharbi, A., Alosaimi, W., Alyami, H. and Ayaz, M., 2021. A systematic literature review on cloud computing security: threats and mitigation strategies. IEEE Access, 9, pp.57792-57807.
Behl, A., Gaur, J., Pereira, V., Yadav, R. and Laker, B., 2022. Role of big data analytics capabilities to improve sustainable competitive advantage of MSME service firms during COVID-19–A multi-theoretical approach. Journal of Business Research, 148, pp.378-389.
Bestak, D.R. and Smys, D.S., 2019. Big data analytics for smart cloud-fog based applications. Journal of trends in Computer Science and Smart technology, 1(2), pp.74-83.
Bisong, E. and Bisong, E., 2019. An overview of google cloud platform services. Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners, pp.7-10.
Chenthara, S., Ahmed, K., Wang, H. and Whittaker, F., 2019. Security and privacy-preserving challenges of e-health solutions in cloud computing. IEEE access, 7, pp.74361-74382.
Ethan, M., 2023. Big Data Processing in the Cloud: Scalable and Real-time Data Analytics.
Gaurav, S., Zhao, X., Narayana, S.S. and Rajkumar, B., 2019. Integration of cloud, internet of things, and big data analytics. Software Practice Experience, 49(4), pp.561-564.
Gupta, A., Goswami, P., Chaudhary, N. and Bansal, R., 2020, March. Deploying an application using google cloud platform. In 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA) (pp. 236-239). IEEE.
Lv, Y., 2022. Cloud Computation-Based Clustering Method for Nonlinear Complex Attribute Big Data. IAENG International Journal of Computer Science, 49(3).
Ren, Q., 2020, April. Design of mobile APP user behavior analysis engine based on cloud computing. In Journal of Physics: Conference Series (Vol. 1533, No. 2, p. 022092). IOP Publishing.
Sahal, R., Breslin, J.G. and Ali, M.I., 2020. Big data and stream processing platforms for Industry 4.0 requirements mapping for a predictive maintenance use case. Journal of manufacturing systems, 54, pp.138-151.
Sun, Z. and Huo, Y., 2021. The spectrum of big data analytics. Journal of Computer Information Systems, 61(2), pp.154-162.
Taher, N.C., Mallat, I., Agoulmine, N. and El-Mawass, N., 2019, April. An IoT-Cloud based solution for real-time and batch processing of big data: Application in healthcare. In 2019 3rd international conference on bio-engineering for smart technologies (BioSMART) (pp. 1-8). IEEE.
Wei, P., Wang, D., Zhao, Y., Tyagi, S.K.S. and Kumar, N., 2020. Blockchain data-based cloud data integrity protection mechanism. Future Generation Computer Systems, 102, pp.902-911.
Xu, X., Liu, Q., Luo, Y., Peng, K., Zhang, X., Meng, S. and Qi, L., 2019. A computation offloading method over big data for IoT-enabled cloud-edge computing. Future Generation Computer Systems, 95, pp.522-533.
Yassine, A., Singh, S., Hossain, M.S. and Muhammad, G., 2019. IoT big data analytics for smart homes with fog and cloud computing. Future Generation Computer Systems, 91, pp.563-573.
Know more about UniqueSubmission’s other writing services:
Dear immortals, I need some wow gold inspiration to create.