Transition From Monitoring to Observability in Modern Service Assurance Through an Empirical Evaluation of Current Practices
DOI:
https://doi.org/10.51699/8w84ma95Keywords:
Monitoring Capability, Observability, Incident Response Efficiency, System Reliability, Service Assurance PerformanceAbstract
Traditional IT checking is failing as a result of the growing complexity systems, exacerbated by the introduction of distributed IT systems, cloud computing, and microservices. Observability is a new comprehensive assurance methodology that adapts monitoring practices to include the analysis of metrics, and traces to enhance visibility and system resilience. A one-time quantitative survey design was applied to a sample of 155 IT professionals based in the USA use cloud infrastructure and service assurance. A five-point Likert-scale questionnaire was the primary method of data collection. The analysis of the data was conducted using descriptive statistics, correlation, and structural statistics using IBM SPSS Statistics 29.0. System Reliability achieved the highest average score (4.02), followed by Observability Capability (3.95). Monitoring Capability achieved the lowest mean score (3.71). Application Performance Monitoring was the most common practice, at 31.6%, while 34.8% of respondents reported partial deployment of observability. The highest correlation was found between Observability Capability and Service Assurance Performance (r = 0.781). In the structural analysis, Monitoring Capability, at least, positively impacted Observability Capability (β = 0.653). The model incorporated the structural equation and reported that the model explained 68.4% of the variance of Service Assurance Performance (R² = 0.684). The analysis conducted showed that observability significantly improves incident response, system reliability, and service assurance performance. This study suggests that the integration of observability should be the primary focus increase the speed at which issues are resolved, in conjunction with improving the reliability of digital services.
References
[1] S. Niedermaier, F. Koetter, A. Freymann, and S. Wagner, “On Observability and Monitoring of Distributed Systems -- An Industry Interview Study,” in Lecture Notes in Computer Science, 2019, pp. 36–52. doi: 10.1007/978-3-030-33702-5_3.
[2] C. Lehrer, A. Wieneke, J. V Brocke, R. Jung, and S. Seidel, “How Big Data Analytics Enables Service Innovation: Materiality, Affordance, and the Individualization of Service,” J. Manag. Inf. Syst., vol. 35, no. 2, pp. 424–460, 2018, doi: 10.1080/07421222.2018.1451953.
[3] A. Kane, O. Chowdhury, A. Datta, and P. Koopman, “A Case Study on Runtime Monitoring of an Autonomous Research Vehicle (ARV) System,” in Lecture Notes in Computer Science, 2015, pp. 102–117. doi: 10.1007/978-3-319-23820-3_7.
[4] C. Sánchez et al., “A Survey of Challenges for Runtime Verification from Advanced Application Domains (Beyond Software),” Form. Methods Syst. Des., vol. 54, no. 3, pp. 279–335, 2019, doi: 10.1007/s10703-019-00337-w.
[5] Q. D. Soetens, S. Demeyer, A. Zaidman, and J. Pérez, “Change-Based Test Selection: An Empirical Evaluation,” Empir. Softw. Eng., vol. 21, no. 5, pp. 1990–2032, 2015, doi: 10.1007/s10664-015-9405-5.
[6] A. Kolk and P. Perego, “Determinants of the Adoption of Sustainability Assurance Statements: An International Investigation,” Bus. Strateg. Environ., vol. 19, no. 3, pp. 182–198, 2008, doi: 10.1002/bse.643.
[7] G. Michelon, D. M. Patten, and A. M. Romi, “Creating Legitimacy for Sustainability Assurance Practices: Evidence from Sustainability Restatements,” Eur. Account. Rev., vol. 28, no. 2, pp. 395–422, 2018, doi: 10.1080/09638180.2018.1469424.
[8] A. M. Almuhaideb and S. Saeed, “Fostering Sustainable Quality Assurance Practices in Outcome-Based Education: Lessons Learned from ABET Accreditation Process of Computing Programs,” Sustainability, vol. 12, no. 20, p. 8380, 2020, doi: 10.3390/su12208380.
[9] M. Warner, “In Search of Confucian HRM: Theory and Practice in Greater China and Beyond,” Int. J. Hum. Resour. Manag., vol. 21, no. 12, pp. 2053–2078, 2010, doi: 10.1080/09585192.2010.509616.
[10] S. Adams, “Post-Panoptic Surveillance through Healthcare Rating Sites,” Information, Commun. Soc., vol. 16, no. 2, pp. 215–235, 2012, doi: 10.1080/1369118X.2012.701657.
[11] F. Hartmann, P. Perego, and A. Young, “Carbon Accounting: Challenges for Research in Management Control and Performance Measurement,” Abacus, vol. 49, no. 4, pp. 539–563, 2013, doi: 10.1111/abac.12018.
[12] M. Steinmeier and M. Stich, “Does Sustainability Assurance Improve Managerial Investment Decisions?,” Eur. Account. Rev., vol. 28, no. 1, pp. 177–209, 2017, doi: 10.1080/09638180.2017.1412337.
[13] D. Berliner and A. Prakash, “``Bluewashing’’ the Firm? Voluntary Regulations, Program Design, and Member Compliance with the United Nations Global Compact,” Policy Stud. J., vol. 43, no. 1, pp. 115–138, 2014, doi: 10.1111/psj.12085.
[14] C. Rainieri, M. A. Notarangelo, and G. Fabbrocino, “Experiences of Dynamic Identification and Monitoring of Bridges in Serviceability Conditions and after Hazardous Events,” Infrastructures, vol. 5, no. 10, p. 86, 2020, doi: 10.3390/infrastructures5100086.
[15] M. Farshchi, J. Schneider, I. Weber, and J. Grundy, “Metric Selection and Anomaly Detection for Cloud Operations Using Log and Metric Correlation Analysis,” J. Syst. Softw., vol. 137, pp. 531–549, 2017, doi: 10.1016/j.jss.2017.03.012.
[16] P. Velte and M. Stawinoga, “Do Chief Sustainability Officers and CSR Committees Influence CSR-Related Outcomes? A Structured Literature Review Based on Empirical-Quantitative Research Findings,” J. Manag. Control, vol. 31, no. 4, pp. 333–377, 2020, doi: 10.1007/s00187-020-00308-x.
[17] J. Bosch and H. H. Olsson, “Digital for Real: A Multicase Study on the Digital Transformation of Companies in the Embedded Systems Domain,” J. Softw. Evol. Process, vol. 33, no. 5, 2021, doi: 10.1002/smr.2333.
[18] M. Döhler, F. Hille, L. Mevel, and W. Rücker, “Structural Health Monitoring with Statistical Methods during Progressive Damage Test of S101 Bridge,” Eng. Struct., vol. 69, pp. 183–193, 2014, doi: 10.1016/j.engstruct.2014.03.010.
[19] P. Castka, C. Searcy, and S. Fischer, “Technology-Enhanced Auditing in Voluntary Sustainability Standards: The Impact of COVID-19,” Sustainability, vol. 12, no. 11, p. 4740, 2020, doi: 10.3390/su12114740.
[20] M. A. Tadesse, B. A. Shiferaw, and O. Erenstein, “Weather Index Insurance for Managing Drought Risk in Smallholder Agriculture: Lessons and Policy Implications for Sub-Saharan Africa,” Agric. Food Econ., vol. 3, no. 1, 2015, doi: 10.1186/s40100-015-0044-3.
[21] K. Foerstl, A. Azadegan, T. Leppelt, and E. Hartmann, “Drivers of Supplier Sustainability: Moving Beyond Compliance to Commitment,” J. Supply Chain Manag., vol. 51, no. 1, pp. 67–92, 2014, doi: 10.1111/jscm.12067.
[22] E. Soltani and A. Wilkinson, “TQM and Performance Appraisal: Complementary or Incompatible?,” Eur. Manag. Rev., vol. 17, no. 1, pp. 57–82, 2018, doi: 10.1111/emre.12317.
[23] L. P. O. Were, E. Were, R. Wamai, J. Hogan, and O. Galarraga, “Effects of Social Health Insurance on Access and Utilization of Obstetric Health Services: Results from HIV+ Pregnant Women in Kenya,” BMC Public Health, vol. 20, no. 1, p. 87, 2020, doi: 10.1186/s12889-020-8186-y.
[24] M. Risius and K. Spohrer, “A Blockchain Research Framework,” Bus. Inf. Syst. Eng., vol. 59, no. 6, pp. 385–409, 2017, doi: 10.1007/s12599-017-0506-0.
[25] L. Fransen and G. LeBaron, “Big Audit Firms as Regulatory Intermediaries in Transnational Labor Governance,” Regul. Gov., vol. 13, no. 2, pp. 260–279, 2018, doi: 10.1111/rego.12224.
[26] A. Duff, “Corporate Social Responsibility as a Legitimacy Maintenance Strategy in the Professional Accountancy Firm,” Br. Account. Rev., vol. 49, no. 6, pp. 513–531, 2017, doi: 10.1016/j.bar.2017.08.001.
[27] L. Di Maso, G. J. Lobo, F. Mazzi, and L. Paugam, “Implications of the Joint Provision of CSR Assurance and Financial Audit for Auditors’ Assessment of Going-Concern Risk,” Contemp. Account. Res., vol. 37, no. 2, pp. 1248–1289, 2019, doi: 10.1111/1911-3846.12560.
[28] R. Lenz, G. Sarens, and K. K. Jeppesen, “In Search of a Measure of Effectiveness for Internal Audit Functions: An Institutional Perspective,” EDPACS, vol. 58, no. 2, pp. 1–36, 2018, doi: 10.1080/07366981.2018.1511324.
[29] N. Khalifa, M. A. Elghany, and M. A. Elghany, “Exploratory Research on Digitalization Transformation Practices within Supply Chain Management Context in Developing Countries Specifically Egypt in the MENA Region,” Cogent Bus. Manag., vol. 8, no. 1, 2021, doi: 10.1080/23311975.2021.1965459.
[30] S. Chakravorti, D. Dey, and B. Chatterjee, “Recent Trends in the Condition Monitoring of Transformers,” in Power Systems, 2013. doi: 10.1007/978-1-4471-5550-8.
