Public cloud providers offer a multi-tenancy model with shared resources. A major concern for many companies is where the data for the AI technologies resides. Heavily regulated industries can face strict data privacy, security and compliance requirements.
Companies comply with data privacy regulations (HIPAA and GDPR) and privacy standards (ISO 31700, ISO 29100, ISO 27701, FIPS 140-3 and NIST Privacy Framework) or risk penalties. AI projects increase the risks because massive amounts of real data dictate the behavior of ML training models. Model developers need to ensure that data is treated with fairness and transparency, said Rob van der Veer, senior principal expert at software assurance platform provider Software Improvement Group, co-editor of the EU's AI Act security standard and advisor to ISO/IEC and Open Worldwide Application Security Project. The data privacy requirements in the EU's GDPR are not specific to AI. Meanwhile, California, one of the U.S. states with data privacy regulations (CCPA and CPRA), is taking the lead in passing GenAI-related bills. Data residency and geolocation are also concerns with cloud AI, especially in Brazil, Singapore and the EU. Companies can set boundaries around data location in their SLAs.
Public cloud providers offer security and compliance frameworks that can aid anomaly detection in real time. Many companies adopt cloud data storage running on CSPs, but their sensitive data remains on-premises to meet information security and compliance requirements. Cloud data storage and analytics platform maker Snowflake, which processes proprietary and sensitive data of many Forbes Global 2000 companies on AWS, Azure and Google Cloud, was breached when, according to the company, a user logged in and failed to use multifactor authentication. Companies that use public cloud AI services should examine the CSP's monitoring and logging tools, employ data encryption at rest and in transit, require strict identity and access management controls, and perform regular audits for compliance.
For many companies, the cost of cloud AI services is difficult to gauge. Shadow AI, similar to shadow IT, is another concern. The FinOps Open Cost and Usage Specification (FOCUS 1.0), released in June, aims to normalize cloud billing for IaaS by using a common taxonomy and metrics for cost and usage data sets. AWS, Google, Microsoft and Oracle contributed to the open source project, which is hosted on GitHub. FOCUS can be extended to SaaS.
Public cloud providers might favor integration with their own services instead of third-party applications, which could lead to vendor lock-in. Data integration remains a major challenge for AI deployments. AI models require massive amounts of structured and unstructured data often coming from fragmented systems whose protocols and APIs might need updating to facilitate data exchange. Many organizations rely on legacy IT systems that may not be compatible with modern AI technologies and standards.
Finding personnel with cloud expertise is challenging enough -- not to mention data scientists, AI and ML engineers, or the nearly impossible-to-find prompt engineer, a role that all too often gets added to the duties of another member of the AI team. Google Cloud offers a prompt-grounding tool designed to address prompting tasks. Project managers need to ensure that AI and software teams follow best practices. Data engineers might not know about standard software development practices, such as versioning, unit testing and keeping documentation up to date -- even when experimenting with AI. Software engineering teams tasked with AI model alignment -- ensuring the AI system matches the designer's goals and is ethically sound -- might lack AI expertise.
Commercial data sets that augment the data pipeline used to train and fine-tune the LLMs needed for AI can help companies get started. But some data sets may offer limited information, which can lead to bias or diversity issues. Improper handling of private, confidential and copyrighted data used to train AI models can result in compliance violations and lawsuits.