Data poisoning occurs when training, fine-tuning or embedding data is tampered with in order to modify the model's behaviour, for example steering it towards specific outputs, degrading performance or introducing backdoors.
Data Poisoning
CCC.GenAI.TH02
Related Capabilities
| ID | Title | Description |
|---|---|---|
| CCC.Core.CP02 | Encryption at Rest Enabled by Default | The service automatically encrypts all data using industry-standard cryptographic protocols prior to being written to a storage medium. |
| CCC.Core.CP06 | Access Control | The service automatically enforces user configurations to restrict or allow access to a specific component or a child resource based on factors such as user identities, roles, groups, or attributes. |
| CCC.GenAI.CP03 | Embedding Model Selection | Ability to select a foundation model used for tasks like semantic search, clustering, and document similarity by converting text into vector embeddings. |
| CCC.GenAI.CP06 | Customizable Model Selection | Provide users the ability to fine-tune models with their own data. |
| CCC.GenAI.CP21 | Generate Content | Ability to generate a response given a foundation model, parameter values, and a prompt. |
| CCC.GenAI.CP22 | Data Control | Ensures prompts, model outputs, embeddings, and training data fed by customers are not used to train foundation models. |
| CCC.GenAI.CP24 | Content Moderation | Ensure the service detects and filters abusive, harmful, and sensitive information to ensure responsible and safe use of the service. |
Related Controls
| ID | Title | Description |
|---|---|---|
| CCC.GenAI.CN03 | Data Provenance and Source Vetting | Ensure that all data for training, fine-tuning or RAG comes from trusted, approved sources and is authorised for the intended purposes in order to prevent the initial introduction of malicious content or leaked sensitive data. |
| CCC.GenAI.CN04 | Sanitisation of Ingested Data | Validate and sanitise all data ingested by GenAI systems from extenal sources or internal knowledge bases, whether for training, conversion to vector embeddings, or real-time retireval, in order to remove or redact poisoned or sensitive data before further processing. |
| CCC.GenAI.CN08 | Quality Control and Red Teaming | Establish a formal program for quality evaluation and adversarial testing (red teaming) to ensure GenAI system meet all business, quality, security and compliance requirements before getting deployed into production environments. |