Learn more about Responsible AI Governance
Download our insight “The future of AI-powered enterprises”
Generative AI: An Overview of its Potential and Threats
The term “Generative AI” refers to various machine learning models that are capable of generating original contents, such as text, images, or audio. These models are trained on large amounts of data, that allow the users to create materials that appear novel and authentic by providing a precise and concise input (named “prompt”).
Large language models (LLMs) are a widely used type of Generative AI models that are trained on text and produce textual content. One prominent example of textual Generative AI application is ChatGPT, a chatbot developed by the AI company OpenAI, that can answer user’s questions and can generate a variety of text contents, including essays, poems, or computer code based on instructions expressed by the users in natural language.
Differently from traditional AI models that are typically customized to execute specific tasks, LLMs can be used for different purposes if properly tuned. The same model can for instance be used for text summarization and question answering. To adapt LLMs to a specific context it is essential to fine-tune the model on proprietary data related to a business-specific tasks. For instance, if a company wants to use LLMs to improve its legal department, models must be tuned with company’s legal documents.
In this early stage of its life, Generative AI has revealed tremendous potential for business and society. Fast content creation can allow companies to save significant amount of business time and resources. Generative AI can be used to power some services, like customer support, improving user experience and reducing labour costs. The fact that these models can be adapted, with proper tuning, to different applications and industries enhances their impact on a greater degree. However, if left unchecked or unproperly used, such AI solutions can also pose significative risks. The potential positive disruption of Generative AI, made of efficiency gains and time savings, may be counterbalanced by risks that should not be underestimated: for this reason, we need to speak about Responsible AI.
Main Sources of Risk for Generative AI
Many challenges and uncertainties may arise in the deployment, in business contexts but not only, of Generative AI tools.
From our point of view, some of the most significative sources of risk within the domain of Generative AI are:
Bias in the algorithm
Biases and discriminations embedded in the data used to train the model could possibly be transferred into the content generated by the AI model, hence a Generative AI application could produce results that are discriminatory or offensive for groups of individuals. Many examples of this behaviour have been collected. For instance, Bloomberg showed in an interactive article how one of the most used AI tools for image generation amplified stereotypes about race and gender.
Many Generative AI tools can generate new, fictional information (for example inventing events that have never taken place) and presenting it as real world, truthful facts. These misleading outputs can cause significant harm to users and third parties. For instance, this could be the case of a financial advice generated by a Gen AI application, based on facts and figures invented by the application itself. During a demo in front of the press, Bing AI, a chatbot created by Microsoft, analysed earning reports from GAP Inc., a clothing retailer. Some information produced by Bing AI appeared to have been made up.
Generative AI can produce harmful content in various forms, including hate speech, violent contents, and discriminatory material. This could happen because the model is trained with data representing negative, hateful, or violent behaviours and ideas. The ethical implications are self-evident, since this could pose serious harm to individuals or groups of individuals. Developers or users of such models could face significant reputational risks, too.
One of the most striking features of Generative AI models is their ability to create fictional contents that appear as realistic, and sometimes even apparently human-made. For this reason, they could be used as the perfect engine to power disinformation campaigns on large scale, deceiving a vast number of individuals with fabricated political, social, or economic messages in order to influence their decision process. This is for example what happened in March of 2023 when AI-generated photos of Donald Trump arrest begun to circulate the web.
Issues related to the infringement of intellectual property can arise in many phases of the lifecycle of a Generative AI model.
During the development, any data protected by intellectual property should be used to train the model only under proper licencing, otherwise there are risks of legal disputes, and dire financial consequences. A notable example is the lawsuit filed by Getty Images against Stability AI, a company that according to the accusations has allegedly used millions of images protected by copyright to train its image-generating AI model. Also, Generative AI models may use the inputs provided by the users to further expand their training data sets. An unexperienced user may imprudently share, in the performance of work-related tasks, confidential information or information that constitute intellectual property of the company.
On the other hand, the information produced by a Generative AI model can infringe intellectual property rights: for instance, a model for image generation could create images and logos that are already commercially used by another party, or an AI chatbot could generate text that has been already published.
Generative AI models could be exposed to personal information due to various reasons, for example because this information is a necessary input to train the model or because a model “captures” information from its users. Handling personal information is always a sensitive topic, especially in the case of personally identifiable information (PII) – that is, the information elements that can be used to identify, locate, or contact a specific individual.
It is paramount to ensure the proper handling of personal information by a Generative AI model: including personal information, especially PII, in the training set of the generative model could constitute a compliance issue and have legal consequences. It could even be more unfavourable if PII are included in the output of the model and shared in an uncontrolled way.
When it comes to cybersecurity, there are different risks that can arise due to the deployment of Generative AI models.
The term “Prompt Injection” refers to a specific new type of Generative AI vulnerability that consists of introducing a malicious instruction into the prompt. In some cases, this injection could cause a disclosure of sensitive information, such as a secret key or private data.
Moreover, these models could be used as tools to craft more efficient and effective cybersecurity attacks, phishing schemes, or social engineering tactics. Another potential case of cybersecurity risk arises when these models are used to support code writing in an organization: the code produced by Generative AI models can potentially introduce security vulnerabilities into the underlying codebase if the organization does not ensure the proper security checks. This is for example what happened in early 2023, when a prompt injection attack was used to discover a list of statements that governs how Bing Chat interacts with people who use the service.