ChatGPT Training and Its Impact on Workflows

The rise of Generative AI tools, has sparked a wave of excitement among employees eager to simplify workflows and enhance creativity. ChatGPT, for instance, is often celebrated for its ability to handle tasks ranging from drafting emails, to generating ideas. However, this enthusiasm often clashes with corporate policies that often restrict its use.

The hesitation arises from significant security concerns, encompassing risks such as copyright infringement, breaches of privacy laws, and the potential exposure of sensitive company trade secrets. However, when restricting ChatGPT usage at work, employees may resort to shadow usage unless the company provides a suitable alternative.

This article explores the challenges and opportunities of ChatGPT in the workplace, balancing its productivity benefits with security concerns. We examine key scenarios and solutions to help companies embrace generative AI while mitigating risks, offering actionable insights for innovation and security.

Everybody Loves ChatGPT!

ChatGPT has quickly become a favourite tool for employees, often surpassing Google Search in usefulness. From drafting emails and summarizing reports to brainstorming ideas, it delivers fast responses or at least a strong starting point. Many employees have also begun exploring its advanced capabilities, such as reasoning and image-to-text conversions for AI content generation tasks across various sales and marketing functions.

Companies are understandably cautious about ChatGPT use, especially amidst rapidly evolving AI regulation. Here's a concise list of the key risks businesses face:

1. Copyright Infringement: This risk arises when employees unknowingly input or generate content that infringes on intellectual property laws. ChatGPT is trained on vast amounts of publicly available data, some of which may include copyrighted material. While OpenAI states that it does not claim ownership of user inputs or outputs, the responsibility for how generated content is used lies with the user, the company is this case.
2. Data Leakage: Employees may unintentionally input sensitive company data into ChatGPT. While OpenAI offers options to disable data retention, misconfigurations or user mistakes could still lead to data exposure, potentially breaching privacy laws and confidentiality agreements. For example, submitting customer data could violate strict privacy regulations in industries like healthcare or finance.
3. Misinformation: ChatGPT can sometimes produce incorrect or misleading information, potentially leading to poor business decisions and errors in customer communication. To mitigate these risks, adopting a Human-in-the-Loop (HITL) approach ensures external communications are validated before being shared.

Alternatives to ChatGPT

Many organizations view Retrieval-Augmented Generation (RAG) as a safer alternative to ChatGPT. RAG combines GenAI with a company’s internal data sources, ensuring secure and controlled retrieval of information to generate responses. While RAG minimizes the risk of hallucinations, it cannot eliminate the risks entirely. To prevent data leaks, companies often restrict the LLM’s direct internet access, limiting the RAG system's ability to retrieve real-time or recent information. Let's not forget that there is a cut-off date for LLMs, leaving it unaware of events or developments beyond that point.

When disconnected from direct internet access, a RAG system retrieves data exclusively from a company's curated internal sources or databases. This ensures the model processes information that is controlled, relevant, and secure. However, employees may continue to rely on ChatGPT, even at work, and may question whether RAG is a viable alternative. This phenomenon is called shadow usage of ChatGPT.

Here are five scenarios to consider, assuming companies have implemented a RAG system:

1. Allow ChatGPT use, but tightly managed
2. On-premises LLM
3. Direct link to the LLM (Cloud)
4. Indirect link to the LLM through your cloud provider
5. ChatGPT Enterprise

Scenario 1: On-Premises LLM

In this scenario, you run the LLM on-premises. It will make the system more responsive and the risk of data breaches is significantly reduced. For a large company, the decision to transition to cloud or stay on-prem is often driven by data sensitivity, regulatory constraints, long-term cost efficiencies, and the need for customization and independence. Investment in infrastructure or a datacenter may be expensive, but it could be beneficial in the long term.

One of the key components of RAG is the famous vector database. A vector database stores and organizes the company's private data as chunks (vectors) to quickly find and retrieve relevant information, given a prompt. There might be a business case to deploy this vector database on-prem. If on-prem isn't feasible, consider using a classifier between the user and the retriever, to determine whether a retrieval operation is needed, given a prompt. This reduces API calls to the vector database, hence cloud costs.

Scenario 2: Direct link to the LLM (Cloud)

A company may opt for a direct and secure connection to the LLM, hosted in the cloud. The direct API access to GPT-4 is secure if you configure it properly:

1. Encryption: Ensure all API calls use TLS (Transport Layer Security) to encrypt data in transit
2. Access Controls: Use GPT-4 API keys or tokens with strict access controls to authenticate requests.
3. Data Retention: OpenAI offers options to disable data logging for API calls. This ensures that your input/output data isn't stored or used for model training. Explicitly confirm with OpenAI's documentation or support team that the data isn't being retained.

Scenario 3: Indirect but Secure link to the LLM through your cloud provider

Companies may choose to access the LLM through their cloud partner. Cloud providers provide encryption, audit trails, and regulatory compliance, offering peace of mind for many enterprises. However, keep in mind that every API call carries a cloud fee. It is good practice to implement cloud FinOps processes to prevent bill shock.

Scenario 4: ChatGPT Enterprise

For companies wanting a plug-and-play option, ChatGPT Enterprise could be worth considering. However, while OpenAI assures data protection, many businesses remain sceptical, particularly in highly regulated industries like finance or healthcare.

Scenario 5: Allow ChatGPT use but tightly managed

Generative AI isn't going away and blocking ChatGPT outright isn't a sustainable strategy. Employees will find workarounds, and might potentially expose the company to even greater risks. The key is to strike a balance between enabling employees to harness these powerful tools and ensuring security, compliance, and cost efficiency.

It is essential to educate employees on the risks of ChatGPT and provide clear guidance on safe usage. Companies should establish a comprehensive AI policy, incorporating AI risk management into their risk register, with oversight from the risk committee to ensure continuous monitoring and mitigation.

This issue ultimately points to a deeper organizational gap: AI Training. Without proper training, employees often resort to shadow usage simply because they lack the skills, guidance, or sanctioned tools to work efficiently with AI. Companies that proactively invest in AI Training empower their staff to use generative tools safely, responsibly, and in line with internal policies. This training becomes particularly critical when organizations deploy AI Agents for sales and marketing automation, where proper governance ensures both productivity gains and regulatory compliance.

These five scenarios are not mutually exclusive and many companies will adopt a hybrid approach, depending on their priorities.

Employees must understand the difference between legitimate AI capabilities and misconceptions about a model's reasoning capabilities and consciousness, to set realistic expectations and appropriate governance frameworks.

Conclusion

While corporate policies are understandably cautious, outright restrictions may inadvertently push employees towards shadow usage. Educating employees on responsible AI use, investing in robust infrastructure, and embracing hybrid models are critical steps toward fostering a secure, innovative workplace.

The path forward requires comprehensive AI Training programs that address both technical competencies and ethical considerations, ensuring employees can leverage AI tools effectively while maintaining security and compliance standards.

Need help with AI Training? Rainmakers SG offers a variety of training programs including 1:1 Executive Training.