How to mitigate the risk from AI-generated apps built by your 'citizen coder' employees

AI tools let non-technical employees build workplace apps in minutes with natural language prompts. While these AI-generated apps boost productivity, they can create severe security and data risks. As Cybersecurity Awareness Month kicks off, we’re sharing how Tenable adopted a structured governance framework that allows our “citizen coders” to build secure and compliant apps with AI.

Key takeaways

1. Unsanctioned applications that non-technical staff build using AI tools often bypass quality assurance and testing, creating shadow AI risks. Without IT and security oversight, these applications can go into use with critical vulnerabilities and misconfigurations, as well as weak data protection.
2. Even when “citizen coders” comply with their organizations’ policies and processes, the number of applications they build using AI tools can become overwhelming and disruptive for the IT and security teams.
3. Governance works better than blanket bans, because prohibiting AI-aided development usually causes employees to keep their app development activities hidden. Tenable has found that implementing a structured, multi-tiered governance framework gives employee citizen coders the security and compliance guardrails they need.
4. An effective governance framework should combine peer leadership and training. While AI leaders in each department are tasked with overseeing tool approvals, managing app dev request queues, and tracking applications’ ROI, “citizen coders” should receive mandatory security and compliance awareness training.

AI tools are making it easier than ever for non-technical employees to spin up workplace applications in minutes. Even staffers who have never written a single line of code, let alone set up a database, are jumping on the AI vibe-coding bandwagon, often encouraged by their team leaders

First, the good news: These AI-generated applications can help boost employees’ productivity by automating workflows and streamlining processes. And if employees can build their own applications, the workload on your perennially busy professional developers shrinks.

Now the bad news: If employees whip up and deploy them without the oversight of the IT and security teams, these applications become shadow AI assets with security and compliance issues that put your systems and data at risk.

Even in scenarios where employee citizen coders adhere to company guidelines, the sheer number of AI-built applications they produce can easily swamp IT and security teams, a trend that’s intensifying as business leaders urge non-technical staffers to use AI to develop their own software tools.

So how can an organization address this threat from employee citizen coders? Spoiler alert: A draconian, across-the-board prohibition won’t work, and in fact will likely backfire, as employees may then deliberately try to hide their AI-aided application development activities. 

In this blog, the first in Tenable’s weekly Cybersecurity Awareness Month series, we’ll share lessons learned and concrete best practices from Tenable’s internal cybersecurity team aimed at establishing policies and controls designed to curb this shadow AI risk from your employee citizen coders.

The risks from citizen coders’ applications

The cyber risks of unsanctioned applications that employee citizen coders build aren’t new. The issue rose to prominence years ago when low-code / no-code development platforms gained widespread popularity. 

Of course, generative AI tools that can create fully-functioning applications from a natural-language prompt have exacerbated the issue by turning practically anyone into a developer. And unlike low-code / no-code platforms hosted as software-as-a-service (SaaS) and monitored by the organization, generative AI products tend to be consumer-grade tools that employees can access and use individually.

These citizen coder applications can create a wide variety of security and compliance risks, as organizations, including the Open Worldwide Application Security Project (OWASP) in its OWASP Citizen Development Top 10 list, have documented.

Below, we highlight common security and compliance issues in AI-generated citizen-coder applications that employees create without permission and oversight from the IT and security departments:

• Due to lack of testing, scanning, and quality assurance checks, their code can have critical vulnerabilities and dangerous misconfigurations, as well as contain risky open-source components.
• They may insecurely access company critical systems and store sensitive data.
• They will not be updated, patched, monitored, logged, nor be included in backup and disaster recovery plans.
• They may have excessive permissions and privileges, and the organization’s identity and access management (IAM) systems won’t protect them.

Even when employee “citizen coders” alert and involve the IT and security teams, organizations are finding that the number of these applications has spiked to such a degree that it’s become burdensome to review, approve, and onboard them. 

In other words, in an average large organization there may be hundreds of employees building applications that they feel are worth seeing the light of day. IT and security suddenly face the daunting task of assessing each one, and deciding which are truly worth the cost and effort of securely deploying and maintaining them throughout their lifecycle.

If these applications consume AI tokens to function, costs can add up quickly. For example, it’s not uncommon for citizen coders to leverage their legitimate access to business applications and create their own local data lakes to help power their vibe-coded application. In addition to the cost issue, the creation of these siloed data lakes creates dangerous data sprawl. 

A five-tier governance model for AI use

At Tenable, we allow non-technical citizen coder employees to develop workplace applications, and as such, we’re not immune to the risks outlined above. However, we have found that establishing a strong governance foundation goes a long way towards preventing problems, not just the ones associated with citizen coders but with overall employee AI usage.

Specifically, Tenable has implemented a comprehensive five-tier AI governance framework, where at one end the executive leadership sets the direction, while at the other end lies daily community use guided by clear policies and peer oversight.

In the case of citizen coders specifically, this means, for example, that the do’s and don’ts of AI-aided citizen coding aren’t determined by individual business units independently, but are rather established uniformly company-wide.

These are the five tiers of Tenable’s AI Governance Model.

• Tier 1: Strategy 

The Executive Staff charts the course by providing strategic alignment, investment guidance, and prioritization for AI initiatives.

• Tier 2: Governance

An AI Governance Board and AI Technical Council create AI policies and guidance documentation, such as lists of approved AI tools. They explicitly own compliance, data privacy, and model risk, while also overseeing AI tool enablement, architecture, and design.

• Tier 3: Execution

AI Functional Leads and R&D Champions drive specific use cases, manage departmental adoption, and measure productivity results.

• Tier 4: Enablement

The Enablement Working Group, led by IT, Learning & Development (L&D) and Corporate Communications, handles hands-on training, resource distribution, and enterprise demonstrations.

• Tier 5: Community

Dedicated Slack channels for AI usage questions, AI engineering, and other topics offer employees support and a forum for crowdsourcing solutions.
 

With respect to citizen coders specifically, this AI governance model helps Tenable ensure that our IT and security experts establish foundational protections, guardrails, and controls for AI-generated applications. That’s because groups within the governance framework own the requirements for issues such as data privacy, compliance, and model risk, as well as for AI tool enablement, design, and architecture.

The critical role of AI Functional Leaders

A key element for mitigating citizen coder risks are Tenable’s AI Functional Leaders, which sit on the governance framework’s third tier — execution. 

Originally conceived as simple project coordinators, this group now operates within a strict governance role with direct authority and accountability over AI skill approvals, operational queues, and organization-wide AI adoption.

These leaders are accountable for all AI use, development, and deployment across their respective departments. For example, there are AI Functional Leaders in sales, tech support, marketing, human resources, legal, infosec, finance, engineering, product management, and several other departments.

AI Functional Leads are heavily involved in their department’s use cases. By overseeing all requests for new skills, connectors, and data access, the AI Functional Leads can flag, for example, the use of unvetted external AI components in citizen coder applications before they reach production.

The importance of AI security awareness training for citizen coders

Security and compliance issues related to AI usage, including applications citizen coders build, usually stem from a lack of knowledge about how to prevent these problems. That’s why Tenable has tied AI tool access directly to mandatory security training and responsible usage training, and in turn complements it with live webinars, pre-built courses, tool-specific deep dives, and weekly show-and-tell sessions. This way, employees in general, and citizen coders in particular, know they have access to vetted resources and expert peer support.

Looking ahead

The citizen coder revolution is here to stay. Non-technical employees equipped with generative AI tools will continue to build applications, especially as their team leaders nudge them to experiment with vibe coding.

At Tenable, we have found that implementing a solid governance framework provides a critical foundation for mitigating the risks associated with employees’ AI use, and specifically offers citizen coders guardrails to securely build AI-generated applications that help boost their productivity and efficiency. 

Have we completely solved the security and compliance challenges from citizen coder-built applications? Not by a long shot. The risks from employee vibe coding, along with those from AI use in general, are constantly morphing. But we feel that our governance framework gives us a solid foundation for tackling these and other AI usage challenges, as they emerge and evolve. 

Our Cybersecurity Awareness Month blog series continues next week, when we’ll tackle the challenges that critical infrastructure organizations face today.