Close today

Go to Top

Go to Top

Security Insights

Security Insights

Security Insights

Can We Use AI-Generated Programs Directly in the Workplace?

Can We Use AI-Generated Programs Directly in the Workplace?

Can We Use AI-Generated Programs Directly in the Workplace?

Kim In-soon

Kim In-soon

Content

Content

Content

These days, companies often hold internal hackathons to encourage employees to use AI more in their daily work. Staff describe their ideas to AI and quickly build and showcase programs or websites that assist with their tasks. It is a welcome shift that reduces repetitive work and lets them test new ideas.

Some tools made in these contests get great reviews and end up used in actual work. The UI loads well, and the necessary features run smoothly. Teammates try it out, say, “This is so convenient!”, and start using it regularly.

Yet, does this software handle internal company data or customer details securely? If the creator moves teams or resigns, can someone else maintain it? If an issue arises, who takes responsibility, and how can it be shut down?

As 'vibe coding'—creating software by describing desired features to AI—spreads, these questions become critical. Enabling non-developers to rapidly build tools and websites is a huge plus. It also cuts the time and cost to test ideas. 

However, just because a program works well does not mean it is secure. To use AI-built tools in business, one must check security and operations, not just basic functionality.

Working well is different from being safe

AI helps quickly build functions requested by users. However, it does not automatically guarantee that the output is secure or maintainable by the company over the long term.

For example, even if a login screen works normally, vulnerabilities may exist that allow viewing info via another user's account. Attackers might access the database if input values are not properly validated. Such issues are hard to catch just by looking at screens; both code and real behavior must be verified.

If a company uses AI-generated software without thorough review, the following issues can occur.

First, security vulnerabilities may remain.

Insufficient input validation or account privilege checks can expose user data incorrectly or let unauthorized persons access features.

Second, passwords or API keys might be hardcoded.

If credentials required to connect to external services are written directly in code, exposing or sharing that code can create a pathway into company systems. Credentials must be kept separate from code and managed securely.

Third, company information may be leaked outside.

Programs connected to AI services may be set to send employee input or company docs to external AI. You must verify what data goes where first. Check if the AI service is approved, and whether entered data is stored or reused.

Fourth, unmanaged software may proliferate across the company.

If a simple tool built by an employee connects to internal databases or external services, it can quickly become vital to business operations. However, if left out of formal development processes and asset management, it may miss security reviews or updates. Shadow IT increases. 

Fifth, updates may be discontinued.

If open-source components or external services used in a program fail, fixing them is hard without a designated owner. If only the creator understands its structure and moves to another role or leaves, diagnosing failures becomes difficult. 

Sixth, humans may not fully understand the AI-generated code.

AI can generate and modify a vast amount of code in a short time. However, if humans do not review the changes, new bugs arising from interactions among various functions may go undetected.

Ultimately, what matters is not just whether AI wrote the code. You must also verify which information and systems it links to, who holds permissions, and who updates and takes responsibility for it.

Personal tests and company services differ

Personal trial tools differ greatly from company tools used across teams. Even in personal tests, using company accounts or real customer data carries risk. Thus, you must use mock or limited test data from day one and test within environments isolated from production.

Once used company-wide, far more factors must be considered. Employees depend on it, and customer or company data may flow through. A crash can halt work. Moving a personal prototype into official service requires security verification and assigned owners.

There is no need to ban vibe coding. Keep the benefit of rapid prototyping and testing, but set review processes before adopting it into company workflows.

The first step is building a prototype.

Bring ideas to life fast with AI. Use test data and isolated setups here, never inputting sensitive files like customer or HR records.

The second step is a small-scale pilot.

Rather than launching company-wide, involve only a few users. Limit the scope, blocking access to live data or critical systems. Test not just intended features, but also check for any unexpected behaviors.

The third step is security review.

At this stage, inspect the code alongside real behavior. Check for unauthorized account access, hardcoded passwords or API keys, and data leaks to external services. Verify that the tool holds only the bare minimum permissions needed.

Automated security tools can also help. Static Application Security Testing (SAST) inspects code for vulnerabilities. Dependency checks scan external libraries or open source for known issues. Dynamic Application Security Testing (DAST) runs the app live to test logins or input handling.

While these scans help catch issues, tool results alone do not guarantee full safety. Owners must review code, permissions, and data flows, consulting experts if needed. Security needs are best checked iteratively in small steps during development, rather than all at once at the end.

The fourth step is running official operations.

After security reviews, set operators and update flows. Define incident responders, kill switches to stop services before damage spreads, and disaster recovery plans. Prepare code and operational handovers in case creators switch roles or leave.

Follow this order: Prototype → Pilot → Security Review → Official Operations. Security is not limited to step three alone. Restrict data and permissions from day one, and verify continuously through testing and release.

AI checks are the start, the final call is human

You can ask an AI to review the code it wrote. For example, you can request: "Explain the issues in this code by risk level from a security officer's view." You can also compare review results by using another AI.

Still, an AI's review should not be taken as final approval. Even if another AI checks the code, both systems may miss the exact same issues. Also, looking at code alone rarely reveals actual company access rules or data flows.

Use AI checks as a first step to find issues, but the final call must be made by a human. Check which info and systems the code accesses, granting only needed rights. Human sign-off is also key for high-impact tasks like sending vital data out or changing account rights.

Questions leaders must ask

When an organization evaluates new software, asking "Does it work well?" is simply not enough. 

You must consider the fallout if it is hacked or fails, and what company data and systems it accesses. You need to know who can fix it if the author is absent, and if work halts when external AI or tools go down. Know who manages updates and security audits and how often, as well as how to stop the tool and roll it back if issues arise. 

If you cannot answer these questions, it is not ready for official company rollout.

AI can write code, but the organization bears the duty to approve it and run it safely. Small pilots, ongoing audits, clear owners, and update pipelines. Building this process protects vital company data while keeping the brisk pace of vibe coding.

Kim In-soon

Kim In-soon

Start-up College Adjunct Professor at Gachon University
Start-up College Adjunct Professor at Gachon University

Former desk member of the Electronic Newspaper ICT Convergence Department, active as a cyber security journalist and communication expert for 20 years.

Former desk member of the Electronic Newspaper ICT Convergence Department, active as a cyber security journalist and communication expert for 20 years.

The Beginning of Flawless Security System, From the Expertise of the No.1 White Hacker

Prepare Before a Security Incident Occurs

The Beginning of Flawless Security System, From the Expertise of the No.1 White Hacker

Prepare Before a Security Incident Occurs

The Beginning of Flawless Security System, From the Expertise of the No.1 White Hacker

Prepare Before a Security Incident Occurs

Subscribe

Find this content useful?
Subscribe to the Enki Letter!

Copyright © 2025. ENKI WhiteHat Co., Ltd. All rights reserved.

Copyright © 2025. ENKI WhiteHat Co., Ltd. All rights reserved.

Copyright © 2025. ENKI WhiteHat Co., Ltd. All rights reserved.