As the automation of workflow progresses from simple notification triggers to core business logic, the ready to go off the shelf SaaS solution will have some limitations on the process. The automation system hosted on cloud will always be tied to execution limits, limit payload sizes, and follow billing model which discourages higher throughput. The self-hosting of n8n will give the freedom to manage data sovereignty, regulatory compliance, and cost of infrastructure when doing continuous API polling, large data processing, or webhook processing.
But operating n8n in the production environment at scale is not the same as operating a single instance of it in the local machine for home automation use cases. As the number of executions per hour increases to thousands, poorly configured instances suffer from database locking, Node.js memory leak issues, and missing webhook executions.
Decoupling Architecture: Moving to Queue Mode
The standard one-process deployment mode (EXECUTIONS_MODE=regular) operates the web GUI interface, webhook listeners, and the execution engine within a single Node.js process. With such an architecture, one big task, like processing a 100MB CSV file or image files in high resolution, can cause the triggering of the V8 garbage collector, 100% CPU load, and freezing of webhook listener ports. In such instances, webhooks are lost without any notification or get 504 Gateway Time-out responses.In order to work effectively with high traffic, the production environment needs to implement the Queue Mode, which separates the n8n app into specific functional units:
Main Node
The node responsible for the GUI, editing visual workflows, synchronization of schemas, and scheduling tasks similar to a cron schedule.
Webhook Nodes
Lightweight worker nodes dedicated only to listening for the incoming HTTP/webhooks. These nodes do not process any data but merely add payloads into the message queue and return an immediate response (200 OK/202 Accepted).
Worker Nodes
Standalone nodes for execution, which consume messages from the message queue, processes payloads, executes workflow nodes, and writes the results back into the primary database.
Redis Message Broker
The common message buffer running BullMQ where jobs are kept in memory until they are acknowledged and processed by the worker nodes. This separation prevents heavy execution tasks from degrading the responsiveness of the UI or causing webhook timeouts. When workload spikes occur, administrators can scale out the fleet of worker containers horizontally without modifying the underlying primary configuration.
Choosing Infrastructure Sizing and Optimizing Compute Instances
Given that n8n runs in Node.js, there is no way to go beyond the performance of one thread because of the single threaded nature of V8. The constraints are crucial to take into account when choosing infrastructure.
For instance, an average Node.js application cannot allocate more than 1.4 – 2GB of heap memory. Any large JSON objects will lead to OOMKilled immediately. Although it is possible to extend this constraint with the help of the “NODE_OPTIONS=”–max-old-space-size=4096″” command, the excessive heap memory that is not limited by architectural constraints will lead to inefficiency. Therefore, it is critical to choose compute instances with great performance of single core and fast IOPS.
In terms of planning the future infrastructure to ensure its reliability, one should opt for an appropriate n8n hosting service that has unmetered bandwidth along with fast NVMe storage to support webhook concurrency. In the case when the organization is running its processes with continuous streams of data, it is vital to have separate compute instances for the Redis broker, PostgreSQL database, and n8n workers in order to avoid competition between DB queries and workflows memory usage.
For organizations requiring predictable overhead in their resources without being confined by SaaS, the deployment of dedicated virtual instances through efficient n8n Cloud Hosting becomes a necessity.
Database Maintenance and Log Deletion
Out of the box, n8n keeps detailed logs of all executions in the database: parameters of nodes, input items, output items, time it took to execute, and error log. In an environment that executes 100,000 executions per day, the unattended PostgreSQL database will quickly grow into hundreds of gigabytes of binaries and JSONs.
Such buildup affects performance as follows:
- Indexing of primary keys for the
execution_entitytable grows in size, hampering read/write performance. - PostgreSQL connection pools run out of memory in keeping active tables.
- Backup and disaster recovery restoration takes unacceptably long.
To stop the exponential growth of the database, use strict pruning settings from within the n8n environment variables:
# Automated execution data pruning
EXECUTIONS_DATA_PRUNE=true
# Maximum age of execution data in hours (168 is 7 days)
EXECUTIONS_DATA_MAX_AGE=168
# Prune finished execution data, but keep failed runs for debugging purposes
EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000
EXECUTIONS_DATA_SAVE_ON_ERROR=all
EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=truePruning of successful execution bodies (EXECUTIONS_DATA_SAVE_ON_SUCCESS=none) and keeping failed execution bodies saves disk space because only important debugging data is kept. In addition, make sure to maintain the
Hardening Production Systems and Security Controls
Self-hosted automated systems often function as the nervous system of an engineering team, managing API keys, database connection strings, and OAuth tokens. Securing the host environment from any form of penetration involves the following steps:
Internal Network Segmentation
The PostgreSQL database and the Redis message broker must be contained within a network segment isolated from other services using a Docker bridge or a VLAN. They do not need to have any ports accessible to the outside world.
External Reverse Proxy
Use an external reverse proxy like Nginx, Traefik, or Caddy for terminating TLS connections. Use proper SSL cipher suites, use HTTP/2 or HTTP/3, and perform rate limiting on endpoints such as /webhook/*.
Encryption Key
The N8N_ENCRYPTION_KEY environment variable ensures that all sensitive credentials within the PostgreSQL database get encrypted. This key needs to be stored safely in an external secrets manager (HashiCorp Vault, AWS Secrets Manager). Loss of this key will result in loss of access to stored credentials.
Sandboxing of Execution
The default behavior of n8n is to allow for the execution of system commands through the Execute Command node. However, in a multi user environment, ensure that there is no arbitrary code execution and internal network communication to avoid lateral network traversal.
Operational Stability
Running an automation system that operates at high capacity needs continuous visibility. Make N8N_METRICS endpoint available in order to track the length of active queues, worker latency, and memory usage of n8n through the Grafana dashboard. Along with this telemetry, add external uptime monitoring for the webhook listener end points to avoid any transient failure affecting your business processes.
In an isolated queue architecture, aggressive database pruning, and secured boundaries, self-hosted n8n is able to offer stability, sovereignty, and economic advantage while dealing with millions of automated workflows.
